explainx.ai350K+ learn from us
TrendingAI News TodayExplainx for Business
Pricing
explainx.ai

Upskill in AI — 16 free pathways, live workshops & bootcamps, and 50+ courses from practitioners. Plus the skills, tools, and MCP servers to practice on.

follow us

follow on google

Add explainx.ai as a preferred source

corporate training

support@explainx.ai

get started

Find your pathTake Free Evaluation

community

Join the community

learn

mind: share how you thinkpathways — start freeworkshopsbootcampscoursescompare Explainxcertificationsmock testsexplainx universitycorporate traininglearn skills & mcp

discover

skillsmcp serversexplainx mcpestacktoolsmdx readeragentsllmsdesignsdictionarypeopleagi trackerfelony benchranks

company

aboutvisionmissionteaminstructorsteach on explainxpartnershipscommunityhackathonscareers

content

daily AI newsstate of AI — live resultsblogreleasespromptsgeneratorsresource libraryfor LLMsexplainx.ai kids

solutions

all solutionsdeveloper upskillingmarketing upskillingproduct manager upskillingleadership upskilling

newsletter · weekly

Get AI news, tools, and insights in your inbox.

supportcontactprivacytermsdata rightshow we create contentsubmission guidelines

© 2026 AISOLO Technologies Pvt Ltd

explainx.ai

On this page

  • Two panels: EF and FCD
  • Quick start without spending API dollars
  • Live evaluation: authorization and judge setup
  • Built-in search adapters
  • Why freshness benchmarks matter for agents
  • EF judge pitfalls
  • FCD timing constraints
  • How this fits Microsoft's grounding product line
  • Builder workflow
  • Step-by-step live pilot (authorized only)
  • Weather versus Finance judge shapes
  • Custom providers and BYO search
  • Security and data handling for eval teams
  • When not to use FreshSearch
  • Agent harness integration
  • Microsoft WebIQ in context
  • Repository layout (benchmark directory)
  • Head, torso, and tail entities
  • Finance task window cheat sheet
  • FCD five-minute deadline in plain language
  • Contributing and license
  • Open questions for Microsoft
  • Comparison to other freshness evals
  • Limitations Microsoft documents
  • Related reading
  • Workshops
← Back to blog

explainx / blog

Microsoft FreshSearch: Benchmark Toolkit for Stale Search Grounding

Microsoft, RAG, Benchmarks, Search

Part of Microsoft, Apple and Amazon AI

Microsoft open-sourced web-grounding-evals on GitHub to score Evidence Freshness and Fresh Content Discovery for AI search providers with offline smoke tests.

Nov 19, 2026·8 min read·Yash Thakker
add explainx.ai
go deep
Microsoft FreshSearch: Benchmark Toolkit for Stale Search Grounding

Microsoft open-sourced FreshSearch — a search-grounding benchmark in the web-grounding-evals GitHub repository — to measure whether AI search providers return fresh evidence and find newly published feed articles. The README frames the problem plainly: developers need to know if grounding keeps answers current, not only if models answer familiar questions.

The repo is a new Microsoft org release (candidate feed lists it November 18, 2026) with offline smoke tests that never call providers, plus authorized live evaluation paths that demand --live and --confirm-authorized-use flags.

Two panels: EF and FCD

Search-source threads linking a query to evidence cards, illustrating how grounding picks cited materialSearch-source threads linking a query to evidence cards, illustrating how grounding picks cited material

table · 3 cols
PanelDatasetWhat it measures
Evidence Freshness (EF)current-launch-ef120.json — 60 finance + 60 weather tasksAge of evidence supporting time-sensitive answers
Fresh Content Discovery (FCD)current-launch-fcd96.json — 96 feeds (4 paywalled feeds excluded at assembly)Retrieval of ten newest eligible feed items, weighted toward newer posts

Microsoft combines EF and FCD 20% EF / 80% FCD when cohort requirements in METHODOLOGY.md are met. Subsets are exploratory.

This is complementary to Google AI Mode temporary threads product news: FreshSearch is measurement, not a consumer feature launch.

Quick start without spending API dollars

A hand pulling the freshest search result card to the front of a stackA hand pulling the freshest search result card to the front of a stack

From the benchmark/ directory:

bash
npm ci --ignore-scripts
npm test
npm run smoke:ef
npm run smoke:fcd

Expected offline scores: EF 62.5 and FCD 51/93 on synthetic fixtures. Outputs land in local-runs/ef/ and local-runs/fcd/.

Builders already running RAG pipelines can treat smoke tests as CI guards before wiring a provider into production agents.

Live evaluation: authorization and judge setup

Live runs require:

  • Provider API keys (WEBIQ_API_KEY, EXA_API_KEY, etc.)
  • A judge configuration (Azure OpenAI templates ship in examples/)
  • --confirm-authorized-use — your attestation that publisher and provider terms allow the crawl pace you set

Microsoft stresses robots rules are not licenses. Metadata-only output files cannot replay captured publisher text — important for compliance teams comparing FreshSearch to internal crawlers.

Budget example for five providers on the full cohort (from README):

  • 4,800 FCD searches (96 feeds × 5 providers × 10 targets)
  • 600 EF searches (120 tasks × 5 providers)
  • Judge calls can exceed 1,200 with confirmation and repair steps

Finance tasks only run 10:00–16:00 America/New_York on exchange days — a detail agents automating benchmarks must schedule around.

Built-in search adapters

table · 3 cols
ProviderEnv keyNotes
WebIQWEBIQ_API_KEYhttps://api.microsoft.ai/v3/search/web
WebIQ AutoWEBIQ_API_KEYAuto endpoint variant
Exa AutoEXA_API_KEYThird-party
ParallelPARALLEL_API_KEYThird-party
PerplexityPERPLEXITY_API_KEYThird-party

Microsoft notes WebIQ is Microsoft-developed but not an official ranking. Compare with Bing grounding stories on how Microsoft ships eval culture internally.

Why freshness benchmarks matter for agents

A search bar feeding one answer card with source markers, like AI Overviews-style groundingA search bar feeding one answer card with source markers, like AI Overviews-style grounding

Coding agents increasingly call search tools (OpenAI research acceleration documents soaring internal agent spend). Stale snippets produce confident wrong answers on earnings, weather warnings, and policy shifts.

FreshSearch encodes head/torso/tail entity tiers so providers cannot overfit famous tickers alone. That mirrors jagged frontier language across OpenAI and Google science reports.

EF judge pitfalls

README warns a successful "return JSON" probe does not prove scoring compatibility. Pilot at least one scored Finance and one scored Weather task before full runs — categories use different judge budgets and evidence shapes.

Teams using Microsoft Aion browser models for local judging still need parity checks against Microsoft's reference judge configuration.

FCD timing constraints

Each feed run has a five-minute execution deadline. Provider searches must finish before reference material ages out; aggressive --search-delay-ms pacing can leave targets held rather than scored. README says never shorten pacing below account-required intervals to force scores.

How this fits Microsoft's grounding product line

FreshSearch arrives beside:

  • HorizonDB for agents at Ignite — different layer (database versus search)
  • Microsoft IQ and Foundry connectors at Ignite

Nothing in the repo ships a hosted leaderboard URL; you self-host HTML via node report.mjs.

Builder workflow

  1. Clone github.com/microsoft/web-grounding-evals
  2. Smoke offline in CI
  3. Authorize one feed and one EF task in a pilot directory
  4. Scale providers only after summary.json shows status: "complete"
  5. Log outputSha256 and git SHA when publishing results internally

Step-by-step live pilot (authorized only)

Microsoft's README walks operators through a small live pilot before the full 4,800-search FCD marathon:

  1. List feed IDs offline with the bundled Node one-liner
  2. Pick one feed your contracts allow
  3. Run evaluate.mjs with --max-fcd-calls 10, --max-ef-calls 1, and documented --search-delay-ms
  4. Inspect local-runs/live-pilot/summary.json for status: "complete"

Pilot failures categories — rate limits, judge truncation, held FCD targets — are documented under failures and partial runs in the runner guide. Do not treat a failed pilot as a zero score; treat it as incompatible pacing or judge config.

Weather versus Finance judge shapes

Weather EF tasks carry timezone and ICU versioning notes in the README because evidence timestamps depend on regional weather services. Finance tasks face NYSE window constraints and early-close calendar rules baked into the dataset through 2028.

Agents automating benchmark runs should tag runs with Node and ICU versions the way the README recommends; otherwise year-over-year comparisons lie.

Custom providers and BYO search

Microsoft ships examples/provider.mjs as a custom adapter contract. Enterprises with private crawl indices can plug internal search without publishing captures, but they still owe publisher authorization for any live FCD feed they include.

That pattern mirrors private RAG deployments (vector DB comparisons) where legal review precedes engineering sprints.

Security and data handling for eval teams

Live runs keep provider and publisher content in memory only during execution. Output artifacts are metadata-only and cannot rebuild HTML bodies. Teams sharing results.json externally still leak URLs, timings, and fingerprints — review before posting to blogs.

Built-in requests ask caches not to store responses; EF judge calls set store:false. Microsoft explicitly says that does not guarantee third-party retention — align with your DPA before pointing FreshSearch at customer-owned feeds.

When not to use FreshSearch

Skip FreshSearch as a daily CI job on live keys — costs explode. Use offline smoke for regression and schedule quarterly live runs when providers change ranking models.

Skip comparing FreshSearch scores to chatbot leaderboards; Microsoft disclaims ranking intent.

Agent harness integration

If your harness calls search via MCP:

  • Log EF/FCD proxies on golden tasks even when not running the full benchmark
  • Cache reference snapshots with TTL aligned to your product SLA
  • Fall back to primary sources when FCD scores drop — pattern used in OpenAI research acceleration internal eval culture

Microsoft WebIQ in context

WebIQ endpoints (api.microsoft.ai/v3/search/web and /v3/auto) sit in the same ecosystem as Copilot grounding refreshes covered during Ignite 2026. FreshSearch is how Microsoft invites third parties to compare against its adapters — not how consumers see rankings in Bing.

Repository layout (benchmark directory)

table · 2 cols
PathRole
datasets/EF120 and FCD96 JSON
providers/Adapter modules
examples/Synthetic captures and judge JSON templates
run.mjsOffline scorer
evaluate.mjsLive orchestrator
report.mjsHTML report builder

Clone URL in candidates: github.com/microsoft/web-grounding-evals (README clone example also references grounding-evals naming — verify which repo your tarball uses before filing issues).

Head, torso, and tail entities

EF tasks bucket entities into head, torso, and tail frequency tiers so providers cannot optimize only for mega-cap tickers or flagship cities. METHODOLOGY.md warns subsets are not population estimates — interpret exploratory cuts carefully when blogging leaderboard takes.

Finance task window cheat sheet

table · 2 cols
SituationFreshSearch behavior
Weekday before 10:00 ETFinance EF tasks held
Exchange holidayHeld per bundled 2026–2028 calendar
Early close dayWindow may end 13:00 ET per README
Custom --ef-input fileOrder preserved; windows still apply to Finance category

Schedule cron jobs in America/New_York, not UTC, when running unattended eval agents.

FCD five-minute deadline in plain language

Reference capture starts a clock. Each provider search must finish while the snapshot is fresh. Slow --search-delay-ms values or two-minute attempt timeouts can leave targets held — visible in issues.csv — rather than scored poorly. That is missing data, not a low FCD score.

Teams publishing internal leaderboards should attach held-rate percentages beside FCD points so executives do not confuse infrastructure timeouts with retrieval quality. Export those fields from issues.csv whenever an automated weekly report goes to leadership.

Contributing and license

The README points contributors to standard Microsoft open-source governance in-repo. Forks should not republish live capture metadata containing publisher text; respect the metadata-only live output rule.

Open questions for Microsoft

The repository does not yet publish a hosted results dashboard or monthly reference scores. Watch the GitHub releases tab for dataset version bumps to current-launch-ef120.json and current-launch-fcd96.json.

Comparison to other freshness evals

table · 3 cols
ApproachFreshSearchTypical RAG unit tests
ScopeMulti-provider search + feedsSingle index snapshot
JudgeLLM-scored evidence ageString match
CostPotentially thousands of paid callsCheap
GoalProvider comparisonRegression guard

Pair FreshSearch with application-level evals (Supercite shopping study) when your agent calls search mid-task.

Limitations Microsoft documents

  • Combined scores need full cohort eligibility
  • Interrupted runs may incur paid provider charges without complete scores
  • --continue-after-failure yields partial status with exit code 2
  • Timezone semantics bind Weather tasks — do not pool across ICU versions blindly

Related reading

  • Vector database RAG compared
  • Google AI Mode privacy threads
  • OpenAI agent spend surge
  • Microsoft GitHub AI findings pipeline
  • Higher-scoring coding models natural experiment
  • What is an agent harness?
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

Workshops

Teams benchmarking search for Copilot Studio agents should pair FreshSearch pilots with explainx.ai's Microsoft Copilot workshop for governance patterns.

Spotted something out of date? Let us know.
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

  • ✓ Free
  • ✓ One email a day, no spam
  • ✓ Unsubscribe anytime
Yash Thakker

Written by

Yash Thakker

Yash is an AI expert with over 300K learners. Join his workshops →

View Yash Thakker in People in AI →

Related posts

Sep 10, 2026

Perplexity Q2D-Web: 190M-Doc Benchmark for Agentic RAG Retrieval

On September 10, 2026, Perplexity published Q2D-Web (Query2Doc-Web) — a large-scale benchmark for first-stage retrieval in agentic RAG systems. Built from 23,000 PII-free production searches over nine months, it pairs 190 million web documents with 69,721 agent-reformulated queries, ten languages, and an average of 99.6 positive relevance judgments per query. explainx.ai breaks down why the benchmark exists, how it differs from MS MARCO Web, and what it means if you ship embedding models or agent search stacks.

Nov 11, 2026

Token Pooling-Aware Training: Cheaper ColBERT-Style Search on Hugging Face

Multi-vector retrievers like ColBERT assign one embedding per token, which explodes index size. A November 11 Hugging Face blog trains models to stay accurate after Ward clustering merges similar token vectors, with open checkpoints on the Hub.

Oct 4, 2026

ThinkingBox: Microsoft Grades AI Agents on Database State, Not Chat

Microsoft researchers released ThinkingBox, an open sandbox and benchmark that checks what an agent actually changed in a backend instead of what it said. Across 121,680 trials, 79,853 failed attempts, and two-thirds of those ended with no tool error. Here is what the numbers show and how to apply the idea to your own agents.