More accurate and less expensive: grounding, benchmarked
We asked four models twenty live-data questions three ways — alone, with web search, and grounded through Pipeworx. Grounded, they answered 96% correctly and got nothing wrong. The receipts are public.
There’s an assumption baked into most AI budgets: better answers cost more. Bigger model, higher token rate, better result. For reasoning and writing, that’s often true. For questions about the live world — what a company just filed, what a rate is today, what an agency approved this week — we kept seeing something different in our own usage, so we measured it.
We took twenty questions with verifiable, current answers — the kind agents get asked every day: a figure from the latest SEC filing, a drug shortage status, a court’s holding, an inflation print, a market price. Five finance, five pharma, five legal, five everything else. Ground truth came from the primary sources themselves, captured the same day. Then we asked four models — Claude Fable 5, Claude Opus 5, GPT-5, and Kimi K2 — each question three ways: the model alone, the model with the best web search it can actually run, and the model connected to Pipeworx over MCP.
The full run, per-question receipts included, is at pipeworx.io/benchmark. Here’s the shape of it.
Alone, the models got 1–3 of 20
Not because they’re bad models — because the answers aren’t in the weights. Most attempts were honest refusals: “I don’t have real-time access,” “check EDGAR.” Honest, but not an answer. Across all four models, 86% of questions got no answer at all. If your product needs live facts, model scale is not the lever.
Web search helped — and started inventing things
With search, scores rose to 6–13 of 20. Real improvement, and it costs real money: Fable 5 with server-side search that fetches whole pages spent $13.00 on twenty questions, because search stuffs pages into an expensive context window and hopes the right line is in there somewhere.
The more interesting number is the one accuracy alone hides. Alone, the models declined 86% of the time and stated a wrong value on 3%. With search, declines fell to 36% — and wrong answers rose to 15%. Search converts silence into speech. It does not convert it into accuracy. A model that says “I don’t know” is behaving correctly about the limits of what it can see; a model that returns a confident wrong figure is the failure that actually costs you something.
Grounded, every model got better and less expensive at once
Connected to Pipeworx, the models scored 18–20 of 20 — above every one of their own search runs — while spending a fraction of the money. Across all four, 96% correct, 4% declined, and not one wrong answer. What they missed, they said they missed.
Claude Fable 5 grounded went 20/20 for $1.26, against 12/20 for $13.00 with search. Same model, same questions; the only thing that changed is where the facts came from.
And the result that reframes the budget conversation: Kimi K2, an open-weights model, answered 19 of 20 for $0.14 — one behind the frontier models, at roughly a hundredth of what searching cost them. Once the facts come from the source, the model’s job shrinks to reading and judgement, and a small fast model reads records just fine.
The part that decays
One question had to be thrown out of this set, and why is worth saying: “how many orbital launches are scheduled in the next seven days” has no single true answer. Our capture read 25, three grounded arms independently read 10, and another counted 5 — every one of them defensible, because schedules move and different sources count different things. It was replaced rather than scored.
That is the honest edge of a benchmark like this. A score is a snapshot; freshness is a property. The questions here were chosen because their answers move, and answers built on live records stay right after the blog post about them is published.
What we’d tell a builder
Pick your model for what models are good at — reasoning, judgement, writing. Pick grounding for facts, and stop paying frontier token rates for recall that no model can do. A routed, cited answer through the gateway costs about half a cent, and every response carries _meta.source, _meta.fetched_at, and a pipeworx:// citation URI your agent can show its user as evidence.
Caveats, because we’d rather you trust the next number too
Twenty questions is enough to see gaps of this size; it is not a precise accuracy claim. Every answer was graded blind — the judge sees the question, one answer and the ground truth, never which model or arm produced it. Search costs are not comparable across models: each one uses the strongest search it can run, and those differ enormously in how much of a page they pull into context, which is most of what the cost column measures. Compare each model against itself. The question set deliberately measures the freshness gap, not general intelligence. All of it, including the raw rows, is on the benchmark page.
See it yourself
The free model on pipeworx.io/try is the same grounded setup. Ask it something that changed this week — no signup, and check the citation that comes back. Or point your agent at the gateway and ask through your own stack.