Can an AI search agent learn to look for evidence, not just get the answer right?
A peer-reviewed four-benchmark study found that rewarding useful search steps improved unfamiliar question answering and evidence retrieval. The tests used fixed academic datasets, GPT-4o annotations and extra inference compute—not live web research or outcomes for real users.
The short version
- What happened
- Researchers evaluated 3,398 questions across HotpotQA, 2WikiMultihopQA, Bamboogle and the English MedQA exam set, using different metrics for open-answer and multiple-choice tasks.
- Who it affects
- For clinicians, teachers, analysts and public-service teams, the practical lesson is to inspect the search trail as well as the answer. Process supervision may help, but consequential decisions still need source checks, domain review and a way to reject incomplete evidence.
- What remains unproven
- Fixed academic question-answering benchmarks rather than live web research
Keep in mind
Fixed academic question-answering benchmarks rather than live web research
No production users, decisions or harms measured
Most process labels generated by GPT-4o
Best-of-ten critic-guided inference adds compute and latency
Different benchmarks use different metrics and cannot be collapsed into one accuracy rate
No independent replication
English-language tasks dominate and MedQA is an exam set rather than clinical care
Earlier preprint existed before the 28 October peer-reviewed publication
Multi-source analysis · 3 linked sources. Primary analysed source date: 28 October 2026. Earliest linked supporting source: 16 May 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
The agent was rewarded for how it searched
A newly peer-reviewed study suggests that information-seeking agents improve when training evaluates their intermediate searches, not only whether the final answer matches a key. That distinction matters for people using AI to assemble clinical, educational, policy or business evidence: a plausible answer can be right by luck or by relying on memorised model knowledge, while an auditable search process exposes what was actually retrieved.
The researchers built RAG-Gym, a common testbed for agent design, post-training and inference. Evaluation covered 3,398 questions: 1,000 each from HotpotQA and 2WikiMultihopQA, all 125 Bamboogle questions and 1,273 English MedQA questions. The first three are multi-step open-answer tasks scored with exact match and F1; MedQA is a multiple-choice medical exam set scored by accuracy. Separate samples of 1,000 HotpotQA and 1,000 MedQA training questions supplied process-reward data.[1][2]
Better benchmark scores came with useful evidence—and more machinery
The Re²Search agent drafts a line of reasoning, identifies the first unsupported claim and turns that gap into a targeted query. Its strongest version combines post-training with a critic that ranks candidate next actions. On HotpotQA, the full Re²Search++ system reached 60.19 F1, compared with 41.09 for a ReAct baseline. Across the study's comparisons with recent methods, the authors report average relative F1 gains of 3.2% to 11.6%, depending on the benchmark and configuration.
Some transfer results are encouraging but modest. A critic trained around Llama-3.1-8B improved GPT-4o-mini's MedQA accuracy from 76.8% to 78.1%. On 2Wiki, GPT-4o process annotations achieved 84.5% retrieval recall and supported 57.9 F1 downstream. The study therefore measured both answers and retrieved evidence—but did not ask independent experts whether the complete evidence trail would support a real decision.[1][2][3]
The distinction that matters
What changed in the training signal
The study separates a correct final answer from a search path that found relevant support.
Outcome only
Reward the final answer
A lucky or memorised answer may look successful
Process reward
Score each query and retrieved step
Useful evidence-seeking becomes trainable
Critic-guided
Sample ten next actions and rank them
Higher benchmark scores, with extra compute
The result is a research direction, not a reliability certificate
The critic normally chose among ten sampled actions at every step, increasing inference cost. More selection was not always better: MedQA accuracy fell from 72.8% with 15 candidates to 71.2% with 20, which the authors associate with reward hacking. An outcome-only method also beat one rollout-level process-reward variant on 2Wiki, 55.6 versus 49.6 F1. These counter-results make the central claim narrower than a universal win for process supervision.
GPT-4o generated most process labels. A medically trained author subset agreed with those labels 85.8% of the time on MedQA, but that is neither perfect agreement nor independent replication. The tasks were static benchmarks, not a changing web, long research session or live service. The work was funded by NIH and NSF awards; the authors declare no competing interests. Confidence would rise with independent replication, open-ended multilingual searches, cost and latency reporting, adversarial sources, calibrated citations and prospective tests measuring whether people make better decisions—not simply whether an answer key is matched.[1][2]