Did a GPT-4o virtual patient prepare medical students better? This study found no measurable gain
Only 49 of 96 responding students offered the tool used it. Nervousness, preparedness, self-efficacy and perceived competence were no better than in the comparison group, despite positive user ratings.
The short version
- What happened
- The University of Amsterdam allocated 383 first-year medical students by class schedule; 180 answered the post-class questionnaire and 49 used the virtual patient.
- Who it affects
- Medical students may value private rehearsal before a stressful consultation. Educators still need evidence of observed skill and safe feedback before using virtual patients to replace scarce human practice.
- What remains unproven
- Allocation by class schedule was not random, and the study was not prospectively registered in a public trial registry.
Keep in mind
Allocation by class schedule was not random, and the study was not prospectively registered in a public trial registry.
Only 180 of 383 eligible students responded; 49 intervention respondents used the virtual patient.
The outcomes were self-reported after one activity, with no blinded assessment of communication performance or longer-term learning.
The study tested one institution, one case and a voluntary text interface; results may not transfer to other curricula or multimodal systems.
Source analysis · 1 linked source. Primary analysed source date: 19 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
Positive reactions did not become a measured learning benefit
Offering first-year medical students an optional GPT-4o virtual patient did not make them less nervous or more prepared for their first standardised-patient consultation, according to a peer-reviewed Dutch study. The University of Amsterdam allocated 383 students by class schedule: 182 received the usual e-learning and 201 also received access to a text-based virtual patient built with retrieval-augmented generation. Only 180 students completed the post-class questionnaire—84 controls and 96 in the intervention group—and 49 intervention respondents actually used the tool.
Across nervousness, preparedness, self-efficacy and self-perceived competence, the allocation-based effects were close to zero and every confidence interval included no difference. Comparing the 49 users directly with controls did not change the conclusion. The tool was liked: 63%–71% of users gave positive ratings, and 71% called it a valuable addition. That contrast matters for education buyers: satisfaction and plausibility are not evidence that a product improves skill.[1]
What the comparison did—and did not—test
Students could practise history-taking with one simulated case before a class involving a trained standardised patient. Most users tried the chatbot once: 35 of 49 had one qualifying conversation, eight used it twice and only six used it three or more times. The researchers powered the study to detect a moderate effect, used non-parametric comparisons and corrected 19 item-level tests for multiple comparisons. The observed effects were smaller than the pre-specified small-effect threshold.
This was not a randomised trial. Class schedules determined access, use was voluntary, and only 47% of eligible students answered the questionnaire. Outcomes were students' own ratings after the activity; independent assessors did not score consultation recordings, knowledge, empathy or clinical decisions. The study was not prospectively registered in a public trial registry, and the allocation-based analysis was described as post-hoc. Those limits make the null result informative but not proof that higher-dose or better-designed virtual-patient practice cannot work.[1]
The distinction that matters
From offer to evidence
The participant flow shows why access, use and measured benefit are different denominators.
383 allocated
Split by class schedule
182 control; 201 offered AI
180 responded
Complete post-class survey
47% response rate
49 used it
Hold a qualifying conversation
Mostly one session
4 outcomes
Compare self-ratings
No measurable difference
A supplement needs a purpose, feedback and enough practice
For students and educators, the safest interpretation is that an accessible chatbot can provide low-pressure rehearsal, but one optional text session should not displace trained actors, observation or feedback. The tested system did not automatically assess performance, and the study did not show that it reduced teaching costs. A school adopting a similar tool would need privacy controls, transparent case design, staff oversight and a way to detect clinically unsafe or unrealistic responses before students learn from them.
Amsterdam UMC and University of Amsterdam grants funded the work; the authors declared no commercial conflicts. Confidence would rise with a prospectively registered randomised trial that ensures minimum practice, uses several cases, records engagement and assesses blinded communication performance over time. It should also report withdrawals and compare feedback quality, not merely whether students opened the system in practice. Until then, the study supports usability interest—not a claim that GPT-4o-trained rehearsal improves medical communication.[1]