Did a grounded medical AI earn more clinician trust than ChatGPT? Yes—but the study measured perceptions, not accuracy
At a German internal-medicine congress, 64 physicians and medical students rated paired answers from AMBOSS's grounded LiSA system and ChatGPT. LiSA scored higher for perceived quality and trust, but the non-blinded evaluation did not independently score factual correctness, clinical decisions or patient outcomes.
The short version
- What happened
- Sixty-four physicians and medical students each rated one paired response set drawn from 15 real-world internal-medicine questions; the design was paired but non-blinded.
- Who it affects
- Clinicians may find grounded answers more usable, but must still verify evidence and patient-specific context before acting.
- What remains unproven
- The study included 64 congress participants, each evaluating one response pair from 15 questions.
Keep in mind
The study included 64 congress participants, each evaluating one response pair from 15 questions.
It was non-blinded and measured perceptions, not independently adjudicated factual accuracy or patient outcomes.
AMBOSS supplied the questions, helped select the final set and provided LiSA, although it did not fund, analyse or write the study.
The comparison bundles grounding with product, interface, content and familiarity differences.
Source analysis · 1 linked source. Primary analysed source date: 4 November 2026.
Read the full report · 8 min The evidence, methods, implications and limitations
The direct answer: clinicians trusted the grounded system more in this small, non-blinded comparison
A peer-reviewed Scientific Reports study published on 4 November found that a knowledge-base-grounded clinical assistant received stronger perceived quality and trust ratings than ChatGPT from 64 physicians and medical students. Participants attended the 2026 congress of the German Society of Internal Medicine. Each saw paired responses from LiSA, a clinical system developed by AMBOSS, and ChatGPT for one of 15 real-world internal-medicine questions. Mean overall quality was 3.96 for LiSA and 3.48 for ChatGPT, while mean trust was 5.11 and 3.67 respectively. The reported differences were statistically significant.
The result is useful because clinicians assessed material resembling questions they may encounter in practice. It does not show that LiSA produced more factually correct diagnoses or safer care. Raters knew what they were evaluating, and the outcomes were perceptions. A structured, clinically familiar answer can feel more dependable even when an unnoticed error remains. The defensible finding is narrow: in this setting and question set, participants preferred and trusted the grounded system's answers. It is not evidence that either tool should autonomously guide a patient or replace clinical verification.[1]
What the paired design measured—and what its denominator limits
Each participant considered both systems' answers to the same question, helping control for question difficulty within that judgement. The study reports overall quality, trust, clarity, completeness, actionability, perceived medical accuracy and safety concerns. LiSA scored higher for completeness, actionability and perceived accuracy; clarity did not differ significantly. The authors interpret that pattern as evidence that the perceived advantage was not simply smoother wording. Yet every person assessed only one of 15 question pairs, so the dataset is not a comprehensive examination of internal medicine and does not give each question a large independent panel.
Participants were recruited at a professional congress rather than sampled across clinical roles, specialties and health systems. The exercise captures a moment of judgement, not repeated use during ward pressure. It cannot show whether familiarity improves calibration or encourages automation bias. A stronger crossed design would have multiple clinicians independently rate every question while blinded to system identity, then separate variation attributable to raters, cases and systems. A larger case set should include ambiguous presentations, missing context, rare conditions and changing guidance rather than only questions selected for this product comparison.[1]
The distinction that matters
What the study found—and what it did not test
The paired evaluation separated perceived qualities, but did not independently adjudicate clinical correctness or outcomes.
Participants
64 physicians and medical students
Each rated one of 15 paired responses
Higher ratings
Quality, trust and actionability
LiSA scored higher on perceived measures
No difference
Clarity
No significant difference reported
Uncertain gap
Safety concerns: 15.6% versus 26.6%
Difference was not statistically significant
Not measured
Objective accuracy and outcomes
No effectiveness conclusion
The safety result is more uncertain than the trust result
Safety concerns were recorded in 15.6% of LiSA evaluations and 26.6% of ChatGPT evaluations. Although the raw percentages look different, the reported p value was 0.143. The study therefore did not establish a statistically significant safety-concern difference under its analysis. It would be wrong to convert the percentages into a known reduction in unsafe answers. The paper also does not provide a clinical adjudication showing whether every concern represented a genuine hazard or whether unflagged answers contained hidden errors.
Trust is not automatically beneficial. Appropriate trust helps clinicians use a reliable tool; excessive trust can suppress checking. A grounded answer may expose recognisable concepts and appear more complete, which could improve scrutiny, but it can also sound authoritative. The safety question is whether users know when confidence should fall, whether sources support the exact claim, and whether a system signals uncertainty or missing context. Those properties require objective error analysis and prospective workflow testing. Perceived safety is a relevant human-factors outcome, but it is not a surrogate for misdiagnosis, medication harm or patient benefit.[1]
Grounding was not isolated as the cause
LiSA is described as knowledge-base-grounded, whereas ChatGPT is a general-purpose system. The comparison therefore bundles source material, retrieval, prompting, product interface, medical editing, response format, model choice and participant familiarity. Researchers did not take one model and randomly switch grounding on or off while holding everything else constant. They cannot establish that retrieval alone caused the higher ratings. Nor did they test whether citations faithfully supported claims, whether the knowledge base was current for every question, or how either system behaved when evidence was incomplete.
A post hoc analysis found larger quality and trust differences among current AMBOSS users than among non-current users. That may reflect useful familiarity with the product's structure, but it may also reflect recognition, expectations or preference. Because outputs were non-blinded, those explanations cannot be separated. Researchers could isolate the mechanism with blinded outputs, common formatting, an identical base model, controlled retrieval conditions and answer-level adjudication against a predefined reference. Until that is done, product teams should not present this paper as a clean causal test proving that grounding itself created superior clinical answers.[1]
The commercial role is disclosed and should shape interpretation
AMBOSS provided the clinical questions, contributed to study design and selection of the final question set, and supplied access to LiSA. The paper says AMBOSS did not fund the work, had no access to study data and was not involved in analysis, interpretation or manuscript preparation. The authors declare no other competing interests. Open-access publication was supported by Philipps-University Marburg. These disclosures do not invalidate the result, but they are central context because the company behind the evaluated system influenced which questions entered the comparison.
Question selection can shape a benchmark without touching the final statistics. A grounded knowledge product may perform especially well where its curated content is strong, while results may differ on rarer conditions, incomplete prompts or rapidly changing guidance. A robust follow-up should preregister selection criteria, include independently sourced cases, preserve difficult items and report results by question. External replication with other knowledge bases and general models would test whether the pattern travels beyond one product. Commercial clinical evidence becomes more persuasive when the evaluated company cannot choose the test set and scoring material can be independently inspected.[1]
What changes for clinicians—and what would change this assessment
For clinicians, the study supports a hypothesis rather than a procurement verdict: answers anchored in a clinical knowledge base may feel more usable than general-purpose output. Before consequential use, a clinician still needs to verify the evidence, patient-specific contraindications, local guidance and whether the question omitted decisive context. A high perceived-quality score does not transfer accountability. Health organisations should distinguish educational queries from medication, diagnosis and escalation decisions, and require stronger checks where an answer could directly affect a patient.
Confidence would rise with a preregistered, blinded, multi-centre study using an independently assembled question set. Several clinicians should adjudicate each answer for factual errors, missing qualifications, harmful recommendations and citation support. A prospective trial could then measure whether decisions improve, information is found faster and errors are caught without over-reliance. The assessment would weaken if higher trust persisted despite equal or worse accuracy. This is consequential research but not Breaking: it is a controlled perception study, not an urgent clinical warning, regulatory approval or active patient-safety incident.[1]