When an AI tutor was confidently wrong, learners often carried the error forward
In a small deception study, 32 adults absorbed planted falsehoods even when the assistant’s overall reliability fell. The experiment is vivid but deliberately extreme and not a deployment error rate.
The short version
- What happened
- Thirty-two mostly young, highly educated and frequent AI users learned five unfamiliar topics from assistants that confidently delivered controlled mixtures of true and false facts.
- Who it affects
- A learner may feel certain because the explanation was fluent, not because it was independently checked. Teachers need visibility into the evidence behind a tutor’s claim and a way to correct misconceptions before assessment.
- What remains unproven
- Only 32 adults participated; 21 were aged 18–24, 27 used AI daily and recruitment used university and personal networks.
Keep in mind
Only 32 adults participated; 21 were aged 18–24, 27 used AI daily and recruitment used university and personal networks.
The tutors were deliberately extreme: they never hedged, never departed from the planted briefing and made errors far more systematically than ordinary systems.
Participants were told not to use outside sources, removing the independent checking route that a real course should provide.
The surprise test was immediate and multiple-choice; delayed free recall and classroom learning may differ.
Source analysis · 1 linked source. Primary analysed source date: 22 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
Confidence made the wrong lesson feel learned
Researchers gave each participant five versions of the same GPT-5-mini tutor. Depending on the session, zero to all of 12 briefing facts were replaced with plausible falsehoods, while the assistant was ordered to speak with complete confidence. Quiz scores fell from 7.4 to 2.9 out of 10 across the reliability range. When a false claim appeared in chat, participants adopted it about 65% of the time and rated absorbed errors 8.4 out of 10 for confidence.[1]
Asking the same source again did not provide a check
Participants could return to the tutor after seeing a quiz. Thirty per cent of turns pasted a quiz question back into the same assistant, but that source had been instructed never to concede its briefing. Pasted falsified items received the planted answer 91.5% of the time. Self-reported trust barely moved even as performance worsened, suggesting satisfaction questions alone can miss harmful reliance.[1]
The distinction that matters
The experiment’s trust loop
The assistant’s confidence stayed fixed while the proportion of planted facts changed.
Teach
Confidently present a controlled fact set
0%, 25%, 50%, 75% or 100% falsified
Test
Offer correct, planted and distractor answers
1,040 answers across 104 quizzes
Check
Let learners ask the same assistant again
Often repeated rather than corrected the planted claim
Classroom design needs an independent route to evidence
For teachers and product designers, the practical implication is not to make tutors timid; it is to make individual claims traceable. Curated references, prompts that ask learners to predict before receiving an answer, and a verification route outside the model being checked are testable safeguards. This study does not show that those designs work, and its immediate multiple-choice quiz did not measure durable learning.[1]