Can people still tell AI from humans? They struggled with saved snippets—but got every GPT-3.5 live test right
Across six constrained tasks, people often misclassified machine output. Active judges perfectly identified GPT-3.5 in one live setup, while a later Llama follow-up again produced mistakes.
The short version
- What happened
- The study assembled 36,499 answers from 636 people and 37 AI systems, then recorded 72,191 Turing-like judgements from 1,916 human judges across six language and vision tasks.
- Who it affects
- People assessing messages, reviews or media should not treat writing style alone as proof of human authorship; platform provenance and contextual verification remain more useful than confident guessing.
- What remains unproven
- Controlled tasks with balanced human/machine trials do not represent real-world prevalence
Keep in mind
Controlled tasks with balanced human/machine trials do not represent real-world prevalence
Evidence spans older datasets and model generations; the preprint began in 2022
Live tests mix language ability with timing, safety refusals and typing-style clues
Imitation is not a measure of intelligence, accuracy, deception intent or safety
Crowdsourced and laboratory samples may not transfer across languages and cultures
Source analysis · 1 linked source. Primary analysed source date: 26 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
Fixed snippets were difficult to classify
A peer-reviewed study found no single answer to whether people can distinguish AI from humans: performance changed sharply with the task. Researchers assembled 36,499 answers from 636 people and 37 AI systems across image captioning, word association, conversation, colour estimation, object detection and attention prediction. Another 1,916 people made 72,191 balanced human-or-machine judgements.
In fixed language samples, judges often struggled. Machine image captions and word associations were correctly labelled as machine 46% and 47% of the time; machine contributions in human–AI conversations were identified 56% of the time. Because trials were balanced, those results describe controlled discrimination—not the probability that any post encountered online is synthetic.[1]
Active questioning made GPT-3.5 easier to spot
The result reversed in a mobile test where judges questioned one person and GPT-3.5 directly: they identified both correctly in every reported conversation. Judges could probe personal details and observe response timing and style. Those clues were part of the test, so the finding does not isolate language quality alone. In a later 28-person follow-up using Llama-3-70B without safeguards, 188 trials no longer produced perfect identification, although detection improved with longer exchanges.
For readers, journalists and platforms, the practical lesson is narrower than 'AI passes the Turing test': style-only guesses from isolated content are unreliable, while interaction and provenance can add evidence. The paper's preprint first appeared in 2022 and was revised in 2025 before today's peer-reviewed version; several model generations and datasets are therefore historical. Imitation also does not establish intelligence, truthfulness, harmful intent or real-world prevalence. Replication with current systems, multiple languages and realistic base rates would strengthen the claim.[1]
The distinction that matters
Why the test format changed the answer
The experiments tested controlled imitation under different conditions, not a universal detector for real social feeds.
Saved snippets
Judges classified fixed language samples
Machine answers were often missed
Live GPT-3.5
Judges asked their own questions and saw timing cues
Every reported identity judgement was correct
Live Llama follow-up
28 judges completed 188 trials without safeguards
Perfect identification did not persist