Can AI spot a lie? Strong synthetic scores collapsed on real military interviews
Models reached 88%–93% accuracy on generated text but only 35.5%–48.7% on 76 real participants. The study does not support operational lie detection.
The short version
- What happened
- Four classifiers scored 88.0%–93.0% accuracy on a held-out set of 300 synthetic participant records, then only 35.5%–44.7% on 76 real military-exercise participants.
- Who it affects
- A deception label can influence investigations, employment and security decisions. Below-chance or near-chance accuracy in a small real-world sample is a reason to withhold operational use, not to tune a threshold around the same data.
- What remains unproven
- 76 real participants
Keep in mind
76 real participants
30 deceptive participants
Military training exercise, not operational screening
Synthetic training text
No language marker survived multiplicity correction
Company-linked evaluated system
Government-contract funding
Provisional typeset version
Source analysis · 1 linked source. Primary analysed source date: 28 October 2026.
Read the full report · 2 min The evidence, methods, implications and limitations
Generated language made the task look much easier
Researchers trained logistic regression, random forest, naive Bayes and support-vector models on 1,000 synthetic participant records—8,000 generated answers in total—split 700 for training and 300 for testing. Logistic regression led the synthetic test with 93.0% accuracy and a 0.978 ROC AUC.
Applied without retraining to interviews from 76 US Army training participants, including 30 assigned to deceive, every classifier fell below 45% accuracy. FranklinAsk, a system grounded in 21 deception papers, reached 48.7%. With wide uncertainty and a small sample, those point estimates do not establish operational value.[1]
The practical warning is about validation, not better lie detection
For investigators and people being assessed, a false label can carry serious consequences. The study suggests synthetic text encoded cleaner stereotypes of deception than stressed, spontaneous speech did. No language feature survived the study's multiple-testing correction.
The US Defense Counterintelligence and Security Agency funded the work. Three authors worked for EntigenLogic, whose FranklinAsk system was evaluated. Larger independent studies with representative interviews, pre-specified thresholds and real decision outcomes would be needed before deployment claims become credible.[1]
The distinction that matters
Why the headline score did not travel
The same models faced two very different kinds of language.
Synthetic
Test on 300 generated participant records
88.0%–93.0% classifier accuracy
Real exercise
Apply models to 76 military participants
35.5%–48.7% accuracy across systems
Decision
Ask whether results support screening
No operational validation demonstrated