A cancer-image model scored 98.5%—then fell to 82.0% on a second dataset
A peer-reviewed lung-and-colon pathology study found a hierarchical model beat a flat classifier on two public datasets. The lower second-dataset result is the more useful warning: this was still an image-level experiment, not a clinical diagnostic test.
The short version
- What happened
- The model scored 98.50% accuracy on the 25,000-image LC25000 benchmark, only 0.24 percentage points above the strongest flat classifier tested on the same split.
- Who it affects
- A pathology aid could help prioritise slides, but a false negative could delay investigation and a false positive could intensify anxiety or testing. This study did not evaluate either outcome in care.
- What remains unproven
- Public image datasets; shuffled image-level splits; patient-separated testing not reported; no unchanged cross-dataset deployment, pathologist comparison, prospective workflow or patient outcome.
Keep in mind
Public image datasets; shuffled image-level splits; patient-separated testing not reported; no unchanged cross-dataset deployment, pathologist comparison, prospective workflow or patient outcome.
Source analysis · 1 linked source. Primary analysed source date: 17 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
The second dataset mattered more than the near-perfect benchmark
Researchers built a coarse-to-fine system that first separates normal from cancerous tissue and then routes images to organ- and subtype-specific classifiers. On LC25000—5,000 images in each of five lung and colon classes—it reached 98.50% accuracy, compared with 98.26% for the best flat ResNet101 baseline. That is a 0.24-point gain on an unusually balanced public benchmark, not a clinical breakthrough.
The same procedure was evaluated separately on LungHist700, which contains 691 lung histology images from 45 patients at two magnifications. On its 139-image test portion, the hierarchy was correct on 114 images and the flat model on 100. Accuracy was 82.01% versus 71.94%; the paired bootstrap interval for the 10.07-point difference ran from 2.88 to 17.99 points. The fall from 98.50% also shows how strongly results depend on the dataset.[1]
Why this cannot yet guide a patient's diagnosis
The paper calls LungHist700 an independent dataset, but it did not take one model trained on LC25000 and deploy it unchanged elsewhere. It applied the framework independently, using a shuffled image-level split for training, validation and testing. Because multiple images came from 45 patients and patient-level separation was not reported, the test may not measure transfer to wholly unseen patients. That is a material limit for pathology evidence.
Four explanation tools highlighted pixels associated with predictions, but the authors did not compare those maps with expert-annotated disease regions. There was no blinded pathologist benchmark, whole-slide workflow, pathology-confirmed prospective cohort or measurement of changed treatment decisions. Severe synthetic image noise also cut accuracy as low as 48.20%, showing that visual robustness is not assured.[1]
The distinction that matters
Why the two accuracy figures answer different questions
The headline benchmark was large and balanced; the second test was smaller, imbalanced and closer to real dataset variation.
Benchmark
25,000 LC25000 images across five equal classes
98.50% accuracy; 0.24 points above the flat baseline
Second dataset
691 LungHist700 images; 139 in the test split
82.01% accuracy; 114 of 139 correct
Clinical gap
No reported patient-separated or prospective test
Performance on new patients and hospitals remains unknown
The useful finding is a research direction, not a diagnostic claim
Breaking a difficult classification task into staged decisions can outperform one flat five-class model, and the second dataset supplies more credible evidence than LC25000 alone. For pathology teams, however, the relevant question is whether the method holds across laboratories, staining protocols, scanners and unseen patients—and whether it adds value beside a pathologist without delaying care.
The assessment would strengthen with locked models tested at patient and whole-slide level across multiple hospitals, prespecified thresholds, confidence intervals for every clinically important class, expert review of explanations and a prospective comparison with routine practice. Until then, the model is an engineering result on public image collections, not an automated cancer diagnosis.[1]