A retinal AI changed with the camera—what should screening programmes test?
In 8,524 Brazilian patients, performance stayed similar for some retinal tasks but dropped sharply for others when the camera platform changed. The unreviewed study cannot separate hardware effects from image-quality differences or show patient benefit.
The short version
- What happened
- An unreviewed University of British Columbia study re-analysed 16,266 retinal photographs from 8,524 patients at three Sao Paulo clinics, comparing Canon CR-2 and Nikon NF5050 images.
- Who it affects
- A screening programme using a cheaper or locally available camera could widen access, but a model that misses camera-specific degradation may give patients inconsistent results. The study supports local validation before clinical reliance, not rejection of lower-cost equipment.
- What remains unproven
- The new analysis is an unreviewed preprint by one author; the underlying BRSET dataset paper was peer reviewed, but that does not peer-review this re-analysis.
Keep in mind
The new analysis is an unreviewed preprint by one author; the underlying BRSET dataset paper was peer reviewed, but that does not peer-review this re-analysis.
Only two camera platforms and one Brazilian dataset were studied, with camera, image quality, acquisition and patient characteristics confounded.
The model was used as a frozen feature extractor; results may not transfer to fine-tuned systems, other foundation models or deployed screening software.
AUROC and quadratic-weighted kappa are benchmark metrics, not evidence of better referrals, treatment, vision or patient safety.
The preprint does not state funding or competing-interest disclosures in its available HTML text.
Source analysis · 2 linked sources. Primary analysed source date: 21 October 2026. Earliest linked supporting source: 11 July 2024.
Read the full report · 4 min The evidence, methods, implications and limitations
One model, two cameras and uneven changes across diseases
The study used BRSET, a peer-reviewed Brazilian dataset containing 16,266 retinal photographs from 8,524 patients seen at three outpatient centres in Sao Paulo between 2010 and 2020. Researchers froze a pretrained retinal foundation model, trained downstream classifiers on Canon CR-2 images and compared performance on a patient-disjoint Canon test set with a Nikon NF5050 cohort the classifiers had not seen during training. They repeated the analysis across 12 tasks, four image resolutions and five classifiers—240 comparisons in all.[1][2]
Some headline tasks travelled well; others did not
For the validation-selected configurations, area under the receiver operating characteristic curve was similar for diabetic-retinopathy detection (0.940 on Canon and 0.946 on Nikon), scar (0.964 and 0.955) and macular oedema (0.952 and 0.952). Larger gaps appeared for haemorrhage (0.804 and 0.569) and hypertensive retinopathy (0.864 and 0.690). Five-stage diabetic-retinopathy grading fell from a quadratic-weighted kappa of 0.692 to 0.527. Higher resolution and a different classifier did not consistently remove the gaps.[1]
The distinction that matters
What changed when the camera changed?
The same frozen representation did not shift uniformly across retinal tasks.
Little change
Test diabetic retinopathy, scar and macular oedema
Selected-task performance remained similar
Larger drop
Test haemorrhage, hypertension and DR staging
Performance fell on the unseen Nikon cohort
Unresolved cause
Compare camera and image-quality labels
Hardware, acquisition and quality remain confounded
The camera comparison is also an image-quality comparison
Nikon images were more than twice as likely to be rated inadequate overall: 18.6% compared with 8.8% for Canon. Most of the difference came from image-field adequacy, at 15.84% versus 4.76%. Because camera platform, acquisition conditions, image quality and patient mix are intertwined in this retrospective dataset, the analysis cannot show that hardware alone caused the performance changes. The study also uses only two cameras and one dataset, reports no prospective workflow test and measures no missed referrals, treatment decisions, visual outcomes or clinician workload.[1][2]
What screening services can take from the study now
A service should not accept one aggregate accuracy figure as proof that a retinal model will transfer to every device. Before deployment, performance should be reported by camera and disease task using patient-level splits, with image-quality failures and uncertain outputs counted. The assessment would strengthen with prospective multi-site testing across more cameras, matched acquisition protocols and evidence that quality control or domain adaptation prevents clinically important errors. Until then, the work is a useful warning about hidden domain shift, not evidence that one camera or model should be chosen for patient care.[1][2]