What do FDA records show about updates to AI medical devices?
A peer-reviewed audit found that 56 of 163 sampled AI-enabled devices had at least one recorded algorithm update. Public files often omitted participant details, sample sizes or evidence of clinical benefit, limiting what clinicians and patients can learn after a device changes.
The short version
- What happened
- Researchers traced 352 FDA 510(k) entries for 163 AI or machine-learning devices and identified 56 devices with at least one recorded algorithm update, represented by 106 entries.
- Who it affects
- Clinicians need version-specific evidence when a device's inputs, features or model change; an original clearance summary may no longer describe the system in use.
- What remains unproven
- The cohort was historical and narrower than the FDA's full current list of AI-enabled devices.
Keep in mind
The cohort was historical and narrower than the FDA's full current list of AI-enabled devices.
The analysis used public 510(k) records and could not observe confidential evidence or changes that did not generate a separate modification request.
One reviewer performed the database search before two reviewers classified candidate algorithm updates, leaving some risk of missed records or misclassification.
The study describes reporting and update frequency; it does not measure diagnostic accuracy, patient harm, safety events or clinical benefit.
Most devices analysed medical images, which limits generalisation to other specialties and forms of clinical AI.
Source analysis · 2 linked sources. Primary analysed source date: 5 November 2026. Earliest linked supporting source: 4 September 2026.
Read the full report · 8 min The evidence, methods, implications and limitations
The finding: algorithm changes were common, but their evidence trail was uneven
A peer-reviewed JAMA Network Open research letter published on 5 November followed a defined historical cohort of 163 medical devices that explicitly used artificial intelligence or machine learning and had entered the US Food and Drug Administration's 510(k) pathway. The researchers retrieved 352 public database entries linked to those devices, including original clearances and later modification requests. They found 56 devices—34.4% of the cohort—with at least one recorded algorithm update, represented by 106 entries and ranging from one to thirteen updates per device.
The median time to a first identified update was 1.8 years, with an interquartile range from 0.9 to 2.5 years. Among the 56 updated devices, 42 had at least one change that altered how the previous version was used, nine had updates described only as improving algorithm performance, and five had changes whose nature was unclear in the public record. The study therefore documents a lifecycle problem: a cleared AI device can change materially after its first authorization, while the public explanation of that change may remain incomplete.
This does not mean a third of all current AI medical devices were secretly changed, nor that the identified updates harmed patients. It is a document audit of a historical cohort, not a clinical outcomes study. The denominator includes devices initially cleared from May 2015 through December 2021, and the search followed their related entries only through 31 March 2025. Its value is in showing what could and could not be reconstructed from the regulator's public files.[1]
How the audit was done—and what counted as an update
The authors began with a previously identified cohort of devices explicitly reporting AI or machine learning and cleared through 510(k) between 9 January 2019 and 29 December 2021; some devices in that cohort had earlier initial clearance dates. One reviewer searched the FDA database and extracted every matching entry. Two reviewers then judged whether each modification request involved an algorithm update and extracted its characteristics. The study followed the STROBE reporting guideline and used descriptive analysis in R.
Updates were separated into changes to previous usage and changes described only as performance improvements. Across 81 entries, the researchers identified 95 usage changes because one entry could contain more than one change. Twenty-three added features, such as an alert capability, while 72 changed algorithm input, such as adding blood-pressure data to a system that already used electrocardiogram signals. Twenty-one entries described performance-only improvement: ten added data, eight added neural networks and three changed architecture.
Most of the cohort—89%—analysed medical images. That concentration matters because reporting patterns in radiology and imaging software may not transfer to every clinical AI product. The paper also relies on what manufacturers and the FDA put into public summaries. A modification described in vague language can be misclassified, while an internal change that never generated a relevant submission cannot appear in the audit at all.[1]
The distinction that matters
What the FDA record audit could reconstruct
The denominators narrow at each step; a recorded update is not the same as a measured clinical benefit.
163 devices
Trace 352 linked 510(k) entries
Historical study cohort
56 devices
Identify 106 algorithm-update entries
At least one recorded update
42 devices
Classify changes to prior use
Use changed at least once
8 devices
Find clinical or organizational benefit
Benefit reported in public records
The missing information clinicians would need to judge a changed system
For each updated device, the researchers assessed whether public records described the data and people used for performance evaluation, the sample size, metrics and results, and any clinical or organizational benefit. Participant information was absent from 41 of 106 update entries. At the device level, 24 of 56 devices—42.9%—had no participant information in any update entry. Sample size was missing from 38 entries, leaving 15 devices with no sample-size information across their update records.
Metrics and results were more often present, but not universal. Fourteen entries omitted both, and eight devices had no entry reporting them. Evidence closest to patient care was rarest: 86 of the 106 entries did not report an organizational or clinical benefit, and 48 of 56 updated devices lacked that information across all associated update entries. A technical score can establish that a revised model performs on a test set; it does not show that the change improves decisions, workflow or health outcomes.
These are reporting gaps, not proof that evaluations were never conducted. Some information may exist in non-public material reviewed by the FDA or in manufacturer files. But public evidence has a distinct purpose: it lets clinicians, hospitals, patients and independent researchers understand what changed and whether evidence still matches the population and setting where the device is used. Missing denominators or participant descriptions make that scrutiny harder.[1]
Why the 510(k) route and change-control plans complicate the picture
The 510(k) pathway generally asks a manufacturer to establish substantial equivalence to a legally marketed predicate device. The paper examines updates that were visible through modification requests in that system. It does not cover every route by which a medical device can reach the US market, every AI feature inside a broader product or every modification made after deployment. The FDA's current public AI-enabled device list is broader and continuously updated, so it should not be treated as the study denominator.
The authors highlight a newer source of undercounting: a predetermined change control plan can set boundaries for future modifications, allowing specified changes without a separate new marketing submission. That can make planned learning and maintenance more manageable, but it also means a search for modification requests will not necessarily reveal every later version. The audit cannot determine whether devices without an identified update stayed unchanged or were modified through another permitted route.
A practical lifecycle record would connect a released software version to its intended use, inputs, evaluation population, sample size, comparator, metrics, known limitations and monitoring plan. Hospitals also need to know when the version they evaluated has changed. Procurement and governance teams should ask for version-specific evidence and change logs rather than assume the original clearance summary permanently describes the product in service.[1][2]
What would make the evidence more useful
The study supports standardized public reporting, but it does not test which template or enforcement mechanism works best. A useful minimum would state the old and new version, reason for the change, affected use, training and evaluation data, patient and site characteristics, sample sizes, uncertainty, subgroup results and whether clinical workflow or outcomes were assessed. Records should distinguish a new input, a new feature and a model retraining rather than collapsing them into a generic software modification.
Independent follow-up should link regulatory records to deployed versions, recalls, adverse-event reports and prospective performance monitoring. It should examine whether reporting quality differs by specialty, manufacturer, risk class and use of a change control plan. Patient-facing communication also matters: a hospital may know that software was updated while a patient affected by its output has no practical way to see what changed or how the new version was evaluated.
The assessment would improve if future cohorts captured all relevant devices and update routes, reviewers reported agreement on update classification, and outcome studies tested whether transparent records change procurement, clinical oversight or safety. It would weaken if missing public details consistently corresponded to strong confidential evidence with effective monitoring, although that would still leave a public accountability gap. For now, the finding is about traceability—not a verdict that every updated device is unsafe.[1][2]