Can a foundation model track tumours during MR-guided radiotherapy?
On 38 held-out cine-MR sequences, fine-tuned SAM2 variants reduced average boundary and centre errors and ran fast enough for the tested image stream. The work remains a retrospective technical test: it did not steer radiation or measure patient safety.
The short version
- What happened
- TrackRAD2025 supplied 50 labelled training sequences with 5,027 frames and 38 held-out test sequences with 2,905 frames from six international institutions.
- Who it affects
- Patients could benefit if reliable tracking keeps radiation on a moving tumour, but this study did not measure delivered dose, harm or outcomes.
- What remains unproven
- Retrospective segmentation and tracking accuracy do not establish safe radiation delivery or patient benefit.
Keep in mind
Retrospective segmentation and tracking accuracy do not establish safe radiation delivery or patient benefit.
The 38-sequence test set is small for characterising rare but serious tracking failures.
The semi-automatically propagated labels were not manually reviewed, and the training comparison covered one benchmark and model family.
Runtime was measured on a single A100 GPU and excludes much of the clinical pipeline.
Some reference contours represented a visible surrogate structure rather than the tumour itself.
Source analysis · 1 linked source. Primary analysed source date: 13 October 2026.
Read the full report · 8 min The evidence, methods, implications and limitations
The answer: promising tracking accuracy, without evidence of clinical readiness
A family of open foundation vision models tracked tumours and surrogate anatomy across short magnetic-resonance image sequences with high overlap and low average centre error in a retrospective benchmark. In the paper's combined analysis, fine-tuning increased mean Dice overlap from 0.874 to 0.891, reduced the 95th-percentile Hausdorff boundary distance from 9.069 to 4.378 millimetres and reduced centroid distance from 5.299 to 1.769 millimetres. All three changes met the Bonferroni-adjusted significance threshold. Inference took roughly 0.04 to 0.08 seconds per frame on one Nvidia A100 GPU, which is fast enough to keep pace with the image streams examined by the researchers.
Those results support technical feasibility, not a claim that SAM2 can safely guide a patient's radiation today. The system was tested after the fact against curated contours. It did not decide whether to interrupt or redirect a beam, and investigators did not measure missed targets, irradiated healthy tissue, treatment time or patient outcomes. The authors explicitly call for prospective testing and evaluation of worst-case failures. A good mean can hide a short tracking loss at precisely the moment accuracy matters most, so clinical validation needs safety rules and case-level failure analysis as well as average segmentation scores.[1]
What data the models saw
The study used TrackRAD2025, an open collection of two-dimensional sagittal cine-MR sequences from 0.35-tesla and 1.5-tesla MR-linac systems at six international institutions. The labelled training set contained 50 sequences and 5,027 frames; the held-out test set contained 38 sequences and 2,905 frames. One institution appeared only in the test set, giving the evaluation some protection against simply memorising every site's image style. Each frame contained one contour, usually the gross tumour volume. When a lesion could not be seen reliably, a more visible surrogate such as the whole liver was used instead.
Medical and dental students, physicists or radiation oncologists created the original frame-by-frame contours, depending on the contributing institution. A medical-physics doctoral researcher with four years of tumour-tracking experience reviewed and corrected them. That creates a stronger reference than unreviewed crowd labels, but it is not an infallible biological truth. The reported inter-observer range itself shows that trained people disagree. Surrogate structures also change the task: tracking a high-contrast organ boundary is not equivalent to tracking a low-contrast tumour that the radiation team must keep inside a treatment margin.[1]
How less than 1% manual labelling was tested
The unlabelled part of TrackRAD2025 contained 2,000 sequences from 477 patients and more than 2.8 million frames. The researchers built a browser tool in which medical students without specialist MRI training drew a box around an object on the first frame. SAM2 produced the first mask and then propagated it through the sequence. Using that process, the team labelled 200 sequences and 33,149 frames while people directly marked less than 1% of frames. The final propagated masks were not manually reviewed, an important limitation when interpreting the apparent saving in expert labour.
Five starting models—four sizes of SAM2.1 and the medically adapted MedSAM2—were evaluated zero-shot and after fine-tuning on manual labels, semi-automated labels or both, producing 20 model-and-training combinations. The researchers found no statistically significant accuracy difference between models fine-tuned on the 5,027 manually labelled frames and those fine-tuned on the much larger semi-automated set. That finding suggests a practical route to cheaper pilot datasets. It does not prove that automatic labels are interchangeable with expert contours for every anatomy, scanner or disease, because the comparison concerned one benchmark and one family of related models.[1]
What the statistics establish—and what they do not
Accuracy distributions were strongly non-normal, so the authors used Kruskal–Wallis tests and generalised linear mixed models rather than relying on ordinary mean comparisons. Each fine-tuning model used 760 values: 20 model-configuration pairs evaluated on 38 test sequences. A Bonferroni-adjusted alpha of 0.016 covered the three similarity metrics. There was no significant zero-shot accuracy difference among the five model variants, and no significant difference across all 20 model-and-training combinations in the omnibus tests. The clearest signal was therefore the broad contrast between zero-shot and fine-tuned use, not a victory for one branded model size.
Mean fine-tuned performance approached the benchmark's reported inter-observer variability, but 'approached human variability' should not be translated into 'as safe as a clinician'. Dice overlap gives larger targets more tolerance than small ones, while boundary and centroid distances condense an entire motion sequence into averages. Runtime was measured on one high-end GPU configuration, and the study did not report an end-to-end latency budget including image acquisition, data transfer, verification and beam-control safeguards. Hardware, software and clinical workflow can each introduce delay or failure beyond the model's inference time.[1]
Why this matters to patients and radiotherapy teams
MR-guided radiotherapy can show a moving target while radiation is delivered. Better tracking could reduce interruptions and help a team keep a tumour inside the planned field as breathing changes its position or shape. For patients, the potential benefit is more confident targeting with less unnecessary exposure of nearby tissue. The risk is equally concrete: if a tracker silently loses a low-contrast lesion, a fast answer can be worse than a slower system that stops and asks a specialist to check. The paper noted one zero-shot sequence where a model lost a low-contrast object early; fine-tuning prevented that particular error, but the benchmark cannot enumerate every rare failure.
For medical physicists and oncology services, the immediate result is about development workflow. A small team may be able to create a research dataset by asking novices to mark starting frames and using a model to propagate masks, reserving expert time for targeted review. Any clinical programme would still need documented acceptance tests, uncertainty or failure detection, escalation rules and validation on its own machines and patient mix. Cloud-hosted labelling also needs a privacy and governance review before protected images are uploaded; this study used a fully de-identified public dataset, which is not the same as routine clinical data.[1]
What would change the assessment
The next persuasive evidence would be a locked evaluation on new institutions, including small and low-contrast targets, irregular breathing, large deformation and sequences deliberately chosen for difficult failures. Results should be reported per patient and per anatomy alongside worst-case deviations, not just pooled means. Lower-cost GPUs and complete clinical pipelines should be timed. Independent teams should also compare semi-automated labels with expert-reviewed contours to learn when propagated mistakes survive fine-tuning and whether targeted quality control can detect them efficiently.
Prospective testing must begin in shadow mode, where the tracker runs alongside treatment without controlling it, before any interventional trial. Investigators should record alarms, human overrides, target loss, beam interruptions, dose to tumour and nearby organs, and any safety incident. The work was conducted by Mayo Clinic and University of Iowa researchers and included US National Institutes of Health support for one author; the authors declared no competing interests. That transparent funding record does not substitute for independent replication. Until those steps are completed, the study supports further evaluation and cheaper data preparation, not clinical deployment.[1]