AI-personalised courses looked promising—but four studies left the effect uncertain
A meta-analysis of 420 computing and engineering students found a large pooled difference, but a small-sample correction and prediction interval both included no benefit.
The short version
- What happened
- Only four controlled studies involving 420 students met the review criteria: one cluster-randomized study and three studies with weaker non-randomized or historical comparisons.
- Who it affects
- A student may receive more tailored feedback, but a university cannot yet promise better learning from AI personalisation. Teachers need results from their own course design, assessment and student population.
- What remains unproven
- Four studies
Keep in mind
Four studies
420 students
Three non-randomized comparisons
Two studies at serious risk of bias
I² 89%
Very-low-certainty evidence
No long-term retention
Source analysis · 1 linked source. Primary analysed source date: 29 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
A large average with very wide uncertainty
AI-personalised instruction was associated with higher post-course performance across four controlled studies, but the evidence is too small and inconsistent for a confident causal claim, according to a peer-reviewed Frontiers in Education review published on 29 October. The standard random-effects model produced a large pooled standardized difference of Hedges' g 1.01 (95% CI 0.33–1.68). A small-sample Hartung–Knapp adjustment widened that interval to −0.10–2.11, which includes no benefit.
The 95% prediction interval—an estimate of where a comparable new study might land—also crossed zero, from −0.42 to 2.43. Results varied sharply between studies, with I² of 89%. For a lecturer or student, the useful conclusion is therefore not that personalised AI adds one dependable unit of learning. It is that a few small implementations sometimes produced higher scores, while the result expected in another course remains uncertain.[1]
Four studies from 84 search records
The reviewers searched Scopus and Web of Science in April 2026 and screened 84 unique records. Eligible studies needed an AI system that adapted content, feedback, pacing or recommendations, a concurrent or historical comparison group, and an objective academic-performance outcome. Four studies with 420 students qualified, all in computing or engineering courses in Egypt, China or Morocco; no intervention lasted longer than one semester.
Only one study randomized intact groups. One claimed random allocation without explaining the procedure, one used a non-randomized concurrent control and one compared with a previous year's students. Two studies were judged at serious risk of bias; the other two had moderate risk or some concerns. One paper's reported effect size did not match its own means, standard deviations and test statistic, while another depended on an unblinded historical control.[1]
The distinction that matters
Why the headline average is not a dependable forecast
Different uncertainty checks change how confidently the pooled effect can be read.
Ordinary model
Pool four studies
g 1.01; 95% CI 0.33–1.68
Small-sample adjustment
Use Hartung–Knapp
95% CI −0.10 to 2.11
New-study forecast
Calculate prediction interval
−0.42 to 2.43
What teachers still need before changing a course
The review rated the overall certainty very low because of bias, inconsistency and imprecision. It also could not assess publication bias with fewer than ten studies, and its protocol was not registered before the authors knew the data. Some interventions bundled AI with prompt training or other support, so the observed difference cannot always be assigned to personalisation itself. The findings do not address long-term retention, independent exams, accessibility or teacher workload.
The authors reported no financial support and no commercial conflicts. Stronger evidence would come from preregistered multi-course trials that randomize enough classrooms or students, keep grading independent, separate the AI component from co-interventions and report retention and subgroup outcomes. Until then, institutions can pilot carefully and measure local results, but should not market AI-personalised learning as proven to improve achievement.[1]