More GenAI use did not predict stronger independent reasoning in Saudi students
A seven-university study found that frequency, duration and weekly time using generative AI were not associated with scores on a new case-based assessment. University context explained substantially more variation—but the cross-sectional, unproctored study cannot establish why.
The short version
- What happened
- Among 431 health-informatics students at seven Saudi universities, 425 (98.6%) reported prior GenAI use; the mean score on a new 90-point case-based assessment was 57.14.
- Who it affects
- Teachers and course leaders should not treat frequent chatbot use as evidence that students are developing independent reasoning. Assessment design, curriculum and supervised practice remain central—and this study does not justify restricting useful AI support on its own.
- What remains unproven
- Convenience sampling and self-reported exposure limit generalisability and may introduce selection or recall bias.
Keep in mind
Convenience sampling and self-reported exposure limit generalisability and may introduce selection or recall bias.
The one-time design cannot establish direction or causality, and the remote assessment was not proctored.
The case assessment was newly developed for this study; its problem-solving subscale was less reliable than the overall instrument.
University was strongly associated with scores, but the study did not identify which institutional differences mattered.
Source analysis · 1 linked source. Primary analysed source date: 24 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
Near-universal use was not the same as stronger assessed reasoning
Researchers surveyed 431 second-year-to-internship health-informatics students across seven Saudi universities between May and July 2026. They measured self-reported GenAI experience and administered a newly developed, expert-reviewed 30-item case assessment covering analysis, evaluation and problem-solving. Scores ranged to 90; the mean was 57.14 with a standard deviation of 15.41.
Duration, frequency and weekly hours of GenAI use did not predict higher scores. Academic-use intensity had a small negative correlation before adjustment, but the association disappeared after university was included. Formal AI training was also associated with a 3.48-point lower score after adjustment, an unexpected result that could reflect who sought training rather than an effect of training itself.[1]
The clearest signal was variation between universities
Average scores ranged from 45.8 to 68.0 between universities, and the reported intraclass correlation was 0.27. That makes curriculum, assessment culture and student mix more plausible targets for follow-up than raw AI-use time. It does not prove any particular university practice caused better performance.
The sample was voluntary and cross-sectional, the use measures were self-reported, and the remote assessment was not proctored. The instrument was new: overall reliability was strong, while its problem-solving subscale was less consistent. A longitudinal, independently proctored study using validated tasks and observed AI-use behaviour would materially change the assessment.[1]
The distinction that matters
What was measured—and what was not
The study linked reported use habits to one independent assessment at a single point in time.
Exposure
Survey GenAI duration, frequency, hours and academic use
Self-reported habits, not observed use quality
Outcome
Complete a 30-item case assessment
Mean 57.14/90 across 431 students
Interpretation
Adjust for student and university factors
No independent advantage from more use; university differences remained