AI tutor access lowered grades in a university trial — but few students used it
A randomized Maryland study found lower grades and course-platform activity after instructors received a course-integrated assistant. Uptake was only about 15%, and the paper is not peer reviewed.
The short version
- What happened
- The instructor-level randomized trial covered 30 instructors, 34 course sections and 2,379 undergraduates at one US university during autumn 2025.
- Who it affects
- Students may receive faster course-specific answers while spending less time in the wider learning environment; universities need to check both convenience and learning outcomes.
- What remains unproven
- The working paper is not peer reviewed, comes from one institution, randomizes only 30 instructors across 34 sections, records low student uptake and cannot separate the effect of direct-answer design from broader changes in AI use.
Keep in mind
The working paper is not peer reviewed, comes from one institution, randomizes only 30 instructors across 34 sections, records low student uptake and cannot separate the effect of direct-answer design from broader changes in AI use.
Source analysis · 2 linked sources. Primary analysed source date: 30 September 2026. Latest linked supporting source: 16 October 2026.
Read the full report · 5 min The evidence, methods, implications and limitations
Access, rather than regular use, was randomized
Giving instructors a course-integrated AI assistant lowered student grades and online course activity in a randomized University of Maryland trial. The result is a warning against assuming that access alone improves learning, not evidence that every AI tutor harms students. Only about 15% of eligible students used the tool, and most participating instructors left it in a mode designed to provide direct explanations and answers rather than guided practice.
Researchers assigned instructors within 13 matched blocks to receive the assistant or continue teaching without it during autumn 2025. The final study covered 30 instructors, 34 course sections and 2,379 undergraduates across 11 of the university's 12 schools. The assistant used GPT-4o with retrieval from course materials. Students in both groups could still use public tools such as ChatGPT or Gemini, so the experiment measured the effect of offering this particular course tool, not AI use versus no AI use.[1][2]
Grades and course-platform activity both moved down
In the authors' exact-match analysis of 22 instructors and 1,353 students, access reduced final grades by an estimated 4.12 percentage points, equivalent to 0.37 standard deviations. It also corresponded to about 721 fewer course-page views, 7.19 fewer active days and 26.39 fewer participation actions. The full 2,379-student analysis estimated a smaller 2.88-point grade decline with weaker statistical precision, while the engagement measures remained negative. Homework scores did not show a clear improvement.
The activity pattern matters because it offers a plausible route from the intervention to the result: students may have substituted quick answers for course engagement. But the trial did not directly establish that mechanism. Roughly three-quarters of recorded requests sought information, explanations or solutions; only 11% concerned practice or test preparation. By the term's end, two instructors were using the more scaffolded tutor mode. Average use among users was about four sessions, leaving a large gap between being assigned access and relying on the tool.[1]
The distinction that matters
What the randomized trial actually compared
The trial changed whether an instructor's class received one integrated assistant. It did not prevent students in the comparison group from using other AI.
Assignment
Randomize instructors within 13 matched blocks
Treatment classes received the course assistant; comparison classes did not
Actual use
Students chose whether to open the assistant
About 15% used it, averaging roughly four sessions among users
Semester outcomes
Compare grades and learning-platform records
Final grades and several engagement measures were lower in the access group
Universities should test the teaching design, not just the chatbot
For lecturers, the practical lesson is to decide what kind of learning behaviour an assistant should encourage, then measure whether students actually use it that way. A tool that retrieves accurate course material can still change when students open the learning platform, attempt problems or seek feedback. The study supports staged deployment with usage logs, learning outcomes and an alternative for students who do not want an AI assistant; it does not justify a blanket conclusion that guided tutoring cannot work.
This is a working paper from one US public university, not a peer-reviewed multi-campus trial. Assignment occurred at instructor level, leaving only 34 course-section clusters, and the authors warn that conventional standard errors may overstate precision. Their larger estimate for first-generation students is concerning but comes from subgroup tests with additional uncertainty and some imbalance in the matched sample. The university both developed and evaluated the system. A preregistered replication with more instructors, genuinely scaffolded tutor use, independent evaluation and direct measures of study behaviour would materially change confidence in the result.[1][2]