AI feedback matched one teacher in an essay trial—but motivated students less
In a randomized study of 312 Chinese undergraduates, Qwen feedback and one instructor's comments produced equivalent revised-essay scores within a prespecified margin. Teacher feedback led to slightly higher intrinsic motivation; the result covers one task and one revision cycle.
The short version
- What happened
- Researchers randomized 312 first-year undergraduates to locally generated Qwen feedback, feedback from one experienced instructor or a minimal-prompt control; 287 were analysed after 8% attrition.
- Who it affects
- Teachers could use AI to expand the supply of rubric-based comments, but this study supports a blended role rather than removing human contact from feedback.
- What remains unproven
- Single Chinese university, first-year cohort, argumentative prompt, instructor, model and one-week feedback cycle; no course-level learning or retention outcome.
Keep in mind
Single Chinese university, first-year cohort, argumentative prompt, instructor, model and one-week feedback cycle; no course-level learning or retention outcome.
The study was not preregistered. Twenty-five of 312 randomized students were omitted from complete-case analysis, and no intention-to-treat analysis was reported.
Students knew the feedback source, and the relatedness and motivation pathway was measured rather than experimentally manipulated; the paper treats mediation as associational.
The authors reported no funding or commercial conflicts. Data are available by inquiry rather than through a public repository.
Source analysis · 1 linked source. Primary analysed source date: 30 October 2026.
Read the full report · 3 min The evidence, methods, implications and limitations
Comparable revisions, different motivation
AI feedback helped students revise one argumentative essay about as well as feedback from a single instructor, but it supplied less motivational lift. A Chinese university randomized 312 first-year undergraduates to Qwen2.5-27B-Instruct feedback, teacher feedback or a control prompt without essay-specific comments. After 25 students were lost or excluded, 287 revisions were scored by two raters who did not know the assignment.
Controlling for the first draft, adjusted scores were 54.70 for AI, 54.58 for the teacher and 48.28 for control. Both substantive-feedback groups beat control by about 0.6 standard deviations, and the AI-teacher difference fell within a prespecified equivalence margin of ±0.35 standard deviations. On a seven-point motivation scale, however, teacher feedback averaged 5.16 versus 4.90 for AI and 4.61 for control.[1]
A useful classroom tool, not a teacher-replacement result
The locally hosted model used a standardized rubric covering thesis, argument, evidence and clarity; researchers reviewed its output but did not edit or regenerate it. One instructor with six years of university writing experience produced the comparison comments. Students knew which source they received, so the motivation gap may partly reflect expectations about people and machines rather than an unavoidable property of AI feedback.
This was one 400-word task, one institution and one revision a week later. The paper was not preregistered, used complete cases rather than all randomized students, and the revised drafts scored lower than first drafts in every group under different writing conditions—the valid claim is a relative advantage over control, not absolute improvement. Repeated, multi-teacher trials would test whether blended feedback saves time without weakening motivation or learning over a course.[1]
The distinction that matters
What the randomized comparison found
The performance result is narrow; motivation separated the two substantive feedback sources.
AI feedback
Qwen comments on one draft
Adjusted revision score 54.70; motivation 4.90/7
Teacher feedback
One instructor's comments
Adjusted revision score 54.58; motivation 5.16/7
Control
No essay-specific comments
Adjusted revision score 48.28; motivation 4.61/7