Do positive views of AI mean students use it reflectively? A 27-person study says not necessarily
In one Chinese programming course, students generally liked the AI tools even as logged activity became more restricted near the final task. The study was exploratory, did not measure learning gains and cannot show that the tool caused the change.
The short version
- What happened
- The peer-reviewed study followed 27 educational-technology undergraduates across eight sessions in a required Python course at one Chinese university.
- Who it affects
- Teachers and students may need to check task progress and the quality of reasoning directly; neither a positive AI survey score nor a low dashboard count reveals whether learning is deep, complete or stalled.
- What remains unproven
- Exploratory sample of 27 students
Keep in mind
Exploratory sample of 27 students
One Chinese university and one course
Five enrolled students excluded
Small and unequal clusters
Different tasks across stages
Profiles clustered independently at each stage
No control group or causal estimate
No learning-outcome measure
Generated feedback quality not independently rated
Source analysis · 1 linked source. Primary analysed source date: 27 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
Positive ratings did not map neatly onto reflective activity
A small peer-reviewed study suggests that students can value generative AI while making limited use of the reflective features built around it. Researchers followed educational-technology majors in a required Python course at one Chinese university. The students used a DeepSeek-V2 programming assistant and a separate dashboard that displayed their own and classmates' activity. At the final stage, four behaviour profiles ranged from broad exploration to a 13-person Limited Explorer group with little recorded reading or peer-oriented activity.
The 27 students still rated AI generally positively. Conventional and permutation-based tests found no statistically significant differences between the four final-stage profiles in perceived usefulness, ease of use, intention to use or attitude. That null result does not prove the profiles were equivalent: small, unequal groups limit statistical power. It does show why a favourable survey score should not be treated as evidence that students are comparing alternatives, checking feedback or integrating peers' ideas.[1]
The logs show a change in activity, not its cause
The study began with 32 second-year students who already had experience with C programming. Five were excluded because they did not review generated reports or had incomplete logs, leaving 27 complete records: 11 women and 16 men. Across eight one-hour sessions, students moved from structured visualization exercises to an open-ended final project. The dashboard recorded seven measures, including opening, moving and reading self- and peer-generated nodes; researchers also coded AI conversations and administered a post-course questionnaire.
Dashboard actions fell from 1,448 in the initial stage to 770 in the final stage. Separate K-means analyses produced three initial profiles and four final profiles, including the Limited Explorer group. Because each stage was clustered independently, similarly named groups do not necessarily contain the same students. The later task was also different. Lower activity could mean efficient completion, fatigue, loss of direction or disengagement; the logs cannot distinguish those explanations, and the design cannot show that the dashboard or AI caused the shift.[1]
The distinction that matters
What the study observed—and what it did not
The evidence separates logged behaviour, survey responses and unmeasured learning outcomes.
Observed
Count dashboard actions across two task stages
1,448 initial-stage actions and 770 final-stage actions
Compared
Test AI perceptions across four final-stage profiles
No statistically significant differences in four survey measures
Not tested
Measure learning gains or causal effects
No control group, performance outcome or causal estimate
For teachers, task status matters more than a label
The practical lesson is narrower than the profile names suggest. A teacher should not assume that a low activity count means a student is struggling, or that enthusiasm for AI means reflective learning is happening. Near a deadline, a student who has finished may need a prompt to verify and justify the final solution; someone with incomplete work may need diagnostic feedback and a clear next step. The authors present those supports as design hypotheses, not tested interventions.
The study measured platform behaviour and perceptions, not code quality, exam performance or durable learning. It came from one course, one institution and one cohort, and the generated feedback reports were not independently rated for quality. Stronger evidence would retain student-level profiles over time, synchronize events across both systems, compare learning outcomes with a suitable control and replicate the work in larger, more diverse courses. Until then, the result is a warning against equating acceptance with effective use—not proof that AI dashboards improve or damage learning.[1]