Did structured AI preparation help students feel they belonged?
A peer-reviewed trial across six undergraduate classes found a larger 12-week rise in classroom belonging under a guided AI-and-teacher protocol. With only six randomised sections, the exact test could not rule out chance at the conventional 5% threshold.
The short version
- What happened
- Researchers randomised six intact undergraduate class sections at one Chinese university, assigning three sections and 110 students to each condition after a baseline assessment.
- Who it affects
- Students may benefit when AI-supported preparation is designed to expose uncertainty and connect it to responsive teaching, but this study cannot establish that the same gains would occur in another course or university.
- What remains unproven
- Only six intact class sections were randomised, leaving four degrees of freedom for the main section-level contrasts
Keep in mind
Only six intact class sections were randomised, leaving four degrees of freedom for the main section-level contrasts
All exact two-sided randomisation p-values were 0.100, the smallest attainable in the balanced six-section design
One Chinese university limits transfer to other subjects, institutions and education systems
The intervention bundled AI exploration, confidence reflection, question generation and instructor response, so it cannot isolate an AI effect
The confidence–performance discrepancy index was preliminary, and individual-level pathway analyses were exploratory
No measured evidence yet on persistence, attendance, grades or longer-term outcomes
Source analysis · 1 linked source. Primary analysed source date: 26 October 2026.
Read the full report · 5 min The evidence, methods, implications and limitations
The intervention joined AI preparation to teacher follow-through
The study did not test whether giving students a chatbot improves belonging. It tested a combined instructional protocol: students used generative AI to explore material before class, reflected on their confidence, recorded questions they could not resolve and brought those uncertainties into a classroom where the instructor responded. The comparison condition followed the course without that complete structure. The distinction matters because any difference may come from reflection, question generation, teacher responsiveness or their interaction—not from AI alone.
After baseline measurement, the researchers randomised six existing undergraduate class sections at one Chinese university: three sections with 110 students received the structured protocol and three sections with 110 students served as the comparison. The intervention began in week 2, with measures collected in weeks 1, 6 and 12. Analyses gave each class section equal weight, recognising that classmates share an instructor and learning environment and therefore cannot be treated as 220 fully independent randomised units.[1]
The estimated differences favoured the structured protocol
By week 12, the change in classroom-belonging score was 0.361 points larger in the intervention condition. The reported 95% confidence interval ran from 0.145 to 0.576, and the Bonferroni-adjusted parametric p-value was 0.029. The corresponding between-condition changes were 0.282 points for emotional engagement, minus 0.497 points for resistance to teacher guidance, and minus 15.24 points on a preliminary index comparing confidence with performance. In this sample, the package was associated with students feeling more connected and less resistant while judging their own performance more closely.
Those estimates are useful, but the experiment had only four degrees of freedom at the section level. The authors therefore also calculated exact randomisation tests, asking how unusual the observed differences were among all assignments of three of the six sections to the intervention. Every primary two-sided exact p-value was 0.100. That was the smallest attainable value in this design: with so few possible assignments, an exact test cannot cross the conventional 0.05 threshold even when the observed arrangement is the most extreme.[1]
The distinction that matters
Why two statistical readings point to different confidence levels
The same observed difference can look conventionally significant under a parametric model yet remain unresolved under an exact test with only six randomised classes.
Parametric model
Uses section-level estimates and distributional assumptions
Belonging difference: adjusted p=0.029
Exact randomisation
Compares every balanced assignment of six sections
Two-sided p=0.100, the design minimum
Editorial reading
Treats the effect estimate as promising, not settled
Replication needs more independent classes
What a lecturer can use—and what still needs testing
The practical lesson is not that AI creates belonging. It is that an AI preparation task may fit better when students must surface uncertainty and a teacher visibly responds to it. A lecturer considering the model could keep the human chain intact: define the pre-class question, ask students to state confidence and unresolved issues, review the resulting questions, and use class time to address patterns rather than treating chatbot output as finished learning.
The trial was conducted at one university, with six clusters and no independent replication. Students and instructors knew their condition, the confidence–performance discrepancy measure was preliminary, and individual-level pathway analyses were explicitly exploratory and sensitive to measurement timing. The assessment would become stronger with more courses and institutions, preregistered cluster-level analyses, a comparison that separates AI use from reflection and teacher response, and outcomes beyond self-report—such as attendance, persistence, achievement and equitable participation.[1]