Does the way students use AI matter?
A peer-reviewed survey of 713 Saudi undergraduates found that supportive and creative AI use was associated with stronger self-reported creativity and metacognitive awareness, while substitutive use showed negative associations. The one-time self-report design cannot establish what AI caused—or whether skills improved.
The short version
- What happened
- The study surveyed 713 Saudi undergraduate EFL learners once in January 2026, using newly developed five-point scales and structural equation modelling.
- Who it affects
- Students may benefit when AI activities preserve responsibility for planning, judging and revising, but this survey does not prove an instructional effect.
- What remains unproven
- The design is cross-sectional and correlational, so mediation and moderation estimates do not establish causal or temporal pathways.
Keep in mind
The design is cross-sectional and correlational, so mediation and moderation estimates do not establish causal or temporal pathways.
AI use, creativity and metacognition are all self-reported; the study contains no observed prompts, interaction logs, task artefacts, grades or independent performance measures.
The questionnaires were developed for this study and validated in the same broad educational context, limiting claims about other languages, cultures, institutions and age groups.
Participants were Saudi undergraduate EFL learners recruited through courses; access, pedagogy and attitudes may differ elsewhere.
Eligibility required prior AI experience, while 6.7% reported never using AI in the frequency item; the accepted manuscript does not explain that apparent difference.
The accessible publication is an early unedited accepted manuscript and may receive production corrections before the Version of Record.
Source analysis · 1 linked source. Primary analysed source date: 3 October 2026.
Read the full report · 8 min The evidence, methods, implications and limitations
The study separates AI help from AI substitution
Asking whether students use AI can hide very different activities. A learner may request an explanation, generate alternative ideas, copy a finished answer, translate text without reflection, or challenge an output for bias and accuracy. Researchers at Northern Border University and the University of Ha'il in Saudi Arabia built a survey model around those distinctions. Their central finding is an association: students who described more supportive and creative engagement also described stronger creative dispositions and awareness of how they plan, monitor and regulate learning, while those reporting substitutive use tended to score lower on both.
Humanities and Social Sciences Communications published the accepted paper on 3 October 2026 after peer review. Nature calls the current article an early, citable, unedited manuscript that will later be replaced by the Version of Record. The research was funded by Northern Border University's Deanship of Scientific Research under project NBU-FPEJ-2026-2905-01. The authors declare no competing interests and say AI was used only for language refinement, not research design, data collection or analysis.[1]
The sample covers 713 undergraduates, not observed learning sessions
The final sample included 713 Saudi undergraduates enrolled in English-as-a-foreign-language courses. Recruitment was stratified by year of study and self-rated English proficiency, then distributed through courses; participation was voluntary and only complete responses were retained. The group included 352 men and 361 women. Most were aged 18 to 23, 56.4% described their English as intermediate, and every academic year was represented. Data were collected online through Qualtrics from 6 to 21 January 2026.
Eligibility required prior experience with AI-supported English learning, including generative tools, machine translation, grammar or writing assistants and speaking-feedback systems. Yet the demographic table reports that 48 respondents, or 6.7%, selected 'never' for frequency of AI use. That may reflect a difference between prior experience and current frequency, but the accepted manuscript does not resolve the apparent tension. More importantly, the study did not observe a class, inspect prompts, retrieve revision histories or score an English task. Every central variable is a self-report at one moment.[1]
Three new questionnaires supply the measurements
The authors developed three five-point questionnaires. A 16-item AI-use instrument divided responses into cognitive-support, creative-extension, reproductive or substitutive use, and ethical or critical awareness. A 20-item creative-thinking disposition scale covered fluency, flexibility, originality, elaboration and risk-taking. A further 16 items addressed metacognitive planning, monitoring, evaluation and regulation. These measure what learners believe they tend to do, not demonstrated creativity, language proficiency or unaided metacognitive performance.
Before the main survey, 388 EFL learners formed a separate pilot sample. The researchers randomly split it into two groups of 194 for exploratory and confirmatory factor analysis, after expert review of content and wording. They report accepted thresholds for factor loadings, reliability, convergent validity and model fit. The main structural model also fitted the survey covariance reasonably well: CFI was 0.956, TLI 0.941, RMSEA 0.049 and SRMR 0.041. Good statistical fit means the proposed relationships are compatible with these responses; it does not make the conceptual categories objectively true or rule out alternative models.[1]
Supportive and creative use moved with higher self-ratings
Standardised path estimates linked cognitive-support use with creative-thinking disposition at 0.38 and creative-extension use at 0.41; both had p values below 0.001. Their direct associations with metacognitive awareness were 0.26 and 0.22 respectively. Reproductive or substitutive use moved in the other direction, with estimates of -0.29 for creative disposition and -0.24 for metacognitive awareness. Ethical and critical awareness had a positive 0.27 association with creative disposition.
Creative-thinking disposition itself had a 0.49 association with metacognitive awareness. The authors also report indirect paths from cognitive-support and creative-extension use through creativity, and interaction terms suggesting the positive associations were stronger among respondents reporting more critical and responsible engagement. These estimates are internally coherent with the proposed theory, but common wording, social desirability and a learner's general confidence could inflate several scales together. Structural equation modelling can estimate a mediation pattern in cross-sectional data; it cannot establish the time order required to show that one behaviour caused the next.[1]
The safer inference is about learner agency, not effectiveness
A student who already plans carefully and enjoys exploring alternatives may naturally use an AI tool to ask questions and generate options. A less confident learner may be more likely to copy, or may describe their practice more negatively. Teaching quality, prior attainment, access to paid tools, workload and digital literacy could influence both use style and self-ratings. Because participants were not randomly assigned to an interface or pedagogy, the model cannot choose among those explanations.
The study therefore does not show that prompting students to report 'constructive' AI use will improve creativity, English or self-regulation. Nor does it establish that translation, rewriting or direct answers always weaken learning; those functions can be accessible supports when embedded in a designed activity. The useful proposition is more modest: research and policy should distinguish how responsibility for meaning-making is shared, then test those patterns with behavioural and performance evidence rather than treating any AI use as one exposure.[1]
What educators can test without turning the survey into a score
Teachers could design assignments in which students first state a plan, use AI for specified forms of explanation or variation, annotate which suggestions they accepted, and then complete a related unaided task. Comparing learning, retention and transfer under different support rules would answer a stronger question than asking for attitudes after the fact. Revision histories and consented interaction logs could show whether critical evaluation actually occurs, while interviews could explain why students switch between support and substitution.
The new scales should not be used to label an individual as an ethical or unethical AI user. The categories were developed and validated in one Saudi EFL context, rely on self-description and include value-laden distinctions. Turning them into a disciplinary, admissions or surveillance score would extend far beyond the evidence. A constructive classroom use would be voluntary reflection: helping learners notice when a tool expands their thinking and when it removes a practice opportunity they still need.[1]
What would change the assessment
Confidence would rise with preregistered randomised or quasi-experimental studies that vary the permitted AI role, measure baseline ability, collect behavioural traces with consent, and test immediate performance, delayed retention and transfer to a new task. Results should be replicated across languages, countries, universities, access levels and students with different learning needs. Researchers should also separate writing, speaking, reading and grammar tasks because the same tool action may serve different cognitive functions in each.
Longitudinal work could test direction: whether constructive engagement predicts later metacognition after accounting for prior skill, or whether strong self-regulation predicts the later choice of constructive use. Until then, this study makes a useful measurement argument and provides transparent linked data and instruments. Its evidence supports paying attention to the way students use AI; it does not prove which approach improves learning.[1]