Does bilingual AI dialogue improve learning?
In a Karnataka field study, AI dialogue increased interest and self-efficacy but did not produce reliably larger knowledge gains than written responses. Bilingual voice reduced some participation barriers, while dialogue had lower completion. The work is an unreviewed preprint.
The short version
- What happened
- The between-subjects field study assigned 305 students to six written, text-dialogue or voice-dialogue conditions in English or bilingual Kannada-English; 173 completed the programme.
- Who it affects
- Students may gain a less intimidating way to ask questions and express partial understanding.
- What remains unproven
- The paper is an unreviewed preprint and does not establish effects outside this specific Karnataka English-medium context.
Keep in mind
The paper is an unreviewed preprint and does not establish effects outside this specific Karnataka English-medium context.
Only 173 of 305 enrolled students completed the study, with different completion rates by format and small outcome-analysis cells.
The first deployment was disrupted by a funding-related centre closure, although the lower dialogue completion pattern remained in the second deployment.
The activity was optional, low-stakes and short; durable learning, transfer and performance in required courses were not tested.
Immediate feedback accompanied dialogue while writing feedback was delayed, so format and feedback timing were not fully isolated.
Source analysis · 1 linked source. Primary analysed source date: 5 October 2026.
Read the full report · 7 min The evidence, methods, implications and limitations
The answer is engagement, not proven extra learning
Bilingual AI dialogue did not reliably improve measured knowledge over written responses in this study. The clearest positive result was motivational: students who completed text or voice dialogue reported greater interest in the AI-literacy topic and stronger confidence that they could understand it. Bilingual voice also appeared to make participation easier for some Kannada speakers. Those gains matter because willingness to contribute can determine whether a learner gets useful practice, but they should not be translated into a claim that conversation taught more.
The trade-off is unusually important. Dialogic activities elicited more words, more time and richer interaction among completers, yet fewer students finished them. A learning tool can therefore look engaging in session data while excluding or losing more learners before the end. For colleges deciding whether to replace short written work with an AI tutor, the practical question is not simply whether conversation feels better. It is whether the design can sustain completion and produce durable learning for the whole assigned group.[1]
Six conditions separated format, modality and language
The Cornell-led team ran an ethics-approved field study with 305 native Kannada-speaking undergraduates aged 18 to 23 at English-medium institutions in Karnataka. Students were randomly assigned for the duration of the study to one of six conditions: written response, text dialogue or voice dialogue, each delivered in English-only or bilingual Kannada-English form. Everyone watched the same four short bilingual instructional videos and answered the same activity prompts. The dialogue groups interacted with an AI tutor; the writing groups submitted a response independently.
The technology was not one model alone. Gemini 2.5 Flash generated tutoring and feedback. Voice sessions used GPT-4o Transcribe for speech recognition, Gemini for the reply and Cartesia Sonic 3 for speech output. Written and text-dialogue groups used web interfaces. The same content, prompts and feedback rubric were held constant, but feedback timing was not identical: dialogic groups received it immediately, while written-response students waited 12 hours to approximate conventional homework. That difference may itself affect experience and confidence.[1]
The denominator fell from 305 enrolled to 173 completers
The first deployment enrolled 167 students through 25 village Model Digital Inclusion Centers; only 51 completed after the centres shut permanently because of an unexpected funding cut. A second college deployment enrolled 138 and retained 122. Overall, the outcome analyses generally used the 173 completers, while discourse analysis included anyone who generated conversation data. Condition-level completion ranged from 20 of 45 in English voice to 38 of 53 in English writing, so some cells used for outcome comparisons contained only about 20 to 30 students.
Random assignment strengthens causal comparison at enrolment, but differential attrition weakens it after assignment. If the students who persisted in voice or text dialogue were more motivated, better connected or more comfortable with the interface, adjusted post-study comparisons can reflect that selection. The authors checked the college deployment separately, where all writing students completed versus 83.0% of text-dialogue and 79.4% of voice-dialogue students. The format pattern remained, suggesting the first site closure was not the whole explanation.[1]
Learning rose across groups without a reliable winner
Knowledge was measured with 15 multiple-choice and open-ended questions administered before and after the programme. Human researchers scored a subset of 236 open-ended responses, then GPT-5.4-mini applied the rubric to the full set; the paper reports quadratic-weighted kappa and intraclass correlation of 0.85 against the human ratings, followed by manual verification of all model-assisted grades. Regression models adjusted post-scores for pre-scores, activity, language, their interaction and deployment site, using robust standard errors.
Knowledge increased in all six conditions. The bilingual voice group had the highest adjusted post-score at 9.91 out of 15, but differences among the six groups were not statistically reliable. Within voice, the bilingual estimate was 1.89 points above English voice with p=0.052—suggestive, not conventionally significant—and the authors emphasise that small cells limit power. The responsible conclusion is parity on this short assessment, not proof that format or language makes no difference under all conditions.[1]
Dialogue changed interest, confidence and participation
On five-point scales, the combined dialogue formats scored 0.47 points above writing for interest and 0.29 points above it for self-efficacy, both with p<0.001. Text and voice estimates pointed in the same direction. Interaction logs and interviews suggest immediate feedback, follow-up questions and hints helped students continue reasoning without fear of being judged. Bilingual voice reduced articulation difficulties, increased turns coded as showing understanding and reduced abandonment compared with English-only voice.
Language preference was not a simple access story. Students sometimes found Kannada easier for explaining concepts, especially by voice, but many valued English for academic and professional opportunity. Kannada typing could also be cumbersome. The paper therefore proposes learner control over modality, language, persistence and pace, with a local language used as translanguaging support rather than an automatic replacement for English. That is a useful design hypothesis, not yet a tested long-term policy.[1]
Completion is part of effectiveness
Across all enrollees, 68.6% completed written response, 52.3% text dialogue and 48.4% voice dialogue, a significant difference across formats. Among those who did complete and answer the post-survey, 68% in dialogue preferred the study activity over their usual work, compared with 14% in writing. These measures describe different populations: preference among finishers cannot cancel the fact that more assigned learners dropped out of dialogue.
Schools should consequently measure starts, completions, time, learning and accessibility together. Voice can reduce a language-production barrier while adding bandwidth, privacy, noise, latency and turn-taking burdens. Text can be reviewable and editable but harder in a local script. A flexible system may outperform a single mandated channel, yet flexibility also complicates evaluation. Any roll-out should retain a non-AI route and examine who stops using each option rather than reporting only average engagement from successful sessions.[1]
What would change the assessment
Confidence would rise with a preregistered multi-institution replication large enough to estimate format-by-language effects, using an intention-to-treat analysis that keeps all assigned students in the main comparison. Required or credit-bearing settings would test whether the completion gap persists when participation has consequences. Longer follow-up should measure retention and transfer to new problems, and a design that equalises feedback timing would isolate conversation from immediacy.
Researchers should also publish performance by language proficiency, device quality, disability and access conditions; compare a trained human discussion or peer activity; audit speech recognition separately for Kannada and code-switching; and report model failures. Until that evidence arrives, the study supports a narrower claim: AI dialogue can make a low-stakes activity more engaging for completers, and bilingual voice can change how some students participate, without demonstrating superior learning or universal accessibility.[1]