The State of Play

Conversational AI tutoring operates at institutional scale with clear design requirements: restricted, scaffolded systems with human oversight show measurable learning gains, while unrestricted access causes cognitive offloading and exam score decline. The practice uses large language models to deliver subject instruction through Socratic questioning and guided discovery — posing clarifying questions, scaffolding reasoning, and adapting dialogue to learner needs rather than dispensing answers. Khanmigo has reached over one million students across 380-plus U.S. districts; an international RCT across 62 schools in four countries (Nigeria, Spain, Ireland, India) with 14,892 Grade 7-9 students demonstrated 0.27 SD learning gains (0.41 SD for lower-achieving students). However, the critical design distinction is now empirically proven: a 1,000-student Turkish RCT showed unrestricted ChatGPT access produced 17% worse exam performance than controls, while identically-designed systems with step-by-step hint constraints preserved learning gains. These findings establish guardrails as non-negotiable for efficacy. Cognitive offloading—students relying on AI for answers rather than engaging with material—is the root mechanism undermining unguarded deployments. The practice has achieved leading-edge maturity with proof of efficacy in controlled settings but remains constrained by fundamental limitations: teaching depends on human judgment, contextual interpretation, and relational accountability that AI systems cannot replicate. Policy-level adoption has begun (UK commitment to fund AI tutors for 450,000 disadvantaged students by 2027), yet category leaders acknowledge limited transformative impact after four years of rollout.

Deployment scale is substantial but deployment efficacy remains contingent on design: Khanmigo operates across 380-plus U.S. districts with over one million students, processing 269,000 daily interactions and 108 million cumulative interactions; the UK Department for Education is funding eight companies to develop AI tutoring tools targeting 450,000 disadvantaged Year 9-10 students by 2027, signaling policy-level confidence. However, efficacy depends entirely on guardrails. A Turkish RCT with ~1,000 high school students comparing unrestricted GPT access ("GPT Base") with identical interface but Socratic constraints ("GPT Tutor") found: during practice, both AI groups outperformed controls, but on unseen exams with no AI access, GPT Base students scored 17% worse than controls—the cognitive debt from unrestricted answer-seeking erased all gains. The same students using Socratic-constrained systems preserved learning. An international RCT across 62 schools (14,892 Grade 7-9 students) with LLM-based math tutors confirmed effectiveness of structured design: overall 0.27 SD gain, 0.41 SD for low-prior-achievement students. Conversely, engagement remains uneven: a multi-institutional study of 11,406 post-secondary students found 10.4% engaged only shallowly through copy-pasting, with equity gaps by institutional selectivity. Mathematics and law domains show persistent accuracy problems (40-54% error rates in law) requiring manual teacher verification. A 138-student RCT of unguided generative AI tutors in programming showed significant impairment of metacognitive calibration (d=0.75, p<.001) and cognitive offloading via direct answer-seeking. By April 2026, field consensus holds that Socratic design—constrained dialogue forcing student engagement—is the critical success factor, but deployment challenges remain unresolved: only 6% of education organizations conduct red-teaming on student-facing AI, teacher adoption gaps persist despite scale, and category leaders (Khan Academy) have publicly reassessed expectations of transformative learning impact after four years of rollout.

• 2022-H2: Research deployments showed positive learning outcomes in controlled studies (Ghana higher ed, Taiwan children's reading), while technical advances (EMNLP Socratic subquestions) demonstrated feasibility for conversational tutoring. ChatGPT launch late 2022 sparked public attention but also raised awareness of accuracy, bias, and ethical concerns. Field consensus: conversational AI shows pedagogical promise but requires careful integration with human teaching, not replacement.

• 2023-H1: Khan Academy launched Khanmigo (GPT-4 powered) pilot with 500 partner schools, implementing Socratic dialogue design that refuses direct answers and asks guiding questions. Real-world pilot deployments in Brazil (155 students/teachers) showed promise, though Newark school district reported mixed results and teacher concerns about the tool doing too much work. Research revealed significant technical challenges: neural dialog tutoring models performed poorly in less constrained scenarios with 45% showing reasoning errors; assessment systems for evaluating student responses in conversational ITS were advancing but inconsistent. Field remained split between optimism about deployment potential and concerns about fundamental limitations in empathy, consistency, and reliability.

• 2023-H2: Khanmigo scaled from pilot (500 schools) to 30+ districts with 28,000 students/teachers; pricing cut to $35/student annually signaled movement toward broader adoption. Research confirmed positive learning mechanisms (self-efficacy, engagement) and hybrid human-AI models showed efficacy with underserved populations. However, empirical testing revealed critical constraints: LLMs achieved only 82% accuracy in constrained domains (thermodynamics), practitioners noted reliability gaps between GPT-4 and free models, and expert educators raised structural concerns about AI's inability to provide motivational demand-generation and relational consistency human tutors offer. Field converged on integration (hybrid models with human oversight) rather than replacement as the realistic deployment path.

• 2024-Q1: Conversational AI tutoring entered mainstream adoption phase. Syntea's university deployment across 40+ distance learning courses demonstrated 27% study time reduction with hundreds of students. Khanmigo continued product maturation with progress-tracking tools and teacher features signaling enterprise focus. Socratic Mind's large-scale test with 600 students proved dialogue-based assessment could scale. Academic research advanced personalization through student modeling frameworks. Peer-reviewed evaluations of Khanmigo adoption showed strong student acceptance but persistent needs for human oversight, flagging technical constraints and ethical safeguards as adoption prerequisites. The practice had moved definitively from experimental research into commercial deployment, but remained contingent on hybrid human-AI models rather than autonomous tutoring.

• 2024-Q2: Ecosystem expansion accelerated alongside documentation of technical and pedagogical limitations. Microsoft partnered with Khan Academy to make Khanmigo free for all U.S. teachers, signaling major platform commitment and broadening accessibility. Newark Public Schools moved from pilot to districtwide expansion despite documented math errors and feedback that the tool sometimes provided excessive assistance. Market-level adoption evidence emerged: consumer AI tutor apps (Answer AI, Question AI) ranked top education apps with millions of downloads, replacing paid human tutoring. However, the window also surfaced critical constraints: EMNLP research exposed the "Student Data Paradox"—training LLMs on student dialogue degrades model factual knowledge and reasoning. Expert educators (Dan Meyer, Amplify) raised structural concerns about AI tutors' effectiveness for conceptual learning. Critical assessments argued current chatbot tutors are outdated text-based tools unsuited to modern pedagogy. By late June 2024, the field had crystallized around a consensus: conversational AI tutoring worked for specific use cases (homework help, scalable basic question-answering, productivity for teachers) but remained constrained by accuracy, pedagogical design, and the irreplaceable relational and motivational dimensions of human teaching. Deployment continued but increasingly framed as augmentation rather than replacement.

• 2024-Q3: Conversational AI tutoring expanded to state-level adoption and broadened ecosystem integration. Bill Gates documented Khanmigo pilot deployment in Newark schools; New Hampshire committed $2.3M in state funding for Khan Academy across districts. Microsoft and Khan Academy partnership extended Khanmigo for Teachers free globally across 49 countries, signaling major platform ecosystem maturity. Georgia Tech's Socratic Mind platform demonstrated 2,000-student scale pilot using conversational AI for assessment, proving method scalability. Research deepened understanding of critical deployment constraints: Wharton study with 1,000 students showed AI tutors improved in-practice performance (48%) but harmed exam performance (17% worse) without guardrails, establishing that tool design—particularly safeguards limiting direct answers—fundamentally shapes learning outcomes. Education Week documented persistent field barrier: Khanmigo's continued mathematical accuracy problems forced teachers to verify all numerical answers, embedding human oversight as operational necessity. By Q3 2024, the field consensus was clear: conversational AI tutoring demonstrated deployment feasibility and market adoption, but real-world effectiveness remained contingent on integrated design safeguards, human teacher oversight, and realistic framing as augmentation to rather than replacement of human instruction.

• 2024-Q4: Conversational AI tutoring consolidated global scale and institutional adoption while maintaining critical safeguards. Philippines Department of Education partnered with Khan Academy and Smart Communications for nationwide Khanmigo deployment, marking entry into emerging markets and demonstrating model scalability beyond North America. Khan Academy's longitudinal efficacy study of ~350K students documented 20% greater-than-expected learning gains with consistent platform use, providing large-scale adoption evidence. Higher education integration advanced: Indian School of Business deployed customized AI tutor in EMBA courses with reported improvements in primary source engagement and academic performance. Institutional standardization accelerated: University of Genoa launched formal teacher certification program for conversational AI tutoring, structured with three mastery levels, signaling move toward professional competency frameworks. Deployment breadth expanded: Khanmigo operating across 266 U.S. school districts with expanding safety features (emotional distress detection). However, critical skepticism deepened: educational technology experts cited empirical evidence of AI tutoring ineffectiveness, referencing Wharton research showing reduced student achievement with unguarded AI tutors. Teacher adoption barriers persisted: survey data showed slight decline in active teacher AI usage despite increased training, indicating gap between tool availability and classroom integration. By end of 2024, conversational AI tutoring had achieved global deployment scale and institutional legitimacy but remained constrained by design safeguards, teacher adoption barriers, and persistent skepticism about efficacy for conceptual learning—the practice had matured from experimental research to operational deployment contingent on hybrid human-AI models.

• 2025-Q1: Conversational AI tutoring entered a consolidation phase marked by continued deployment gains and deepening understanding of design constraints. Khanmigo deployments expanded across U.S. school systems: Enid High School (Oklahoma) reported remarkable increases in math achievement and doubled engagement through strategic implementation with teacher ownership; Michigan State University scaled a pilot from 80 to 800 students with positive feedback on understanding and performance. Peer-reviewed research reinforced the hybrid-model thesis: a Taiwan study of 230 university students showed ChatGPT provided valuable accessibility and non-judgmental interaction, but students preferred human tutors for tailored feedback, while a Mexico-based trial with doctoral students documented how Socratic Lab AI tool increased participation in synchronous sessions. Critical research clarified the design imperative: synthesis of recent studies revealed unrestricted AI tutors harm learning outcomes (17% worse exam performance with standard ChatGPT) but customized tutors with safeguards substantially boost performance, establishing guardrails as essential for efficacy. Practitioner assessment surfaced a structural limitation: respected math educators noted that AI tutors lack the sensing capacity of human teachers and that individualized learning software has historically shown paltry effect sizes, raising questions about whether conversational AI could achieve transformative outcomes without human mediation. By March 2025, conversational AI tutoring had consolidated a clear deployment model: effective at scale within constrained, guided parameters but requiring human oversight, design safeguards, and realistic expectations about complementing rather than replacing teacher judgment.

• 2025-Q2: Conversational AI tutoring achieved mainstream K-12 and higher education adoption with expanded institutional integration and global reach. Michigan Virtual structured a K-12 Khanmigo pilot with professional development for 25 teachers in grades 6-12, demonstrating institutional pathways for adoption. Alpha School launched full operational deployment claiming 2.3x faster learning gains and 99th percentile standardized test results. Quantitative adoption evidence showed 63% of K-12 teachers incorporating generative AI into teaching (up 12% YoY), indicating sustained adoption momentum across the sector. Systematic review of intelligent tutoring systems in NPJ Science of Learning synthesized empirical evidence on K-12 outcomes, clarifying variable effectiveness across contexts. However, critical barriers to scalable impact persisted: expert educators documented learner readiness as essential—students required guidance for effective AI interaction beyond prompt provision—and teacher customization control was necessary to tailor system behavior to pedagogical goals. Implementation-level constraints deepened: teachers continued to face demands for manual verification of AI accuracy, particularly in mathematics. Critical assessments argued that despite rising adoption, conversational AI tutoring remained fundamentally constrained by its reduction of learning to curriculum-aligned problems, lacking the meaningful context required for genuine conceptual understanding and likely to plateau as a supplement rather than transform educational outcomes. By June 2025, conversational AI tutoring had matured from emerging adoption into established institutional practice, but with clear evidence-based limitations constraining transformative impact without human mediation and careful design guardrails.

• 2025-Q3: Conversational AI tutoring expanded into state-level deployment, demonstrated pedagogical validation of Socratic dialogue, and deepened understanding of adoption constraints. Louisiana launched statewide Khanmigo pilot with 50% student activation and 71% teacher usage, showing integration success across diverse school contexts. Khanmigo's user base grew to 700,000+ across 380+ districts, reflecting sustained commercial adoption momentum. Controlled research validated pedagogical design: a German study of 65 pre-service teachers showed Socratic AI tutors significantly enhanced critical, independent, and reflective thinking compared to unguided chatbots, providing evidence that dialogue structure matters. However, the quarter also surfaced persistent effectiveness limitations: a mixed-methods study of high school students found Khanmigo delivered no performance advantage over control groups; a university law course deployment of Socratic chatbot showed 40-54% error rates; and a comprehensive ITS review synthesized mixed evidence across contexts with calls for greater evaluation rigor. By September 2025, conversational AI tutoring had demonstrated sustainable state-level scaling and pedagogical validation of Socratic method design, but evidence remained divided on learning outcome efficacy—the field converged on the practice as a proven supplement for specific use cases (engagement, accessibility, teacher support) but not as a transformative replacement for human tutoring.

• 2025-Q4: Conversational AI tutoring consolidated mainstream institutional adoption with emerging evidence of supervised AI-tutor efficacy. Khanmigo reached 1M students across U.S. K-12 systems, marking major scaling milestone beyond 700k in Q3. Google LearnLM's peer-reviewed RCT (N=165) demonstrated supervised AI tutors performed at least as well as human tutors with 5.5pp better knowledge transfer on novel problems, providing first rigorous evidence of AI competency parity in controlled settings. Comprehensive literature review of 48 tutoring effectiveness studies documented mixed outcomes: confirmed benefits (STEM improvement, engagement) alongside significant limitations (cognitive offloading, reduced critical thinking, modest gains vs. traditional instruction). Practitioner and research consensus continued to emphasize critical constraints: AI tutoring requires robust design safeguards, human oversight, and accurate student modeling to be effective; current systems remain fundamentally limited compared to human teachers in sensing, motivation, and real-world application. Critical academic assessments persisted: experts argued current systems are pedagogical tools rather than true intelligent tutoring systems, lacking essential student and tutor models. By end of 2025, conversational AI tutoring had achieved mainstream adoption at scale (1M+ users, 380+ U.S. districts) with supervised deployment models showing comparable efficacy to human tutors, but fundamental pedagogical limitations remained: the field remained clear that AI tutoring was an established supplement for specific use cases (homework support, accessibility, teacher productivity) but not a transformative replacement for human instruction.

• 2026-Jan: Conversational AI tutoring entered a phase of scale consolidation and deployment quality validation. Multi-institutional research tracked heterogeneous real-world engagement patterns: a study of 11,406 students across 10 post-secondary institutions found 10.4% exhibited shallow engagement with copy-pasting behavior while students from selective institutions showed deeper engagement, highlighting equity considerations and variability in student agency with AI tutors. Supervised AI tutoring efficacy was reinforced: a UK RCT with 165 secondary students (ages 13-15) showed supervised AI tutors (Google LearnLM with human oversight) outperformed human-only tutoring on problem-solving (66.2% vs 60.7% success), with minimal hallucination (0.1%), validating the hybrid human-AI model that had emerged as field consensus. Khan Academy reported internal metrics demonstrating learning persistence: students reaching 2+ proficient skills weekly on Khanmigo correlated with significant yearly test score gains, and students receiving AI guidance were more likely to solve subsequent problems independently. Industry consensus at AIED 2025 reinforced pedagogical positioning: over 700 researchers and practitioners aligned on shifting from "answer engines" to Socratic dialogue design, with Khan Academy's CLO emphasizing guidance over direct provision. However, critical barriers to scaled deployment remained evident: a global security survey revealed only 6% of education organizations conducted red-teaming for student-facing AI systems, with 84% lacking AI anomaly detection and 79% lacking purpose binding, documenting a severe safety infrastructure gap. Expert skepticism persisted: critical assessment documented AI tutors' inability to read emotion, perceived shallow instructional dialogue compared to human tutors, and equity risks from tool-mediated learning. By end of January 2026, conversational AI tutoring had stabilized as a proven supplement within hybrid human-AI models at institutional scale, with validated efficacy in controlled settings but persistent real-world deployment variability and unresolved safety infrastructure challenges.

• 2026-Feb: Conversational AI tutoring demonstrated strengthened pedagogical validation and framework maturation. Peer-reviewed research from Hong Kong Polytechnic and German universities confirmed Socratic dialogue design effectiveness: a quasi-experimental study with 31 healthcare students showed the Socratic Playground for Learning platform significantly increased self-efficacy (effect size 0.57), while a 80-student programming study validated that Socratic-scaffolded AI (GSL) fostered deeper engagement and critical thinking compared to direct-answer AI (GDL). Framework research from Cornell, University of Adelaide, and Digital Promise synthesized tutoring best practices with generative AI, proposing design principles for scalable pedagogically sound conversational tutors. Analyst synthesis from Brookings Institution reviewed RCT evidence confirming learning gains, knowledge transfer, and psychological safety benefits. However, critical expert assessment persisted: UCL professor Rose Luckin documented that AI tutors address only a narrow fraction of human intelligence (16%), with research showing metacognitive laziness, reduced self-monitoring, and procrastination when AI support is withdrawn. Deployment evidence showed heterogeneous impact: Google LearnLM (165 students) achieved 76.4% AI message approval rates and superior novel problem-solving (66% vs 61%) in supervised settings, while Tutor CoPilot (1,000 elementary students) showed 4pp mastery improvement by augmenting human tutors. By end of February 2026, the field had solidified conviction that conversational AI tutoring was effective as a pedagogically designed supplement, with Socratic dialogue structure as key differentiator, but remained constrained by fundamental limitations in addressing broader dimensions of human learning and requiring sustained human oversight for efficacy.

• 2026-Q1: Conversational AI tutoring demonstrated empirical validation of guided discovery design and solidified institutional scale. Gold-standard RCT (n=334, IZA Institute) published counterintuitive finding: unrestricted AI access outperforms restricted access by 0.21 SD, challenging concerns about overreliance and establishing continuous AI availability as more effective for learning than gated access. Design research synthesis (AEI) showed Turkish RCT (1,000 students) confirming that Socratic guardrails (step-by-step hints vs. direct answers) eliminate negative effects of unguarded AI; adaptive sequencing algorithms with AI yielded 0.15 SD gains equivalent to 6-9 months of additional learning. Deployment evidence solidified: Upper Canada College (1,200 students) documented 23% reduction in remedial support and 82% student helpfulness rating using no-code, teacher-customizable AI tutors; Appalachian State faculty reported higher exam scores for students combining conversational AI tutor with peer discussion. Khanmigo ecosystem matured to 1.4M cumulative users globally; Khan Academy + Google partnership (February 2026) expanded Khanmigo to 40+ languages and 180+ countries via Microsoft infrastructure, positioning conversational tutoring as educational infrastructure. Medical education systematic review (67 studies, 2019-2025) validated pedagogical approach while documenting persistent challenges (algorithmic bias, hallucinations, privacy); recommended human-AI symbiosis model. UK Department for Education announced largest government commitment: trial AI tutoring with 450,000 disadvantaged students by 2027, signaling policy-level conviction in practice's efficacy. By March 2026, conversational AI tutoring had consolidated three years of leading-edge maturity with robust evidence base supporting Socratic dialogue design, institutional deployments showing measurable learning outcomes, and policy commitment signaling preparation for broader adoption—but fundamental constraints (equity variability, human oversight necessity, limited impact on complex conceptual learning) remained unresolved.

• 2026-Q2 (April–May): Conversational AI tutoring demonstrated strengthened evidence of equitable impact while surfacing critical bias, design constraints, and adoption challenges. Peer-reviewed research (Shao & Wang, Guangxi Normal University) showed AI-assisted tutoring significantly enhanced intrinsic motivation and self-efficacy among university students, with pronounced effects for lower-achieving learners. Khan Academy released "Explain Your Thinking" feature in select schools, implementing conversational assessment where AI poses questions to elicit conceptual understanding beyond correct answers. However, Stanford preprint research identified systematic bias in AI tutor feedback: high-achieving and White students received detailed, developmental feedback while Hispanic, ELL, and lower-achieving students received grammar-focused responses. UC San Diego deployed course-grounded Socratic AI tutor to 400-student genetics class; Socratic guardrails (hints vs. direct answers) mediate learning gains through metacognitive engagement, benefiting low-prior-knowledge students.

Recent peer-reviewed evidence (May 2026) refined understanding of conversational AI tutoring limitations and product evolution. A Neuron RCT (57 university students, HKUST) demonstrated AI-led conversational tutoring produces learning outcomes and brain synchrony patterns statistically indistinguishable from human instruction, supporting parity claims within controlled settings. However, large-scale empirical studies identified critical technical and behavioral barriers: an Oxford Internet Institute / Nature study (400,000+ responses across 5 models) documented accuracy-warmth trade-off—7.43 percentage-point error increase when fine-tuning for empathy, directly constraining tutoring chatbot design. ICLR 2026's outstanding paper revealed severe accuracy degradation across 15 major LLMs in multi-turn conversations (39% decline), limiting conversational tutoring reliability in extended dialogue. Empirical analysis of actual student behavior (12,650 messages across 500 conversations) found students extract answers despite pedagogy designed for sustained learning dialogue, fundamentally misaligning with Socratic design intent.

Deployment evidence revealed uneven adoption despite scale: Khan Academy's public admission that only 15% of students with access regularly engage with Khanmigo—despite 108 million cumulative interactions—prompted full summer 2026 platform redesign, signaling that tool availability does not translate to sustained engagement. Vendor optimization data from Khan Academy (6 months of A/B testing) showed +3.4% next-item correctness and +5.09% cognitive engagement improvements, reflecting ongoing product iteration. However, critical classroom-level analysis documented adoption failure: students bypass Socratic prompts, research shows limited reflection and weak knowledge transfer, particularly among struggling learners. Critical analysis from scholars (Stanford, UCL, others) documented fundamental automation limits: teaching fundamentally requires human judgment, contextual interpretation, and relational accountability that AI systems cannot fully replicate. RAND survey of 4,200 K-12 teachers found 27% specifically use Khanmigo, with 68% using AI tools weekly; yet only 34% believed AI made them more effective educators, indicating persistent adoption-efficacy gap.

By May 2026, the field had consolidated conviction that conversational AI tutoring worked effectively within carefully designed Socratic parameters with human oversight, with emerging evidence of parity with human tutors in controlled settings—but the critical gap between test-lab efficacy and real-world deployment remained unresolved. The practice faced persistent headwinds: knowledge tracing and student modeling frameworks lag pedagogical needs, accuracy-warmth design trade-offs constrain friendly tutoring, multi-turn conversation degradation limits extended dialogue, behavioral evidence shows students game systems by extracting answers, engagement remains concentrated among early adopters (15% active usage), and category leaders acknowledge limited transformative impact after four years of rollout. Conversational AI tutoring had solidified as an operational supplement within hybrid human-AI models but demonstrated enduring constraints that prevent transformative replacement of teacher-led instruction.