The State of Play

— Nature Scientific Reports study of seven leading LLMs reveals fundamental failure mode in multi-turn conversations: models oscillate between true/false on identical statements, exhibit sycophancy, and show unpredictable misinformation behavior limiting reliability for extended tutoring dialogues.

— University of Toronto NUMI RCT with 6,000+ middle-school math students: AI tutor withholding answers and coaching through mistakes yielded +3 points on transfer test, validating that well-designed conversational tutors teaching through scaffolding outperform unrestricted assistance.

— Microsoft Research + UIUC framework for training per-student simulators via two-stage learning; RL-optimized tutors trained with StudentSim rewards rated by experts as more accurate, better-guided, and more personalized than GPT-5.4 baselines across chess, writing, and math domains.

— Study of 50 learners comparing unrestricted chatbot, pedagogically-constrained Socratic mode, and adaptive brain-signal tutoring: unrestricted showed higher immediate gains (likely test-timing artifact); Socratic mode showed progressive disengagement; adaptive generated highest EEG-measured engagement.

— Chicago Public Schools abandoned district-wide Gemini rollout to 373,000+ high school students after three-school pilot exposed barriers to scaling; institutional decision classified pilot results as 'too early' for expansion, signaling caution despite early-adopter positioning.

— Bocconi University RCT with 1,000+ students randomly assigned to ChatGPT access, causal-inference training, both, or neither: ChatGPT group scored nearly 1 level higher on clarity/logic with expert-similar answers; students actively scaffolded thinking, not passively delegating work.

— Major philanthropic funder's infrastructure strategy for AI tutoring at scale. Learning Commons emphasizes teacher-in-the-loop conversational tools grounded in learning progressions, directly supporting guided discovery model.

— Synthesis of 2025–26 RCTs documenting critical trade-off: AI assistance improves immediate performance (48% better) but reduces unassisted exam performance (17% worse); design guardrails essential for efficacy.

— Peer-reviewed RCT of Rori WhatsApp AI math tutor in Ghana showing 0.36 effect size (roughly one year of learning) with low-cost supplemental deployment and teacher control retained.

— Credentialed analysis distinguishing pedagogical design from AI capability: custom tutors with tight scaffolding outperform active-learning 2×; Khanmigo showed no advantage over search/paper-only groups; unstructured AI reduces retention.

— World Bank RCT in Nigeria (9 schools, 6 weeks) found AI + teacher instruction yielded 0.31 SD gain, equivalent to 1.5–2 years of normal schooling, requiring teacher mediation to catch model hallucinations.

— Two-year preregistered RCT (18 schools, 6,902 student-term observations): Khanmigo assignment raised math achievement 1.3 percentile ranks/term (0.06–0.08 SD/year); binding constraint identified as student engagement, not model capability.

— RCT with 6,000+ students in Hamilton County: AI embedded in mastery workflow showed marginal delayed-test gain (40.2% vs 37.0%, 3.2pp, p=0.065) concentrated on practiced material; mechanism: improved post-error accuracy at cost of speed.

— World Bank meta-analysis of 191 effect sizes (14 RCTs, 10 economies): adaptive/AI interventions yield 0.125 SD average gain; NO evidence generative AI outperforms older ITS; critical gap: zero studies in low-income countries.

— IES government synthesis of 20 rigorous causal studies finding teacher-mediated tutoring promising, student-facing tools mixed, and general-purpose AI associated with worse outcomes; identifies guardrails as essential.

— RCT (n=117) showing ChatGPT improved essay scores but caused no knowledge gain or transfer; triggers metacognitive laziness and technology dependence—performance illusion masks learning failure.

— RCT (n=1,625) showing students rely on rather than learn from AI assistance; hybrid human-AI approaches no more effective than AI alone—questions assumption that additional support improves learning.

— Allen Institute for AI benchmark of 7 frontier LLMs on 462 authentic math tutoring transcripts (1,500+ teacher-flagged moments) reveals all models default to over-helping with plain instructions; design-critical failure mode.

— Sal Khan admits original Khanmigo version 'was a non-event' for most students; v2 redesign emphasizes teacher integration for engagement—candid founder perspective on real-world adoption barriers.

— Survey of 1,200+ K-12 principals shows 90% adoption of generative AI but only 30% believe it improved student learning; shallow integration despite high accessibility signals adoption paradox.

— Research brief summarising two RCTs: nearly 50% of students never engaged with AI platforms independently; even with human support, weekly usage 2-5 minutes fell short of 30-minute threshold for measurable reading gains.

— Analysis of sycophancy in AI tutors with Stanford studies showing models affirm 49% more than humans and different feedback by student demographics; Turkish study showed 17% exam drop with unrestricted AI.

— Pre-registered RCT (1,763 students, 12 schools, 8 weeks) of Guided Learning in Gemini achieved +0.258 SD math gains with 69% engagement; interaction analysis validated Socratic design (76% scaffolding questions, 2% direct solutions).

— AEFP Live Handbook evidence review identifies evidence gap: only 20 rigorous causal studies of 800+ papers; critical finding that students complete tasks more successfully with AI assistance but perform worse on unassisted assessments, indicating dependency effects.

— Expert-authored benchmark (1,490 prompts, 15,220 rubric criteria) evaluating frontier LLM tutoring capabilities: best model scored 55.7% on tutoring task essentials; systematic failure in pedagogical creativity and adaptive explanation generation—documents frontier model limitations.

— Scientific Reports randomized crossover RCT (194 Harvard undergraduates) comparing AI tutor vs in-class active learning showed median learning gains 2× larger with AI tutor (effect size 0.73–1.3 SD); depends critically on expert instructor-authored prompts, tight structure, and scaffolding.

— Practitioner-researcher analysis of AI tutoring efficacy across 10+ studies, including critical finding that Khanmigo showed no significant advantage over Google search or paper in Yaylali & Mehta 2025 RCT; identifies that strongest effects require teacher involvement and aligned assessment.

— Khan Academy founder's July 2026 public admission that Khanmigo's first iteration (2023-2026) 'did not change student learning as much as many of us hoped'; signals deployment reality vs pedagogical design optimism after three years of scale rollout.

— Longitudinal peer-reviewed study (26,000+ students, 30 months) tracking AI homework tool adoption: short-term homework score +18%, but exam performance declined 20% within 6 months, 24% within 2 years; mechanism analysis showed 81% outsourced (harmed) vs 19% used as tutors (stable gains).

— iLoveStudy AI tutoring agent demonstrated at WAIC 2026 with Socratic method implementation, multimodal interaction (speech, vision), step-by-step reasoning prompts, and sub-second end-to-end latency; shows production-ready system deploying guided discovery at scale.

— Frontiers in Education peer-reviewed analysis of 1,495 real chatbot interactions documenting heterogeneous engagement: small proportion drives disproportionate activity; many users disengaged after initial interaction; chatbot usage concentrated in administrative rather than tutoring domains.

— Founder's reflection: first Khanmigo did not change learning as hoped; core lesson is integrating AI into practice content to prevent cognitive offloading; references Newark deployment with state assessment gains.

— Longitudinal study tracking 26,000 secondary students over 30 months: unguarded AI tutoring showed 18% homework gain but 20% exam decline within 6 months and 24% loss on high-stakes exams after 2 years.

— Naturalistic study of 16,851 conversational tutoring interactions: concrete elaboration predicted next-turn understanding; empathetic language had no effect; shorter responses outperformed longer elaboration.

— North Carolina cut $10M Khan Academy contract 95% to $500k after Sal Khan admitted Khanmigo 'non-event'; only 5% of students achieved recommended dosage—critical signal of real-world adoption barriers.

— Georgia Tech/UC San Diego 173-student quasi-experimental study: Socratic Mind buffered quiz-decline by 3.3 points; 69% reported improved problem-solving, 62% critical thinking from Socratic questioning design.

— Singapore MOE AI-in-Education Framework mandates AI assistants function as Socratic guides, not answer engines; Learning Assistant (LEA) implements scaffolded questioning with zero-data retention.

— Real university physics course deployment (n=47 vs 53 controls) achieving 0.71-1.30 SD effect size with GPT-4 tutor providing context-aware hints without direct answers; self-selection limitations acknowledged.

— ACL 2026: LFTutor with intent-driven Socratic questioning significantly outperforms baseline LLMs on critical-thinking tasks; demonstrates pedagogical scaffolding improves reasoning outcomes.

— ACL 2026: LLMs excel at evidence extraction but struggle to leverage long-term history for knowledge-state diagnosis and adaptive teaching—a critical limitation for sustained conversational tutoring.

— AIED 2026 Best Paper award: Socratic guidance tutor produced higher learning gains and independent transfer to unconstrained LLM use than prompt-refinement tutor, validating Socratic dialogue as superior design.

— ACL 2026 Best Social Impact Paper analyzing 12,650 real student-AI dialogue messages: educators designed for learning dialogue, but students predominantly use conversational tutors for answer extraction.

— Critical analysis documenting fundamental LLM limitations in conversational learning: knowledge distributed in parameters, cannot forget, cannot simulate unstable intermediate knowledge states that characterize learning and scaffold teaching.

— Microsoft's third annual AI in Education Report: 92% of students and educators using AI for school; 78% of leaders implementing/scaling AI; critical finding: 77% of students and 53% of educators have received NO formal training.

— Empirical study testing 4 LLMs (GPT-4o, GPT-3.5, Llama-3.3, Llama-3.1) on 600 eighth-grade essays: identical text produced different feedback by student demographics; documented bias in tools powering MagicSchool and School AI used at scale.

— NYC DOE's ERMA governance framework expanded to require algorithmic bias and equity review for tools affecting 1.1M students across 1,700+ schools; red-flags AI use in academic placement, grading, discipline.

— Stanford SCALE Initiative RCTs: 40–47% of elementary students never logged in; average weekly use 2.18–5.23 minutes vs 30 minutes needed for gains; human support increased engagement 71–80% but insufficient for dosage. Critical adoption-barrier evidence.

— Learning analytics of Socratic AI tutor in Python course (95 interactions): prior experience moderated interaction-performance relationship (p=.045); longer conversations negative for beginners, positive for experienced students—Socratic indirection risks cognitive overload.

— Microsoft announces Study and Learn Agent in Copilot Chat, reframing AI from answer-engine to learning coach; scaffolded questions, interactive practice, feedback designed for retention and independent thinking.

— Analysis of Bastani et al. (PNAS 2025, Turkish RCT ~1,000 students) showing unguarded ChatGPT achieves 48% higher practice scores but 17% worse exam performance; Socratic guardrails preserve learning. OECD-cited design-dependent outcomes.

— Empirical study of CURIOBOT framework operationalizing Berlyne's collative variables (novelty, complexity, conflict, uncertainty) as linguistic interventions in conversational tutoring; 270 conversations showed 2.4x more exploratory turns.

— Systematic scoping review of 104 AI-feedback studies (2008-2024): hybrid human-AI approaches consistently outperform AI-only; effectiveness depends heavily on implementation context and population equity.

— Empirical study showing email-based guidance on Socratic AI-tutor use (ask for explanations, support reasoning) achieved +0.22 SD on final exams concentrated in open-response questions; low-cost intervention addresses usage-quality gap.

— Stanford National Student Support Accelerator publishing peer-reviewed RCTs on AI tutoring effectiveness; establishes AI tutoring efficacy depends on human engagement structures and hybrid human-AI approaches improve scalability.

— UK Department for Science assessment: AI tutoring tools have 'limited quantity, scope and evidence base, with few providing full tutoring capacity.' Government contracts 8 vendors for co-design pilots; critical policy-level skepticism of tools' maturity.

— NPR/Ipsos poll of 500+ K-12 teachers: only 23% use AI weekly for classroom instruction vs 54% for admin; 55% see AI as shortcut avoiding work; critical adoption barrier revealing minimal instructional deployment.

— Peer-reviewed empirical study (N=1,498 undergraduates, 38 classes) introducing 'AI-Learning Gap': AI-assisted work quality (M=7.62) substantially exceeds independent knowledge mastery (M=5.55). Demonstrates AI assistance improves output without strengthening understanding.

— Empirical analysis of 9,490 real-world deployment chats revealing critical gap: students bypass chatbot scaffolding in practice (ICML 2026). Benchmarks assume high uptake; deployments show students drive interactions toward own goals, fundamentally misaligning with Socratic design intent.

— Pre-registered RCT of Gemini Guided Learning with 1,763 students across 12 schools: +0.258 SD math gains (1.2–1.7 years progress in 8 weeks); 69% engagement; 76% scaffolding questions, 2% direct answers—Socratic design validated empirically.

— Statewide deployment: Utah State Board of Education deploying Gemini for Education with Guided Learning to 708,000 K-12 students and 28,000 teachers starting 2026-2027; Socratic method design (step-by-step hints, not direct answers).

— Empirical study (N=98 Grade-9 students, 1,616 conversational turns) showing post-test performance significantly lower than pre-test; students dominated by instrumental help-seeking with no self-regulation; higher cognitive load predicted lower scores—critical failure mode in student-AI tutoring dialogue.

— RCT across 10 Taipei high schools (970 students, 5-month Python course): personalized problem sequencing with Socratic AI tutor guidance achieved 0.15 SD learning gain versus standard curriculum, equivalent to 6–9 months additional learning.

— Iowa State deployment across two years (160–180 students in animal science lab): AI tutor achieved 4.6 percentage-point grade improvement for users, 9.1 points for heavy users (full letter grade), with 40% voluntary adoption rate demonstrating real-world engagement.

— ACL 2026 peer-reviewed paper introducing LFTutor, an LLM-based tutoring system using intent-driven Socratic questioning and critical argumentation; demonstrates pedagogical scaffolding significantly outperforms baseline LLMs.

— Oregon State research documenting cognitive impacts of heavy AI tutoring use: 66% decline in reflection, 41% drop in critical thinking, 21% lower perceived need to understand concepts; mechanism identified as cognitive offloading; tech-savvy students most vulnerable.

— Khan Academy's official reporting: 108M total Khanmigo interactions, 269K daily interactions, but only 15% engagement rate; organization pivoted to measuring 'next-item correctness' and implemented full platform redesign for summer 2026 launch.

— Peer-reviewed framework for training conversational Socratic tutoring agents at scale; proposes student simulation, pedagogical reward modeling, and multi-objective RL achieving competitive performance with proprietary models using 30B parameters.

— National University of Singapore deployment of ScholAIstic across social work and dentistry courses with Socratic dialogue simulations and real-time feedback; received OpenGov Asia Recognition of Excellence Award 2026; demonstrates professional training contexts.

— Practitioner analysis of Khanmigo's adoption failure: identifies structural barriers (anterograde amnesia from session-reset, data-access moats protecting LMS/SIS vendors, cost scaling) and establishes system limitations as adoption constraint rather than pedagogics.

— Critical synthesis of Wharton RCT showing unrestricted AI tutoring harmed exam performance (−17% vs control) while Socratic-guardrailed tutoring preserved learning; explains cognitive offloading mechanism and validates guided discovery design principle.

— Self-distillation framework addressing multi-turn conversation degradation where information is revealed incrementally; recovers 92–100% of single-turn performance across model families (Llama, Qwen, Phi, OLMo); addresses core reliability barrier in conversational tutoring.

— Teacher Tapp national survey (8,000–10,000 teachers): 54% use AI for lesson planning and quizzes, but only 11% for live lesson delivery; adoption barriers are reliability concerns (56%) and academic integrity fears (46%), not confidence.

— Benchmark of 7 LLM tutoring agents on 10,836 solution-feedback pairs revealed systematic failure to distinguish valid-but-suboptimal reasoning from incorrect solutions—precisely where adaptive feedback should discriminate.

— Large-scale RCT (1,662 students, 5 schools, China): adaptive AI tutoring outperformed traditional instruction with +8.78 to +13.84 point gains and Guinness certification; demonstrates deployment efficacy at scale.

— Georgia Tech Socratic Mind deployment: Socratic dialogue design achieved 40% increase in student self-questioning and 20% improvement in metacognition; scaled implementation model demonstrated.

— Deployment priority hierarchy: conversational tutoring is lower-priority adoption target than administrative and teacher-facing AI in 2026 schools; cites NCES, EdSurge, and governance framework data.

— Michigan Virtual longitudinal study (26,106 K-12 students, 2 years): achievement gap between AI users and non-users narrowed by 91%; high performers improved from B+ to A-.

— Rigorous Eedi RCT (20 schools, 3,448 Year 7 students, 2 years): no effect at 6 months, but +0.32 SD learning gains by 18 months with sustained engagement; Socratic AI tutoring outperforms expert human tutors on transfer.

— OECD meta-analysis of RCTs and design experiments: average +4 percentage point tutoring gains with high variability; +9 points when AI supports novice tutors; critical finding of cognitive offloading risks with unrestricted access.

— Founder admission: Sal Khan stated Khanmigo 'was a non-event' for most students despite 700K+ users across 380+ districts; critical signal documenting deployment-engagement gap and structural barriers to transformative impact.

— Quasi-experimental RCT (635 students, grades 5-8): hybrid human-AI tutoring achieved 36% proficiency gains and 61% MAP growth, validating differentiated human-AI model at scale.

— Neuron (top-tier journal) validation: 8-10 minute conversational interactions with AI matched human tutors on recall, comprehension, and knowledge transfer; provides neuroscientific evidence of learning mechanism parity.

— Named deployments with scale metrics: Harvard CS50 Duck answered 800K+ student questions; Georgia Tech, ASU operating conversational AI tutors; provides 2026 institutional adoption landscape data.

— Critical classroom-level analysis documenting adoption failure of Khanmigo: students bypass Socratic prompts, research shows limited reflection and weak knowledge transfer, particularly among struggling learners.

— Neuron RCT (57 university students, HKUST, 2026) showing AI-led conversational tutoring produces learning outcomes and brain synchrony patterns statistically indistinguishable from human instruction.

— Khan Academy vendor update on Khanmigo optimization: 6 months of A/B testing yielded +3.4% next-item correctness and +5.09% cognitive engagement across millions of sessions, documenting ongoing product maturation.

— Khan Academy's public admission of critical adoption barrier: only 15% of students with access regularly engage with Khanmigo despite 108M cumulative interactions, triggering summer 2026 platform redesign.

— Research framework proposing interpretable knowledge tracing for LLM-based tutoring dialogues grounded in Item Response Theory, addressing how conversational tutors assess and adapt to student knowledge state.

— Oxford Internet Institute / Nature study (400k+ responses across 5 models) documenting accuracy-warmth trade-off: 7.43 percentage-point error increase when fine-tuning for empathy, directly constraining tutoring chatbot design.

— ICLR 2026 outstanding paper showing severe accuracy degradation across 15 major LLMs in multi-turn conversations (39% decline), directly limiting conversational tutoring reliability in extended dialogue.

— Detailed case-study of Khan Academy's rigorous A/B testing methodology for Khanmigo: 64 completed experiments, 29 running; demonstrates four-phase evaluation process for continuous product optimization.

— Peer-reviewed empirical study (12,650 messages across 500 conversations) identifying fundamental misalignment: students extract answers despite pedagogy designed for sustained learning dialogue, undermining Socratic design intent.

— Peer-reviewed RCT (PNAS, ~1000 Turkish high school students) showing unrestricted conversational AI tutoring causes 17% exam score decline via cognitive debt, while constrained guided tutoring preserves learning—directly demonstrates critical design requirements.

— Khan Academy vendor update on Khanmigo performance: 269,000 daily interactions, 108 million cumulative interactions, ongoing product improvements based on classroom feedback—demonstrates sustained commercial operation and user engagement at scale.

— Rigorous RCT across 62 schools in 4 countries (Nigeria, Spain, Ireland, India) with 14,892 Grade 7-9 students showing 0.27 SD effect overall, 0.41 SD for low-prior-achievement students, validating effectiveness of LLM tutors when integrated with classroom instruction.

— Multi-site empirical study of conversational AI tutoring in K-12 classrooms identifying design levers that improve dialogue quality, while revealing persistent gap between intended and actual cognitive demand during student-AI conversations.

— UK Department for Education commitment to fund and deploy adaptive AI tutoring tools at national scale: up to 8 companies selected, £300,000 each, targeting 450,000 disadvantaged Year 9-10 students annually—signals policy-level adoption momentum.

— Comprehensive critical assessment by recognized edtech critic documenting Khanmigo engagement failure after three years: subsidy dependency, limited teacher adoption despite scale rollout, analysis of why AI tutors without human relationships fail to sustain adoption.

— Semantic narrative analysis documenting Sal Khan's April 2026 candid retrospective assessment that Khanmigo has not delivered predicted learning revolution after three years of rollout—critical limitation signal from practice's category leader.

— Peer-reviewed RCT (138 undergraduates, programming) showing unguided generative AI tutors significantly impair metacognitive calibration (p<.001, d=0.75) via cognitive offloading, with deficit persisting on transfer tasks—negative evidence for importance of guardrails.

— Systematic review in npj Science of Learning (28 empirical studies, 4,597 K-12 students) finding intelligent tutoring systems produce medium-to-large effects, with effectiveness contingent on pedagogical features, personalization, and implementation conditions.

— Critical analysis arguing teaching fundamentally resists full automation because instructional work depends on human judgment, contextual interpretation, and relational accountability—supports necessity of hybrid human-AI tutoring models.

— RAND survey of 4,200 K-12 teachers: 27% specifically use Khanmigo for student tutoring, 68% use AI tools weekly; only 34% believe AI makes them more effective, balanced against 41% reporting job has become harder.

— Multi-institution deployment of course-grounded Socratic AI tutor across 400-student genetics class and other disciplines; tutor asks questions rather than providing answers, designed by researchers to follow Socratic principles.

— Active research project from ELLIS Alicante developing Socratic chatbot with planned empirical evaluation on critical thinking; represents contemporary institutional direction toward guided discovery design with learner agency.

— Khan Academy CLO describes 'Explain Your Thinking' feature piloted in select schools: AI engages students in dialogue to reveal conceptual understanding beyond correct answers, exemplifying guided discovery design.

— Stanford preprint research identifies systematic bias in AI tutor feedback: high-achieving and White students receive detailed developmental feedback while Hispanic, ELL, and low-achieving students receive grammar-focused responses—critical barrier to equitable adoption.

— Peer-reviewed study shows AI-assisted tutoring significantly enhances intrinsic motivation and self-efficacy, with pronounced effects for lower-achieving students, indicating equitable impact for at-risk populations.

— Quasi-experimental STEM study shows Socratic AI tutoring enhances academic performance especially for low-prior-knowledge learners, with metacognitive engagement as primary mechanism—validates pedagogical design principle.

— Synthesis of design research: Turkish RCT (1,000 students) shows guarded AI with step-by-step hints outperforms unrestricted access; adaptive sequencing trial (700 students) shows 0.15 SD gains from AI-orchestrated productive struggle, validating Socratic scaffolding principle.

— Practitioner evidence synthesis: UK DfE committing £1M+ to trial AI tutoring with 450,000 disadvantaged students by 2027; identifies what makes AI tutoring effective vs. generic tools; cites RCT evidence that unstructured AI harms outcomes.

— Systematic review (67 studies, 2019-2025) on AI-powered Socratic tutoring in medical education: confirms scalability and psychological safety; identifies persistent challenges (bias, hallucinations, privacy); recommends human-AI symbiosis model.

— Appalachian State faculty designed Macro Buddy conversational tutor for economics; empirical finding: students using AI tutor with peer discussion earned higher exam scores than solo study, validating guided discovery + dialogue synergy.

— Market ecosystem scale: Khanmigo 1.4M cumulative users; Duolingo 116M MAU with conversational features; Squirrel AI 24M students. February 2026 Khan+Google partnership signals platform maturity; research confirms hybrid human-AI models deliver strongest results.

— Khan Academy CLO (PhD, educational psychology) discusses learning science foundations for conversational tutoring, cognitive offloading risks, safety guardrails, and responsible design—expert perspective on practice maturity and constraints.

— ECAI 2024 workshop: fine-tuned Llama 2 models designed for Socratic questioning outperform standard chatbots on critical thinking outcomes; validates pedagogical design principle using open-source, privacy-preserving LLMs.

— Gold-standard RCT (n=334) showing unrestricted AI tutoring outperforms restricted access by 0.21 SD; challenges concerns about overreliance and demonstrates effectiveness of continuous AI availability for guided discovery.

— Upper Canada College deployment: teachers built curriculum-aligned AI tutors via no-code platform; outcomes: 23% reduction in remedial support, 78% weekly engagement, 82% student helpfulness rating with explicit Socratic guidance architecture.

— Khanmigo grew to 700,000 users across 380+ U.S. school districts in one year; documented 5 hours/week teacher time savings and RCT evidence of math performance gains, especially for below-grade-level students.

— Critical analysis by UCL Knowledge Lab professor Rose Luckin argues AI tutors address only narrow fraction of human intelligence; cites research on metacognitive laziness, reduced self-monitoring, and procrastination when AI is removed, documenting fundamental pedagogical limitations.

— Quasi-experimental study in Journal of Computer Assisted Learning with 80 college students compared Socratic AI (GSL) vs direct-answer AI (GDL) in programming; Socratic approach fostered deeper cyclical engagement, critical thinking, and reduced frustration vs trial-and-error dependency.

— FutureEd coverage of two RCTs: Google LearnLM (165 students, 76.4% AI message approval, 66% vs 61% novel problem success) vs Tutor CoPilot (1,000 elementary students, 4pp mastery improvement), contrasting AI-substitution and AI-augmentation deployment models.

— Peer-reviewed quasi-experimental study from Hong Kong Polytechnic with 31 healthcare students showed Socratic Playground for Learning platform significantly increased self-efficacy (effect size 0.57, p=0.041), validating AI Socratic method effectiveness in professional education.

— Framework synthesis from Cornell, University of Adelaide, University of Florida, and Digital Promise proposing 'keep, change, center, study' design for conversational AI tutors, integrating human tutoring research with generative AI to produce pedagogically sound systems.

— Brookings Institution synthesis of RCT evidence on generative AI tutoring showing substantial learning gains, knowledge transfer, personalization at scale, and psychological safety benefits, while acknowledging concerns about accuracy and design responsibility.

— Preprint analysis of 11,406 students across 10 post-secondary institutions using GenAI Tutors, identifying heterogeneous engagement patterns (10.4% shallow engagement with copy-pasting) and selectivity effects, offering deployment realities at scale.

— Critical assessment documenting AI tutoring limitations: inability to read emotion, shallow instructional dialogue, equity risks; contrasts with human tutoring meta-analysis (282 RCTs), providing negative signal on current AI effectiveness.

— RCT of 165 British students (13-15) showed supervised AI tutors outperformed human-only tutoring (66.2% vs 60.7% problem-solving success), with 0.1% hallucination rate, validating hybrid human-AI model efficacy.

— Khan Academy CLO details Khanmigo metrics: students reaching 2+ proficient skills weekly see significant yearly gains; next-question correctness after AI assistance indicates sustained learning, providing vendor deployment evidence.

— Global survey of 225 security leaders revealed only 6% of education orgs conduct red-teaming; 84% lack AI anomaly detection, 79% lack purpose binding, documenting critical safety gaps in deployed conversational AI systems.

— Summary of AIED 2025 conference (700+ participants) documenting consensus shift toward Socratic tutoring design; Khan Academy keynote positioned Khanmigo as guiding reasoning rather than answer provision.

— Academic critique arguing current GenAI chatbots are tools, not intelligent tutoring systems, lacking the tutor and student models required for effective teaching; raises concerns about accuracy, completeness, and creating illusion of learning.

— Comprehensive literature review of 48 studies on AI tutoring effectiveness reporting benefits (improved STEM, motivation) and significant limitations (cognitive offloading, reduced critical thinking, modest gains compared to traditional instruction).

— Peer-reviewed RCT (N=165) by Google LearnLM Team showing supervised AI tutor performed at least as well as human tutors, with 5.5 pp better knowledge transfer on novel problems; 76.4% of AI messages required zero or minimal editing.

— Research review showing AI tutoring efficacy for disadvantaged students: Nigeria pilot achieved 0.3 SD gains in 6 weeks; Tutor CoPilot study (900 tutors, 1.8k students) found 4pp mastery improvement, greatest benefit for lower-rated tutors.

— Critical founder perspective detailing widespread problems: ChatGPT 50% math accuracy, Khanmigo struggles with complex math; UPenn study found students solved 127% more practice problems but performed no better on exams.

— Khan Academy founder reports Khanmigo expects to reach 1M elementary and secondary students in 2025-26 school year (up from 700k), discussing Socratic pedagogy, addressing cheating concerns, and role of AI tutoring as teacher's guide.

— Louisiana Department of Education piloted Khanmigo starting January 2025, achieving 50% student account activation and 71% teacher usage across participating schools with 22 professional development sessions, demonstrating state-level deployment pathway.

— University law course deployment of SmartTest Socratic chatbot showed 40-54% error rates in feedback generation, significant integration effort required, and only 27% student preference for AI feedback over human tutors despite 76% wanting tool access.

— Mixed-methods study of 8 Palm Beach County high school students found Khanmigo provided no performance advantage over non-AI control group, offering critical evidence on effectiveness limitations despite adoption momentum.

— Controlled study of Socratic AI Tutor with 65 German pre-service teachers found significant improvements in critical, independent, and reflective thinking compared to uninstructed chatbot, validating pedagogical design of guided dialogue.

— Khan Academy CLO Kristen DiCerbo documents Khanmigo's growth from 68,000 users (2023-24) to 700,000+ (2024-25) across 380+ district partners; addresses persistent challenges including prompt inconsistency and need for rigorous evaluation.

— Systematic review of ITS from 2010-2025 identifies mixed effectiveness across contexts, persistent evaluation rigor gaps, and need for greater scientific testing to validate conversational AI tutoring approaches.

— Alpha School operational deployment of AI-integrated educational model reporting students learn 2.3x faster than statistical predictions with 99th percentile standardized test results, demonstrating claimed efficacy of AI-augmented personalized learning.

— Critical assessment arguing AI tutors fundamentally misunderstand learning by reducing open-ended exploration to curriculum-aligned problems, lacking meaningful context needed for genuine understanding and likely to fail despite adoption momentum.

— Systematic review in NPJ Science of Learning assessing the effects of ITS on K-12 students' learning outcomes and experimental design rigor, providing meta-level synthesis of empirical evidence on conversational AI tutoring efficacy.

— Education Week practitioner analysis identifying key implementation barriers: learner readiness for effective AI interaction, need for teacher customization control, and persistent requirement for human oversight due to AI inconsistency.

— Spring 2025 survey of 3,000+ educators and students showing 63% of K-12 teachers incorporate GenAI into teaching (up 12% YoY), and 67% of HED students use GenAI to summarize concepts, quantifying broader adoption momentum in educational institutions.

— Michigan Virtual structured K-12 Khanmigo deployment pilot with 25 teachers in grades 6-12, including yearlong professional development program and inclusive district pricing strategy, demonstrating integration pathway for educational institutions.

— Synthesis of recent research showing unguarded AI tutors harm learning (ChatGPT: 17% worse on exams) but customized tutors with safeguards boost performance; guardrails and design constraints are essential for efficacy.

— Enid High School (Oklahoma) deployed Khanmigo in geometry classes with remarkable increases in math achievement and doubled engagement; demonstrated strategic implementation with teacher ownership and just-in-time feedback.

— Respected math educator Dan Meyer critiques AI tutors' inability to replicate human teacher sensing; cites nationwide study of individualized learning software showing paltry effect in math and diminished student social connection.

— Mexico-based research with 46 doctoral students showed Socratic Lab AI tool increased participation compared to async forums, with higher satisfaction when combined with teacher mediation in synchronous sessions.

— Peer-reviewed study with 230 university students in Taiwan comparing ChatGPT and human tutors for critical thinking; students valued ChatGPT's accessibility but preferred human tutors for tailored feedback, supporting hybrid model integration.

— Michigan State University piloted Khanmigo with 80 students, achieved impressive results, and expanded to 800 students; students reported better understanding and improved performance on assessments.

— Philippines Department of Education partnership with Khan Academy and Smart Communications for nationwide Khanmigo deployment, providing free access to millions of students through telco data partnership.

— Reporting on Khanmigo's large-scale deployment across 266 U.S. school districts; documents safety feature that detects student self-harm discussions and notifies teachers, extending conversational AI tutoring beyond academic learning.

— Longitudinal efficacy study of ~350K students in grades 3-8 showed 20% greater-than-expected learning gains with 30+ minutes weekly Khan Academy use, including Khanmigo conversational tutoring.

— Indian School of Business case study showing customized AI tutor integrated into EMBA course improved student engagement with primary sources and academic performance, demonstrating higher-ed deployment feasibility.

— Critical assessment by education technology expert arguing AI tutors are ineffective for learning, citing Wharton RCT where ChatGPT reduced student achievement, countering deployment momentum with empirical limitations.

— University of Genoa's formal teacher certification program for conversational AI tutoring, structured with three mastery levels, signaling institutional standardization and professionalization of AI tutoring pedagogy.

— Georgia Tech Socratic Mind platform demonstrated large-scale pilot with 2,000 students using AI-powered Socratic questioning for assessment, showing scalability of conversational tutoring method.

— Education Week documents Khanmigo's persistent math errors and teacher verification strategies, surfacing reliability constraints that shape classroom deployment practices.

— Wharton study with 1,000 students showed AI tutoring improved practice performance (48% better) but harmed exam performance (17% worse) without guardrails; safeguards crucial for efficacy.

— Microsoft and Khan Academy partnership makes Khanmigo for Teachers free globally across 49 countries, signaling major ecosystem integration and broadened accessibility.

— News analysis of New Hampshire's $2.3M state contract with Khan Academy for Khanmigo, documenting state-level adoption alongside critical concerns about AI hallucinations and safeguards.

— Bill Gates documents a pilot deployment of Khanmigo in Newark schools, providing independent third-party validation of real-world classroom adoption and teacher use.

— Critical assessment dismissing current AI tutoring chatbots as outdated text-based tools, arguing they inadequately support modern pedagogy and that multimodal AI represents a more promising direction.

— Balanced expert debate on Khanmigo: Sal Khan advocates for AI tutors while Dan Meyer (Amplify) critiques effectiveness for conceptual learning and average students, surfacing key adoption limitations.

— Market analysis documenting consumer adoption of AI tutor apps (Answer AI 6M downloads, Question AI 12M+), cost displacement of traditional tutoring, and persistent hallucination challenges.

— Khan Academy and Microsoft partnership made Khanmigo for Teachers free for all U.S. educators, signaling major ecosystem investment and shift toward broad accessibility for category-leading product.

— Independent reporting on Newark Public Schools' Khanmigo pilot and districtwide expansion plans; documents pricing ($35/student), persistent math errors, and teacher feedback on tool usability.

— EMNLP 2024 research exposing 'Student Data Paradox': training LLMs on student dialogue data degrades factual knowledge and reasoning, revealing fundamental technical constraints on AI tutoring efficacy.

— ACM CHI 2024 paper addressing student modeling in conversation-based tutoring systems, showing framework effectiveness in facilitating personalization for individual learner needs.

— Khan Academy engineering director detailed Khanmigo's development, safety alignment, and scaling strategy for universal access; positioned conversational AI tutoring as foundational infrastructure for future education systems.

— Peer-reviewed empirical study of student acceptance and strategic analysis of Khanmigo, showing positive engagement with caveats: concerns over technical constraints, ethical dilemmas, and need for human monitoring.

— University study of AI tutor Syntea deployed with hundreds of distance learning students across 40+ courses, showing 27% average study time reduction by third month post-launch.

— Khan Academy announced Khanmigo's progress-tracking and student-identification tools, including automated detection of struggling students and skill gaps, indicating continued product maturation.

— L@S 2024 conference paper presenting Socratic Mind, an LLM-based oral assessment system, tested with 600 students in large classroom deployment, demonstrating scalable Socratic dialogue implementation.

— Peer-reviewed study showing student interaction with generative AI positively influences learning achievement through self-efficacy and cognitive engagement, providing mechanisms-level evidence for AI tutoring effectiveness.

— Respected math educator's critical analysis predicting limited adoption increases because AI tutors lack the 'impatient demand generation' great teachers provide; cites 2018 data showing only 11% of students used Khan Academy as recommended.

— Khan Academy official announcement of Khanmigo pilot integrating GPT-4 as 'virtual Socrates' asking guiding questions; includes explicit acknowledgment of current limitations including math errors and hallucinations.

— Product update reporting Khanmigo adoption scaling to 30+ school districts and 28,000 students/teachers, with price cut from $60 to $35 per student annually to increase accessibility.

— Quasi-experimental study across three urban, low-income schools with 585 middle school students showed hybrid human-AI math tutoring achieved positive effects on proficiency and usage, particularly benefiting lower-achieving students.

— University research study evaluating LLMs as unsupervised tutors in thermodynamics found leading model (GPT-4) achieved only 82% accuracy, falling short of 95% threshold required for educational use, documenting critical domain-specific limitations.

— AIED 2023 conference paper evaluated 13 computational text models for assessing student responses in conversational ITS using 5,166 response pairings; combination models outperformed individual models against human judge assessment.

— Randomized controlled trial with 900 tutors and 1,800 K-12 students from underserved communities showed AI-augmented tutors achieved 4pp higher topic mastery, more likely to use guiding questions and less likely to give answers.

— Education Week expert analysis of AI tutoring limitations: experts emphasized AI lacks empathy and emotional connection, and human tutors provide motivation, accountability, and consistency that technology cannot replicate.

— EACL 2023 analysis of neural dialog tutoring models found poor performance in less constrained scenarios, 45% of conversations showed significant reasoning errors, and human evaluation revealed low performance in equitable tutoring.

— Khan Academy launched Khanmigo pilot using GPT-4, designed as 'virtual Socrates' that refuses to give direct answers and instead asks guiding questions; invited 500 partner schools for limited pilot access.

— Khan Academy deployed Khanmigo AI tutor pilot in Brazil (Paraná and São Paulo) with 155 students and teachers; teachers reported students felt less shame asking questions of AI, with plans to expand to 10,000 users.

— Peer-reviewed controlled study showing undergraduate students in Ghana using AI chatbot tutoring performed better academically than those with human instructors, first such study in Ghana.

— Critical analysis warning that AI-aided emotional regulation in conversational systems has negative consequences, raising design concerns for empathetic tutoring approaches.

— NPR coverage of ChatGPT's educational applications documenting accuracy limitations, hallucination risks, and expert skepticism about AI tutoring capabilities in late 2022.

— Education practitioner analysis documenting specific limitations: error rates, bias risks, and lack of interpersonal relationships as critical barriers for AI tutoring deployment.

— EMNLP 2022 conference paper presenting technical method for AI to auto-generate Socratic subquestions guiding students through math problems, core technique for conversational tutoring.

— Peer-reviewed study demonstrating conversational AI chatbot maintained children's reading interest in book talk, while control group interest faded significantly.

2026

2025

2024

2023

2022