Impactful and Responsible AI Systems for Education.

Groot and McKee will describe an ongoing collaboration between Eedi and Google DeepMind to evaluate a contextualised, misconception-grounded AI maths tutor, and how its evidence base is being built trial by trial. Eedi's diagnostic platform maps every wrong answer to a named misconception, so the AI tutor begins from a theory-driven model of the student's error rather than a blank chat window. Groot will tell the story of the exploratory three-arm randomised trial that contrasted a static pre-written hint, a live human tutor, and a supervised AI tutor working from the diagnosed misconception. In that trial the AI matched qualified human tutors on immediate error correction and on misconception resolution, and showed a directional advantage on knowledge transfer; both interactive conditions clearly outperformed static hints. McKee will then describe the scale-up now running: a four-arm, individually randomised trial across roughly 1,200 secondary maths students in ten English schools over twelve weeks, with STAR Maths growth as the primary outcome. The four arms — a control, a pedagogy-only AI tutor, a pedagogy-plus-hypercontext AI tutor, and a human tutor — are designed to isolate the question at the centre of the study: what does hypercontext, the rich personalisation of scaffolding to the individual student, add on top of a strong pedagogy prompt? The speakers will close on what this staged pipeline, from exploratory signal to confirmatory scale-up, offers as one practical way to keep rigorous evaluation and a fast-moving technology in step.

1. A Multi-Agent System for Feedback Quality Assurance: Evaluate, Revise, and Validate at Scale
   Okan Bulut, Cole Walsh
2. A Classroom Study of LLM-generated Feedback Intervention in Introductory Programming
   Hasnain Heickal, Andrew Lan
3. Knowing When to Defer: Selective Prediction for Responsible Knowledge Tracing
   Joshua Mitton, Prarthana Bhattacharyya, Ralph Abboud, Simon Woodhead
4. Micro-Randomized Trial of an AI-Simulated Practice Tool for Therapeutic Skills
   Ryan Louie, Ellen Converse, Diyi Yang, Emma Brunskill
5. Influence and Negotiation in Co-Designing an AI-Enhanced Multimodal Learning Analytics Platform for Secondary STEM Classrooms
   Hakeoung Hannah Lee, Hyun G. Kwon, Jongheon Kim, JeeHun Sung, Jandi Choi, Hyeri Mel Yang, Chaeyeon Kim, Miguel Lujan
6. More Correcting or Confounding? When an AI Arguing Agent Informed by Common Student Misconceptions Joins Case-Based Peer-Argumentation in a Real Course
   Jérôme Brender, Patrick Jermann, Engin Bumbacher, Francesco Mondada, Chenyang Wang
7. Psychometric Analysis of MRBench V2
   Mayank Sharma
8. Learning in Blocks: A Skill-Driven Framework for Assessment-Centered Agentic AI in Education
   Nicy Scaria, Silvester John Joseph Kennedy, Deepak N. Subramani
9. A Two-Validator Web Interface for Structured Geometry Figure Annotation
   Sabin-Codruț Badea, Adrian Marius Dumitran
10. More Sources, Better AI? When Context Helps and Harms AI-Assisted Discovery in Educational Data Archives
    Xin Wei, Morgan Lee, Jie Min, Jeremy Roschelle

AI presents both significant opportunities and risks for educational assessment including second language assessment. AI promises to produce higher-quality, shorter, and lower-cost language tests than was previously possible. However, educational assessment in high-stakes contexts demands that AI be deployed carefully to ensure tests maintain high standards of fairness and validity. This talk examines how two organizations have built on principles of Responsible AI to explore the use of AI in areas such as item development, test security, scoring, and communication of assessment results.

As LLMs rapidly enter adaptive testing and personalized tutoring, education needs evidence that moves with the technology while remaining grounded in learning theory and classroom realities. In this talk, we present FoundationalASSIST, an English educational dataset developed through collaboration between the ASSISTments team, WPI, and UCF to support continuous evaluation of LLMs for learning. Unlike existing datasets built around question identifiers and binary correctness labels, FoundationalASSIST provides full question text and actual student responses across 1.7 million interactions from 5,000 students. This makes it possible to evaluate whether LLMs can model student knowledge, predict responses, and understand pedagogical properties of assessment items. We evaluate four frontier models on Knowledge Tracing and Pedagogical Grounding tasks. Results reveal major gaps: models barely exceed trivial baselines for knowledge tracing and fall below random chance on item discrimination. These findings show why classroom-ready AI requires open datasets, continuous evaluation, and evidence-based deployment.

Presenters Fancsali and Ribeiro will describe an on-going collaboration between Carnegie Learning and the SCALE Lab at Stanford University to rigorously evaluate Carnegie Learning's MATHstream, a supplement to its blended, core middle math curriculum that is used by hundreds of thousands of learners every year. Fancsali will introduce MATHstream and a generative AI-enabled support within MATHstream called LiveHint AI before presenting preliminary evidence of associations between student learning outcomes and usage of MATHstream and LiveHint AI. Ribeiro will introduce SCALE Lab and describe collaborative efforts to evaluate MATHstream in an upcoming large-scale randomized controlled trial.

Learning is never neutral. Every learning moment is an ecosystem: learners arrive whole, educators arrive shaped by their own histories, and tools arrive encoded with assumptions, yet most systems see only a fraction of this context. We argue that context can and should be an intentional design target for AI in education, not a generic instructional add-on, across the continuum from preservice to inservice teaching. We illustrate this through two cases where instrumenting context is a central design guideline. In New Insights, practicing educators used AI-supported assessment as an in-the-moment guide, describing a shift toward asset-based teaching that made the whole learner — how they work, persist, and self-direct — more visible. In CRP-JeepyTA, preservice teachers learning culturally responsive pedagogy engage a tool grounded in local course knowledge, a persona informed by teacher educators of color, and prompts surfacing their own positionality. Together, these cases show how design can make learners' resources and teachers' contexts salient, rather than flattening them.