The Power of Feedback

Harnessing AI for Scalable, Evidence-Based Mathematics Instruction

Our Goal

The research project aims to achieve two primary objectives. First, it focuses on the impact of GenAI on students’ development and investigates the extent to which automated AI-generated feedback can support student math competence. Second, it examines the impact on teachers’ professional performance, with particular attention to teachers’ needs, attitudes, and practices regarding the integration of GenAI in math classrooms. With these objectives, the project aims to provide evidence-based recommendations for education policy and practice and contribute to the co-design of scalable, teacher-informed AI learning systems.

The Challenge

The rise of AI-powered educational platforms has opened new possibilities for personalized learning and formative assessment. However, robust empirical evidence on the effectiveness and limits of such tools is limited. In response, this international research project, initiated and internationally coordinated by the Organisation for Economic Co-operation and Development (OECD), aims to evaluate the use of generative AI (GenAI) in primary-level mathematics classrooms. For Germany, the project is led by Dr. Tim Fütterer from the Hector Research Institute of Education Sciences and Psychology at the University of Tübingen and funded by the Akademie für Innovative Bildung und Management Heilbronn-Franken gemeinnützige GmbH (aim).

Our Solution

Using the OECD’s open-source Platform for Innovative Learning Assessments (PILA), in this project, we will assess whether and how automated feedback improves students’ mathematics skills. Moreover, the project will explore teachers’ needs, attitudes, and practices regarding the integration of AI in the classroom and contribute to the co-design of scalable, teacher-informed AI learning systems.

Intervention Design

The intervention targets 4^th- and 5^th-grade mathematics classrooms (in Germany, only 4th grade) and utilizes an AI tutoring system (“Betty’s Brain Math”) embedded within PILA (an open-source Platform for Innovative Learning Assessments). The core study design is a randomized controlled trial (RCT) with two conditions:

Treatment group: Students receive formative feedback from the AI tutor while working on the exercises. Teachers have access to a live dashboard that displays all students in real-time, including the tasks they are currently working on, those they have answered correctly or incorrectly, and other performance indicators. Ultimately, a summary board presents aggregated insights, including average performance, frequently missed or successfully mastered tasks, and underlying conceptual difficulties, all based on AI-generated analysis of student data.
Control group: Students complete the same exercises but without receiving AI-based formative feedback. Teachers have access to the same dashboards as in the treatment group, including both the live overview and the summary board.

Teachers will participate in a 2-hour training session, followed by 6–8 weeks of classroom implementation (approx. 2×40 minutes per week). Pre- and post-standardized math tests, as well as student and teacher questionnaires, will assess additional learning outcomes (e.g., behavioral or cognitive engagement in class) and attitudes (e.g., competence beliefs).

Timeline

(Germany)

Q1–Q2 2026

School recruitment

Q3 2026

Teacher training

Q2–Q4 2026

Main study

Q4 2026–Q1 2027

Data analysis

Q4 2026–Q4 2028

Publication

Project Coordination & Partners

International Initiative and Coordination

Organisation for Economic Co-operation and Development (OECD)

Project Lead Germany

Dr. Tim Fütterer
Hector Research Institute of Education Sciences and Psychology, University of Tübingen

National Coordination Germany

Prof. Dr. Charlott Rubach & Henrike Kant,
Institute for School Pedagogy and Educational Research, University of Rostock

Research Team Germany

Prof. Dr. Jochen Kuhn, Chair of Physics Education, Ludwig-Maximilians-Universität München
Prof. Dr. Katharina Scheiter, Chair of Teaching and Learning with Digital Media, University of Potsdam
Prof. Dr. Ulrich Trautwein, Hector Research Institute of Education Sciences and Psychology, University of Tübingen
Prof. Dr. Marlit Lindner, Department of Education, Flensburg Campus, Leibniz Institute for Science and Mathematics Education (IPN)
Prof. Dr. Johanna Fleckenstein, Institute of Educational Science, University of Hildesheim

Financed by

aim – Akademie für Innovative Bildung und Management
Heilbronn-Franken gemeinnützige GmbH