View in EDS

Cheaper (and More Effective) by the Dozen: Evidence from 12 Randomized A/B Tests Optimizing Tutoring for Scale. EdWorkingPaper No. 25-1333

Saved in:
Bibliographic Details
Title: Cheaper (and More Effective) by the Dozen: Evidence from 12 Randomized A/B Tests Optimizing Tutoring for Scale. EdWorkingPaper No. 25-1333
Language: English
Authors: Noam Angrist, Claire Cullen, Janica Magat, Annenberg Institute for School Reform at Brown University
Source: Annenberg Institute for School Reform at Brown University. 2025.
Availability: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
Peer Reviewed: N
Page Count: 43
Publication Date: 2025
Sponsoring Agency: Jacobs Foundation
Document Type: Reports - Research
Education Level: Elementary Education
Descriptors: Tutoring, Tutorial Programs, Cost Effectiveness, Instructional Improvement, Efficiency, Instructional Effectiveness, Educational Experiments, Numeracy, Computer Uses in Education, Handheld Devices, Telecommunications, Foreign Countries, Elementary Education
Geographic Terms: Botswana
Abstract: Over the course of 12 rapid randomized experiments, we optimize an educational tutoring program. Tutoring is one of the most effective educational approaches yet has remained difficult to scale due to high costs. We adaptively test and improve a technology-enabled tutoring program to enhance cost-effectiveness and scalability. Results show that seven of twelve tests led to efficiency improvements, a "rate of discovery" of 58%. This compares favorably to the tech sector where 10-40 percent of tests generate improvements, demonstrating the potential for A/B testing to yield large efficiency gains in the education sector. The largest efficiency gains were driven by cost-reducing modifications that streamlined labor-intensive implementation processes and effectiveness-enhancing innovations that actively involved caregivers in their child's education, more than doubling impact at minimal additional cost. We explicitly measure practitioner prior and posterior beliefs, and find that rigorous testing facilitates more accurate identification of 'what works.' Our findings both reveal the returns to iterative testing in social programs and contribute new evidence on simple, cost-effective strategies to improve learning outcomes. [This report was funded by the UBS Optimus Foundation, the Mulago Foundation, the Douglas B. Marshall Foundation, and the Agency Fund.]
Abstractor: As Provided
Entry Date: 2026
Accession Number: ED678304
Database: ERIC
Description
Abstract:Over the course of 12 rapid randomized experiments, we optimize an educational tutoring program. Tutoring is one of the most effective educational approaches yet has remained difficult to scale due to high costs. We adaptively test and improve a technology-enabled tutoring program to enhance cost-effectiveness and scalability. Results show that seven of twelve tests led to efficiency improvements, a "rate of discovery" of 58%. This compares favorably to the tech sector where 10-40 percent of tests generate improvements, demonstrating the potential for A/B testing to yield large efficiency gains in the education sector. The largest efficiency gains were driven by cost-reducing modifications that streamlined labor-intensive implementation processes and effectiveness-enhancing innovations that actively involved caregivers in their child's education, more than doubling impact at minimal additional cost. We explicitly measure practitioner prior and posterior beliefs, and find that rigorous testing facilitates more accurate identification of 'what works.' Our findings both reveal the returns to iterative testing in social programs and contribute new evidence on simple, cost-effective strategies to improve learning outcomes. [This report was funded by the UBS Optimus Foundation, the Mulago Foundation, the Douglas B. Marshall Foundation, and the Agency Fund.]