Effects of Zearn Math and balancing analyses in large-scale trials

A new EdWorkingPaper from the Annenberg Institute at Brown University highlights not only the effects of Zearn Math, a digital K–8 platform, but also how education researchers balance confirmatory and exploratory analyses in large-scale trials. Conducted by the RAND Corporation, the two-year randomized controlled study involved more than 10,500 students in Grades 3 to 5 across 64 schools in a large urban Texas district. The study’s preregistered confirmatory outcome was the STAAR state assessment, selected for its alignment with grade-level standards and central role in Texas accountability policy. The NWEA MAP served as a preregistered exploratory outcome. Researchers use confirmatory outcomes in preregistration to indicate the primary outcomes they will use to test their key questions and interpret as the main evidence of impact. Exploratory outcomes can still have hypotheses and be preregistered, but they are typically secondary measures (like a district test vs. the state test) and are interpreted with more caution, helping researchers see if patterns replicate across measures without overclaiming from secondary data.

The study provides important context on Zearn Math’s implementation and usage patterns. Schools received beginning-of-year training, biweekly coaching for instructional leads, and classroom-level incentives to promote engagement. In the second year, usage increased substantially. By the final semester, most students met recommended weekly usage goals, and half completed the full sequence of on-grade-level lessons for the year. On the confirmatory STAAR outcomes, students in Zearn schools showed small positive effects, with effect sizes of +0.07 for all students and +0.10 for those who were below proficient at baseline. Although these results were not statistically significant, they may suggest improvement relative to business-as-usual practice.



Exploratory analyses of the MAP adaptive math assessment revealed larger and statistically significant effects, including an effect size of +0.13 for students who began below proficiency. While these results fall outside the preregistered plan and are reported with caution, they suggest that Zearn Math may support broader math learning that extends beyond grade-level benchmarks. Clearly identifying preregistered versus exploratory outcomes helps ground the study’s conclusions in rigor while still surfacing insights that can inform future research. Together, the consistent direction of the findings, strong second-year implementation, and significant exploratory gains suggest that Zearn Math holds promise for helping students, especially those below proficiency, build foundational skills and make progress in math.