We Knew Fire was Hot

Abstract

Lant Pritchett looks at Ludger Woessman’s body of research to emphasise one particular paper of his, written jointly with Martin West, entitled ‘Class-size effects in school systems around the world: Evidence from between-grade variation in Trends in International Mathematics and Science Study (TIMSS)’. When the paper was written in 2002, it already spoke directly to the likely external validity of estimates of causal impact from “education production functions” relating inputs to learning outcomes. If “what works” is truly heterogeneous across contexts—and this Woessmann paper shows that even for simple inputs like “class size” and even in mostly developed country contexts, it is—then aggregating estimates of “what works” is not going to be of much use in deciding what to do in specific circumstances. But Pitchett’s point is that we already knew that was going to be the case, both empirically and logically. We did not need to spend millions of dollars and hundreds of Randomised Control Trials (RCTs) to know they were not going to add up to a single clear answer about “what works” in any given context. The lack of external validity of the emerging RCT evidence is not a surprise or even a “finding” that we could have only reached by doing the RCTs—we knew the eventual outcome, we just didn’t want to know it. This paper is part of the Research on Improving Systems of Education (RISE) Programme