Recently, I was reminded of the commonly used slogan “evidence-based policy.” Except for pure marketing purposes, I find this terminology to be a misnomer, a misleading portrayal of academic discourse and the advancement of understanding. While we want to embrace evidence, the evidence seldom speaks for itself; typically, it requires a modeling or conceptual framework for interpretation. Put another way, economists—and everyone else—need two things to draw a conclusion: data, and some way of making sense of the data.
That’s where modeling comes in. Modeling is used not only to aid our basic understanding of phenomena, but also to capture how we view any implied trade-offs for social well-being. The latter plays a pivotal role when our aim is to use evidence in policy design.
This is intuitive if you think about the broad range of ideas and recommendations surrounding macroeconomic policy and the spirited, sometimes acrimonious way in which they’re debated. If everything were truly evidence based, to the extent we can agree on the accuracy of the evidence, why would there be such heterogeneity of opinion? The disagreement stems from the fact that people are using different models or conceptual frameworks, each with its own policy implications. Each of them might be guided by evidence, but policy conclusions can rarely be drawn directly from the evidence itself.
The interplay between theory and evidence has long been discussed by prominent scholars in economics and other disciplines, including some at the University of Chicago. My colleague Stephen Stigler reminded me of a quote of Alfred Marshall’s from 1885 about the potentially important impact of the choice of evidence to report:
The most reckless and treacherous of all theorists is he who professes to let facts and figures speak for themselves, who keeps in the background the part he has played, perhaps unconsciously, in selecting and grouping them.
This concern has not been erased by our current data-rich environment.
Others have weighed in on how to give policy-relevant interpretations to evidence. Back in 1947, Tjalling Koopmans, a prominent member of the Cowles Commission (an economic-research organization then headquartered at the University of Chicago, and now housed at Yale), wrote an essay called “Measurement without Theory,” exposing the limitations of well-known evidence on business cycles. This same theme was revisited later by other scholars affiliated with the Cowles Commission, namely Jacob Marschak and Leo Hurwicz, and then again in an acclaimed paper by my current and longtime colleague Bob Lucas written in 1976. Of course, the generation and construction of new data adds much richness to economic analyses. For many important economic questions, however, empiricism by itself is of limited value.
For a recent exchange illustrating divergence in opinions given evidence, consider the disparate viewpoints of two excellent economic historians, both working at the same institution: Northwestern’s Joel Mokyr and Robert Gordon. Here’s Mokyr on why we should be optimistic about the long-term prospects for innovation:
There are a myriad of reasons why the future should bring more technological progress than ever before—perhaps the most important being that technological innovation itself creates questions and problems that need to be fixed through further technological progress.
And here’s Gordon, with a markedly less rosy analysis:
. . . the rise and fall of growth are inevitable when we recognize that progress occurs more rapidly in some time periods than others. . . . The 1870–1970 century was unique: Many of these inventions could only happen once, and others reached natural limits.
Gordon warns us that we can’t expect technological progress to keep up with the pace set in the previous century, whereas Mokyr says, to paraphrase, “That century was special, but other special things are likely to happen in the future in ways we can’t fully articulate right now. There’s no reason to be pessimistic about technological progress going forward.” These are two astute scholars relying upon the same historical evidence, yet they’ve drawn different conclusions. Why? The evidence alone does not answer the question they are addressing, and they’re using different subjective inputs to help in extrapolating from the evidence. (For more from Gordon and Mokyr on innovation, watch “Can innovation save the US economy?” part of The Big Question video series.)