The COVID-19 era has been an extraordinary time to be a policy maker. The pandemic has created a decision-making environment that is tremendously difficult to navigate—just look at the vast multitude of policies implemented by the globe’s various mayors, governors, members of parliament, presidents, and other local and national leaders. Faced with the same information, or lack thereof, these policy makers have come to many different, sometimes contrasting, conclusions about how to confront the virus and its various effects on the health and well-being of their constituents.
Should businesses remain open or be forced to close? How about schools? Should borders be closed to international travelers? Should people be required to wear masks outside their homes? Under what circumstances should they be allowed to leave home at all? Policy questions such as these have been occupying politicians and other leaders for the past year.
These issues have been difficult for a variety of reasons. One is the simple dearth of comparable experiences to draw upon for guidance. This certainly applies to the economic crisis induced by the pandemic, but also to the epidemiological side of it: even though we’ve had plenty of experience with various diseases, we’ve had to discover a number of specific details about this virus—how it’s most likely to spread, how contagious asymptomatic people really are, how long people remain contagious after developing symptoms—in a hurry.
This time pressure is another complicating factor for pandemic decisions, as policy makers have felt compelled to make quick judgements in hopes of stemming the spread of the virus before its growth becomes out of control. And they must make these decisions in complex political environments that are, in many cases, characterized by polarization and a lack of trust.
In such a context, there is no way for policy makers to guarantee they’re making the “right” decision in every case. But as someone who has devoted a great deal of study to uncertainty and how to cope with it, I believe there is a way to guarantee they’re making prudent, consistent, transparent, and logically defensible decisions. In a recently published paper, a group of coauthors and I argue that by using what decision theorists call decision rules, policy makers can establish a sound decision-making process that includes a framework for examining and evaluating their policy options. Policy makers at every level and of every political philosophy can use such rules to discipline their decision-making, even if they use them informally.
An irony of policy making in the COVID-19 era is that despite a relative lack of data or comparable experiences for many of the decisions they have to make, leaders are still confronted with many different inputs to their decision processes, which they must try to balance. One type of input is expert opinion, often from experts in different disciplines—epidemiology but also economics and other social sciences, for example—and often not in full agreement. Another type of input is the competing predictions of various quantitative models, which, like the experts, reflect their own modeling assumptions and interpretations of the data that we do have.
Models are stylized renderings of reality that can help illuminate the channels by which alternative policies could alter socially relevant outcomes, and quantify the potential impact of these policies. They give policy makers a way to project the impacts of alternative policies, such as how a broad closure of businesses might affect viral spread, unemployment, and various other outcomes. By design, models are abstract and incomplete. Every model is, therefore, wrong to some degree.
Decision rules are meant to add clarity to the decision-making process and to avoid having the public speculate on future courses of action.
Different models take different inputs and make different assumptions. They also can be used in different ways. They can provide best-guess estimates for the outcomes of a given policy, or they can elucidate the possible bad outcomes that might transpire. While both types of predictions are informative, it is important to distinguish the nature of the prediction.
Thus, quantitative models can be extremely useful for decision-making, even in cases of tremendous uncertainty, as long as they’re used properly—and that often means considering more than one for making assessments. There is the danger in policy contexts that leaders will find a single model that supports their preferred course of action. The model may thus give them unwarranted confidence in a particular policy path. Thomas Aquinas delivered the famous injunction to “beware the man of one book.” I like to issue the same warning about the policy maker of one model.
I think of each model as telling a story. Rational policy making requires understanding and acknowledging the limits to the credibility of any one model’s narrative, listening to the stories told by competing models, and using them to form a more complete understanding of the situation. Decision theory and decision rules can help discipline this conversation between models and decision makers.
Decision rules in practice
One way to reconcile the conflicting predictions made by various models would be to assign probabilities to each of them. A state governor could look at four projections and decide that Model A has a 50 percent chance of being accurate, Model B a 30 percent chance, and models C and D 10 percent each. She could then craft a policy that averages these various outcomes using weights commensurate with their probabilities.
The problem with this in the pandemic context is that policy makers will often have neither the information nor the expertise to make this kind of calculation. Assigning probabilities in anything other than an arbitrary way is quite difficult in a situation that offers so little information or precedent. While speculating about such probabilities might be a good starting point, it is wise to approach this with an open mind and to explore how sensitive policy evaluation is to the weights assigned to different models. Decision rules allow leaders to balance their subjective judgments of the probabilities of various outcomes, and to acknowledge the limited information that underlies such judgements and then move forward in a way that incorporates this uncertainty. They can facilitate decisions that, as my coauthors and I put it, “remain valid for a wide range of futures and keep options open.”
Decision rules are meant to add clarity to the decision-making process and to avoid having the public speculate on future courses of action. A decision rule stipulates a course of action as a function of information that will be available at the time of the decision. Under what set of circumstances will particular businesses be allowed to return to normal? These circumstances could include the number of COVID-19 cases or the fraction of the population that has been vaccinated, or whatever other information is deemed to be of value to the decision makers.
There are many possible decision rules, and which one a policy maker uses is a matter of his own preferences and priorities, as well as those of his constituents. Economists like to think in terms of trade-offs. In effect, there is an uncertainty trade-off when using models to assess alternative policies. Policy could embrace the best-guess outcomes predicted by a model or focus on concerns about the possibility of bad outcomes. Decision theory offers a way to characterize this uncertainty trade-off.
In many cases, the pandemic will impose costs on society no matter what policies are chosen, so it’s important for policy makers to be able to express how they came to their decisions.
In our paper, my coauthors and I demonstrate how different decision rules affect rational policy choices with the example of a school-closure decision. In our exercise, the hypothetical policy maker has to decide whether to close schools in order to stem the pandemic, and if so, for how long. She has three models to consider, all of which are based on different assumptions about how easily children may spread the virus. We consider four different decision rules that lead to four different conclusions about how to proceed, ranging from 10- to 20-week closures.
None of these policies are exclusively the “right” one—the policy maker chooses a decision rule that reflects her personal characteristics and those of her constituency, and the decision rule helps guide her toward a policy appropriate for those characteristics.
What the decision rules have in common is that they provide a way to organize the various contrasting opinions and predictions the policy maker must balance. And critically, they provide a transparent way to work through decisions that makes it possible to communicate to constituents how particular choices were made.
Communication is key
Trade-offs are an important concept in economics, and the trade-offs at play in the pandemic—between economic activity and viral transmission, or between fighting the virus and addressing other medical and emotional concerns, or any number of other sets of priorities—are often stark. In many cases, the pandemic will impose costs on society no matter what policies are chosen, so it’s important for policy makers to be able to express how they came to their decisions.
This doesn’t mean that they will or should make the specifics of their decision rules part of their routine communication with the public. But by using decision theory, even informally, to provide structure to their decision-making, they will likely make it easier to explain in an accessible way how they approached the difficult choices they had to make.
One important part of this communication will be acknowledging the role of uncertainty in the decision-making process. There is an urge, prevalent among both politicians and the experts who advise them, to overstate the certainty that underlies any particular course of action. It is driven by a broad perception that the public does not want politicians or policy makers to acknowledge the uncertainty inherent in predictions about the consequences of alternative courses of action. This often leads their advisors to exude excessive confidence in the outcomes of a particular policy when advocating for or against it.
But the public can recognize overconfidence. People see various experts delivering contrasting opinions with the same unflagging certainty and know that one or all must be wrong. They can review a track record of predictions and decisions made with seeming certainty that didn’t pan out. Over time, false certainty undermines trust in both experts and policy makers.
By incorporating uncertainty into their decision-making process—scrutinizing what they don’t know and factoring that into their choice of decision rules or evaluation of models—leaders can more effectively communicate with the public about the role that uncertainty played in determining their choices. It is an important part of being able to articulate, even to themselves, how they arrived at their decisions.
Those decisions may not always lead to the best outcomes as things play out. Decision theory and decision rules are tools to be used in the policy process; they don’t automate decision-making itself. Policy makers must still rely on experts to guide them on how to use models in a reliable way and how much confidence to place in alternative models when their policy implications differ. There is an unavoidable aspect of subjectivity when our understanding is incomplete and models do not have common quantitative predictions. Policy makers must also articulate their goals or preferences, including their aversion to uncertainty.
But in a pandemic context in which there is often a scarcity of data and an abundance of opinions, and in which policy makers are confronted with a variety of models pointing to sometimes diverging projections, decision rules provide a way to treat these various decision-making assets in a transparent and rational way.
Lars Peter Hansen is the David Rockefeller Distinguished Service Professor at the University of Chicago Departments of Economics and Statistics and at Chicago Booth. He was a 2013 recipient of the Nobel Prize in Economic Sciences.