The danger of making policy based on assumption

Economic reform tends to be undertaken with no economic knowledge

George J. Stigler | Jun 11, 1964

Sections Public Policy

The long-running Selected Papers series features notable work by University of Chicago faculty and other business leaders. This essay is an edited excerpt; the original was based on a talk given in 1964 to alumni of Chicago Booth.

We Americans are a well-meaning people. We are unanimously in favor of a healthy population, also fully employed, well housed, and deeply educated. To a man, we wish prosperous and peaceful nations in the rest of the world, and possibly we are even more anxious that they be prosperous than that they be peaceable. We ooze benevolence, and practice much charity, and could easily become smug in our self-conscious virtue.

The denunciation of American complacency, however, is not my purpose, at least not my explicit purpose. I admire the humane and generous sympathies of our society—sympathies that extend to the uneducated and the uncultured and the unenterprising and even the immoral as well as to the educated and the cultured and the enterprising and the moral. We are a people remarkably agreed on our basic goals, and they are goals which are thoroughly admirable even to one, like myself, who thinks one or two less fashionable goals deserve equal popularity.

Fortunately, our agreement on basic goals does not preclude disagreement on the way best to approach these goals. If the right economic policies were so obvious as to defy responsible criticism, this would be an intolerably dull world. In fact, I believe that each generation has an inescapable obligation to leave difficult problems for the next generation to solve—not only to spare that next generation boredom but also to give it an opportunity for greatness. The legacy of unsolved problems which my generation is bequeathing to the next generation, I may say, seems adequate and even sumptuous.

Goals and methods

It is not wholly correct to say that we are agreed upon what we want but are not agreed upon how to achieve it. When we get to specific goals, we shall find that our agreement does not always extend to orders of importance. For example, some people are willing to preserve personal freedom of choice for consumers even if the choice is exercised very unwisely in some cases, and others will be more concerned with (say) the health of consumers which these unwise choices may impair. Nevertheless, it is roughly true that we know where to go.

We do not know how to get there. This is my fundamental thesis: we do not know how to achieve a given end. We do not know the relationship between the public policies we adopt and the effects these policies were designed to achieve.

This surely sounds absurd: I am saying that although we have had a Securities and Exchange Commission for 30 years, we do not know how to improve the securities markets. I am saying that we have regulated the railroads 77 years and do not know how to achieve a sensible railroad rate structure. I am saying that no one knows whether a fair employment practices act will serve to reduce the discrimination against minorities in the labor markets. We can get on a bus labeled Economic Reform, but we don’t know where it will take us.

You will find it hard to assimilate immediately a challenge to a belief that is so deeply implanted in you that it is simply self-evident. I am reminded of the equally formidable task undertaken in 1819 by a young English clergyman named Richard Whately. In a pamphlet with the title Historic Doubts Relative to Napoleon Buonaparte, he argued that the evidence that Napoleon had ever existed was very unsatisfactory and inconclusive. He recognized, as I have just done, the difficulty of getting men to rethink an undisputed position.

But is it in fact found that undisputed points are always such as have been the most carefully examined as to the evidence on which they rest? That facts or principles which are taken for granted, without controversy, as the common basis of opposite opinions, are always themselves established on sufficient grounds? On the contrary, is not any such fundamental point, from the very circumstance of its being taken for granted at once, and the attention drawn off to some other question, likely to be admitted on insufficient evidence, and the flaws in that evidence overlooked?

Experience will teach us that such instances often occur: witness the well-known anecdote of the Royal Society; to whom King Charles II proposed as a question, whence it is that a vessel of water receives no addition of weight from a live fish being put into it, though it does, if the fish be dead. Various solutions, of great ingenuity, were proposed, discussed, objected to, and defended; nor was it till they had been long bewildered in the inquiry, that it occurred to them to try the experiment; by which they at once ascertained that the phenomenon which they were striving to account for . . . had no existence but in the invention of the witty monarch.

Whately’s case against Napoleon’s existence rested chiefly upon the utter improbability of the man’s career. As just one instance,

Another peculiar circumstance in the history of this extraordinary personage is, that when it is found convenient to represent him as defeated, though he is by no means defeated by halves, but involved in much more sudden and total ruin than the personages of real history usually meet with; yet, if it is thought fit he should be restored, it is done as quickly and completely as if Merlin’s rod had been employed. He enters Russia with a prodigious army, which is totally ruined by an unprecedented hard winter; (everything relating to this man is prodigious and unprecedented;) yet in a few months we find him intrusted with another great army in Germany, which is also totally ruined at Leipsic; making, inclusive of the Egyptian, the third great army thus totally lost: yet the French are so good-natured as to furnish him with another sufficient to make a formidable stand in France; he is, however, conquered, and presented with the sovereignty of Elba; (surely, by the bye, some more probable way might have been found of disposing of him, till again wanted, than to place him thus on the very verge of his ancient dominions;) thence he returns to France, where he is received with open arms, and enabled to lose a fifth great army at Waterloo; yet so eager were these people to be a sixth time led to destruction, that it was found necessary to confine him in an island some thousand miles off, and to quarter foreign troops upon them, lest they should make an insurrection in his favour? Does any one believe all this, and yet refuse to believe a miracle?

Whately was a young divine when he wrote this piece, which I interpret to assert that the evidence a typical Englishman possessed for Napoleon’s existence was no better than the evidence he possessed for Biblical miracles.

I am jealous of Whately. He was arguing for miracles, which everyone wants to believe in, and in fact, everyone wishes to benefit from miracles. Whately soon became an archbishop. I, on the contrary, am compelled to argue against miracles, for I assert that passing a law does not solve a problem. I shall be lucky if I am not fined for loitering on the highway of progress. But on with the task.

The credulity of reformers

I doubt that I can use Whately’s approach. One could indeed marvel at the credulity of reformers. In 1887, the railroads of this nation exceeded 180,000 miles, many times the length of the highways of the Roman Empire. The railroad lines and equipment had a value perhaps in the $10 billions, or more than twice the expenditures of both sides on the Civil War. The railroads employed 700,000 men—itself the largest industrial army that history had ever seen. This stupendously vast empire was ruled by a set of entrepreneurs of great ability and utter determination. To establish an equitable rate structure, to govern this empire in the most minute detail, the Congress in its wisdom created the Interstate Commerce Commission. A committee of five men, aided by a staff of 61 and abetted by an appropriation of $149,000 (in 1889 dollars), was to assume direction of the industry. Could anyone believe that this committee would change much the structure of rates, and not believe in miracles? But since you believe in miracles, I must part company with Whately.

If earlier experts could know that a dead fish weighs rather more than a live one, modern experts surely could know that a commission weights down electrical rates.

When we undertake a policy reform or improve some part of the economy, there is one way, and only one way, to find out whether we have succeeded: to look and see. Now, only a naive person will believe that historical evidence is unambiguous. Some years ago, a young man sued Columbia University, at which I was then professing, for a considerable sum of money because it had failed to teach him wisdom. The fact that he brought the suit was conclusive evidence of Columbia’s failure. Nevertheless, I agree with this befuddled ex-student that colleges should impart wisdom if they possibly can. I challenge anyone in the whole wide world, however, to prove that on average colleges have taught wisdom, or that on average they haven’t. The burden of proof is too heavy for anyone to lift.

Still, it is easy to exaggerate the ambiguity of historical experience; after all, the past is the only source of knowledge of the future. Our trouble, frankly, is less that history speaks obscurely than that we have listened carelessly. We have not studied the experience of economic reform, and know not its successes nor its failures, its lessons on ways to proceed and ways to avoid.

Recording the outcome

And of course, the past is instructive only if we study it. Suppose you are ill and I give you a medicine, chosen at random. You will probably survive, and since most medicines are not very potent, even get well. This is not too different from what medical research must be like, for all research involves the liberal use of trial and error. What turns this nearsighted groping into large progress is the recording of the outcome, so that recoveries due only to chance are separated from those due to the beneficial effects of a particular medicine. In a world without memory, there would be not progress but an endless succession of random moves, lacking any cumulative improvement.

So the results of experiment should be determined, and compiled. This may be Platitude No. 1 to the scientific investigator, but it is no platitude in the formulation of economic policy. In political life, it is an idea of considerable novelty, and there are those who would call it un-American except that it is also un-British and un-Russian and un-Indonesian.

Although we have studied the experience under some of our economic policies, the number and importance of those we have not studied are simply astounding. Let me give just three examples that will, I hope, suggest the problem we face in devising good policies for economic reform.

Regulation of electrical rates

My first example is the regulation of rates for electricity, an area in which modern experimentation began in 1907 in New York and Wisconsin, and for which two-thirds of the states created special public-service commissions as long ago as 1915. Yet when, in 1963, Claire Friedland and I began a study of the impact of these regulatory commissions on the level and structure of rates, we were the first investigators ever to do so on even a moderately comprehensive scale.

It was the implicit verdict of the many economists and political scientists who had studied the regulation of electrical rates during the last half century that a study of the effects of regulation was unnecessary. The bounteous literature implicitly asserts that the influence of the commissions on rates was obvious. The experts knew that of course regulatory bodies are not always competent or honest, but even so, the experts were confident that on average the commissions hold down the prices below what the electrical companies would be able to charge because of their monopoly position in each community. If earlier experts could know that a dead fish weighs rather more than a live one, modern experts surely could know that a commission weights down electrical rates. But our study of the effects of regulation on rates came to the conclusion that the effects of regulation are apparently too small to be detected.

The SEC and investors

A more-recent economic reform was the creation of the Securities and Exchange Commission, some 30 years ago, to protect investors from the flamboyant falsehoods that on occasion appeared in the prospectuses that preceded new stock issues. The prospectuses that are now issued after much delay and very substantial expenditures have substituted grim statistics for the enticing loveliness of a seed catalog. To what end?

Again, my main point is that no one had studied the effects of this elaborate machinery on the fortunes of the buyers of new stock issues until I undertook to do so last year. Neither the security markets nor their regulators nor the academic economists have deemed it necessary to measure the undoubted beneficial effects of three decades of regulation.

Perhaps a word on how one measures the effects of regulation may be useful, for it is no simple task to disentangle one of many influences on the course of events. The SEC study illustrates one approach. Here I hypothetically bought every substantial new issue of industrial common stocks from 1923 to 1928, a period before the SEC, and from 1948 to 1955. The value of the stock in each of the five years following its issues was also ascertained. We can now calculate what happens to our new investment over time. There remains the problem of allowing for the considerable changes in this world between the reigns of Calvin Coolidge and Dwight Eisenhower. The differential effect of the SEC is measured by comparing values of these new investments with the outcome of buying established securities, over which the SEC has no significant control.

The main finding was that there was no important difference between the 1920s and the 1950s! I may add that it was fortunate that the purchases of new stocks were hypothetical: the investor in new issues of common stock lost 20 percent of his shirt after one year in both periods.

The Federal Reserve System

My last instance is the effect of the Federal Reserve System on the stability of the American economy. This system of central banking was created 50 years ago and has controlled our money system ever since. Here economists have made studies of shorter episodes in the history of the system; it is widely accepted, for example, that the restrictive monetary policy of 1931–32 contributed greatly to the financial collapse of 1933. But my colleague Milton Friedman, collaborating with the National Bureau of Economic Research’s Anna Schwartz, has recently published the first full-dress study of the effects of the Federal Reserve System upon the stability of prices and banking institutions throughout its history.

By now you may feel able to predict the results: that the Federal Reserve System has had no effect on monetary stability. But no—this time there was an effect:

The stock of money shows larger fluctuations after 1914 than before 1914 and this is true even if the large wartime increases in the stock of money are excluded. The blind, undesigned, and quasi-automatic working of the gold standard turned out to produce a greater measure of predictability and regularity . . . than did deliberate and conscious control exercised within institutional arrangements intended to promote monetary stability.

Many economists and all bankers will challenge Friedman and Schwartz’s conclusions—in fact, a fair number will challenge them even before they learn what Friedman and Schwartz have written. But no one will be able in good conscience to say that the study was anticipated or has been contradicted by any other study of comparable scope and thoroughness.

Let me assume, tentatively and hopefully, that you are prepared to acknowledge that the relationship of policies to results is surprisingly obscure. I do not say that our knowledge is nonexistent, because that statement would be distinguishably removed from truth. I do say our knowledge is extremely meager. Why are we so poorly informed on the effective weapons of economic reform?

Scholars and political life

It’s worth briefly exploring the reasons we know so little of the effects of past economic policies, because those reasons tell us something about both scholars and political life. The studies that should have been made are the professional responsibility of economists and political scientists. I have no desire to criticize them. Economists are, by their own admission, learned, resourceful, diligent, and benevolent. Political scientists have accused themselves of similar traits. Why have these scholars failed to study much more intensively the relationship between public policies and the course of events? The main answers, I believe, are as follows.

The best scholars are not the best reformers. A scholar ought to be tolerably open-minded, unemotional, and rational. A reformer must promise paradise if his reform is adopted: a candid and qualified estimate of the effects of a given public policy would never arouse a majority from inertia. A reformer should have a low threshold of emotion: I am reminded of Samuel Plimsoll, of the ship line, whose sole stock in trade as a reformer, the London Times reported, was an unrivaled capacity for becoming fervidly indignant upon hearsay evidence. It follows that reformers care little for meticulous scholars—and use only those parts of the scholars’ work that fit their needs, very much the way theatrical advertisements present selected adjectives from the reviews. The scholars are normally contemptuous of the reformers, whose scholarly attainments are indeed usually amateur. Reform and research seldom march arm in arm.

The economists have, until recently, been preoccupied with the workings of a comparatively unregulated economic system, what is loosely described as laissez-faire. They have seldom been in the forefront of economic reform—the two great exceptions being their advocacy of free international trade and policies designed to stabilize aggregate economic activity. They have had a marked preference for free-market organization of economic life.

The reformers, on the contrary, have seldom conceived of any method of achieving a given result except by giving explicit directions to individuals to act in the desired way. When a reform is not achieved by a given regulatory body, the reformers know no other solution than to give this or some other regulatory body more power and more instructions.

Reformers’ edge

Economic reformers, moreover, have had one wondrous advantage for a century or more: the economy was improving in its performance in most ways, so most policies could claim success even if economic progress was quite unrelated to the reform. Some policies were designed to reduce poverty, but the Western economies were all becoming richer, and poverty was diminishing as a result of economic growth. Other policies were designed to improve foods and homes, but technology was also striding forward here. Still other policies were designed to improve markets, but the advance of transportation and communication was also improving markets. It is as if the college dining room were to claim sole credit for the fact that seniors weigh more than freshmen.

If close study of the effects of previous reforms had been demanded by our political conscience, it would have been supplied in the past. There is an economic law, named after [19th century French economist] J. B. Say, to the effect that every offer of goods for sale is an implicit demand for the goods that will be received in exchange. Similarly, there is a Say’s law of scholarship: professors will study any problem that the society really believes in need of study. Our society has not believed that a close study of the process of economic reform is essential to devise effective reforms.

Our long-term prospects for rational reform will be much improved as soon as our young people recognize the complexity of the problem. There is an absurd notion abroad that we mostly understand how our economy works and that a democracy—or, for that matter, a dictatorship—knows how to utilize the accumulated knowledge of the social sciences in legislation and administration. On the contrary, we are far from understanding either our economy or the ways in which to improve it, and the room for creative work in the social sciences is immense. If Mr. Nobel had been a wiser man, he would have directed his prizes to the social sciences to dramatize that really difficult goal of man: the achievement of a civilized society.

George J. Stigler was Charles R. Walgreen Distinguished Service Professor of American Institutions at Chicago Booth. He died in 1991.