However, there’s a crisis brewing. The quality metrics that are being calculated are inaccurate. And as a consequence, some good hospitals are being penalized, and some bad hospitals are being rewarded. Patients are also being directed, in some cases, to providers that are not giving the best care.
Why are the metrics inaccurate? There are two primary reasons. One has to do with data integrity, and especially with internal controls. We lack national standards for how hospitals should control, internally, the collection and maintenance of data. For example, on the death of a patient, some hospitals make sure that conditions such as obesity and diabetes are properly coded, while others don’t. Those that do have seen increases in their mortality quality metric without having necessarily made an actual quality improvement. Differences in internal controls can make hospitals look better or worse than they really are.
There are also data-integrity problems with how quality metrics are externally reported. Some of my collaborators at Rush University Medical Center just published an article demonstrating that the patient-safety indices calculated by U.S. News & World Report have, in the past, suffered from a data-integrity issue.1U.S. News & World Report has actually changed its methodology for how it calculates these scores, and in 2016 there have been some reversals, with formerly high-ranking hospitals now near the bottom along some metrics. And of course, clinically, nothing has changed.
The second major problem with quality metrics involves methodology. A couple of years ago the Institute of Medicine published a report saying that hospitals should begin to collect data on social and behavioral determinants of health. It’s been well understood that these determinants actually have significant impact on readmission rates, mortality, and other things. But they aren’t being accurately captured right now, and so it is believed that safety-net hospitals in particular are being unfairly penalized.
Similarly, ProPublica has begun ranking surgeons across the nation, and the methodology they’re using for risk adjustment, while sophisticated, was not properly vetted in the medical community before their rankings were released.2 Nonetheless, their ranking affects where the public gets its care.
As a result of all this, my second piece of advice to the president is that the government should set up for health care something similar to the Financial Accounting Standards Board.3 We need a nonprofit, independent authority to serve two basic roles. One is setting standards for the internal controls of hospitals around data collection and data integrity, as well as standards for external reporting of quality metrics, much as the National Quality Forum does today. Its second role should be as an auditor. It should be able to go into hospitals and verify that the standards established for internal controls are in fact being followed. In addition, any agency, governmental or private, that’s involved in rating hospitals should be subject to audit to make sure it’s following proper methodological standards and being transparent.
What I’m describing is not a small job. But taking these steps would allow us to build on the strides we’ve already made toward better quality and patient protection in heath care, and it would allow hospitals to compete with one another on a level playing field.
1 Bala Hota, Thomas A. Webb, Brian D. Stein, Richa Gupta, David Ansell, and Omar Lateef, "Consumer Rankings and Health Care: Toward Validation and Transparency," The Joint Commission Journal on Quality and Patient Safety, October 2016.
2 Mark W. Friedberg, Peter J. Pronovost, David M. Shahian, Dana Gelb Safran, Karl Y. Bilimoria, Marc N. Elliott, Cheryl L. Damberg, Justin B. Dimick, Alan M. Zaslavsky, "A Methodological Critique of the ProPublica Surgeon Scorecard," RAND Corporation, 2015.
3 This idea builds on views expressed by David M. Shahian, Elizabeth A. Mort, and Peter J. Pronovost in “The Quality Measurement Crisis: An Urgent Need for Methodological Standards and Transparency,” The Joint Commission Journal on Quality and Patient Safety, October 2016.