More Research Is Needed Before Hospital Quality Metrics Are Implemented

Updated:May 19,2014
Disclosure: Dr. Messe is a member of the Get With the Guidelines Stroke Steering Committee. He has no relevant relationships with industry to disclose. Dr. Mullen has no no relevant relationships with industry to disclose.
Pub Date: Thursday, Jan. 23, 2014
Author: Steven R. Messe, MD and Michael T. Mullen, MD, MSCE
Affiliation: Hospital of the University of Pennsylvania

Citation

Katzan IL, et al; on behalf of the American Heart Association Stroke Council, Council on Quality of Care and Outcomes Research, Council on Cardiovascular and Stroke Nursing, Council on Cardiovascular Radiology and Intervention, Council on Cardiovascular Surgery and Anesthesia, and Council on Clinical Cardiology. Risk adjustment of ischemic stroke outcomes for comparing hospital performance: a statement for healthcare professionals from the American Heart Association/American Stroke Association [published online ahead of print January 23, 2014]. Stroke. doi: 10.1161/01.str.0000441948.35804.77.
http://stroke.ahajournals.org/lookup/doi/10.1161/01.str.0000441

Article Text

The American Heart Association/American Stroke Association have developed a Scientific Statement addressing the challenges of measuring and risk-adjusting ischemic stroke outcomes in order to compare quality of care across US hospitals.1  At first glance this may not appear to be a compelling issue  for practicing clinicians; however, quality and outcome data obtained from administrative claims are likely to become increasingly consequential as public reporting of hospital-level quality metrics and pay-for-performance initiatives are developed.  The discussion and recommendations in this Statement are therefore of tremendous importance to patients, providers, payers, and health policy-makers.
   
Stroke is highly prevalent and is a leading cause of mortality and adult onset disability.2  As a result, care for patients with stroke accounts for a meaningful portion of expenditures by the Centers for Medicare and Medicaid Services (CMS).3,4   With the untenable growth in health care costs there have been concerted efforts by government and private payers to tie reimbursement to the quality of care received.5  Pay-for-performance programs provide a financial incentive for hospitals to deliver higher quality care.  These programs are predicated upon an ability to accurately assess clinical outcomes. The CMS Hospital Inpatient Quality Reporting Program has proposed two stroke outcome measures to be utilized beginning in 2016:  The 30-day risk-standardized stroke readmission rate and the 30-day risk-standardized stroke mortality rate.  On the surface, these outcomes appear to be reasonable choices with high face validity.  Both are objective and readily measured in existing administrative claims databases.  Death has an obvious and profound impact on the patient and his/her family.  Hospital readmission adds dramatically to healthcare cost and may be avoidable, at least in some instances.6,7  Unfortunately, as outlined in the scientific statement, there are major methodological challenges which limit the utility of these outcome measures. 
 
It is impossible to fairly compare outcomes across hospitals without adequately adjusting for differences in case mix and disease severity.  Risk adjustment is necessary for all disease states, but in stroke there are unique challenges that limit the effectiveness of claims-based outcome measures.  The authors of the Scientific Statement performed a literature review to determine factors associated with outcome from stroke and identified age, gender, comorbid conditions, vascular risk factors, and pre-stroke function as significant predictors.  However, overwhelming each of these factors in determining outcome is the severity of the stroke symptoms at presentation to the hospital.8  It is intuitive that your final stroke outcome is strongly and directly linked to how severe it was at onset.   Measures of stroke severity are not captured in administrative databases.  Risk-adjustment models created by CMS, which do not account for stroke severity, have only modest discrimination (c-statistics 0.71 for 30-day mortality and 0.59 for 30-day readmission).9,10  For 30-day mortality, a claims-based model that does not include a measure of stroke severity will misclassify a large proportion of hospitals.11 The most commonly used measure to quantify stroke severity is the National Institutes of Health Stroke Scale (NIHSS) score.  In the AHA/ASA Get With The Guidelines – Stroke database, where hospitals have by their inclusion already demonstrated an abiding interest in improving outcomes from stroke, the NIHSS was not documented in over 40% of patients.12 In clinical practice, the availability of stroke severity measures is sure to be far less.  As a result, adding these data to existing administrative datasets is not likely, at least in the short term. 

Public reporting of hospital quality and pay for performance should lead to more informed healthcare consumers and improved quality of care.13 However, there are potentially serious unintended consequences to these initiatives if risk adjustment is inadequate, particularly if there are systematic differences in disease severity across hospitals.  This is likely to be the case for stroke, since EMS policy in many states preferentially routes the most acutely ill patients to specialized stroke centers.14  Developing primary and comprehensive stroke centers is a labor- and resource-intensive endeavor.  Inadequate risk adjustment could result in financial penalties for these hospitals, which could then result in hospitals avoiding the sickest stroke patients or electing not to pursue stroke center status.  Similarly, referral centers and primary/comprehensive stroke centers could be less inclined to accept transfers of patients with a devastating stroke.  These negative incentives could result in exacerbation in disparities for vulnerable populations, and the rapidly developing stroke systems of care would be undermined.   Finally, basing these programs on mortality alone, without accounting for the functional status of survivors, could reduce the use of palliative care in the most severely injured patients.  Given the extremely low quality of life associated with severe disability and the high costs of long- term care this would be harmful to the patient and to society.15,16 As the statement points out, functional status is not uniformly captured in administrative, or even clinical, databases and there is currently no way to incorporate a measure of functional status into hospital-level quality metrics using claims. 

There has been an impressive and heartening trend in reducing stroke mortality over the past 4 decades.17   While much of this is attributable to more aggressive management of vascular risk factors, there is likely a meaningful contribution from the development of specialized stroke centers.  Stroke centers have lower mortality and utilize more acute stroke therapies, which reduce disability after stroke.18-21  Appropriate and accurate measurement of hospital-level outcomes may allow for further optimization of stroke care systems by directing patients to the hospitals best equipped to care for them.  However, public reporting and pay-for- performance programs should not be initiated until there is a better understanding of hospital-level outcomes and risk standardization models have been more carefully validated as the potential for negative consequences is too great.    As the AHA Scientific Statement concludes, there is an urgent need for additional research in this area.


References

  1. Katzan IL, et al; on behalf of the American Heart Association Stroke Council, Council on Quality of Care and Outcomes Research, Council on Cardiovascular and Stroke Nursing, Council on Cardiovascular Radiology and Intervention, Council on Cardiovascular Surgery and Anesthesia, and Council on Clinical Cardiology. Risk adjustment of ischemic stroke outcomes for comparing hospital performance: a statement for healthcare professionals from the American Heart Association/American Stroke Association [published online ahead of print January 23, 2014]. Stroke. doi: 10.1161/01.str.0000441948.35804.77.
    http://stroke.ahajournals.org/lookup/doi/10.1161/01.str.0000441948.35804.77
  2. Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Blaha MJ, Dai S, Ford ES, Fox CS, Franco S, Fullerton HJ, Gillespie C, Hailpern SM, Heit JA, Howard VJ, Huffman MD, Judd SE, Kissela BM, Kittner SJ, Lackland DT, Lichtman JH, Lisabeth LD, Mackey RH, Magid DJ, Marcus GM, Marelli A, Matchar DB, McGuire DK, Mohler ER, 3rd, Moy CS, Mussolino ME, Neumar RW, Nichol G, Pandey DK, Paynter NP, Reeves MJ, Sorlie PD, Stein J, Towfighi A, Turan TN, Virani SS, Wong ND, Woo D, Turner MB. Heart Disease and Stroke Statistics--2014 Update: A Report From the American Heart Association. Circulation. 2014;129:e28-e292.
  3. The Agency for Healthcare Research and Quality. Total Expenses and Percent Distribution for Selected Conditions by Type of Service: United States, 2011. Available at: http://meps.ahrq.gov/mepsweb/data_stats/tables_compendia_hh_interactive.jsp., 2013.
  4. Centers for Disease Control and Prevention (CDC). Prevalence and most common causes of disability among adults: United States, 2005. Morbidity and Mortality Weekly Report. 2009;58:421-426.
  5. Institute of Medicine (IOM). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC 2001.
  6. Medicare Payment Advisory Commission (MedPAC). Report to the Congress: Promoting Greater Efficiency in Medicare. Available at: http://www.medpac.gov/documents/jun07_entirereport.pdf.
  7. van Walraven C, Bennett C, Jennings A, Austin PC, Forster AJ. Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ. 2011;183(7):E391-402.
  8. Fonarow GC, Saver JL, Smith EE, Broderick JP, Kleindorfer DO, Sacco RL, Pan W, Olson DM, Hernandez AF, Peterson ED, Schwamm LH. Relationship of national institutes of health stroke scale to 30-day mortality in medicare beneficiaries with acute ischemic stroke. J Am Heart Assoc. 2012;1(1):42-50.
  9. Bernheim S, Wang C, Wang Y, Bhat K, Savage S, Lichtman J, Phipps MS, Drye EE, Krumholz HM. Measure Methodology Report: Hospital 30-Day Readmission Following Acute Ischemic Stroke Hospitalization Measure. Centers for Medicare & Medicaid Services. Available at: https://www.qualitynet.org/dcs/ContentServer?cid=1228773364875&pagename=QnetPublic%2FPage%2FQnetTier4&c=Page.  Accessed January 20th, 2014.
  10. Bernheim S, Wang C, Wang Y, Bhat K, Savage S, Lichtman J, Phipps MS, Drye EE, Krumholz HM. Measure Methodology Report: Hosptial 30-day Mortality Following Acute Ischemic Stroke Hospitalization Measure. Centers for Medicare & Medicaid Services. Available at: https://www.qualitynet.org/dcs/ContentServer?cid=1228773364875&pagename=QnetPublic%2FPage%2FQnetTier4&c=Page.  Accessed January 20th, 2014.
  11. Fonarow GC, Pan W, Saver JL, Smith EE, Reeves MJ, Broderick JP, Kleindorfer DO, Sacco RL, Olson DM, Hernandez AF, Peterson ED, Schwamm LH. Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity. JAMA : the journal of the American Medical Association. 2012;308(3):257-264.
  12. Tong D, Reeves MJ, Hernandez AF, Zhao X, Olson DM, Fonarow GC, Schwamm LH, Smith EE. Times from symptom onset to hospital arrival in the Get with the Guidelines--Stroke Program 2002 to 2009: temporal trends and implications. Stroke. 2012;43(7):1912-1917.
  13. Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, Bratzler DW. Public reporting and pay for performance in hospital quality improvement. N Engl J Med. 2007;356(5):486-496.
  14. Song S, Saver J. Growth of Regional Acute Stroke Systems of Care in the United States in the First Decade of the 21st Century. Stroke. 2012.
  15. Post PN, Stiggelbout AM, Wakker PP. The utility of health states after stroke: a systematic review of the literature. Stroke. 2001;32(6):1425-1429. 16. Gemworth 2013 Cost of Care Survey.
  16. Gemworth Life Insurance Company. Available at: https://www.genworth.com/dam/Americas/US/PDFs/Consumer/corporate/130568_032213_Cost%20of%20Care_Final_nonsecure.pdf.  Accessed January 18th, 2014, 2014.
  17. Lackland DT, Roccella EJ, Deutsch AF, Fornage M, George MG, Howard G, Kissela BM, Kittner SJ, Lichtman JH, Lisabeth LD, Schwamm LH, Smith EE, Towfighi A. Factors influencing the decline in stroke mortality: a statement from the american heart association/american stroke association. Stroke. 2013;45(1):315-353.
  18. Organised inpatient (stroke unit) care for stroke. Stroke Unit Trialists' Collaboration. Cochrane Database Syst Rev. 2000 (2):CD000197.
  19. Mullen MT, Kasner SE, Kallan MJ, Kleindorfer DO, Albright KC, Carr BG. Joint commission primary stroke centers utilize more rt-PA in the nationwide inpatient sample. J Am Heart Assoc. 2013;2(2):e000071.
  20. Xian Y, Holloway RG, Chan PS, Noyes K, Shah MN, Ting HH, Chappel AR, Peterson ED, Friedman B. Association between stroke center hospitalization for acute ischemic stroke and mortality. JAMA : the journal of the American Medical Association. 2011;305 (4):373-380.
  21. Lees KR, Bluhmki E, von Kummer R, Brott TG, Toni D, Grotta JC, Albers GW, Kaste M, Marler JR, Hamilton SA, Tilley BC, Davis SM, Donnan GA, Hacke W, Allen K, Mau J, Meier D, del Zoppo G, De Silva DA, Butcher KS, Parsons MW, Barber PA, Levi C, Bladin C, Byrnes G. Time to treatment with intravenous alteplase and outcome in stroke: an updated pooled analysis of ECASS, ATLANTIS, NINDS, and EPITHET trials. Lancet. 2010;375(9727):1695-1703.

-- The opinions expressed in this commentary are not necessarily those of the editors or of the American Heart Association -- 
 

AHA Scientific Journals

AHA Scientific Journals


Connect with AHA Science News

Follow AHAScience on Twitter (opens in new window)
Like AHA Science News on Facebook (opens in new window)

AHA Science News on YouTube



Subscribe to AHA's Science News Channel (opens in new window)
on YouTube for exclusive, late-breaking coverage of over 200 video interviews, panel discussions, welcome messages and more.