2016 Revision of the SCAI position statement on public reporting

Citations Over TimeTop 10% of 2016 papers

Abstract

The public reporting of institutional and individual operator results of percutaneous coronary interventions (PCIs) is intended to provide meaningful information to the public and enhance the delivery of superlative health care. By giving consumers specific outcome data 1, patients will be empowered to participate more fully in decisions concerning their medical care. The influence that public reports wield could increase if publicly reported information proves to be an accurate representation of “value” in health care delivery, and if third-party payers use this information to allocate reimbursement in a value-based system 2. Despite these well-intended goals, there is uncertainty whether existing programs correctly identify high- and low-performing PCI centers and operators. Moreover, there is emerging evidence that public reporting can deleteriously influence case selection by encouraging risk avoidance behaviors. Thus, potentially beneficial procedures might be withheld from high-risk patients who can derive the greatest benefit, because operators and facilities fear being labeled as outliers 3-8. This position statement updates the prior Society for Cardiac Angiography and Interventions (SCAI) Policy on Public Reporting 1. SCAI continues to endorse public reporting, provided the reports are not misleading, deliver meaningful information to consumers to help inform their choices, and facilitate quality improvement. Offering the public accurate and understandable metrics, including measures to assess the appropriateness of case selection, are essential to achieve this aim. Mandatory public reporting highlights the dilemma of balancing the provision of necessary revascularization procedures to high-risk patients with the consequences of negative outcomes. Table 1 summarizes the data presented to the public in 4 states with active programs 5-9. Other states (e.g., Pennsylvania [PA], California [CA]) have rudimentary programs. A summary of the experience of the two states with the most established programs follows. New York State's (NY) annual reports contain every hospital's and operator's PCI volume, unadjusted (observed) in-hospital/30-day mortality rate (OMR), expected mortality rate (EMR), risk-adjusted mortality rate (RAMR), and outlier status (significantly higher, lower, or not different from the statewide mortality rate), calculated on a rolling 3-year basis. The OMR is the observed number of deaths divided by the total number of cases, whereas the EMR is the sum of the predicted probabilities of death (as a function of patients' own risk factors) for all patients divided by the total number of patients. The RAMR is then obtained as the OMR/EMR ratio of the provider multiplied by the registry's overall mortality rate. Non-federal hospitals in Massachusetts (MA) performing PCIs annually report Standardized Mortality Incidence Rates (SMIR), using a Bayesian hierarchical modeling approach (see below) analogous to using OMR/EMR. However, in lieu of OMR, the predicted mortality rate is used instead. By employing hierarchical modeling, fewer hospitals are identified as outliers than in NY. SMIR is also reported separately for patients presenting with ST-elevation myocardial infarction (STEMI) or cardiogenic shock. In the future, the state will shift to reporting risk-adjusted 30-day mortality. For public reporting to be effective, the information reported should (1) address the consumer's important questions and priorities; (2) present credible information that is interpretable by the consumer; (3) stimulate the consumer to act on the information; and (4) avoid generating false perceptions regarding individual and/or institutional quality of care. Although public reporting is intended to assist patients in making better choices and support the identification of outliers with poor performance 10, there are few data to confirm this actually occurs 11. The reasons include an apparent lack of consumer interest, difficulty interpreting the presented information, perceived inaccuracies, and a lack of opportunity for patients to act on public reports (e.g., in emergent clinical situations or when care is directed by payers). RAMR after 30 days has been the principal metric presented in public reports. Although patients are understandably concerned about procedural mortality, SCAI believes that the RAMR is insufficient as a summary measure of quality. The primary reason is that mortality after PCI depends more on the clinical acuteness at presentation and comorbidities than on technical proficiency or operator judgment; that is, the variation between providers is much smaller than the variation between patients. Patient severity of illness and comorbidities are highly predictive of mortality after PCI, especially acute myocardial infarction (AMI), cardiogenic shock, and other high-risk settings. The principle underlying risk adjustment is that correcting for confounding factors can mathematically equalize these factors. In theory, RAMR should provide complete correction and be used to identify “outliers” above an arbitrary (e.g., >95th) percentile in order to identify variations in interventional program quality 12. Many interventionists lack confidence that risk adjustment is precise at the high end of risk 5, 7. They have doubts about whether risk adjustment models include some factors frequently used to make clinical decisions (unmeasured confounders). Clinicians are concerned that misinterpretation by the public and regulators will lead to unwarranted damage to professional and institutional reputations. This could result in some excellent operators and hospitals with apparently higher mortality rates being perceived as having poor quality and outcomes because of a high-risk case mix or referrals to their facility 11, 13. Notably, when a complex PCI concludes without mortality, no additional “credit” for high technical competence is assigned to the operator; yet, when a death occurs related to the seriousness of the underlying condition, it is imputed to be a flaw of the operator 14. Each dataset identifies predictive variables based on its own modeling and weights them according to their population, which might not be applicable to all datasets. Their predictive accuracy when applied to a validation subset is frequently omitted 20. Although 1.3% is the median RAMR for PCI nationally, individual facility values above or below the median might not be significantly different. Notably, it has never been demonstrated that programs identified in this manner have demonstrably lower quality compared with other hospitals. Additionally, because RAMR is low for most institutions, it is a weak discriminator of programs and operators. This occurs because the low mortality achieved with PCI requires large numbers of patients to reduce variance and narrow the confidence intervals (CIs). Even when a hospital's OMR greatly exceeds its EMR, it is unclear if it can be appropriately labeled an outlier hospital that delivers poor quality care 12. Using National Cardiovascular Data Registry (NCDR) data, Sherwood et al. 21 found that hospitals performing PCIs in a greater number of high-risk cases did not have a poorer RAMR than facilities treating lower-risk patients. The risk adjustment model used was well calibrated even among very high-risk patients. In fact, hospitals treating the highest risk patients actually had better RAMR than facilities treating patients with a lower severity of illness. However, it is critical to recognize that this study assessed risk through measured variables; unmeasured factors are not assessed by published risk scores and might result in penalization by RAMR statistics. Risk adjustment is performed with logistic regression models or hierarchical multilevel models. The advantages and disadvantages of these methods have been debated 22, 23, but both types of models might be suboptimal discriminators between good and poor performers. Hierarchical modeling is currently the preferred method for both hospital and physician quality comparisons because it accounts for the “clustered” nature of observations. When comparing hospitals, individual procedures within a particular hospital are not truly independent from one another. Rather, they are related because of the characteristics associated with the particular facility. Hierarchical models are given this nomenclature because they account for the fact that the data are organized in tiered fashion: procedures are clustered within patients, patients are clustered around physicians, and physicians are clustered around institutions. Another advantage is that these models can limit widely varying estimates that can occur due to small sample size. Hierarchical models that use “Bayesian” or “empirical Bayesian” methods employ “shrinkage,” in which point estimates for RAMR are moved closer toward the mean. The rationale is to reduce wide fluctuations in RAMR that might occur due to chance, and to mitigate potential type I error due to multiple hypothesis testing. Consequently, Bayesian hierarchical modeling approaches tend to identify fewer statistically significant differences than classical modeling, but it is uncertain whether this correction is an accurate reflection of risk. If an outlier is identified, however, it is more likely that the observation is real; conversely, this type of model makes it very unlikely for a low-volume hospital or provider to be identified as an outlier. Because RAMR using hierarchical models might preferentially identify larger volume facilities as outliers, a “level playing field” might not exist if such statistics are used to compare hospitals with one another. Some models report only point estimates because the 95% CIs are wide (due to low death rates). Point estimates can mislead the public by appearing to delineate differences in RAMR, when in reality; showing statistical significance might be nearly impossible when sample sizes are small. Rank ordering or categorization of provider quality is misleading when based on uncertainly associated RAMR point estimates and should be discouraged because it falsely implies a high level of precision. An important reason why very high-risk patients are not removed from most risk adjustment models is that, if allowed, there would be considerably fewer deaths remaining in the dataset to evaluate. Censoring deaths at the high end of risk could render the models nonpredictive, causing the regression equations to lose stability. It is possible that there might be no remaining statistical differences among programs or operators. For example, the mortality in AMI is nearly 10 times higher than all other patient subgroups, largely driven by poor outcomes in cardiogenic shock. If these cases are excluded, residual mortality becomes very low, and the number of lower-risk patients needed is too large to allow meaningful discrimination among institutions. It is unusual for the public to possess the statistical background necessary to accurately interpret RAMR and to use the data to choose the best provider. One reason is that they are unable to identify optimal outcomes for a particular dataset (Table 2). Once a death occurs, regardless of the clinical circumstances, the correction factor does not adjust the RAMR to zero. It is unlikely that the public understands the proper context of nonzero values, and might misperceive that >0 mortality necessarily signifies poor quality. It is crucial to recognize that patient mortality might occur despite the best clinical decision-making and technical skill. In current public reporting programs, the guidelines that inform consumers how to interpret the data are difficult to comprehend due to the complexity of statistical methodology; the fact that crude mortality rate depends strongly on case mix is not clarified. The public does not appreciate the limitations of risk adjustment to completely correct for differences in patient populations 5, 7, 24. Experience in states that have implemented reporting programs confirms that facilities and operators develop risk-averse behaviors to avoid being identified as outliers. This reaction can lead to poorer patient outcomes when patients who have the most to gain from a high-risk procedure do not receive treatment. Reluctance to perform high-risk procedures that might save lives, based on apprehension of reported outcomes, is an unintended detrimental consequence of public reporting. In comparing PCI outcomes in the states of Michigan (a state with collaborative quality improvement [CQI]) and NY (which mandates public reporting) using NCDR registry data, Boyden et al 25 found that patients undergoing PCI in NY had a lower-risk profile, with a lower proportion of patients with AMI and cardiogenic shock, compared with Michigan. In a propensity-matched analysis, patients in NY who had PCI were less likely to be referred for emergent, urgent, or salvage procedures (odds ratio [OR] 0.67; 95% CI 0.51–0.88; P < 0.0001), and had lower in-hospital mortality (OR 0.72; 95% CI 0.63–0.83, P < 0.0001). Thus, mandated public reporting in NY was associated with fewer high-risk patients undergoing PCI compared with CQI in Michigan. Moscucci et al. 26 compared the pre-procedural severity of illness of PCI patients in NY with eight Michigan hospitals. These investigators found that NY patients were less likely to undergo PCI for AMI, cardiogenic shock, and cardiac arrest. The unadjusted in-hospital mortality rate was significantly lower in NY than in Michigan (0.8% vs. P < 0.0001), whereas there was no significant in RAMR between the two P These results could be of risk avoidance of high-risk patients driven by the fear of public reporting of procedural mortality rates in NY. In et al. used the NCDR to PCIs in a comparing outcomes from states that had mandated public reporting with that did They found that, to patients in reporting who PCI in public reporting states had predicted in-hospital mortality vs. P but lower observed in-hospital mortality vs. 95% CI P < patients in states with mandated public reporting PCI for cardiogenic or salvage risk avoidance in PCI case However, of PCIs were because risk is low in such patients, risk avoidance is had in reporting and few were of these were to assess risk avoidance the study populations were to only patients undergoing patients not undergoing PCI were not evidence of risk avoidance from physician et al. reported that a of interventional that or that of statewide PCI report in their to perform of interventionists or that patients who might from PCI might not receive the procedure as a result of public reporting of mortality Risk avoidance is also through an of patients with AMI that both undergoing and not undergoing et al. patients with AMI in 10 states and found that in states with public reporting of outcomes, fewer PCI procedures were with the greatest in and cardiogenic patients, compared with In the of PCI for AMI were to the in states prior to public reporting vs. 95% CI However, after of public reporting, of undergoing PCI in compared with states vs. 95% CI P for in was a significant in the proportion of patients presenting with AMI who were with PCI in which with the of public reporting with the differences most in with cardiogenic or cardiac arrest. However, 30-day mortality in PCI patients did not between public reporting and the unadjusted rates were vs. This study that public reporting does not result in outcomes or but lead to avoidance of high-risk in of AMI with PCI in and in with can be at evidence of in the of public reporting from of patients presenting with emergent and high-risk such as AMI, shock, and cardiac arrest. et al. 4 the for registry NY patients were less likely than patients to undergo coronary (OR 95% CI P < and PCI (OR 95% CI P for shock. Although and mortality was lower for centers in NY only public reporting compared with centers in other RAMR was significantly higher in NY 2). The RAMR of patients was greater in NY patients patients. Moreover, NY patients to receive only of NY patients vs. of patients < had within days of Mortality in the registry for NY the of 4 with can be at et al. data from the in patients presenting with AMI and found that states with public reporting and to have lower rates of PCI, for and cardiac Although mortality was lower in patients with AMI undergoing PCI in reporting states et the overall mortality for patients with was significantly higher in reporting states 95% CI with the for risk mortality advantage by PCI with public reporting to be by the mortality in patients who did not undergo PCI for AMI in public reporting states mortality for patients presenting with AMI by Public Reporting and PCI with can be at approaches have been to mitigate risk avoidance very high-risk patients from the reported or one for cases and one for high-risk cases to be the of the model depends on the numbers of cases in risk the of the variables the and whether the models account for case there is insufficient discrimination in the and high-risk of most especially when high-risk patients are in the sample used to the model This is when high-risk patients are from reporting. Many contain patients at the low and the high of risk but are not well in the because such patients are referred for or are Because the discrimination in this is low, the model would be for these patients. Although of patients can be NY patients with cardiogenic and with cardiac and from RAMR This was operators could use their clinical and such patients without fear of being labeled an outlier in a public In two additional variables were using clinical information that was not in the data and that are used in the of The the use is based on presentation with at the of of the or high-risk including in with AMI or PCI of the remaining coronary A of high-risk cases, is as (1) having high-risk not by current risk adjustment or (2) procedures for which PCI was the only or best for of risk cases are from public reporting. who PCI for use were found to have a increase in the of in-hospital death after adjustment for other risk factors and Although these cases only of all the of a to their risk the predictive accuracy of the risk model from to P < et al. used the data to compare data from NY with other including and Michigan 4 and The results demonstrated that, the of in in the NY PCI for patients presenting with AMI and by compared with only in other states < overall in-hospital mortality in patients with presenting with by in NY compared to in et al the rates of PCI, or in NY vs. the states of New and and after the Once a significantly higher proportion of the patients PCI, or or revascularization or after the of cardiogenic from public reporting in NY. risk of patients with cardiogenic and myocardial infarction PCI compared to York states in NY and states 95% confidence on data from can be at risk of in-hospital death compared for patients with cardiogenic and myocardial infarction compared to York states in NY and states 95% confidence on data from can be at Another are revascularization who were for are multiple reasons for including and other clinical and status is associated with greater in-hospital mortality and greater mortality These high-risk patients are not for by current methods because the do not the necessary an unmeasured status be significant that it might the apparently higher mortality observed with PCI compared with in outcome whether high-risk cases or the of RAMR models to assess quality. Patient characteristics that are not for in risk models have the potential to results that falsely or lower measured especially if such characteristics are both important and of the patients from RAMR would increase the that such patients receive needed care without fear on the of providers that an outcome would their own report A is whether this could a operators can the by as patients as possible as high-risk This might independent and of correct Public reporting at the operator level implies that poor outcomes are related to operator or clinical However, most poor outcomes are due to acuteness of and than factors An approach to an annual is to the NY reports the individual operator risk-adjusted mortality calculated from the data the however, the of this approach is individual physician data to increase sample is not necessarily that occur in medical and a (e.g., make individual physician data and In an of hospitals, et al questions about the of statistical to provider performance and the high variance of for low-volume data are unlikely to identify outliers or in a SCAI of public reporting as a to stimulate public in health care. the and of clinical data to the public will and providers to for in outcomes. A summary of to public reporting methods is found in Table In to the RAMR and its EMR, and their SCAI strongly that public reporting programs a of outcome (Table a complete of measures that assess case and such as and would be Although the public reporting of RAMR to be a necessary of reporting outcomes, SCAI does not support its use as the or summary measure of quality. RAMR PCI is a but and metric for the of operator and hospital quality. It is strongly that RAMR be because its in performance quality is uncertain and to risk avoidance in case comparisons among providers and hospitals accurate risk adjustment to account for case selection and a presentation of its An or of risk can the quality of care being Risk adjustment the of mortality, but for a complete of all of care in complex This can only be by a and of procedural mortality at the level of the facility. The unintended consequence of reporting RAMR is the risk-averse it Risk avoidance is to public and it can no be that public reporting to this Public reporting of outcomes in high-risk patients can lead to behaviors at the It might risk-averse who are as being operators. For these SCAI reporting RAMR both and of high-risk SCAI that patients with do not and who have cardiac or cardiogenic should be from public reporting after confirms the accuracy of this high-risk of patients. SCAI of data within The reason for a to might be highly predictive and should be in SCAI in this to allow the of cases to be in and to identify that should be in the It is strongly that public reporting from to institutional risk-adjusted outcomes reporting for mortality measures are and are to a of SCAI about the of reporting the RAMR for individual operators. Public reporting of outcomes in high-risk patients, if at should accurately the performance of operators and institutions. in the there will be to mortality statistics that are for individual operator quality. the present however, such methods do not especially for the highest risk patients.

Related Papers

• → Interleucinas na encefalopatia hipóxico-isquêmica(2003)14 cited
• Susquehanna Chorale Spring Concert "Roots and Wings"(2017)
• → Genentech taps Affimed’s antibodies(2018)
• Enxerto livre de arteria epigastrica inferior: avaliacao hemodinamica e angiografica pre e pos-operatoria(1988)
• → KOMPLIKASI REGIONAL ANESTESI PADA PEDIATRIK(2023)