RiskAdjusted Control Charts Base on work of Eric L. Eisenstein, D.B.A.,^{1} Charles F. Bethea, M.D.^{ 2 }Published in Health Care Management Science From: ^{1}Duke University Medical Center, Durham, NC; ^{2}Baptist Medical Center, Oklahoma City, OK Acknowledgements: The authors wish to acknowledge Tracey A. Simons, M.Ed., M.A. for her excellent technical and editorial assistance. This article is published in the Journal of Health Care Management Science Spring 2000 issue. INTRODUCTION Quality improvement professionals use control charts to examine whether changes in care processes has led to improvements in outcomes of care. Historically control charts were first used in manufacturing. Health care differs from manufacturing in that its primary input, patients, are not uniform [15,16]. Even within commonly performed procedures (e.g., CABG surgery or percutaneous coronary intervention), there are large differences in patient characteristics (i.e., patient mix) leading to different resource use [16, 2225]. For this reason, control charts should be adjusted for patient characteristics (often referred to as risks). Otherwise, lower acuity cases (with lower resource requirements) may be incorrectly identified as examples of excellent cost control while higher acuity cases (with higher resource requirements) may be incorrectly identified as examples of poor cost control. While riskadjustment techniques have been developed for mortality [1718, 2728] and costs of care [2325, 29], they are not generally applied in analysis of control charts. In fact, none of the leading books on analysis of control charts teach how risk adjustments should be made. Techniques for riskadjusting pcharts have been developed [15]. Nonetheless, xbar charts in health care continue to be presented without adequate riskadjustment [14, 31]. In this paper we begin by introducing techniques for riskadjusting Xbar charts. We will then apply these techniques to the inhospital care costs for CABG surgery (a common but expensive procedure) and will then compare differences between unadjusted and riskadjusted Xbar chart results. METHODS Study Population Our study population included all CABG surgery patients admitted to Baptist Medical Center, Oklahoma City, Oklahoma during the first ten months of fiscal year 1996. CABG surgery patients were defined as having an ICD9CM procedure code between 36.10 and 36.19. Patients with incomplete cost data and with concurrent percutaneous coronary intervention or valvular surgery procedures were excluded. Data Sources Clinical information used in our riskadjustment model came from Baptist Medical Center’s implementation of the Duke Information System for Clinical Cardiology (DISCC). Both the DISCC system [32] and the riskadjustment model used in this study [29] have been previously described. The patient characteristics selected for this model came from previous CABG surgery riskadjustment models (cost and mortality) [2324, 2728] and from the expert opinion of clinical cardiologists and outcomes researchers at Baptist and Duke University Medical Centers. All cost information used in our analyses came from Baptist Medical Center’s implementation of the Transition One cost accounting system. Transition One estimates hospital costs using a bottomup approach that is based on resources consumed and their unit prices [33]. Our control chart analysis used the natural log of inhospital costs because CABG cost data is skewed to the right and must be transformed to meet the assumptions of Normal distribution [23, 3435]. Methods For Developing Control Charts Unadjusted Control Chart The following formula detail the steps required to calculate lower and upper control limits for continuous variables in unadjusted Xbar control charts. Each time period is shown by the suffix "i", each patient is shown by the suffix "j". Thus, X_{ij} is the observation for the jth patient in the ith time period. Suffixes are dropped to designate an average; thus, X_{i }is the average of observations for all patients in time period "i", and X_{ }is the average for all patients in all time periods. The number of cases in each time period is shown as n_{i}, the log costs of care for individual patient cases (x_{ij}), and the sum of log costs of care in time period "i" is shown as l_{i}. The formulae for calculating the population average X, the population standard deviation S, average for time period "i" shown as X_{i}, standard deviation for time period "i" shown as S_{i}, upper control limits for time period "i" shown as UCL_{i}, and lower control limits for time period "i" shown as LCL_{i} are given below as equations (1) through (6). An example of our calculations for the first month of our study (July) is shown below in equations (7) through (12). The degrees of freedom for the tdistribution are calculated as one less than the population sample size. The complete calculations for all study months are reported in Table 4 and the resulting unadjusted control chart is shown as Figure 1. X = 5616.38 / 574 (7) S = (55.49 / 573) ^{0.5 } (8) X_{1}= 474.21 / 49 (9) S_{1}= 0.31 / (49)^{0.5 } (10) UCL_{1} = 9.7846 + (1.645 * 0.0445) (11) LCL_{1} = 9.7846  (1.645 * 0.0445) (12)
RiskAdjusted Control Chart The following formulae detail the steps required to calculate lower and upper control limits for continuous variables in riskadjusted Xbar control charts. The log of costs of care for case "j" in time period "i" is X_{ij} and the expected log of costs of care for case "j" in time period "i" is shown as E_{ij}. The expected costs are predicted from the severity of the patient conditions using procedures described in citation [29]. The steps in calculating risk adjusted control limits are given in equations (13) through (17). å
_{j=1, …,m} E_{ij} /n_{i
(13)
}D_{ij }= X_{ij }E_{ij
}D_{ }= å
_{i=1, …nj} _{j=1,
…,m} D_{ij } /n_{
}_{(14)
}S = [å
_{i=1, …nj} _{j=1,
…,m} (D_{ij} –D)^{2}/(n1)]^{ 0.5}
An example of our calculations for the first month of our study (July) is shown below in equations (18) through (22). The degrees of freedom for the tdistribution are calculated as one less than the month’s sample size. The complete calculations for all study months are reported in Table 5 and the resulting unadjusted control chart is shown as Figure 2. E_{1 }= 477.6779 / 49 (18) d_{1} = 3.4639 / 49 (19) S_{d1} = 0.064 / (49) ^{0.5 } (20)
RESULTS InHospital Costs and Length of Stay During the ten months of this study, inhospital costs of CABG surgery at Baptist Medical Center averaged $18,859 (standard deviation $8,932, range $9604 to $126,830) and median costs were $16,773 (interquartile range $14,598 to $20,335). As anticipated, CABG surgery costs were skewed to the right with 73.9% of cases having costs less than $20,000, 20.4% having costs between $20,000 and $30,000, and 5.7% having costs > $30,000 (Figure 1).
The number of cases per month ranged from 47 in March to 71 in September (Table 2). There was significant variation in average costs by month (range from $16,621 in July to $21,841 in February). However, there was less variation in average length of stay (range from 6.7 days in January to 8.4 days in February). Inhospital costs were correlated with the progression of months (July to April) (p = 0.017), whereas, length of stay was not (p = 0.796).
Risk Characteristics There also was significant monthly variation in the variables used in our CABG surgery riskadjustment model (Table 3). Average age at admission varied in a rather tight band between 61.7 and 65.8 years. However, the percentage of patients with prior CABG surgery varied from 4.0% to 16.4% and patients with a history of congestive heart failure ranged from 1.6% to 18.0%. With regard to hospital process factors, the percentage of patients with acute myocardial infarction as the reason for admission ranged from 3.2% to 19.3% while those with an emergency admission ranged from 9.1% to 28.6%. Additionally, the percentage of patients with coronary angiography in the same admission as their CABG surgery procedure varied from 42.9% to 72.3%. Month of discharge independently explained 3.2% (p = 0.027) of the variation in log costs and our riskadjustment model explained 32.0% (p = 0.001). However, month of discharge was not a significant predictor of log costs (p = 0.2044) after adjusting for differences in risk.
Unadjusted Control Chart We used formulae (1) through (6) in calculating the unadjusted inhospital CABG surgery costs by month (Table 4). These results are shown graphically in Figure 1.
The middle line in Figure 1 is the population’s average log costs for all ten months, the dashed lines are its 90% upper and lower confidence limits, and the small squares are the average log costs for each month. There is a trend toward increasing CABG costs over time in Figure 1 with the actual costs for three months (July, January, and February) being beyond the upper or lower control limit boundaries. Since only one month would be expected to be beyond these control limits by chance, this is an indication that the underlying CABG surgical process may be out of statistical control. riskAdjusted XBar Chart We used formulae (13) through (17) in calculating the riskadjusted inhospital CABG surgery costs by month (Table 5). These results are shown graphically in Figure 2.
The middle line in Figure 2 is the riskadjusted average log costs for each month, the dashed lines are its 90% upper and lower confidence limits, and the small squares are the average log costs for each month. After riskadjustment, only two months (July and November) are beyond the upper or lower control limit boundary. Since only one month would be expected to be beyond these control limits by chance with a 90% confidence limit, there may be some cause for concern. Additionally, two of the months whose average CABG surgery costs were beyond the 90% control limit boundary without riskadjustment (January and February) are now within these limits and may be instances of wider risk variation rather than cost outliers. DISCUSSION As financial pressures continue to increase in the US health care environment, hospitals will continue to investigate ways to control their costs of care. Because CQI techniques have been successfully applied in the manufacturing sector of the US economy, there has been great hope that they can achieve equivalent success in managing the ever escalating costs within the health care sector. Unfortunately, health care has an additional source of variation (patient mix) which has not typically been included in health care control charts. Our study demonstrates the value of riskadjustment when developing control charts for health care. Without riskadjustment, three of the months we studied had average costs beyond the lower or upper control limit boundaries and there appeared to be a trend toward higher (and potentially out of statistical control) costs. After riskadjustment, only two months had average costs beyond their control limit boundaries and our trend toward higher costs over time was seen as a trend to increasing patient acuity, which is associated with higher expected health care costs. Previous CABG Cost Studies Although the value of riskadjustment in control charts has not been previously demonstrated, there is evidence that it could be beneficial when studying CABG surgery outcomes. Previous studies have demonstrated the value of riskadjustment when making comparisons among surgeons, hospitals, and payers [23, 24, 29]. Thus, we can assume that riskadjustment techniques would be beneficial in other types of analysis in which continuous outcome variables may be expected to vary based upon differences in baseline patient characteristics. Limitations Implementation Requirements While the methods used in our analyses are simple, our information requirements may be beyond those available to many community hospitals. Our analyses require information from a surgical information system to construct a riskadjustment model as well as information from a hospitalwide cost accounting system to measure total inhospital costs. Additionally, our methods may not be feasible for other types of procedures which have lower average costs of care or which occur less frequently. Log Transformation Although log transformation is required when cost data is highly skewed to the right, the resulting data is extremely difficult to present to health care administrators. We experimented with smearing as a method for retransforming the data [36]. The retransformed data was more understandable, but its spread was greater than the logged data and incorrectly produced a greater number of months with average cost beyond the upper and lower control limit boundaries. Since the log transformed and retransformed data did not identify the same outliers, we would not recommend smearing as a means for retransforming data for control charts. Sample Size Sample size is an obvious limitation of our analysis. If we had more months and more cases per month, our results would have been more compelling. Nonetheless, ten months of data did demonstrate the value of riskadjustment in control charts. We are currently analyzing three years of CABG surgery data from Baptist Medical Center and hope to present these results in the near future. Conclusions Although riskadjustment may not be required for all health care control charts, it proved valuable in the present analysis by allowing us to adjust our results for monthly differences in patient mix. riskadjustment demonstrated that the trend we saw in CABG costs paralleled the increasing acuity in CABG patients. Thus, the higher costs in succeeding months were the result of increasing patient acuity and not the result of a surgical process which was potentially out of control. REFERENCES
