
In this section, we show how the concepts we have learned regarding the confidence intervals and hypotheses testing can be applied to monitoring the performance of an organization over time. In particular, we show how an Xbar control chart can be constructed. The objectives of this section are to:
To understand assumption of Xbar chart
To be able to construct an Xbar chart using Excel.
To be able to interpret the meaning of findings from Xbar chart
There are two reasons why to construct a control chart.
To discipline intuitions. Data on human judgment show that we, meaning all of us including you, have a tendency to attribute system improvements to our own effort and skill and system failure to chance events. In essence, we tend to fool ourselves. Control charts help see through these gimmicks. It helps us see if the improvement is real or we have just been lucky.
An example may help you understand my point. In an air force, a program was put in place to motivate more accurate bombing of targets during practice run. The pilot who most accurately bombed a target site, was given a $2,000 bonus for that month. After a year of continuing the program, we found an unusual relationship. Every pilot who received a reward did worse the month after. How is this possible? Rewards are supposed to encourage and not discourage positive behavior. Why would pilots who did well before do worse now just because they received a reward? The explanation is simple. Each month, the pilot who won did so not because he/she was better than the others but because he/she was lucky. We were rewarding luck; thus, it was difficult to repeat the performance next month. Control chart helps us focus on patterns of changes and go beyond a focus on luck.
In a field like medicine, over time a poor outcome occurs the natural tendency is to think of it as poor care. But such rash judgments are misleading. In an uncertain field such as medicine, from time to time there will be unusual outcomes. If you focus on these occasional outcomes, you would be punishing good clinicians whose efforts have failed by chance. Instead, focus on patterns of good or bad outcomes. Then you know that the outcomes are associated with the skills of the people and the underlying features of the process and not just chance events. Control charts help you see if there is a pattern in the data and move you away from making judgments about quality through case by case review.
To tell a story. Pcharts display the change over time. These charts tell how the system was performing before and after we changed it. They are testimonials to the success of our improvement efforts. Telling these types of stories helps the organization to:
celebrate small scale successes, an important step in keeping the momentum for continuous improvement.
communicate to others not part of the cross functional team. Such communications help pave the way for eventual organization wide implementation of the change.
You can of course present the data without plotting it and showing it. But without charts and plots, people not be able to see the data. Numbers makes people understand the data but plots and charts, especially those drawn over time, make people connect a story with the data, they end up feeling what they have understood. For many people seeing the data is believing it. When these charts are posted publicly in the organization, control charts prepare the organization for change. They transfer and explain the experience of one unit of the organization to other units.
In analyzing satisfaction ratings, Xbar charts are used to track changes in average satisfaction ratings over time. In analysis of health status of a group of patients, Xbar charts are used to trace the group's health status over time. In this section, we introduce Xbar control charts one step at a time. It is important that you take each step and complete the assignment in the step before proceeding. To help you understand the steps, we provide you with data and ask you to analyze this data as we proceed. The lecture ends when you have completed an Xbar chart. The only way you can learn about control charts is to do some. You learn by doing. You gain confidence in your own ability to create control charts when you do some. Do not just read these pages. Get a piece of paper and do the exercises as you proceed.
Here is the data we need to analyze. Over several months, we tracked the satisfaction with our unit. The following Table shows the data for these months.
Time Period 
Ratings of 1st Subject 
Ratings of 2nd Subject 
Ratings of 3rd Subject 
Ratings of 4th Subject 
Average Rating 
1 
80 
84 
82 
80 
? 
2 
70 
72 
74 
70 
? 
3 
76 
78 
76 
78 
? 
4 
80 
78 
78 
80 
? 
The question we have is whether the unit has improved over time. How would you answer this question? Look at the data. There are wide variations in the data. Could these variations be due to chance? The first step is to calculate the average for each time period. Calculate average rates by summing the ratings in each time period and dividing by the number of cases in that period. Thus, for the first time period, the average will be 80+84+82+80 divided by 4, which is 81.5.
Do not proceed until you have calculated the averages for all time periods. Then, you need to plot the data. Numbers are deceiving. They hide much. To understand numbers you must see them. The next step is to create a xy plot; where the xaxis is time and the yaxis is the average satisfaction rating. Create a xy plot of the data before proceeding further. Before we go further, let me plot the data so that you can compare your work against mine. In the picture below, you see the data for the four time periods.
Figure 1: Average Satisfaction Rating Over Time
The xy plot you have created already tells you a lot. But it will tell you more, if you add to the plot the upper and lower control limits, between which one expects 95% of the data. The upper and lower control limits in an Xbar chart is based on the assumption that data are normally distributed. So before we calculate these limits, we need to check and see if the assumptions are met.
When the assumptions of Normal distribution are met, we can proceed to the next step of calculating upper and lower control limits.
Calculate and plot the grand average. The grand average is the average of all ratings across all time periods. Do not calculate this by averaging the mean of each time period. The correct way to do this is to sum all the ratings for all time periods and divide the sum by the number of ratings. In the example provided here, it makes no difference how you calculate the grand average, but in situations where the number of cases in each time period is changing, it does make a difference.
Please calculate the central tendency of the satisfaction ratings and plot it on your xy plot before proceeding further. This is how it will look like now:
Figure 2: Averaged Ratings Are Better Understood When Central Tendency Is Plotted
With the central tendency line in, we have a visual line to compare the data to. It already tells us a lot, it gives us a sense of which time periods are closer to our central tendency line.
If you notice, in some time periods you have larger differences from the central tendency line than in others. The question arises whether these differences are so large that they are beyond what could be expected from chance events. To answer this question we need to add two other lines to our plot. These are upper and lower control limits. Then points beyond these limits are differences that could not be due to chance. They indicate real changes in the patients' satisfaction with our services.
To calculate the upper control limit follow these steps:
On your chart, plot the upper and lower control limits. Note that the control limits are the same at each time period and the control limits are symmetrical around the grand average. Calculate and plot the upper and lower control limits for each time period. This is how the control chart will look like when you complete your work.
Figure 3: Adding in Control Limits
Note that the control limits are straight lines in this example because in every time period we sampled the same numbers of cases. If this were not the case, the control limits would be tighter when the sample size was larger. If observations fall within the control limits, then the change in the observed rates may be due to chance. Note also that the second time period is lower than the lower control limit. Therefore, patients rated our services worst in this time period and the change in the ratings were not due to chance events. It marks a real change in satisfaction with our services. We would not have known this until we created a control chart.
Examine the plot that you created. Remember that we are trying to answer the question of whether there has been improvements in the process. Look at the plot. What does it tell you. Is it easier to understand the data now that you have plotted it and put limits around the data?