George Mason University

 

HAP 525: 

Risk Analysis in Healthcare

 

Causal Modeling & Root Cause Analysis

By Farrokh Alemi, Ph.D.
Jee Vang

 
 

Summary

Root Cause and Failure Mode Analyses are commonly performed in hospitals to understand factors that contribute to errors and mistakes. Despite the effort that healthcare professionals are putting into creating these analyses, few models of root causes are validated or used to predict future occurrences of adverse events.  We review the literature on Causal Networks and Bayesian Probability models and show how these tools can be used to improve Root Cause Analysis.  In particular, we show that more in-depth insight can be gained by (1) testing the proposed root causes for structural assumptions of independence (root causes should be conditionally independent of sentinel events given direct causes) and by verifying that the root cause model implies probabilities that are comparable to those reported in the literature or known through experience of the health care organization.  We show how both assumptions and conclusions of Root Cause Analysis can be verified against observed data.

Introduction

Root Cause Analysis, according to the Joint Commission on Accreditation of Health Care Organizations is a "process for identifying the basic or causal factors that underlie variation in performance, including the occurrence or possible occurrence of a sentinel event."   Sentinel events include medication errors, patients' suicide, procedure complications, wrong site surgery, treatment delay, restraint death, elopement death, assault or rape, transfusion death, and infant abduction.  Direct causes bring about the sentinel event without any other intervening event.  Most direct causes are physically proximate to the sentinel event.  The effect of root causes on sentinel events are always through some direct cause.  Because of accreditation requirements and due to renewed interest in patient safety, many hospitals and clinics are actively conducting Root Cause Analyses.  

When a sentinel event occurs, most employees are focused on the direct causes that have led to the event.  For example, many will claim that the cause of medication error is a failure to check label against the patient's armband.  But this is just the direct cause.  To get to the real reasons, one should ask why did the clinician not check the label against the armband.  The purpose of Root Cause analysis is to go beyond direct and somewhat apparent causes and figure out the underlying reasons for the event.  The objective is to force one to think harder about the source of the problem.  It is possible that the label was not checked against the armband because the label was missing.  Furthermore it is also possible that the label was missing because the computer was not printing.  Then, the root cause is computer malfunction and the direct cause is the failure to check the label against the armband.  Exhorting employees to check the armband against the label is a waste of time, if there is no label to check in the first place.  A focus on direct causes may prevent the sentinel event for a while, but sooner or later the root cause will lead to a sentinel event.  Inattention to root causes promotes palliative solutions that do not work in the long run.  The value of root cause analysis lies in identifying the true, underlying causes.  An investigation that dos not do this is at best a waste of time and resources, and at worst can exacerbate the problems it was intended to fix.  But how do we know if our speculation about the causes of an event are correct?

To make the situation worse, almost all who conduct Root Cause analyses become overconfident about the accuracy of their own insights.  No matter how poorly an analysis is carried out, since there is no way of proving a person wrong, people persist in their own fallacies.   Some are even incredulous about the possibility that their imagined causal influences could be wrong.  They insist on the correctness of their insights because "it is obvious."  Unfortunately, it is not clear why a complex problem, which has led to a sentinel event, which has not been corrected for years, which has been left unaddressed by hundreds of smart people should have such an obvious solution.  After all, if the solution was so obvious why was it not adopted earlier?  Search for obvious solutions contradicts the elusiveness of correcting for sentinel events.  If a sound and reliable method existed for checking the accuracy and consistency of Root Cause Analysis, then employees might correct their misperceptions and not be so overconfident.  

One way to check on accuracy of Root Cause Analysis is to examine time to next sentinel event.  Unfortunately, because sentinel events are rare, one has to wait a long time to see rare events occur again, even if no changes were made.   Thus, the organization may have little solace by marking time as long periods of time are no sign of success and the event may reoccur any day.  An alternative needs to be found to check the accuracy and consistency of Root Cause Analysis without having to wait for the next sentinel incidence. 

Simple methods for checking the accuracy of a Root Cause analysis have not been available to date.  This paper suggests a method for doing so.  As before, clinicians propose a set of causes.  But now several additional steps are taken.  First, probabilities are used to quantify the relationship between causes and effect.  Then, the laws of probability and causal diagrams are examined to see if the suggested causes are consistent with the clinician's other beliefs and with existing objective data.  Through a cycle of testing model assumptions and conclusions against observed data, one improves the accuracy of the analysis and gains new insights into the causes of the sentinel event.

Bayesian Networks

We use a set of techniques that have been developed for analysis of Bayesian Causal Networks to validate Root Cause Analysis.  A Bayesian Causal Network is a mathematical model of the cause and effect relationships and the way these relationships lead to observed statistical patterns of an event.   It consists of a set of nodes, typically pictured as ovals, connected by directed arcs.   Each node represents a mutually exclusive and collectively exhaustive set of possible events.  For example, Figure 1 shows a Bayesian network with two nodes.  The node "Armband legibility?" has three possible values, exactly one of which must occur and no two can coincide.   These states are "No armband," "Poor" and "Good."  The other node "Armband checked?" has two possible values, "Ok" and "Not ok."  A node with two possible values is called a binary node.  Binary nodes are common in root cause analysis. 

Figure 1:  A simple Bayesian Causal Model with a Local Probability Table for "Armband checked?"

A Bayesian Network is cyclical directed graph, meaning that you cannot start from a node and follow the arcs and arrive back to where you started.  In a Bayesian Network, the relationship between any three nodes can be expressed in one of the following three ways:  serial, diverging or converging structures.  Each of these graph structures can be verified through tests of conditional independence and are further explained through examples below.

The relationship between “Armband legible?” and “Armband Checked?” in Figure 1 is a direct causal relationship.  Bayesian networks can also represent indirect causal relationships, through the concept of conditional independence, as shown in Figure 2. In this example, the root cause “Understaffing” is an indirect cause of the sentinel event. There is no direct arc from the root cause to the sentinel event. This means that the action of the root cause on the sentinel event is indirect, operating through an intermediate cause. That is, the direct cause of a medication error is a fatigued nurse. The root cause, “Understaffing,” is conditionally independent of the sentinel event given the intermediate cause. This means that if we intervene in any given instance to relieve a fatigued nurse, we can break the link from the root cause to the sentinel event, thus reducing the probability of the sentinel event to its nominal level. However, this solution is a palliative one and will not produce a long-term solution unless the root cause is addressed.  Figure 2 illustrates a serial graph structure.  In these structures the sentinel event is independent of the root cause given the known value for direct cause .  

Figure 2: Serial Example of Direct and Root Cause of Medication Error 

Another type of conditional independence occurs when a cause gives rise independently to two different effects, as depicted in Figure 3. This type of graph structure is known as diverging.  In this example, “High blood pressure” and “Diabetes” are conditionally independent given the value of “Weight gain,” but are correlated due to the influence of the common cause. That is, the two effects typically either co-occur (when the common cause is present) or are both absent (when the common cause is absent). This type of conditional independence relationship is quite useful for diagnosing the presence of root causes that can lead to multiple independent effects that each influence different sentinel events. For example, understaffing might lead to several different intermediate causes, each of which could be a precursor of different sentinel events. If several of these precursor events were to be observed, one could infer that the understaffing problem was sufficiently severe to impact patient care. Proactive remediation could then be initiated prior to the occurrence of serious adverse medical outcomes.


Figure 3:  Conditional Independence is Assumed in Diverging Structure

Figures 2 and 3 illustrate serial and diverging causal structures, respectively. As we have seen, a serial structure represents the action of an indirect causal relationship, and a diverging structure represents multiple independent effects of a single cause. In both these cases, the two terminal nodes are conditionally independent of each other given the middle node. A different kind of causal structure, the converging structure, is shown in Figure 4. A converging structure occurs when two different causes can produce a single effect, as when either a fatigued nurse or a missing armband can cause a medication error. Notice that in this case, the terminal nodes are not conditionally independent given the middle node. For example, if the sentinel event is known to occur, and we learn that the armband was present, this will increase our probability that the nurse was unacceptably fatigued. Likewise, if we find that the armband was missing, this will reduce the likelihood that the problem was due to fatigue.


Figure 4:  Two Causes Converging into a Common Effect 

Data can be used, if available, to validate the graph structure of a causal Bayesian network. As we noted above, when a connection is serial or diverging, the terminal nodes are conditionally independent given the intermediate node. This relationship can be evaluated by holding the value of the intermediate node fixed and examining whether the terminal nodes are correlated. In the example of Figure 3, we would compute the correlation of “High blood pressure” and “Diabetes” for all cases in which there was weight gain. We would then calculate the correlation for all cases in which there was no weight gain. If both correlations were statistically indistinguishable from zero, we could conclude that the conditional independence relationships were satisfied. Similar tests can be used to evaluate conditional independence relationships for a diverging structure. In general, in a Bayesian network a node is conditionally independent of all its non-descendents given its parents. This general condition implies a set of correlations that should be equal to zero if the structure is correct. Although it is tedious to verify all these relationships by hand, it is straightforward to automate the verification process, and computer programs have been written to accomplish the task.  

Given the structure of an acyclic causal graph, one can read off the assumed conditional independencies.  Conditional independencies are identified through examining serial or diverging graphs in causal model so that removal of the condition would sever the directional flow from the cause to the effect.  Often, a complicated Root Cause Analysis can be broken into smaller components containing serial and diverging structures.  If these structures are observed and if removal of the condition in these structure would sever the link between the other two nodes, then a conditional dependencies has been identified.  Careful examination of conditional independence relationships is an important element of specifying and validating a Bayesian Network for root cause analysis. 

Validation of Conditional Independence

Once conditional independencies have been identified, the assumptions can be verified by examining data or by querying experts.  If data is available, in a serial structure, the correlations between root cause and sentinel event should equal to the correlation between root cause and direct cause times the correlation between direct cause and sentinel event. 

R root cause, sentinel event = R root cause, direct cause * R direct cause, sentinel event 

Where:  
R root cause, sentinel event is the correlation between root cause and sentinel event.
R root cause, direct cause is the correlation between root cause and direct cause
R direct cause, sentinel event  is the correlation between direct cause and sentinel event

In a diverging structure, a similar relationship should hold.  In particular, correlation between the two effects should be equal to the multiplication of the correlation between the cause and each effect:

R effect1, effect2 = R cause, effect1 * R cause, effect2 

Where:  
R effect1, effect2 is the correlation between the two effects
R cause, effect1 is the correlation between cause and first effect
R cause, effect2  is the correlation between cause and second effect

If data are not available, the analyst can ask the investigative team to verify assumptions of conditional independence based on their intuitions.  For example in the serial structure in Figure 2, if we know that the nurse was fatigued, would information about staffing add much to our estimate of the probability of medication error.  Another way to ask the same questions is to ask if understaffing affects medication errors only through creating a fatigued nurse.  In this method, the exclusivity of the mechanism of change is checked.  Still another way of verifying conditional independence is through asking for estimates of various probabilities.  One might ask:

Question:  What do you think is the probability of medication error when the nurse is fatigued?
Answer:  It is higher than when the nurse is not fatigued but still relatively low.
Question:  What do you think is the probability of medication error when the nurse is fatigued and working in understaffed unit.
Answer:  Well I think that understaffing leads to a fatigued nurse but you are not asking about that, are you?
Question:  No, I want to know about probability of medication error in these circumstances. 
Answer:  I would say it is similar to the probability of medication error among fatigued nurses.

If conditional independence is violated, then the serial or diverging structures in the graph are incorrect.  If these conditions are met, then the causal graph is correct.

Lets look at slightly more complicated set of causes.  Figure 4 shows four proposed causes for medication error: understaffing, fatigued nurse, vague communication, and similar medication bottles.  Two root causes (understaffing and vague communications) are shown to precede the direct cause "fatigued nurse."  Removing the node "fatigued nurse" would stop the flow from these two root causes to the medication error.  Therefore, a conditional independence is assumed.  This assumption can be verified either through data or through experts judgments.  Let us assume that if we know that the nurse is fatigued, understaffing adds no additional information to probability of medication error. So this independence is verified.  But suppose that even when the nurse is not fatigued, vague communications may lead to medication errors.     Therefore, the assumption of conditional independence of vague communication and medication error is not met. 

Figure 4:  Four possible causes of medication error & their relations

Therefore, the causal network needs to be modified.  Further exploration may indicate that vague communications, similar bottles, and fatigued nurse directly affect medication errors.  This example shows how verifying conditional independence could help us revise root cause analysis. 

Predictions from Root Causes

The causal model behind the root cause analysis can be used to predict the probability of the sentinel event and this probability can then be compared to the intuitions of the investigative team. The probability of sentinel event can be calculated from each of the direct causes and the probability of direct causes can be calculated from their root causes.

p (Sentinel event, Various causes) =
p (Sentinel event | Direct causes) * p (Direct causes | Direct root causes) * p (Root causes)

To calculate, the probability of sentinel event, S, given a set of different unobserved (CU) and observed causes (Ci),  we can use the following formula:

The above formula requires one to carefully track numerous probabilities.  Because these calculations are tedious, investigative teams can use widely available software, e.g. Netica (TM), to simplify the calculations.  An example can demonstrate how such calculations are made using the software .  Suppose that Figure 5 shows root causes for wrong side surgery in a hospital.  First, note that the root causes listed are poor MD training, and over staffing as it contributes to fatigued nurse. These are the root causes because they are independent of sentinel event given the various direct causes.  The direct causes listed are nurse marking the patient wrong, surgeon not following the markings and patient providing wrong information.  These are direct causes because an arc connects them to the sentinel event.

Figure 5:  Root causes for wrong side surgery

Given the Root Cause Analysis in Figure 5, the next step is to estimate the probability of the various cause and effects.  These probabilities are obtained by asking the expert to assess the conditional probabilities implied in the graph.  Each node is conditioned on its direct causes.  For example, to estimate the probability of having a fatigued nurse, the facilitator needs to ask the investigative team the following two questions:

  1. In 100 occasions in which the unit is understaffed, what is the frequency of finding a nurse fatigued?

  2. In 100 occasions in which the unit is not understaffed, what is the frequency  of finding a nurse fatigued?

Obviously, estimates of probabilities from experts is subjective and therefore may be unreliable.  But if experts are provided with tools (calculators, paper, pencils), brief training in concept of conditional probabilities, available objective data (e.g. JCAHO's reports on prevalence of various causes), and if experts are allowed to discuss their different estimates, then experts' estimates are sufficiently accurate and reliable to provide a useful model.  These probabilities may not be accurate to the last digit, but can provide for a test of consistency.  Suppose that through interviewing experts or through analysis of data of the hospital, the investigative team has estimated the following probabilities:

p(Over staffing) = .40
p( Patient provided wrong information) = .05
p( Poor training) = .12
p(Fatigued nurse | Over staffing) = .30
p(Fatigued nurse | No over staffing) = .05
p( Nurse marked patient wrong | Fatigued nurse) = 0.17
p( Nurse marked patient wrong | Not fatigued nurse) = 0.01
p( Surgeon did not follow markings | Poor training) = 0.10
p( Surgeon did not follow markings | Good training) = 0.01
p( Wrong side surgery | Patient provided wrong information, Nurse marked patient wrong & Surgeon did not follow markings)  is given as in Table 1

Conditions

Probability of wrong side surgery given conditions

Patient provided wrong information

Surgeon did not follow markings

Nurse marked patient wrong

True

True

True

0.75

True

True

False

0.75

True

False

True

0.70

True

False

False

0.60

False

True

True

0.75

False

True

False

0.70

False

False

True

0.30

False

False

False

0.01

Using these estimates we can calculate the probability of wrong side surgeries when no information about any cause are present as 0.07 (see Figure 6 for an example analysis using Netica (TM) software available through Norsys).  Does this seem reasonable to the investigative team?  If the probability is higher by an order of magnitude from what the investigative team expected, then perhaps important constraints that prevent wrong side surgeries have been left out of the analysis.  If it is too low by an order of magnitude, then an important cause or mechanism by which wrong side surgeries occur might have been missed.  If it is in the ball park but not exactly what we expected, then perhaps the estimated probabilities might be wrong.  In any case, when there is no correspondence between the probability of the sentinel event and the investigative team's intuition, it is time to re-think the analysis and its parameters. 

Figure 6:  Application of Netica Software to Root Cause Analysis in Figure 5

Other probabilities can also be calculated and compared to the experts' intuitions.  Suppose on a particular unit in a particular day, we find the nurse was fatigued but the clinician was well trained and the patient provided accurate information.  Given the above estimates and the root cause in Figure 5, the probability of wrong side surgery on this day is calculated as: 0.03 (See Figure 7).  If this corresponds to the investigative team's expectation, then the analysis is consistent and we can proceed.  If not, we need to examine why not and look for adjustments that would fit the model predictions to experienced historical rates. 

Figure 7:  Probability of wrong side surgery when the patient has provided
correct information, the surgeon is well trained but there is over staffing

Reverse Predictions

The Bayesian network can also be used to calculate the probability of observing a cause given an effect has occurred.  This is the reverse of how most people think about causes and effects.  Most people start with a cause and want to predict the probability of the effect.  Bayesian probability models allow us to do the reverse.  One can start with known sentinel events and ask about the prevalence of a particular cause among them.  Since causes are not as rare as sentinel events, this procedure allows us to check on the adequacy of the analysis without having to wait a long time for reoccurrence of the sentinel event.  To make matters easier, the Joint Commission on Accreditation of Healthcare Organizations publishes prevalence of categories of causes among sentential events.  These data can be used to examine the consistency of the Root Cause Analysis.  Large discrepancy between observed prevalence of causes among sentinel events and assumed prevalence of causes in the investigative team's model suggest errors in assignments of probabilities as well as possible missed cause or constraint.

Figure 8:  JCAHO's Report on Various Categories of Causes for Sentinel Events

There are a number of software systems that allow the calculation of reverse probabilities in a Root Cause Analysis.  Using Netica software we calculated the prevalence of over staffing in our model of wrong side surgeries.  We started by setting the probability of wrong side surgery to 100%.  Then we allowed the software to calculate the prevalence of over staffing.


Figure 9:  Prevalence of Over Staffing Among Wrong Side Surgeries

The software calculated that overstaffing is present in 44% of wrong side surgeries.  Is this a reasonable estimate?   In contrast, JCAHO reports staffing levels to be a cause of sentinel event less than 20% of the time (see Figure 8).  Obviously, there are many reasons for a health care organization to differ from other aggregate data reported by JCAHO.  But JCAHO's data can be used as a rough benchmark.  The two probabilities differ considerably.  These differences suggest the need to think again through the analysis.

Summary of Proposed Method for Root Cause analysis

Sentinel events can be reduced if health care organization create a blame-free environment, conduct Root Cause Analysis and take concrete actions on the basis of the analysis.  To conduct root cause analysis we propose the following steps:

  1. Before a sentinel event occurs, an investigative team is organized.  The team should include a facilitator and a team leader.  The facilitator's responsibility is to organize tasks, serve as staff to the team, and conduct team meetings in an efficient and effective method.  The facilitator should be trained in probability models.  The leader's responsibility is to make sure that the investigation is carried out thoroughly and to provide content expertise. 
  2. When a sentinel event is reported, the employees closest to the incident are asked to record facts (not accusations) about the event, including what happened, who was present, where did the event occur, when did it occur and what was the time sequence of the events that preceded the sentinel event.
  3. The investigative team meets and brainstorms (1) potential causes for the incident, and (2) key constraints that if they were in place would have prevented the incident from occurring.  Two steps are taken to make sure the listing is comprehensive.  First, the framing bias is reduced by asking for a list of causes with alternative prompts.  Thus, since constraints can be thought of reverse causes, the team should be asked to list both the constraints and causes.  Furthermore, because the team is focused on conditions that led to the sentinel event, they should be asked to examine also conditions that prevented sentinel events in other occasions. 
  4. The facilitator interviews the investigative team or uses existing data to assign a probability to each cause and a conditional probability for each effect. 
  5. The facilitator checks the accuracy of the causal model and asks the investigative team to revise their model.  The following steps allows one to check the accuracy or consistency of the causal model:
    1. The facilitator uses the model to predict the probability of the sentinel event.  If this probability is several magnitudes higher than historical pattern or investigative team's intuitions, the facilitator seeks additional constraints that would reduce the probability of the sentinel event.  If the probability is lower than historical experience or the investigative team's intuitions, the team is asked to describe additional mechanisms and causes that may lead to the sentinel event.
    2. The facilitator uses the model to calculate the prevalence of the causes among sentinel events.  These data are checked against the investigative team's intuitions as well as against observed rates published by the JCAHO. 
    3. The facilitator checks that claimed root causes are conditionally independent from the sentinel event.  If a root cause is directly linked to the sentinel event, the investigative team is asked to redefine the direct cause to be specific to the mechanism used by the root cause to affect the sentinel event.  If few root causes have been specified, the investigative team is asked to think again through reasons why direct causes occur. 
    4. The facilitator checks the marginal probabilities against objective data.  If the probabilities do not match, the facilitator should use the objective probabilities whenever available.
  6. Document the findings.  A chart is organized showing all the nodes and arcs.  The root causes, direct causes and sentinel events are shown in the chart.   Arrows are drawn from root causes to direct causes and from direct causes to sentinel events.    

Discussion

Investigative teams often rely on their own intuitions for listing the root causes of a sentinel event.  They rarely check the validity of their analysis.  In this paper, we have shown how Bayesian networks can be applied to root cause analysis to test the validity and/or consistency of the analysis.  Real analysis should be a careful examination of facts and not a cover for wishful speculation.  By creating a Bayesian Network and estimating the probabilities of various events, one can scrutinize assumptions made in root cause analysis.  In particular, one can check to see if important root causes have been missed, if the analysis is focused on root causes or direct causes, if frequency of sentinel event corresponds to expectations and experienced rates, if prevalence of causes of sentinel events correspond to known rates, and if assumptions of dependence or independence are wrong.  These are not exact ways of checking the accuracy of the analysis.  But these methods allow us to check the intuition of investigative teams and help them think through the implication of their analysis. 

Examples of Root Cause Analysis

Example of Survey of Safety Team to Construct A causal Model

This section provides an example of interaction with members of safety team after they have identified a potential set of causes.  The section headings in this survey correspond to the direct causes of medication error as envisaged by the safety committee in prior meetings.  This survey is part of step 4 in the proposed method for root cause analysis.  This email communication is followed by a face to face meeting of the safety committee which focuses on responses to this emailed survey.

In anticipation of the committee’s xxx xxth meeting, I wanted to contact you and ask for your ideas regarding how to reduce medication omission errors.  If you recall our analysis had shown that medication omission errors occur once every xx days.  Our task is to agree on steps that could increase the time between errors to longer number of days.   Our analysis had shown that in the past there were x time periods in which time between medication omission errors exceeded xx days.  We want to make this occur much more often.

This email is designed to solicit your ideas about what would help.  It is organized in sections that describe various causes envisioned in our last meeting.  For each cause, we ask you to think through possible solutions.  Please reply to this email with your ideas (if you wish you can print and fax your response to xxx xxx xxxx).  Read the remainder of this email; reply to this email and insert within each section in the email the first idea that comes to you.  Do not try to search for the best idea but just what you think might work.  Please try to give at least one idea for each section.  Please make sure that you give us an estimate of the frequency of occurrence of different events as these numbers will be used to set priorities for what we need to discuss first (you do not need to be very precise in your estimates, given a number that is roughly in the ball park of what you think is the frequency of the event).  We will collate the ideas we have received from everyone and then start our meeting with a discussion of the ideas.  Please do not delay in responding.  I know this is a short notice but we were hoping that you will respond to this email today or tomorrow.  If you are having difficulty answering these questions, please contact me on my cell phone at xxx xxx xxxx

Section 1:  Order Not Taken Off the Chart

In 100 patients within your organization, how often “order is not taken off the chart?”

In 100 patients for whom order is not taken off the chart, how often do these patients have a medication omission error?

What do you think could reduce the frequency of times that order is not taken off the chart?  In our last meeting you had mentioned that distractions, frequency of change in dose, patient overload and illegible orders lead to miscommunications and miscommunications lead to orders not being taken off the chart. 

Section 2:  Failure to Reconcile Physician Order

In 100 patients within your organization, how often “physician orders are not reconciled?”

In 100 patients for whom physician orders were not reconciled, how often do these patients have a medication omission error?

What do you think could reduce the frequency of times that physician orders are not reconciled?  In our last meeting you had mentioned that distractions, frequency of change in dose, patient overload and illegible orders lead to miscommunications and miscommunications lead to physician orders not reconciled. 

Section 3:  Misplaced Medication Order

In 100 patients within your organization, how often “medication orders are misplaced?”

In 100 patients for whom medication orders were misplaced, how often do these patients have a medication omission error?

What do you think could reduce the frequency of times that medication orders are misplaced?  In our last meeting you had mentioned that distractions, frequency of change in dose, patient overload and illegible orders lead to miscommunications and miscommunications lead to order written on wrong page and order written on wrong page leads to misplaced medication errors.  You had also mentioned that faxed orders and protocol not being followed also led to misplaced medication errors.

Section 4:  Incorrect Administration of IV

In 100 patients within your organization, how often “IV is administered incorrectly?”

In 100 patients for whom IV is administered incorrectly, how often do these patients have a medication omission error?

What do you think could reduce the frequency of incorrect administration of IV?  In our last meeting you had mentioned that distractions, frequency of change in dose, patient overload and illegible orders lead to miscommunications and miscommunications lead to incorrect administration of IV.

Section 5:  Order Not Written

In 100 patients within your organization, how often “orders are not written?”

In 100 patients for whom orders are not written, how often do these patients have a medication omission error?

What do you think could reduce the frequency of times that medication orders are misplaced? 

Section 6:  Computer Related Errors

In 100 patients within your organization, how often “computer related errors” occur?

In 100 patients with computer related errors, how often do these patients have a medication omission error?

What do you think could reduce the frequency of computer related errors?  In our last meeting you had mentioned that software discontinued, down time, and changing previously entered field lead to computer errors.

Section 7: Pharmacy Order Delayed 

In 100 patients within your organization, how often “pharmacy order is delayed?”

In 100 patients for whom pharmacy order is delayed, how often do these patients have a medication omission error?

What do you think could reduce the frequency of delay of pharmacy orders?  In our last meeting you had mentioned that phone interactions and multiple task affect the pharmacy distractions, which in turn affects order delays.

Section 8:  Patient Did Not See the Order

In 100 patients within your organization, how often “patients do not see their medication order?”

In 100 patients who had not seen their medication order, how often do these patients have a medication omission error?

What do you think could reduce the frequency of patients not seeing their medication orders?  In our last meeting you had mentioned that phone orders and multiple tasks distract the pharmacist.  These distractions and orders arriving after portion of the order has been filled affect whether the patient sees the medication order.

Section 9:  Pharmacy Order Written Incorrectly

In 100 patients within your organization, how often “pharmacy order is written incorrectly?”

In 100 patients for whom pharmacy order is written incorrectly, how often do these patients have a medication omission error?

What do you think could reduce the frequency of orders written incorrectly?  In our last meeting you had mentioned that phone orders and multiple task distract the pharmacist.  These distractions lead to orders written incorrectly.

Section 10:  Prescription is Incorrect

In 100 patients within your organization, how often “prescription is incorrect?”

In 100 patients for whom prescription is incorrect, how often do these patients have a medication omission error?

What do you think could reduce the frequency of incorrect prescription?  In our last meeting you had mentioned that patients giving wrong information leads to incorrect prescriptions.

Section 11:  Patient off the Unit

In 100 patients within your organization, how often “patient is off the unit?”

In 100 patients off the unit, how often do these patients have a medication omission error?

What do you think could reduce the frequency of patients being off the unit? 

Section 12:  Patient in Boarding Unit

In 100 patients within your organization, how often “patients are in the boarding unit?”

In 100 patients in the boarding unit, how often do these patients have a medication omission error?

What do you think could reduce the frequency of patients being in the boarding unit? 

Section 13:  Patient NPO Option not Addressed

In 100 patients within your organization, how often “patient’s NPO option is not addressed?”

In 100 patients for whom patient’s NPO address was not addressed, how often do these patients have a medication omission error?

What do you think could reduce the frequency of patient’s NPO option not being addressed?

Section 14:  Medication Held for Proceeding

In 100 patients within your organization, how often “medication held for proceedings?”

In 100 patients for whom medication is held for proceedings, how often do these patients have a medication omission error?

What do you think could reduce the frequency of medication held for proceedings?  

Section 15:  RT Treatment Omitted

In 100 patients within your organization, how often “RT treatment was omitted?”

In 100 patients for whom RT treatment was omitted, how often do these patients have a medication omission error?

What do you think could reduce the frequency of RT treatment being omitted?  In our last meeting you had mentioned that one factor was no RT consult being in the computer.

Section 16:  Ambiguous Dose/Numbers

In 100 patients within your organization, how often “medication orders have ambiguous dose or numbers?”

In 100 patients for whom medication orders have ambiguous dose or numbers, how often do these patients have a medication omission error?

What do you think could reduce the frequency of ambiguous dose or numbers?  In our last meeting you had mentioned that new computer cut and past option might be a factor.

This ends our survey  Please do not forget to send this survey back to us today or tomorrow so that we can collate the ideas in the survey and bring it to the meeting.

Results of the survey

The safety team met and responded to above questions, first individually and then collectively.  The Table below shows the average responses:
 

Causes of Medication
Omission Error

Priority

Prevalence of the cause*Probability of omission error given presence of the cause = Probability of medication omission

Ways to reduce omission errors due to this cause
Order not taken off chart 0.04*0.76=0.03
  1. Chart check at end of shift
  2. One designated person to check all charts
  3. Automated process for order entry
  4. Computerized physician order entry
  5. Legibility audit process
  6. Consistent process for signing off orders
Failure to reconcile physician order  0.09*0.68=0.06
  1. NCR forms (eliminate faxing)
  2. Computerized physician order entry
  3. Rcopia implementation
  4. Weekend pharmacy staffing
  5. Improve communication between pharmacist and nurse
  6. Automate order entry process to create less burdensome process
  7. Legibility checks
  8. Outgoing nurse to check with incoming nurse
  9. Consistent process for signing off orders
Misplaced medication order 0.03 x 0.65 = 0.02
  1. Order must be written in actual chart not on separate order sheet not already in the chart
  2. NCR forms
  3. Consistent order processing
  4. Computerized physician order entry
  5. Eliminate faxing orders
  6. Fax order to pharmacy immediately
  7. Orders should be placed in designated areas for pharmacy pick up
  8. Verify receipt by pharmacy
  9. RCopia implementation
  10. weekend pharmacist on-site
Incorrect administration of IV 0.04 x 0.46 = 0.02
  1. Education for RNs and attention to detail when giving IV medication
  2. Safety teams to improve attention to detail
  3. Check drug against order
  4. Follow up
  5. Piggy back labeled incorrectly
  6. Nurse verification by reading, pharmacy label and bag label
  7. Correct use of mini-bag system,
  8. Nurse confirm IV
  9. Minimize distraction
  10. Legibility audits
Order not written 0.03 x 0.29 = 0.01
  1. Reinforce to staff about writing orders at time of receiving telephone orders
  2. Do not take verbal orders
  3. Education for verbal order process for RNs and MDs
  4. Remove meds from override list in pyxis so that nurse cannot remove med without an order being entered into system
  5. Policy to not take verbal orders except for when MD scribbled
  6. MD education
Computer related errors 0.06 x 0.30 = 0.02
  1. Friendlier computer system
  2. Have a back up plan ready and everyone knows how to use it
  3. Improve downtime process and entry of meds after system back up
  4. Reinforce with nurses that orders must be acknowledge in computer before administering medication
  5. More attention to detail by person entering order into computer
  6. Safeguards built into software
Pharmacy order delayed 0.09 x 0.41 =0.04
  1. Verification by nursing staff before a certain time frame if order does not appear in MAR
  2. Orders to pharmacy in timely fashion
  3. Consistency in order process
  4. More pharmacy staff when census is up
  5. Medications arriving on the unit in a timely manner
  6. Consistent communication between RN and Pharmacy
  7. Computer reminders for meds not received
  8. Speed up order entry process
  9. Increase pharmacy staff on weekends and holidays
  10. Examine process efficiencies in pharmacy to prioritize tasks
Pharmacist did not see the order 0.06 x 0.63 = 0.04
  1. Nurse verification of order on chart
  2. Orders consistent fashion to pharmacy
  3. No phone orders to pharmacy
  4. Call pharmacy and follow up in a timely manner
  5. Clinical staff pharmacist on each unit
  6. Automate order entry process
  7. Standardize paper process to using all NCR forms
  8. Shift chart check
  9. Prioritization of pharmacy tasks
Pharmacy order written incorrectly 0.06 x 0.36 = 0.02
  1. Rcopia
  2. Computerized physician order entry
  3. No verbal orders
  4. Comply with TOR process
  5. Chart check process
  6. Computerized physician order entry
  7. Awareness training of the effect of what is being written
  8. Pharmacist to double check their own orders or have another pharmacist do another pharmacists order entry
  9. Telephone orders should be repeated back to the physician
  10. Change physical location of pharmacist on unit to avoid distraction
  11. Educate staff to avoid distracting pharmacist while writing orders
  12. More attention to detail by pharmacy staff
  13. Increase staffing in pharmacy
Prescription is Incorrect 0.09 x 0.33 = 0.03
  1. Meds (spelled correctly) and doses confirmed by pharmacy
  2. Computerized physician order entry
  3. Add pharmacist to other areas of the hospital such as ED, OR
  4. Have pharmacy check all prescriptions
  5. Rcopia
  6. Having pharmacist more involved
  7. Medication reconciliation form filled out correctly
  8. Ability to get information from local pharmacies online
  9. Computerized health histories with patient pharmacist in ER
  10. Utilize peneral methods to verify patient medication information
Patient off the unit 0.31 x 0.44 = 0.14
  1. Create a better system to remind nurses to go back and give med
  2. Group testing when possible to avoid frequency of off-unit time
  3. More bedside procedures
  4. Group procedure together
  5. Portable films
  6. Give meds ASAP when patient is back
  7. Communication with other departments that patient has traveled to
  8. Improve scheduling of test
  9. Minimize number of times patient must leave unit
Patient in Boarding Unit 0.08 x 0.33 = 0.03
  1. Continue to educate nurses who are taking care of boarding patients to use EMR to document med administration
  2. Have nurses that work the floor and know the computer to work boarding unit not nurses that don’t
  3. Quicker through put of patients
  4. Reduce LOS
  5. Have a designated boarding area with nursing personnel assigned
  6. More beds available
  7. Improve patient flow process
  8. Expand unit size to accommodate increased population
  9. Educate staff on process frequently
Patient NPO option not addressed 0.07 x 0.29 = 0.02
  1. Re-educate staff on procedure that require the patient to be NPO
  2. Educate nurses and pharmacy to seek clarification on NPO orders
  3. Design process to re-evaluate all medications when patient is again eating so that oral meds can be resumed
  4. Communication of report
  5. Daily rounds
  6. Clear physician orders
  7. Better communication from secretary to RN when patient resume from test and or NPO orders rescinded
  8. Notify pharmacy ASAP
  9. Training for better communication between dietician and nurse
Medication Held for Procedure 0.31 x 0.24 = 0.08
  1. All orders must be rewritten completely
  2. Training for better communication when procedure completed
  3. Re-educate staff to frequently recheck MARs
  4. Less procedure
  5. Reduce unnecessary procedures
  6. Education to order specifically to either hold or give meds when patient NPO for procedure
  7. Pharmacy flagging medications in Meditech so that nursing staff will review
  8. Improved scheduling of procedures
RT treatment omitted 0.16 x 0.89 = 0.15
  1. No longer a very bad problem, developed computerized alert which fixed the problem
  2. Educate secretarial staff to put orders in computer
  3. Redesign the process
  4. RT to chart medication when done not at the end of shift
  5. Consistent order process
  6. Nursing verification of RT
  7. Implement "flag" in medication system to ensure RT consult occurs
Ambiguous dose/numbers 0.12 x 0.42 = 0.05
  1. Computerized physician order entry
  2. Chart check
  3. Train for familiarity with appropriate doses
  4. Clear physician orders
  5. Eliminate cut and paste option on order entry
  6. Train MD for appropriate dosages
  7. Pharmacy clarification with physician
  8. Rcopia

Comparison of Survey Results to Objective Data

To check the accuracy of the survey results, we examined the last 30 medication omissions.  The analysis indicated that the daily probability of various medication of omissions were radically different from estimates obtained through survey of safety team members.  Following Table shows the results.

 

Cause of Medication Omission Subjective
Daily
Probability
Objective
Daily
Probability
(number of errors)
Subjective
Rank
Objective
 Rank
Pharmacist did not see the order 0.037 0.045 (10) 7 1
RN failed to give (no cause specified)   0.022 (5) 17 2
Incorrect administration of IV 0.019 0.013 (3) 13 3
Physician could not be reached to clarify order Not mentioned 0.013 (3) 17 4
Pharmacy order delayed 0.039 0.013 (3) 6 4
Pharmacy order written incorrectly 0.022 0.009 (2) 11 4
RN miscommunication between shifts Not mentioned 0.009 (2) 17 4
Misplaced medication order 0.018 0.004 (1) 14 8
Order not taken off chart 0.03 0.004 (1) 8 8
RN and MD miscommunication Not mentioned 0.004 (1) 17 8
RT treatment omitted 0.145 0.004 (1) 1 8
Misread labels Not mentioned 0.004 (1) 17 8
Put as given but failed to give Not mentioned 0.004 (1) 17 8
Medication held for proceeding 0.075 0.004 (1) 3 8
Patient NPO option not addressed 0.021 0 (0) 12 9
Order not written 0.008 0 (0) 16 9
Failure to reconcile physician order 0.06 0 (0) 4 9
Computer related errors 0.018 0 (0) 15 9
Prescription is incorrect 0.029 0 (0) 9 9
Patient in boarding unit 0.026 0 (0) 10 9
Ambiguous dose/numbers 0.052 0 (0) 5 9
Patient off the unit 0.137 0 (0) 2 9

These data suggest that the safety team should focus on solving the following top five priority areas:

  • Pharmacist did not see the order
  • Incorrect administration of IV
  • Physician could not be reached to clarify order
  • Pharmacy order delayed

What Do You Know?

Advanced learners like you, often need different ways of understanding a topic. Reading is just one way of understanding. Another way is through writing. When you write you not only recall what you have written but also may need to make inferences about what you have read.

  1. Interview a colleague to analyze root causes of an adverse outcome (not necessarily a sentinel event).  Make sure that you list at least 3 direct causes or constraints.  Make sure that you include the categories suggested by JCAHO.  Draw a flow chart. 

  2. Indicate what are the direct and root causes of the sentinel event.

  3. Give an example question that can check the conditional independence assumption associated with root causes.  Make sure the question is not awkward

  4. Verify all assumptions of conditional independence in your model by interacting with your expert.  Show what assumptions were checked and which assumptions were violated. 

  5. Estimate marginal and conditional probabilities by interviewing your expert.

  6. Use a software to estimate the probability of the sentinel event.

  7. Use a software to calculate the probability of sentinel event under at least 3 different scenarios (combination of causes occurring or not occurring).

  8. Ask your expert if the various estimates in questions 6 through 7 are within your expert's expectations.

  9. Calculate the prevalence of root causes for the sentinel event in your analysis.  Compare these data to JCAHO's reports on prevalence of causes of sentinel events.  Report the difference between your model assumptions and the JCAHO's data.

  10. Suggest how you would change the causal model to better accommodate your expert's insights.  Show how your root cause analysis changed as a consequence of the data you examined.

Bring your work to class on disk.  See an example by I. J. on root causes of rape on campus.   See another example on root causes of medication error by David Pattie.  See a similar analysis of prescription errors by SR.  See Hoda's root cause analysis of falls (in Arabic). 

Presentations

There are three sets of presentations for this lecture:

  1. See slides for this lecture

  2. See a video on how to set up a root cause model using Netica software.

  3. See a video on how to analyze root cause model using Netica software

Narrated slides and videos require Flash.

You can also follow the same narrated lecture on YouTube in two parts:

Part one:  introduction to the concept

 

Part two:  use of Netica software and validation of causal models

 

 

 

More


Copyright 2004 Farrokh Alemi, Ph.D. Created on 3/24/2004. Most recent revision 10/22/2011.  This page is part of the course on Risk Analysis.