George Mason University

HAP 525:
Risk Analysis in Healthcare


Introduction

These lectures start from introduction to concept of probability and take you all the way to probabilistic risk analysis. Along the way we cover concepts such as subjective probability assessment, conditional likelihood ratios, causal modeling, Bayesian networks and assessment of probabilities through time to the event. The goal is to provide you with tools you can use at work. The topics are covered without mathematical proofs but with logical reasoning to support the use of the tools. We want you to both understand the logic of probability tools and to be proficient in applying these tools to predict rare sentinel events such as wrong side surgery or security violations.

Unique Approach 

Our approach to probability and mathematics is different from most college level courses. First, we teach by application. From the very fist concepts of probability to complicated causal networks, we introduce all ideas by providing you with realistic applied situations. In our view, the problem that most people have with math is that it is too abstract. By extensive use of examples we hope to make it easier for new students to discover the utility of probabilistic modeling.

To many, formulas look esoteric and irrelevant to their everyday work life. We show how math helps track conflicting ideas and make sense of complex real situations. We hope to breath life into various concepts by showing their utility in actual practice within health care environment.

This focus on what is practical has led us to a number of changes in the content of the book. Unlike many probability courses we do not cover continuous distributions, the ideas are important but in practice most people have only access to discrete distributions. We build everything on discrete variables. Instead of saying that someone�s age could be anywhere from 0 to 100, we only deal with age groupings (e.g. under 18, above 65, etc.). The reason for this is rather simple. This is the data we have access to. To this date, we have yet to see age described as a continuous variable (e.g. 9.45 years). Of course it can be done but it is not done. This kind of simplification of data is not related to age alone. Many variables are exclusively measured on discrete levels (e.g. presence of diabetes). When we give out the notion of continuous variables and distributions, the set of tools we can use changes. As you will see, we focus a lot on contingency tables showing the relationship between two discrete variables. These are the building blocks of data readily available and this is what the course is built on.

The key to successful use of probability models is not in knowing the rules of probability but knowing the causes and effects we are observing. Rules of probabilities are just an instrument for expressing these causes and effects. For risk analysis to work, you must know or have a good idea about causes of the adverse event and be willing to use probability models to test your knowledge. To do so you need a model of reality. Effective risk analysis is built on accurate model of the various causes of the adverse event. Then it should not be surprising that these lectures spend a great deal of time helping you focus on how to model complex tasks. We show you behavioral techniques that could help you obtain estimates from an expert. We show you how to check the assumptions inherent in visual models of cause and effect. We show you how to use a mathematical model you have constructed to answer �what if� questions about the future. All of this takes us well beyond the mathematics of risk analysis and into modeling the reality you face. While these lectures seem to be about probability, it really is about building useful models of reality.

We provide a number of features to assist you in your learning.  Each lecture is followed by a series of assignments that we will complete in class. Each lecture is accompanied with a question and answer section that you can use to ask questions. Answers will be posted on the web under the lecture area. Each lecture is followed by a take home message summarizing the single lesson learned in the lecture. To assist interaction around a topic, we provide an area for commenting on the topic including the ways the presentation of a topic can be improved.

Comparison to Others

A number of other books on risk analysis are available and have been useful to us in preparation of this book. A useful book is Feldman and Valdex-Florez�s Applied Probability and Stochastic Processes. They provide a very accessible introduction to the concept of the probabilities, though they do not cover risk analysis in depth. Feldman and Valdex-Florez�s book is a good example of a series of books intended to introduce the concepts of probability with numerous examples but mostly outside its application. In contrast we introduce probability from the perspective of risk analysis and within health care applications, hopefully this applied perspective will make it more meaningful to some readers.

An excellent book on risk analysis is Quantitative Risk Analysis for Environmental and Occupational Health by William Hallenbeck. This book focuses on health effects of long term exposure to hazards. So for example, the book discusses analytical models for showing how a toxin spreads and gets absorbed by the population. This book a good example of a large number of books focused on risk analysis within environmental and occupational health. In contrast to this book, we do not cover toxicology nor is our coverage as mathematical.

Another excellent book on risk analysis is Probabilistic Risk Analysis: Foundations and Methods by Tim Bedford and Roger Cooke. This book is an excellent book on methods of risk analysis but the approach is not based on causal modeling and recent advances in probability modeling. In contrast, we use causal modeling as the foundation of our advice on how to complete risk analysis. Still another book, and one that I recommend to you is Louis Anthony Cox�s Risk Analysis: Foundations, Models and Methods. This book covers almost everything we do here. Like us it covers probability and causal modeling. Like us it is focused on health care applications. But we take slightly different focus. Our focus is on behavioral aspect of modeling cause and effects. We cover the mathematics of risk analysis to the extent needed for completing and understanding an application. We do not provide, for example, mathematical proofs. When introduce the ideas with simple calculations and use software to track complicated calculations in actual risk analysis. In contrast, we go into much depth about how do you interact with experts, how do you use language to revise models to be easier to understand, how do you interpret a risk analysis findings and present the results. All of this advice on procedures go beyond the mathematics of risk analysis. In short, we put less emphasis on mathematics and more on behavioral aspects of modeling.

Why Do Risk Analysis?

Risk assessment helps decision makers allocate resources to reduce the risk, to manage it if it occurs or to insure against it. An effective risk analysis provides an organization with insight. Some risks they can probably ignore. Other risks can be reduced with improved security steps. Think for a second about recent events. Which is worse, Hurricane Katrina or mailed biohazards. If you had to choose to protect against one hazard and not the other, which one would you choose? Risk analysis helps managers allocate their limited resources to tasks that presents the largest likelihood of occurring and causing the largest damage.

Why Probabilistic Risk Analysis?

Assuming that you are going to do a risk assessment, why not just list the risks. What is the point of conducting probabilistic analysis?

  • First and foremost, effective risk analysis requires us to estimate the relative probability of occurrence of the adverse event. Without quantification of this probability, we would not be able to compare it against other adverse events. We will end up trying to protect the organization from all risks, which is euphuism for doing nothing at all. After all, no one has the resources to protect against all risks.
  • Second, probabilistic risk analysis allows to have a consistent method of aggregating the risk of several events co-occurring. The logic of probability models can be used to combine the effect of a sequence of events. True that no one may be exactly sure about the probability of an event but at least we are sure that these probabilities are combined with each other in a consistent and logical fashion. This is a big advantage, when risk analysis has to consider a large number of events. Probability models enable us to remain consistent across these large number of events.

Criticism of Probabilistic Risk Analysis

There are three criticism of probability models.

  1. First, some believe that rare probabilities cannot be estimated accurately. To some extent they are right as large databases are needed in order to have sufficient occurrences of a rare event. But as we will see, time to the event can be used to calculate the probability of rare events. Time to an event, even rare events, is readily available and therefore can be accurately measured. It can be measured objectively or through experts� consensus opinion. Therefore, this first criticism is a red herring and not relevant in practice.
  2. Second, some argue that the probabilistic risk analysis is not practical and will divert attention and resources from the task of risk modification. Surprisingly, probabilistic risk analysis takes little time to do because it focuses on a limited set of risks. In contrast, the traditional risk analysis focuses on a comprehensive list that wastes considerable effort and ends up not providing any specific direction for change. As you will see later, probabilistic risk analysis is mathematically difficult but practical to implement and use, specially when one has access to modern software and tools.
  3. Third, some argue that history is not relevant in predicting future events. For example, they argue that terrorists will revise their strategies and strikes in ways that are not anticipated. By relying on historical patterns we will miss the novel vulnerabilities we have. We agree that we will miss some future vulnerabilities. The past patterns are not indicative of all future events. But consider what these critics propose should be done. They suggest that a comprehensive analysis be done. All vulnerabilities be identified based on experts� consensus. This sounds good but it is really a nightmare. People like to be protected from all risks and they would like to trust that their consultants are pursing such efforts. But to do so, consultants must think of what might go wrong. For example, a recent study examined what might go wrong if a tank of milk is contaminated with a bio hazard. How many people will die before it is detected. Another consultant looked at what will happen if a person infected with a biohazard would walk through metro stations. The list goes on. In the absence of historical patterns, we are subject to the imagination of the security consultant regarding what might go wrong. The more imaginative the more fearful the scenario. In these circumstances, organizations like little children will be fighting against endless imaginary foes. Each organization remains as vulnerable as the imagination of their consultant. The more vivid this imagination, the more creative the consultant, the more the organization is in fear. This is not a recipe for action but for paralysis and constant fear. Consider the alternative we propose, to look at what has happened to date and to predict what might occur. Surely, we will miss some risks but over time we get better and we do not need to live in constant fear. To us, relying on data seems the only sane way out of a perpetual imaginary fear of what might happen.

Short History of Risk Analysis

In recent years, there have been many occasions in which risks of rare events have been assessed and subsequent events have helped confirm the accuracy of the risk analysis or improve aspects of the analysis.  Probabilistic risk analysis originated in aerospace industry.  One of the earliest comprehensive studies was started after the loss of life due to a fire in Apollo flight AS-204 in 1967.  In 1969, the Space Shuttle Task Group in the Office of Manned Space Flight of NASA suggested that the probability of loss of life should be less than 1 percent.  Colglazier and Weatherwax (1983) conducted a probabilistic risk analysis of shuttle flights.  But overtime, NASA administrators abandoned numerical forecast of risks as the projected risks were so high to undermine the entire viability of the operations.  Cooke (1991) and Bell and Esch (1989) report that NASA administrators "felt that the numbers could do irreparable harm."  But subsequent shuttle accidents returned the emphasis to probabilistic risk analysis.  Today almost all components of space shuttle go through independent risk analysis (Safaie 1991, 1992, 1994; Hoffman 1998; Planning Research Corporation, 1989, Science Applications International Corporation, 1993, 1995).  A good example of such risk analysis can be found in the work of Pate-Cornell and Fischbeck (1993, 1994), where they assessed the risk of tiles breaking away from the shuttle.  In this award winning study, the authors linked management practices to risks of various tiles on the shuttle.

In nuclear safety, several studies have focused on reactor safety.  The first such study was the Reactor Safety Study (1975).  The study was followed by a series of critical reviews (Environmental Protection Agency, 1976; Union of Concerned Scientists, 1977, American Physical Society, 1975), including in 1997 a Congressional bill to mandate a review panel  to examine the limitations of the study.  The near failure of reactor core at Three Miles Island, however, proved that the scenarios anticipated in the study were indeed correct, though the probability of human failures were underestimated.   Not surprisingly, reviews of Three Miles Island re-emphasized the need for conducting probabilistic risk analysis (Rogovin and Frampton, 1980; Kemeny et al. 1979).  Kaplan and Garrick (1981) conducted a study of probability of reactor melt down.  In 1983, the U.S. Nuclear Regulation Commission issued a manual for how to conduct Probabilistic Risk Analysis.     Probabilistic risk analysis has also been used by the energy firms not focused on nuclear power to predict catastrophic events (Cooke, Jager 1998; Rasmussen, 1981; Ortwin, 1998)

Probabilistic risk analysis has been applied to a variety of natural disasters including earthquake predictions (Chang, Shinozuka, Moore 2000), predicting floods and coastal designs (Voortman, van Gelder, Vrijling 2002; Mai, Zimmermann, 2003; Kaczmarek 2003 ), environmental pollution (Slob, Pieters 1998; Moore, Sample, Suter, Parkhurst, Scott, 1999).  A large number of studies focus on waste disposal and environmental health (Ewing, Palenik, Konikow 2004; Sadiq, Husain, Veitch, Bose. 2003; Cohen 2003; Garrick, Kaplan 1999).   In health care probabilistic risk analysis has focused on analysis of root causes of sentinel adverse events such as wrong side surgery or failure mode and effect analysis of near catastrophic events (Bonnabry, et. al 2005).   Amgen pharmaceutical has also used the procedure for deciding on new product development (Keefeer, 2001).  In failure mode analysis within health care most often the rank order of rare probabilities are assessed and the magnitude of the probability is ignored (DeRosier, Stalhandske, Bagian, Nudell 2002). 

The application to terrorism is new.  Taylor, Krings and Alves-Foss (2002) have applied probabilistic risk analysis to assessment of cyber terrorism risks.  Others have suggested the use of these techniques in assessment of terrorism ( Apostolakis, Lemon 2005; Haimes, Longstaff 2002).

Requirements & Expectations

These lectures assume that you know algebra, though not calculus. They assume that you are facile with numbers and counting. They assume that you have access to a software for Bayesian analysis (Netica) and that you can after a brief training use the software. They assume that you can use Excel or other software to make contingency tables. No prior course in probability is required. No course in statistics is required. No computer programming is needed. Some knowledge of health care systems is required and you do need to have access to an organization to try out your ideas. But these can also be arranged while you go through the course.

You are expected to role play probabilistic risk analysis during the class sessions. You will be presented with problems and ask to solve them and learn by doing them. Later you are expected to apply the methods learned in the course to a realistic problem and help evaluate the effort of others in the class. There are no exams, no other assignments. Just a series of role playing exercises followed by a month long field project.

Intended Audience

These lectures are organized for students in graduate programs taking a first course on risk analysis, vulnerability assessment or security analysis. Professionals in the field may also find the material useful as a way of improving their skills and increasing quantification of risks.  These lectures may also be of use to students of probability and causal modeling.

This web site is provided as a free service to the Internet community.  You are encouraged to link to this site and to use this site in your own work.

What Do You Know?

Here are some questions you should be able to answer based on this lecture:

  • Why is probabilistic risk analysis preferred to comprehensive lists of vulnerabilities?
  • How is our approach to risk analysis different from other books on probability or on risk analysis?
  • What is assumed and required prior to start of these lectures?
  • What is expected from you prior to end of this course?

References

American Physical Society, Study group on light water reactor safety: Report tot he American Physical Society, Review of Modern Physicians Vol. 47, Supplemental No. 1, 1975.

Apostolakis GE, Lemon DM. A Screening Methodology for the Identification and Ranking of Infrastructure Vulnerabilities Due to Terrorism. Risk Analysis 2005, 25:2, 361-376

Bell TE, Esch K. The space shuttle: A case study of subjective engineering. IEEE Spectrum, 1989, 42-46.

Bonnabry P, Cingria L, Sadeghipour F, Ing H, Fonzo-Christe C, Pfister RE. Use of a systematic risk analysis method to improve safety in the production of paediatric parenteral nutrition solutions. Qual Saf Health Care. 2005 Apr;14(2):93-8.

Catrambone R., Beike D., Niedenthal P. (1996) Is the self-concept a habitual referent in judgments of similarity? Psychological Science; 7 (3): 158-163.

Chang SE, Shinozuka M, Moore JE. Probabilistic Earthquake Scenarios: Extending Risk Analysis Methodologies to Spatially Distributed Systems. Earthquake Spectra, 2000, 16: 3, pp. 557-572.

Cohen BL. Probabilistic risk analysis for a high-level radioactive waste repository. Risk Anal. 2003 Oct;23(5):909-15.

Colglazier EW, Weatherwax RK. Failure estimates for the space shuttle. Abstracts for Society for Risk Analysis Annual Meeting 1986, Boston MA, p 80, Nov 9-12, 1986.

Cooke R, Jager E. A probabilistic model for the failure frequency of underground gas pipelines. Risk Anal. 1998 Aug;18(4):511-27.

Cooke RM. Experts in uncertainty: Opinion and subjective probability in science, Oxford university Press, New York, 1991.

DeRosier J, Stalhandske E, Bagian JP, Nudell T. Using health care Failure Mode and Effect Analysis: the VA National Center for Patient Safety's prospective risk analysis system. Jt Comm J Qual Improv. 2002 May;28(5):248-67, 209.

Environmental Protection Agency, Reactor Safety Study Oversight Hearings Before the Subcommittee on Energy and the Environment of the Committee on Interior and Insular Affairs, House of Representatives, 94th Congress, Second Session, Serial No. 84-61, Washington DC, June 11, 1976.

Ewing RC, Palenik CS, Konikow LF. Comment on "Probabilistic risk analysis for a high-level radioactive waste repository" by B. L. Cohen in Risk Analysis, volume 23, 909-915. Risk Anal. 2004 Dec;24(6):1417-1419.

Fox EP. SSME Alternate Turbopump Development Program—Probabilistic Failure Methodology Interim Report. FR-20904-02, 1990.

Garrick BJ, Kaplan S. A decision theory perspective on the disposal of high-level radioactive waste. Risk Anal. 1999 Oct;19(5):903-13.

Glynn PW, Iglehart DL. Importance sampling for stochastic simulations. Management Science 35: 11 (November 1989), 1367 - 1392.

Haimes YY, Longstaff T. The Role of Risk Analysis in the Protection of Critical Infrastructures Against Terrorism. Risk Analysis, 2002, 22:3, pp. 439-444.

Heidelberger P. Fast simulation of rare events in queuing and reliability models. ACM Transactions on Modeling and Computer Simulation (TOMACS) archive 5: 1 43 - 85, 1995

Hoffman CR, Pugh R, Safie FM. Methods and Techniques for Risk Prediction of Space Shuttle Upgrades. AIAA, 1998

Kaczmarek Z. The impact of climate variability on flood risk in Poland. Risk Anal. 2003 Jun;23(3):559-66.

Kaplan S, Garrick B. On the quantitative definition of risk. Risk Analysis, 1981, 1: page 11-27.

Keefeer DL. Practice abstract. Interfaces 31: 5, 2001, pp 62-64.

Kemeny J. Report of the President's Commission on the Accident at Three Mile Island, Washington DC, 1979.

Krouwer JS. Managing Risk In Hospitals Using Integrated Fault Trees And Failure Mode Effects And Criticality Analysis. AACC Press, 2004.

Marx DA, Slonim AD. Assessing patient safety risk before the injury occurs: an introduction to sociotechnical probabilistic risk modelling in health care. Qual Saf Health Care. 2003 Dec;12 Suppl 2

Mai S, Zimmermann C. Risk Analysis-Tool for Integrated Coastal Planning. Proc. of the 6th Int. Conf. on Coastal and Port Engineering, 2003.

Mobus C. (1979) The analysis of non-symmetric similarity judgments: Drift model, comparison hypothesis, Tversky's contrast model and his focus hypothesis. Archiv Fur Psychologie; 131 (2): 105-136.

Moore, DRJ, Sample BE, Suter GW, Parkhurst BR, Scott TR. A Probabilistic risk assessment of the effects of Methylmercury and PCBs on mink and Kingfishers along East Fork Poplar Creek, Oak Ridge, Tennessee, USA. Environmental Toxicology and Chemistry, 18: 12, pp. 2941-2953, 1999.

Ortwin R. Three decades of risk research: accomplishments and new challenges. Journal of Risk Research, 1998, 1:1 pp 49 - 71.

Pate-Cornell ME, Fischbeck PS. Probabilistic Risk Analysis and Risk--Based Priority Scale for the Tiles of the Space Shuttle. Reliability Engineering and System Safety. Vol. 40, no. 3, pp. 221-238. 1993.

Pate-Cornell ME, Fischbeck PS. Risk management for tiles of the space shuttle. Interfaces, 1994, 24: 1, pp 64-86.

Planning Research Corporation, Independent Assessment of Shuttle Accident Scenario Probabilities for Galileo Mission and Comparison with NSTS Program Assessment, 1989.

Rogovin M, Frampton GT. Three Mile Island, a Report to the Commissioners and to the Public, Government Printing Office, 1980.

Rasmussen NC. The Application of Probabilistic Risk Assessment Techniques to Energy Technologies. Annual Review of Energy, 6: 123-138, 1981.

Sadiq R, Husain T, Veitch B, Bose N. Distribution of arsenic and copper in sediment pore water: an ecological risk assessment case study for offshore drilling waste discharges. Risk Anal. 2003 Dec;23(6):1309-21.

Safie FM. A Statistical Approach for Risk Management of Space Shuttle Main Engine Components. Probabilistic Safety Assessment and Management, 1991

Safie FM. A Risk Assessment Methodology for the Space Shuttle External Tank Welds. Reliability and Maintainability Symposium, 1994.

Safie FM, Fox EP. A Probabilistic Design Analysis Approach for Launch Systems. AIAA/SAE/ASME 27th Joint Propulsion Conference, 1991.

Safie FM. Use of Probabilistic Design Methods for NASA Applications. ASME Symposium on Reliability Technology, 1992.

Siegel P.S., McCord D. M., Crawford A. R. (1982) An experimental note on Tversky's features of similarity. Bulletin of Psychonomic Society; 19 (3): 141-142.

Schwarz G, Tversky A. (1980) On the reciprocity of proximity relations. Journal of Mathematical Psychology; 22 (3): 157-175.

Science Applications International Corporation, Probabilistic Risk Assessment of the Space Shuttle Phase 1: Space Shuttle Catastrophic Failure Frequency Final Report, 1993.

Science Applications International Corporation, Probabilistic Risk Assessment of the Space Shuttle, 1995.

Slob W, Pieters MN. A probabilistic approach for deriving acceptable human intake limits and human health risks from toxicological studies: general framework. Risk Anal. 1998 Dec;18(6):787-98.

Srinivasan R. Importance Sampling. Springer, 2002.

Taylor C, Krings A, Alves-Foss J. Risk Analysis and Probabilistic Survivability Assessment (RAPSA): An Assessment Approach for Power Substation Hardening.� @ Proc. ACM Workshop on Scientific Aspects of Cyber Terrorism, 2002.

Tversky A. (1977) Features of similarity. Psychological Review; 84 (4): 327-352.

Union of Concerned Scientists. The risk of nuclear power reactors: a review of the NRC reactor study, WASH-1400, 1977.

U.S. NRC, Reactor Safety study. U.S. Nuclear Regulatory Commission, WASH-1400, NUREG-751014, 1975.

U.S. NRC, PRA Procedures Guide, U.S. Nuclear Regulatory Commission, NUREG/CR-2300, 1983.

Voortman HG, van Gelder P, Vrijling JK Risk-based design of large-scale flood defense systems. 28th International Conference on Coastal Engineering, 2002.

Presentations

 

To assist you in reviewing the material in this lecture, please see the following resources:

  1. See the slides for the lecture

Narrated lectures require use of Flash.


Copyright © 2006 by Farrokh Alemi, Ph.D.  Created on Tuesday October 4th, 2006.  Most recent revision 10/22/2011. This page is part of a Course on Risk Analysis