|
|
These lectures start from introduction to concept of probability and take you
all the way to probabilistic risk analysis. Along the way we cover concepts such
as subjective probability assessment, conditional likelihood ratios, causal
modeling, Bayesian networks and assessment of probabilities through time to the
event. The goal is to provide you with tools you can use at work. The topics are
covered without mathematical proofs but with logical reasoning to support the
use of the tools. We want you to both understand the logic of probability tools
and to be proficient in applying these tools to predict rare sentinel events
such as wrong side surgery or security violations.
Our approach to probability and mathematics is different from most college
level courses. First, we teach by application. From the very fist concepts of
probability to complicated causal networks, we introduce all ideas by providing
you with realistic applied situations. In our view, the problem that most people
have with math is that it is too abstract. By extensive use of examples we hope
to make it easier for new students to discover the utility of probabilistic
modeling.
To many, formulas look esoteric and irrelevant to their everyday work life.
We show how math helps track conflicting ideas and make sense of complex real
situations. We hope to breath life into various concepts by showing their
utility in actual practice within health care environment.
This focus on what is practical has led us to a number of changes in the
content of the book. Unlike many probability courses we do not cover continuous
distributions, the ideas are important but in practice most people have only
access to discrete distributions. We build everything on discrete variables.
Instead of saying that someone�s age could be anywhere from 0 to 100, we only
deal with age groupings (e.g. under 18, above 65, etc.). The reason for this is
rather simple. This is the data we have access to. To this date, we have yet to
see age described as a continuous variable (e.g. 9.45 years). Of course it can
be done but it is not done. This kind of simplification of data is not related
to age alone. Many variables are exclusively measured on discrete levels (e.g.
presence of diabetes). When we give out the notion of continuous variables and
distributions, the set of tools we can use changes. As you will see, we focus a
lot on contingency tables showing the relationship between two discrete
variables. These are the building blocks of data readily available and this is
what the course is built on.
The key to successful use of probability models is not in knowing the rules
of probability but knowing the causes and effects we are observing. Rules of
probabilities are just an instrument for expressing these causes and effects.
For risk analysis to work, you must know or have a good idea about causes of the
adverse event and be willing to use probability models to test your knowledge.
To do so you need a model of reality. Effective risk analysis is built on
accurate model of the various causes of the adverse event. Then it should not be
surprising that these lectures spend a great deal of time helping you focus on
how to model complex tasks. We show you behavioral techniques that could help
you obtain estimates from an expert. We show you how to check the assumptions
inherent in visual models of cause and effect. We show you how to use a
mathematical model you have constructed to answer �what if� questions about the
future. All of this takes us well beyond the mathematics of risk analysis and
into modeling the reality you face. While these lectures seem to be about
probability, it really is about building useful models of reality.
We provide a number of features to assist you in your learning. Each
lecture is followed by a series of assignments that we will complete in class.
Each lecture is accompanied with a question and answer section that you can use
to ask questions. Answers will be posted on the web under the lecture area. Each
lecture is followed by a take home message summarizing the single lesson learned
in the lecture. To assist interaction around a topic, we provide an area for
commenting on the topic including the ways the presentation of a topic can be
improved.
A number of other books on risk analysis are available and have been useful
to us in preparation of this book. A useful book is Feldman and Valdex-Florez�s
Applied Probability and Stochastic Processes. They provide a very accessible
introduction to the concept of the probabilities, though they do not cover risk
analysis in depth. Feldman and Valdex-Florez�s book is a good example of a
series of books intended to introduce the concepts of probability with numerous
examples but mostly outside its application. In contrast we introduce
probability from the perspective of risk analysis and within health care
applications, hopefully this applied perspective will make it more meaningful to
some readers.
An excellent book on risk analysis is Quantitative Risk Analysis for
Environmental and Occupational Health by William Hallenbeck. This book focuses
on health effects of long term exposure to hazards. So for example, the book
discusses analytical models for showing how a toxin spreads and gets absorbed by
the population. This book a good example of a large number of books focused on
risk analysis within environmental and occupational health. In contrast to this
book, we do not cover toxicology nor is our coverage as mathematical.
Another excellent book on risk analysis is Probabilistic Risk Analysis:
Foundations and Methods by Tim Bedford and Roger Cooke. This book is an
excellent book on methods of risk analysis but the approach is not based on
causal modeling and recent advances in probability modeling. In contrast, we use
causal modeling as the foundation of our advice on how to complete risk
analysis. Still another book, and one that I recommend to you is Louis Anthony
Cox�s Risk Analysis: Foundations, Models and Methods. This book covers almost
everything we do here. Like us it covers probability and causal modeling. Like
us it is focused on health care applications. But we take slightly different
focus. Our focus is on behavioral aspect of modeling cause and effects. We cover
the mathematics of risk analysis to the extent needed for completing and
understanding an application. We do not provide, for example, mathematical
proofs. When introduce the ideas with simple calculations and use software to
track complicated calculations in actual risk analysis. In contrast, we go into
much depth about how do you interact with experts, how do you use language to
revise models to be easier to understand, how do you interpret a risk analysis
findings and present the results. All of this advice on procedures go beyond the
mathematics of risk analysis. In short, we put less emphasis on mathematics and
more on behavioral aspects of modeling.
Risk assessment helps decision makers allocate resources to reduce the risk,
to manage it if it occurs or to insure against it. An effective risk analysis
provides an organization with insight. Some risks they can probably ignore.
Other risks can be reduced with improved security steps. Think for a second
about recent events. Which is worse, Hurricane Katrina or mailed biohazards. If
you had to choose to protect against one hazard and not the other, which one
would you choose? Risk analysis helps managers allocate their limited resources
to tasks that presents the largest likelihood of occurring and causing the
largest damage.
Assuming that you are going to do a risk assessment, why not just list the
risks. What is the point of conducting probabilistic analysis?
- First and foremost, effective risk analysis requires us to estimate the
relative probability of occurrence of the adverse event. Without
quantification of this probability, we would not be able to compare it
against other adverse events. We will end up trying to protect the
organization from all risks, which is euphuism for doing nothing at all.
After all, no one has the resources to protect against all risks.
- Second, probabilistic risk analysis allows to have a consistent method
of aggregating the risk of several events co-occurring. The logic of
probability models can be used to combine the effect of a sequence of
events. True that no one may be exactly sure about the probability of an
event but at least we are sure that these probabilities are combined with
each other in a consistent and logical fashion. This is a big advantage,
when risk analysis has to consider a large number of events. Probability
models enable us to remain consistent across these large number of events.
There are three criticism of probability models.
- First, some believe that rare probabilities cannot be estimated
accurately. To some extent they are right as large databases are needed in
order to have sufficient occurrences of a rare event. But as we will see,
time to the event can be used to calculate the probability of rare events.
Time to an event, even rare events, is readily available and therefore can
be accurately measured. It can be measured objectively or through experts�
consensus opinion. Therefore, this first criticism is a red herring and not
relevant in practice.
- Second, some argue that the probabilistic risk analysis is not practical
and will divert attention and resources from the task of risk modification.
Surprisingly, probabilistic risk analysis takes little time to do because it
focuses on a limited set of risks. In contrast, the traditional risk
analysis focuses on a comprehensive list that wastes considerable effort and
ends up not providing any specific direction for change. As you will see
later, probabilistic risk analysis is mathematically difficult but practical
to implement and use, specially when one has access to modern software and
tools.
- Third, some argue that history is not relevant in predicting future
events. For example, they argue that terrorists will revise their strategies
and strikes in ways that are not anticipated. By relying on historical
patterns we will miss the novel vulnerabilities we have. We agree that we
will miss some future vulnerabilities. The past patterns are not indicative
of all future events. But consider what these critics propose should be
done. They suggest that a comprehensive analysis be done. All
vulnerabilities be identified based on experts� consensus. This sounds good
but it is really a nightmare. People like to be protected from all risks and
they would like to trust that their consultants are pursing such efforts.
But to do so, consultants must think of what might go wrong. For example, a
recent study examined what might go wrong if a tank of milk is contaminated
with a bio hazard. How many people will die before it is detected. Another
consultant looked at what will happen if a person infected with a biohazard
would walk through metro stations. The list goes on. In the absence of
historical patterns, we are subject to the imagination of the security
consultant regarding what might go wrong. The more imaginative the more
fearful the scenario. In these circumstances, organizations like little
children will be fighting against endless imaginary foes. Each organization
remains as vulnerable as the imagination of their consultant. The more vivid
this imagination, the more creative the consultant, the more the
organization is in fear. This is not a recipe for action but for paralysis
and constant fear. Consider the alternative we propose, to look at what has
happened to date and to predict what might occur. Surely, we will miss some
risks but over time we get better and we do not need to live in constant
fear. To us, relying on data seems the only sane way out of a perpetual
imaginary fear of what might happen.
In recent years, there have been many occasions in which risks of rare events have been assessed and subsequent events have helped confirm the accuracy of the risk analysis or improve aspects of the analysis. Probabilistic risk analysis originated in aerospace industry. One of the earliest comprehensive studies was started after the loss of life due to a fire in Apollo flight AS-204 in 1967. In 1969, the Space Shuttle Task Group in the Office of Manned Space Flight of NASA suggested that the probability of loss of life should be less than 1 percent. Colglazier and Weatherwax (1983)
conducted a probabilistic risk analysis of shuttle flights. But overtime, NASA administrators abandoned numerical forecast of risks as the projected risks were so high to undermine the entire viability of the operations. Cooke (1991) and Bell and Esch (1989) report that NASA administrators "felt that the numbers could do irreparable harm." But subsequent shuttle accidents returned the emphasis to probabilistic risk analysis. Today almost all components of space shuttle go through independent risk analysis (Safaie 1991, 1992, 1994; Hoffman 1998; Planning Research Corporation, 1989, Science Applications International Corporation, 1993, 1995). A good example of such risk analysis can be found in the work of Pate-Cornell and Fischbeck (1993, 1994), where they assessed the risk of tiles breaking away from the shuttle. In this award winning study, the authors linked management practices to risks of various tiles on the
shuttle.
In nuclear safety, several studies have focused on reactor safety. The first such study was the Reactor Safety Study (1975). The study was followed by a series of critical reviews (Environmental Protection Agency, 1976; Union of Concerned Scientists, 1977, American Physical Society, 1975), including in 1997 a Congressional bill to mandate a review panel to examine the limitations of the study. The near failure of reactor core at Three Miles Island, however, proved that the scenarios anticipated in the study were indeed correct, though the probability of human failures were underestimated. Not surprisingly, reviews of Three Miles Island re-emphasized the need for conducting probabilistic risk analysis (Rogovin and Frampton, 1980; Kemeny et al. 1979). Kaplan and Garrick (1981) conducted a study of probability of reactor melt down. In 1983, the U.S. Nuclear Regulation Commission
issued a manual for how to conduct Probabilistic Risk Analysis. Probabilistic risk analysis has also been used by the energy firms not focused on nuclear power to predict catastrophic events (Cooke, Jager 1998; Rasmussen, 1981; Ortwin,
1998)
Probabilistic risk analysis has been applied to a variety of natural disasters including earthquake predictions (Chang, Shinozuka, Moore 2000), predicting floods and coastal designs (Voortman, van Gelder, Vrijling 2002; Mai, Zimmermann, 2003; Kaczmarek 2003 ), environmental pollution (Slob, Pieters 1998; Moore, Sample, Suter, Parkhurst, Scott, 1999). A large number of studies focus on waste disposal and environmental health (Ewing, Palenik, Konikow 2004; Sadiq, Husain, Veitch, Bose. 2003; Cohen 2003; Garrick, Kaplan 1999). In health care probabilistic risk analysis has focused on analysis of root causes of sentinel adverse events such as wrong side surgery or failure mode and effect analysis of near catastrophic events (Bonnabry, et. al 2005). Amgen pharmaceutical has also used the procedure for deciding on new product development (Keefeer, 2001). In failure mode analysis within health care most often the rank order of rare probabilities are assessed and the magnitude of the probability is ignored (DeRosier, Stalhandske, Bagian, Nudell
2002).
The application to terrorism is new. Taylor, Krings and Alves-Foss (2002) have applied probabilistic risk analysis to assessment of cyber terrorism risks. Others have suggested the use of these techniques in assessment of terrorism ( Apostolakis, Lemon 2005; Haimes, Longstaff
2002).
These lectures assume that you know algebra, though not calculus. They assume
that you are facile with numbers and counting. They assume that you have access
to a software for Bayesian analysis (Netica) and that you can after a brief
training use the software. They assume that you can use Excel or other software
to make contingency tables. No prior course in probability is required. No
course in statistics is required. No computer programming is needed. Some
knowledge of health care systems is required and you do need to have access to
an organization to try out your ideas. But these can also be arranged while you
go through the course.
You are expected to role play probabilistic risk analysis during the class
sessions. You will be presented with problems and ask to solve them and learn by
doing them. Later you are expected to apply the methods learned in the course to
a realistic problem and help evaluate the effort of others in the class. There
are no exams, no other assignments. Just a series of role playing exercises
followed by a month long field project.
These lectures are organized for students in graduate programs taking a first
course on risk analysis, vulnerability assessment or security analysis.
Professionals in the field may also find the material useful as a way of
improving their skills and increasing quantification of risks. These
lectures may also be of use to students of probability and causal modeling.
This web site is provided as a free service to the Internet community.
You are encouraged to link to this site and to use this site in your own work.
Here are some questions you should be able to answer based on this lecture:
- Why is probabilistic risk analysis preferred to comprehensive lists of
vulnerabilities?
- How is our approach to risk analysis different from other books on
probability or on risk analysis?
- What is assumed and required prior to start of these lectures?
- What is expected from you prior to end of this course?
American Physical Society, Study group on light water reactor safety: Report tot he American Physical Society, Review of Modern Physicians Vol. 47, Supplemental No. 1,
1975.
Apostolakis GE, Lemon DM. A Screening Methodology for the Identification and Ranking of Infrastructure Vulnerabilities Due to Terrorism. Risk Analysis 2005, 25:2,
361-376
Bell TE, Esch K. The space shuttle: A case study of subjective engineering. IEEE Spectrum, 1989,
42-46.
Bonnabry P, Cingria L, Sadeghipour F, Ing H, Fonzo-Christe C, Pfister RE. Use of a systematic risk analysis method to improve safety in the production of paediatric parenteral nutrition solutions. Qual Saf Health Care. 2005
Apr;14(2):93-8.
Catrambone R., Beike D., Niedenthal P. (1996) Is the self-concept a habitual referent in judgments of similarity? Psychological Science; 7 (3):
158-163.
Chang SE, Shinozuka M, Moore JE. Probabilistic Earthquake Scenarios: Extending Risk Analysis Methodologies to Spatially Distributed
Systems. Earthquake Spectra, 2000, 16: 3, pp. 557-572.
Cohen BL. Probabilistic risk analysis for a high-level radioactive waste repository. Risk Anal. 2003
Oct;23(5):909-15.
Colglazier EW, Weatherwax RK. Failure estimates for the space shuttle. Abstracts for Society for Risk Analysis Annual Meeting 1986, Boston MA, p 80, Nov 9-12,
1986.
Cooke R, Jager E. A probabilistic model for the failure frequency of underground gas pipelines. Risk Anal. 1998
Aug;18(4):511-27.
Cooke RM. Experts in uncertainty: Opinion and subjective probability in science, Oxford university Press, New York,
1991.
DeRosier J, Stalhandske E, Bagian JP, Nudell T. Using health care Failure Mode and Effect Analysis: the VA National Center for Patient Safety's prospective risk analysis system. Jt Comm J Qual Improv. 2002 May;28(5):248-67,
209.
Environmental Protection Agency, Reactor Safety Study Oversight Hearings Before the Subcommittee on Energy and the Environment of the Committee on Interior and Insular Affairs, House of Representatives, 94th Congress, Second Session, Serial No. 84-61, Washington DC, June 11,
1976.
Ewing RC, Palenik CS, Konikow LF. Comment on "Probabilistic risk analysis
for a high-level radioactive waste repository" by B. L. Cohen in Risk Analysis, volume 23, 909-915. Risk Anal. 2004
Dec;24(6):1417-1419.
Fox EP. SSME Alternate Turbopump Development Program—Probabilistic Failure Methodology Interim Report. FR-20904-02,
1990.
Garrick BJ, Kaplan S. A decision theory perspective on the disposal of high-level radioactive waste. Risk Anal. 1999
Oct;19(5):903-13.
Glynn PW, Iglehart DL. Importance sampling for stochastic simulations. Management Science 35: 11 (November
1989), 1367 - 1392.
Haimes YY, Longstaff T. The Role of Risk Analysis in the Protection of Critical Infrastructures Against
Terrorism. Risk Analysis, 2002, 22:3, pp. 439-444.
Heidelberger P. Fast simulation of rare events in queuing and reliability
models. ACM Transactions on Modeling and Computer Simulation (TOMACS) archive 5:
1 43 - 85, 1995
Hoffman CR, Pugh R, Safie FM. Methods and Techniques for Risk Prediction of Space Shuttle Upgrades. AIAA,
1998
Kaczmarek Z. The impact of climate variability on flood risk in Poland. Risk Anal. 2003
Jun;23(3):559-66.
Kaplan S, Garrick B. On the quantitative definition of risk. Risk Analysis, 1981, 1: page
11-27.
Keefeer DL. Practice abstract. Interfaces 31: 5, 2001, pp 62-64.
Kemeny J. Report of the President's Commission on the Accident at Three Mile Island, Washington DC,
1979.
Krouwer JS. Managing Risk In Hospitals Using Integrated Fault Trees And Failure Mode Effects And Criticality
Analysis. AACC Press, 2004.
Marx DA, Slonim AD. Assessing patient safety risk before the injury occurs: an introduction to sociotechnical probabilistic risk modelling in health care. Qual Saf Health Care. 2003 Dec;12 Suppl
2
Mai S, Zimmermann C. Risk Analysis-Tool for Integrated Coastal Planning. Proc. of the 6th Int. Conf. on Coastal and Port Engineering,
2003.
Mobus C. (1979) The analysis of non-symmetric similarity judgments: Drift model, comparison hypothesis, Tversky's contrast model and his focus hypothesis. Archiv Fur Psychologie; 131 (2):
105-136.
Moore, DRJ, Sample BE, Suter GW, Parkhurst BR, Scott TR. A Probabilistic risk assessment of the effects of Methylmercury and PCBs on mink and Kingfishers along East Fork Poplar Creek, Oak Ridge, Tennessee,
USA. Environmental Toxicology and Chemistry, 18: 12, pp. 2941-2953, 1999.
Ortwin R. Three decades of risk research: accomplishments and new challenges. Journal of Risk Research, 1998, 1:1 pp 49 - 71.
Pate-Cornell ME, Fischbeck PS. Probabilistic Risk Analysis and Risk--Based Priority Scale for the Tiles of the Space Shuttle. Reliability Engineering and System Safety. Vol. 40, no. 3, pp. 221-238.
1993.
Pate-Cornell ME, Fischbeck PS. Risk management for tiles of the space shuttle. Interfaces, 1994, 24: 1, pp
64-86.
Planning Research Corporation, Independent Assessment of Shuttle Accident Scenario Probabilities for Galileo Mission and Comparison with NSTS Program Assessment, 1989.
Rogovin M, Frampton GT. Three Mile Island, a Report to the Commissioners and to the Public, Government Printing Office,
1980.
Rasmussen NC. The Application of Probabilistic Risk Assessment Techniques to Energy
Technologies. Annual Review of Energy, 6: 123-138, 1981.
Sadiq R, Husain T, Veitch B, Bose N. Distribution of arsenic and copper in sediment pore water: an ecological risk assessment case study for offshore drilling waste discharges. Risk Anal. 2003
Dec;23(6):1309-21.
Safie FM. A Statistical Approach for Risk Management of Space Shuttle Main Engine Components. Probabilistic Safety Assessment and Management,
1991
Safie FM. A Risk Assessment Methodology for the Space Shuttle External Tank Welds. Reliability and Maintainability Symposium,
1994.
Safie FM, Fox EP. A Probabilistic Design Analysis Approach for Launch Systems. AIAA/SAE/ASME 27th Joint Propulsion Conference,
1991.
Safie FM. Use of Probabilistic Design Methods for NASA Applications. ASME Symposium on Reliability Technology,
1992.
Siegel P.S., McCord D. M., Crawford A. R. (1982) An experimental note on Tversky's features of similarity. Bulletin of Psychonomic Society; 19 (3):
141-142.
Schwarz G, Tversky A. (1980) On the reciprocity of proximity relations. Journal of Mathematical Psychology; 22 (3):
157-175.
Science Applications International Corporation, Probabilistic Risk Assessment of the Space Shuttle Phase 1: Space Shuttle Catastrophic Failure Frequency Final Report,
1993.
Science Applications International Corporation, Probabilistic Risk Assessment of the Space Shuttle,
1995.
Slob W, Pieters MN. A probabilistic approach for deriving acceptable human intake limits and human health risks from toxicological studies: general framework. Risk Anal. 1998
Dec;18(6):787-98.
Srinivasan R. Importance Sampling. Springer, 2002.
Taylor C, Krings A, Alves-Foss J. Risk Analysis and Probabilistic Survivability Assessment (RAPSA): An Assessment Approach for Power Substation Hardening.� @ Proc. ACM Workshop on Scientific Aspects of Cyber Terrorism,
2002.
Tversky A. (1977) Features of similarity. Psychological Review; 84 (4):
327-352.
Union of Concerned Scientists. The risk of nuclear power reactors: a review of the NRC reactor study, WASH-1400,
1977.
U.S. NRC, Reactor Safety study. U.S. Nuclear Regulatory Commission, WASH-1400, NUREG-751014,
1975.
U.S. NRC, PRA Procedures Guide, U.S. Nuclear Regulatory Commission, NUREG/CR-2300,
1983.
Voortman HG, van Gelder P, Vrijling JK Risk-based design of large-scale flood defense
systems. 28th International Conference on Coastal Engineering, 2002.
To assist you in reviewing the material in this lecture, please see the following resources:
-
See the slides
for
the lecture
Narrated lectures require use of Flash.
Copyright ©
2006 by
Farrokh Alemi, Ph.D. Created on Tuesday October
4th, 2006. Most recent revision
10/22/2011. This page is part of a
Course on Risk Analysis
|
