Introduction to Decision Analysis

Welcome to the world of decision analysis. This section will introduce you to the purpose and organization of this book.   This book describes how analytical tools can be used to help healthcare managers and policymakers make complex decisions. It provides numerous examples in health care settings including benchmarking performance of clinicians, project implementation, scenario planning, resource allocation, analyzing impact of HMO penetration, setting insurance rates, root cause analysis and negotiating employment agreements.

Nearly 20 years ago, Alemi (1986) wrote an article arguing for training health care administrators in decision analysis. Despite widespread acceptance of the idea at the time, as demonstrated by published commentaries, decision analysis has not caught on with health care administrators as much as it has in other industries.  Overall, application of decision analysis in other industries is growing (Keefer, Kirkwood, Corner 2004). MBA students are more likely to receive instruction in decision analysis and when they go to work, they are more likely to use these tools. Goodwin and Wright (1998) give several descriptive examples of use of decision analysis by managers: DuPont uses it to improve strategic decision making, nuclear planners used decision analysis to select counter measures to Chernobyl disaster. ICI America uses it to select projects, Phillips Petroleum uses it to make oil exploration decisions, US military uses it to acquire new weapon systems, EXEL Logistics uses it to select a wide area network, ATM Ltd uses it for scenario planning. The list goes on. In contrast, there are few applications to health care management reported in the literature. This is not so ironic if it were not for the fact that there are numerous applications of decision analysis to clinical decision making and an increasing emphasis in health care to base clinical decisions on evidence and data (Detsky et al. 1998). Decision analysis is used widely by clinicians to understand best course of treatment. This book hopes to change the situation in one of two ways. First, the book will highlight the application of decision analysis to health care management. Health care managers can see for themselves how useful analysis can be in central problems they face.  Second, this book covers decision analysis in enough depth so that readers can apply the tools to their own settings.

This book  is ideally suited for students in health care administration programs.  It may help these programs to develop a courses in decision analysis.  At the same time, the book will be useful for existing survey courses on quantitative analysis in terms of providing a more in-depth understanding of decision analysis so that students feel confident in their abilities to apply these skills in their careers.

The book is also intended for clinicians interested in application of decision analysis to improving quality of care.  Often, practicing physicians, Medical Directors, Nurse Mangers and Clinical Nurse Leaders need to take a system perspective of patient care.  This book provides them with analytical tools that can help them understand systems of care and evaluate the impact of these operations on patient outcomes.    There are a number of books on clinical decision analysis.  This book includes applications to quality improvement that are not typically discussed in other clinical decision analysis books, including root cause analysis, assessing severity of patients' illness and benchmarking performance of clinicians.   These are novel tools that can serve clinicians well if they want to improve health care settings.

Finally, this book may be useful in training of health care policy analysts.  Policy analysts have to provide accurate analysis under time pressure.  Decision analysis is one tool that can help them provide relevant analysis in a timely fashion.  The book contains a number of application of decision analysis to policy decisions including the design of health insurance programs and security analysis.

Organization of the Book

This book is organized in two broad sections. In the first section various analytical tools (multi-attribute utility models, Bayesian probability models, and decision trees) are introduced. In particular, the following chapters are organized:

2. Modeling Preferences
   A demonstration of how to model a decision-maker's values and preferences. This chapter shows how to construct multi-attribute value and utility models, tools that are helpful in evaluation tasks. In particular, it shows how to use multi-attribute value model in constructing severity indices. 
3. Measuring Uncertainty
   An introduction to concepts of probability and causal networks. This chapter lays the ground work for measuring uncertainty by subjective assessment of probability. It shows how to assess the concept of probabilistic independence, a concept central to model building.
4. Modeling Uncertainty
   A demonstration of how to assess the probability of uncertain, rare events based on several clues. This chapter introduces Bayes odds form and shows how it can be used in forecasting future events. In particular, the chapter applies the Bayes Odds form to assessment of market for a new type of Health Maintenance Organization.
5. Decision Trees
   A demonstration of how to combine utility and uncertainty in analyzing options available to healthcare managers. Includes analyzing sensitivity of conclusions to errors in model parameters. It shows how a decision tree can be used to analyze the impact of a Preferred Provider Organization on an employer.
6. Group Decision Making
   Advice on how to obtain the preferences and uncertainties of a group of decision makers. This chapter describes the Integrative Group process.

In the second section, the tools in Part I are applied to various management decisions, including:

7. Root-Cause Analysis
   Application of Bayesian networks to root-cause modeling. Use of causal networks to conduct root-cause analysis of sentinel events.
8. Cost-Effectiveness Analysis
   Use of decision trees for analyzing cost effectiveness of clinical practices and cost of programs.
9. Risk Analysis
   Application of Bayesian probability models to assessment of privacy and security risks.
10. Program Evaluation
    Use of decision-analysis tools for program evaluation. Use of Bayesian probability models to analyze markets for new health services.
11. Conflict Analysis
    Use of multi-attribute value modeling in analyzing conflict and conflict resolution. This chapter shows how multi-attribute value models could be used to model conflict around family planning program. In addition, an example is given of negotiation between an HMO manager and a physician.
12. Rapid Analysis
    This chapter shows how subjective and objective data can be combined in order to conduct a rapid analysis of business and policy decisions.
13. Clinical benchmarking
    Use of decision analysis to construct measures of severity of illness of patients and to compare clinicians' performance across different patient populations.

It is imperative that health care managers become facile with formal decision analysis techniques so that they will be competent to analyze and devise effective solutions to the complex situations they will encounter in their careers. This book provides the basis for the acquisition of skills which will help them do so. Good decisions based on a systematic consideration of all relevant factors and stakeholder opinions and values lead to good outcomes, both for those involved in the decision making process, and for the customers who are directly impacted by the consequences and effects of such decisions.

Introduction to Decision Analysis

This chapter introduces the idea behinds decision analysis, the process of analysis and its limitations.  The discussion is directed toward decision analysts who help individuals in healthcare institutions responsible for making tough choices. Anytime a selection must be made among alternatives, a decision is being made, and it is the role of the analyst to assist in this process.  When decisions are complicated and require careful consideration and systematic review of the available options, the analyst role becomes paramount. As you will learn in this chapter, the analyst’s job is to separate complex decisions into component parts and then reconstitute the final decision from those parts. Some decisions are harder than others to make, and this chapter provides a discussion of why that might be. For instance, some problems are poorly articulated. In other cases, the causes and effects of potential actions are uncertain. There may be confusion about what environmental factors could impact the decision. Given these uncertainties, you will learn how to clarify and simplify such problems without diminishing the usefulness or accuracy of the analysis. As an analyst, it will be your job to provide structure to the problem, to reduce uncertainty about potential future events that may impact the situation, to help decision makers clarify their values and preferences, and to help reduce conflict among decision makers who may have different opinions about the utility of various options. This chapter goes on to outline specifically the steps involved in decision analysis including problem exploration and goal clarification, identification of decision makers, problem structuring, quantifying values and uncertainties, analyzing the courses of action and finally recommending the best course of action. This chapter will provide you with a solid foundation in your understanding of the purpose and process of decision analysis. Later chapters will introduce more specific tools and skills which are meant to build upon this foundation.

Who is an Analyst?

This book is addressed to analysts who are trying to assist health care managers in making complex and difficult decisions.  The Webster Merriam's dictionary definition of system analysis can be used to define who is an analyst.   An analyst studies the choices between alternative courses of actions, typically by mathematical means, in order to reduce uncertainty and align it with decision makers' goals. �

This book is laid out as if the decision maker and the analyst are two different people.  Of course, a decision maker might want to self analyze their own decision.  In these circumstances, the tools described in the book can be used but the decision maker must play both the role of the analyst and the decision maker.  When mangers want to think through their problems, they can use the tools in this book to analyze their own decisions without the need for an analyst.  

Who is a Decision Maker?

One of the first tasks of an analyst is to clarify who makes the final decision and on what timetable.  Many chapters in this book  assume that a single decision maker is involved in the process.  Sometimes more than one decision maker may be involved.  Chapter 11, on Conflict Analysis, and chapter 6, on Group Decision Making, are intended for situations when multiple decision makers are involved.,

Through out the book, the assumption is that at least one decision maker is always available to the analyst.  This is an over simplification of the reality of organizations.  Sometimes it is not clear who the decision maker is.   Other times, an analysis starts with one decision maker who leaves his/her position midway through the analysis; one person commissions the analysis and another person receives the findings.   Sometimes, an analyst is asked to conduct an analysis from a societal perspective, where it is difficult to identify clear decision makers.   All of these variations make the analysis process more difficult.   �

What is a Decision?

This book is about using analytical models to finding solutions to complex decisions.  Before proceeding, various terms should be defined.  Lets start with a definition of a decision.  Most individuals go through their daily work without making any decisions.  They react to events without taking the time to decide about them.  When the phone rings and if they are available, they pick it up and answer.  In these situations, they are not deciding but just working.  Sometimes, however, they need to make decisions.  If they have to hire someone and there are many applicants, they need to make a decision.

One is making a decision as opposed to following routines, when one has several plausible courses of actions.  According to the Webster Dictionary, deciding is to arrive at a solution that ends uncertainty or dispute about what to do.  A decision is made when a course of action is selected among alternatives.   More formally, a decision has following components:

• A set of alternatives or options

• Each alternative leads to a series of consequences over which the decision maker has little control

• The decision maker is uncertain about what might happen.

• The decision maker has different preferences about outcomes associated with various consequences.

A decision involves choosing among uncertain outcomes with different values.

What is Decision Analysis?

The Webster Dictionary defines analysis as separation of a whole into its component parts.  Decision analysis is the process of separating a complex decision into its component parts and using a mathematical formula to reconstitute the whole decision from its parts.  It is a method of helping decision makers make simple and familiar choices and using a mathematical model to infer from these choices what would the decision maker had preferred to do in the complex intractable decision.

An Example

The means by which the analysts can capture uncertainties and values is exemplified by a hypothetical situation faced by the head of the state agency responsible for evaluating nursing home quality: A nursing home has been overmedicating its residents in an effort to restrain them, and the administrator of state agency must take action to improve care at the home. The possible actions include fining the home, prohibiting admissions, and teaching the home personnel more appropriate use of psychotropic drugs.

Any real‑world decision has many different effects. For instance, the state could institute a training program to help the home improve its use of psychotropic, but the state's action could have effects beyond changing this home's drug utilization practices. The nursing home could become more careful about other aspects of its care such as how it plans care for its patients. The nursing home industry could become convinced that the state is enforcing stricter regulations on administration of psychotropic drugs. All these effects are important dimensions that should be considered during the analysis and in any assessment performed afterward.

The problem becomes more complex because the agency administrators must consider which constituencies' values should be taken into account and what their values are regarding the proposed actions. For example, the administrator may want the state to portray a tougher image to the nursing home industry, but one constituent, the chairman of an important legislative committee, may object to this image. So the choice of action will depend on which constituencies' values are considered and how much importance each constituency is assigned.

Why are Some Decisions Hard to Make?

Decisions are hard to make because of uncertainty about outcomes of the decisions and confusion about value of various outcomes.   Three types of uncertainty typically confront the decision maker:

• Poorly articulated problem.  The decision maker is unsure or unaware of the nature of the problem. This causes difficulties because a successful solution must start with correct identification of the problem. For example, if the nursing home was simply ignorant of how to use psychotropic drugs appropriately, a decision to prohibit further admissions would just alienate the nursing home without solving the problem. In this case, training would not only solve the problem but also foster a positive relationship between the home and the state. If, however, the nursing home knew exactly what it was doing‑cutting costs with chemical restraints‑‑a consultation and training program could be fruitless. But while it is important to know as much as possible about the nursing home's reasons for its drug policy, one can never be certain about those reasons, and almost any decision includes an element of diagnostic uncertainty.

• Poorly understood cause and effects.  The decision maker is not sure what action will lead to the desired outcome.  In general, too little is known about most problems‑‑even those that seem clearly understood‑‑to be sure that a specific action will lead to a particular outcome. This is true even if no major events intervene. In the nursing home example, even if the inappropriate chemical restraint was clearly based on ignorance and the state had already won the court case, one cannot be sure training would correct the problem. Perhaps the home personnel did not have the education or motivation to use the drugs properly.

• Poorly understood environment.  The decision maker is not sure about what external events may change the alternatives available.  In characterizing a decision problem, the analyst must identify which future events will affect the relationship between action and outcome, and analyze what the impact of those events will be.  In short, the analyst must have a crystal ball of sort to see the future.  Suppose that during the time the decision maker is deciding to prohibit further admissions, the constitutionality of state regulation of admissions is being challenged in court.  If the state loses the court test, then selecting that option (prohibiting admissions) could be futile.  Future events can be divided into two important groups: (1) unlikely events that would have monumental consequences and (2) likely events that would have lesser but significant consequences.

Decisions are also hard to make because the values of different outcomes may not be clear or different constituencies may disagree on value of various outcomes.   The more the confusion about what the decision makers really want, the more difficult to analyze it.:

Prototypes for Decision Analysis

         Real decisions are complex.  The purpose of analysis is not to capture decisions in all its complexity.  The goal is not to impress, and in the process overwhelm, the decision maker about the analyst's ability to capture all possibilities.  The goal of analysis is to simplify the decision enough to meet the decision maker's needs.  An important challenge then is to determine how to simplify an analysis without diminishing its usefulness and accuracy.

         A useful simplification is to ignore some uncertainties, so the value of an action is assumed to be more "certain" than it really is‑in other words, the chance of an event is either near zero or one. For instance, in deciding which departments need additional funds, the decision maker might choose to assess current levels of needs and ignore the uncertainty about future needs. Of course, such simplifications are only appropriate when using them will make little difference in the results of the analysis.

          Alternatively, the analyst may assume that uncertainty is the only issue and that the other values and actions can be addressed without the help of analysis. For example, the principal challenge in strategic planning may be diagnosing what would our target customers need. Presumably, after knowing unmet customers needs, the decision maker's action would be relatively clear and the analyst would not need to examine the decision maker's preferences over different outcomes.

          Over the years, as analysts have applied various tools to decisions some prototypes have emerged.  If an analyst can recognize that a decision is like one of the prototypes in his/her arsenal of solutions, then he/she can quickly address the problem.  Each prototype leads to some simplification of the problem and specific analytical solution.  The existence of these prototypes helps in addressing the problem with known tools and methods.   These prototypes include:

1. Identify the problem

2. Reduce uncertainty about future events

3. Clarify values

4. Reduce conflict among various constituencies

5. Do it all

Prototype 1. Identify the problem

Analysts can help decision makers by providing a structure to the problem.  Sometimes decision makers do not truly understand the problem they are addressing.  This lack of understanding can be manifested in disagreements about the proper course of action. Each member of a decision making team may prefer a reasonable action based on his or her limited perspective of the issue. An analyst can promote better understanding of the decision by helping policy makers to explicitly identify:

• Individual assumptions about the problem and its causes
• Objectives being pursued by each decision maker
• Constituencies having different perceptions and values
• Options available
• Factors that influence the desirability of various outcomes
• Principal uncertainties that complicate the problem

Figure 1: Decision Maker Faces a Difficult Decision As Options, Values &
Uncertainties in the Decision are not Clear

The analyst can listen to the decision maker and let him or her articulate various aspects of a problem.  As Figure 1 shows, the analyst usually seeks to understand the nature of the problem by clarifying the values and uncertainties involved in the problem.  When the problem is fully described, the analyst can provide an organized summary to the decision makers, helping them see the whole and its parts.

Prototype 2. Reduce Uncertainty

An analyst can help decision makers by reducing their uncertainty about future events.  Decision makers are sometimes not sure what will happen if an action is taken.  They may not be sure about the state of their environment. What is the chance that a fine will really change the way the nursing home uses psychotropic drugs? What is the chance that if a hospital administrator opens a stroke unit their competitors will not do the same?

             Through the process of analyzing uncertainty, the decision's options and their relative desirability can be clarified. What causes system malfunctions? What are the chances a similar event will recur? How likely is action A to lead to outcome B? Often the answers to such questions are vague at best. Returning to our example, although you probably have some clues about whether the nursing home's overmedication was caused by ignorance or greed, usually the clues are neither equally important nor measured on a common scale. The analyst helps to compress the evaluations to a single scale for comparison.  The analyst uses various tools to forecast the future. 

Figure 2: Decision Maker Faces a Difficult Decision Because
Outcomes of their Action is Uncertain

The forecast is based on a set of clues.  Some clues suggest the target event might occur, other clues suggest the opposite.  The analyst distills the implications of often contradictory clues into a single forecast.  Deciding on the nature and relative importance of these clues is difficult because people tend to assess complex uncertainties poorly unless they can divide them into manageable components. Decision analysis can help make this division by using probability models that combine components after their individual contributions have been determined. This book addresses such a probability model‑Bayes' theorem‑‑in chapter 4.

Prototype 3.  Clarifying Values

An analyst can help decision makers clarify their values and preferences.  In some situations the options are clearly identified and uncertainty plays a minor role.  In these decisions the primary concern is to examine the options in terms of a complex set of values that have differing levels of importance and are measured on different scales. The Decision Maker's actions will have many outcomes some of which are positive and others negative (see Figure 4).  One option may be preferable on one dimension but unacceptable on another.  Somehow the decision maker must trade off the gains in one dimension with losses in another.

Figure 3:  Decision Maker Faces a Difficult Choice
Because their Actions May Lead to Positive & Negative Outcomes

In traditional attempts to debate an option, advocates of one option focus on the dimensions that show it having a favorable outcome, while opponents attack it on dimensions on which it performs poorly. Optimally, a decision analysis provides a mechanism to force consideration of all dimensions‑a task that requires answers to these questions:

• Which objectives are paramount?

• How can an option's performance on a wide range of measuring scales be collapsed into an overall measure of relative value?

For example, a common value problem is how to allocate limited resources to various individuals or options. The British National Health Service, with a fixed budget, deals with this issue quite directly. Some money is allocated to hip replacement, some to community health services, and some to long‑term institutional care for the elderly. Many people who request a service after the money has run out must wait until the next year.  A chief financial officer has to tradeoff various projects in different departments and decide on allocation of a budget for the unit.  The decision analysis approach to these questions uses a process called multi-attribute value (MAV) modeling, which is introduced in chapter 2. 

Prototype 4.  Reduce Conflict

An analyst can  help decision makers understand conflict better by modeling the uncertainties and values that different constituencies see in the same decision.  Common sense tells us that people with different values tend to choose different options. The principal challenge facing a decision making team may be understanding how different constituencies view and value a problem and determining what trade‑offs will lead to a win-win, instead of a win‑lose,  solution.  Decision analysis addresses situations like this by developing a MAV model for each constituency and using these models to generate new options that benefit all. 

Figure 4:  Decision Makers Face a Difficult Decision
Because they Differ in the Outcomes they Prefer

Consider for example a contract between an HMO and a clinician.  The contract will have many components.  There are at least two different perspectives on almost every issue.  The parties would need to balance cost, benefits, professional independence, required practice patterns, and many other issues.  An analyst can identify the issue and highlight the preferences of the parties.  Then conflict can be understood and steps can be taken to avoid escalation of conflict to a level that disrupts the negotiations. 

Prototype 5.  Do it all

Of course, a decision can have all of the elements of the last four prototypes.  In these circumstances, the analyst must:

• Identify who are the various decisions makers and their differences.

• Identify possible actions, of which usually there are many.

• Identify possible outcomes of an action, cost is one outcome but other outcomes are also relevant

• Identify major uncertainties (events that, if they were to occur, would interfere with an action's leading to a certain outcome)

• Attach values to those outcomes

• Analyze the data available to recommend a course of action

Figure 5:  Components of a Decision

In this prototype, the analyst needs to address both uncertainty about future outcomes as well as confusion about value of these outcomes.  An example is making a decision about merger between two hospitals.  There are many decision makers.  The decision structure itself is not clear.  There are many uncertain consequences associated with the merger.  The impact of merger on various outcomes (cost, access, quality, etc.) are not clear nor is it clear how much each outcome should be emphasized and why.

Steps in Decision Analysis

         Good analysis is about the process not the numbers.  One way to analyze a decision is for the analyst to conduct an independent analysis and present the results to the decision maker in a brief paper. This is usually not very helpful and emphasizes the findings as opposed to the process.  Decision makers are more likely to accept an analysis in which they have actively participated.

          The preferred method is to conduct decision analysis is as a series of increasingly more sophisticated interactions with the decision maker.   At each interaction, the analyst listens and summarizes what the decision maker speaks about.  In each step, the problem is structured and an analytical model is created.  Through these cycles, the decision maker comes to self insight and the analysis documents his/her conclusions.

           Of course, sometimes decisions are made in groups.  Here again the goal is to conduct a decision conference that engages the decision makers in various steps of the analysis.  A decision conference starts with a day‑long retreat during which the decision making team agrees upon the conceptual structure of the problem. In the next day, the group agrees on objectives, possible actions, uncertainties, outcomes, values, probabilities, and perhaps other topics.  Sometimes values and uncertainties are quantified.  The conference prods decision makers to think carefully about the issues and the tradeoffs they are willing to make.  It structures the problem.  The team emerges from this short retreat with better understanding of their decision.

            Whether the analysis is done for one person or several people, there are several distinct steps in decision analysis.  A number of investigators have suggested steps in conducting decision analysis (Soto, 2002; Philips, et al. 2004; Weinstein et al. 2003).  Soto, working in the context of clinical decision analysis, recommends that all analysis should take the following steps:

1. "State clearly the aim and the hypothesis of the model

2. Provide the rationale of the modeling

3. Describe the design and structure of the model

4. Expound the analytical time horizon chosen

5. Specify the perspective chosen and the target decision makers

6. Describe the alternatives under evaluation

7. State entirely the data sources used in the model

8. Report outcomes and the probability that they occur

9. Describe medical care utilization of each alternative

10. Present the analyses performed and report the results

11. Carry out sensitivity analysis

12. Discuss the results and raise the conclusions of the study

13. Declare a disclosure of relationships

Based on commonly suggested steps, we suggest the following:

Step 1. Explore the Problem & the Role of the Model

Problem exploration is the process of understanding why the decision maker wants to solve a problem.  The analyst needs to understand what would the resolution of the problem achieve. This understanding is crucial because it helps identify creative options for action and sets some criteria for evaluating the decision.  The analyst also need to clarify what is the purpose of the modeling effort ( to keep track of ideas, to have a mathematical formula that can replace the decision maker in repetitive decisions, to clarify issues to the decision maker, to help others understand why the decision maker chose a course of action, to document the decision, to help the decision maker arrive at self insight, to clarify values, to reduce uncertainty, etc.). 

Let's return to the head of the Division of Nursing Home Administration who was trying to decide what to do about the nursing home that was restraining its residents with excessive medication. The problem exploration might begin by understanding the problem statement: "Excessive use of drugs to restrain residents." Although this type of statement is often taken at face value, several questions could be asked. How should nursing home residents behave? What does restraint mean? Why must residents be restrained? Why are drugs used at all? When are drugs appropriate, and when not? What other alternatives does a nursing home have to deal with problem behavior?

The questions at this stage are directed at (1) helping to understand the objective of an organization, (2) defining frequently misunderstood terms, (3) clarifying the practices causing the problem, (4) understanding the reasons for the practice, and (5) separating desirable from undesirable aspects of the practice. 

During this step, the decision analyst must determine which ends, or objectives, will be achieved by solving the problem. In the example, the policymaker must determine whether the goal is primarily to 

• Protect an individual patient without changing overall methods in the nursing home

• Correct a problem facing several patients‑in other words, change the home's general practices

• Correct a problem that appears to be industry wide

Once these questions have been answered, the decision analyst and policymaker will have a much better grasp on the problem. The selected objective will significantly affect both the type of actions considered and the particular action selected.

Step 2. Identify Decision Makers, their Perspective and Timeframe

Who makes the decision is not always clear.  Some decisions are made in groups, others by individuals.  For some decisions there is a definite deadline for others there is no clear timeframe.  Some decisions have already been made before the analyst came on board; others involve much uncertainty and the analyst needs to sort them out.  Sometimes the person who sponsors the analysis is preparing a report for a decision making body, not available to the analyst.  Other times the analyst is in direct contact with the decision maker.  Decision makers may also differ in the perspective they want the analysis to take.  Sometimes providers costs and utilities are central, other times patients' values derive the analysis.  Sometimes societal perspective is adopted other times the problem is analyzed from the perspective of a company.  Decision analysis can help in all of these various situations but in each of them the analyst should explicitly specify the decision makers, the perspective of the analysis and the time frame for deciding.  The analyst should sort out who are the real decision makers, what is their perspective and when should the decision be made.

The specification of decision maker does not stop with identifying the individual's involved.  It is important to understand the constituencies, whose ideas and values must be present in the model.  A decision analysis can always assume that only one constituency exists and that disagreements arise primarily from misunderstandings of the problem, not from different value systems among the various constituencies. But when several constituencies with different assumptions and values, the analyst must examine the problem from the perspective of each constituency.

A choice must also be made about who will provide input into the decision analysis. Who will specify the options, outcomes, and uncertainties? Who will estimate values and probabilities? Will outside experts be called in? Which constituencies will be involved? Will members of the policymaking team provide judgments independently, or will they work as a team to identify and explore differences of opinion?  Obviously, all of these choices depend on the decision maker and an analyst should simply ask question and not supply answers to the questions.

Step 3. Structure the Problem

Problem structuring adds conceptual detail to the general structure provided by step 2. The goals of structuring the problem are to clearly articulatey

• What the problem is about, why it exists, and whom it affects

• The assumptions and objectives of each affected constituency

• A creative set of options for the decision maker

• Outcomes to be sought or avoided

• The uncertainties that affect the choice of action

Structuring is the stage in which the specific set of decision options is identified. Although generating options is critical, it is often overlooked by decision makers‑a pitfall that can easily promote conflict in cases where diametrically opposed options falsely appear to be the only possible alternatives. Often, creative solutions can be identified that better meet the needs of all constituencies.  To generate better options, one must understand the purpose of analysis.  The process of identifying new options relies heavily on reaching outside the organization for theoretical and practical experts, but the process should also encourage insiders to see the problem in new ways.

It is important to explicitly identify the objectives and assumptions of the decision makers. Objectives are important because they lead to the preference of one option over the other. If the decision making team can understand what each constituency is trying to achieve, the team can analyze and understand its preferences more easily. The same argument holds for assumptions. Two people with similar objectives but different assumptions about how the world operates can examine the same evidence and reach widely divergent conclusions.

Take, for example, the issue of whether two hospitals should merge.   Assume that both constituencies‑those favoring and those opposing such merger‑‑want the hospital to grow and prosper.  One side believes that the merger will help grow faster and the other side believes that the merger would make the organization lose focus.  One side believes that the community will be served better by competition and another side believes the community will benefit from collaboration between the institutions.  In each case, the assumptions (arid their relative importance) influence the choice of objectives and action, and that is why they should be identified and examined during problem structuring.

Problem structuring is a cyclical process‑the structure may change once the decision makers have put more time into the analysis.  The cyclical nature of the structuring process is desirable, not something to be avoided. � An analyst should be willing to go back and start all over with new structure and new set of options.

Step 4. Quantifying Values

The analyst should help the decision maker break complex outcomes into their components and weight the relative value of each component. The components can be measured on the same scale, called a value scale, and an equation can be constructed to permit the calculation of the weighted average of the scores.

Value has two sides: cost and benefits. Cost is typically measured in dollars and may appear straightforward.  But true costs are complex measures and difficult to measure.  This is the case because certain costs, such as loss of goodwill, are non-monetary and difficult to track from operation budgets. Furthermore, even monetary costs may be difficult to allocate to specific operations as overhead and other shared cost may have to be allocated in methods that seem arbitrary and not precise. 

Benefits need to be measured based on various constituencies' preferences.  Assuming that benefits and the value associated with the benefits are un-quantifiable can be a major pitfall because it can place them subservient to costs in a formal analysis of policy, even though values often drive the actual decision.  By assuming values cannot be quantified, the analysis may ignore concerns most likely to influence the decision maker.

Step 5. Quantifying Uncertainties

The analysts interacts with decision makers and experts to quantify uncertainty about future events. For example, if the nursing home inspectors were asked to estimate the chance that the home's chemical restraint practice resulted from ignorance or knowing intention to save money, they might agree that the chances were 90 percent ignorance and 10 percent intent. In some cases, additional data are needed to assess the probabilities. In other cases, there is too much data available. In both cases, the probability assessment must be divided into manageable components. Bayes' theorem (see chapter 4) provides one means for disaggregating complex uncertainties into their components.

Step 6. Analyze & Recommend Course of Action

Once values and uncertainties are quantified, the analyst uses the model of the decision to score the relative desirability of each possible action. This can be done in different ways depending on what type of a model has been developed.  One way is to examine the expected value of the outcomes.  Expected value is the weighted average of the values associated with outcomes of each action.  Values are weighted by the probability of occurrence of each outcome. Suppose, in the nursing home example, two actions are possible:�

A1 = Consult
A2 = Stop admission

The possible outcomes are:

1. Industry changes:  Chemical restraint is corrected and industry "gets the message that the state intends tougher regulation" 

2. Only one home changes:  The home receiving the citation changes but the rest of industry does not "get the message"

3. No change:  The home ignores the citation and there is no impact on the industry

Suppose the relative desirability of each outcome is:

• Industry changes has best value, 100

• Only one home changes, 25

• No change, 0

The probability that each action will lead to each outcome is shown in the six cells of the matrix in Figure 3.

Figure 3:  A Decision Matrix For Nursing Home Regulatory Actions

The expected value principle says the desirability of each action is the sum of the values of each outcomes of the action weighted by probability of the outcome.  If  p_ij is the probability of action "i" leading to outcome "j" and V_j is the value associated with outcome "j", then expected value is calculated as:

Expected value of action "i"  =∑p_ij V_j

In the case of our example, expected values are:

Expected value of consultation =  0.05 * 100+ 0.60*25 + 0.35*0 = 20F

Expected value for stopping admission = 0.40 *100 + 0.20 *25 + 0.40 * 0 = 45

This analysis suggests that the most desirable action would be to stop admissions because its expected value is larger than consultation.

Step 7: Conduct Sensitivity Analysis

The analyst interacts with the decision maker to identify how various assumptions in the analysis affect the conclusion.  The previous analysis suggests that "Consultation" is inferior to "Stopping Admission". But this should not be taken at face value because the utility and probability estimates might not be accurate. Perhaps the source of those estimates was guesses, or the estimates were average scores from a group, some of whose members had little faith in the estimates. In these cases, it would be valuable to know whether the choice would be affected by using a different set of estimates. Stated another way, it might make sense to determine how much an estimate would. have to change to alter the choice of "preferred" action.

Usually, one estimate is changed until the expected value of the two choices become the same.  Of course, several estimates can also be modified at once, especially using computers. Sensitivity analysis can be vital not only to examining the impact of errors in estimation but also to determining which variables need the most attention (e.g., reduction in disagreement and/or increase in confidence).

At each stage in the decision analysis process, it is possible and often essential to return to an earlier stage to

• Add a new action or outcome
• Add new uncertainties
• Refine probability estimates
• Refine utility estimates

This cyclical approach offers a better understanding of the decision problem and fosters greater confidence in the analysis. Often the decision recommended by the analysis is not the one implemented, but the value of the analysis remains because it increases understanding of the issues. Phillips refers to this as the theory of requisite decisions‑once all parties agree that the problem representation is adequate for reaching the decision, the model is "requisite":

From this point of view, decision analysis is more an aid to problem solving than a mathematical technique, Considered in this light, decision analysis provides the decision maker with a process for thinking about their actions.  It is a practical means for maintaining control of complex decision problems that involve risk, uncertainty, and multiple objectives. (Phillips 1984, p. 26)

Step 8:  Document and Report Findings

Even thought the decision maker has been intimately involved in the analysis and is probability not surprised at its conclusions, it is important to document and report the findings.  An analysis has its own life cycle.  It may live well beyond the current decision makers.  It is important to document all considerations that were put into the analysis.  Individuals who did not participate in the decision may wonder how it was arrived at.  A clear documentation, one that uses multi-media to convey the issues, would help create a consensus behind a decision. 

Limitations of Decision Analysis

It is difficult to evaluate the effectiveness of decision analysis because often no information is available on what might have happened if decision makers had not followed the course of action recommended by the analysis.   One way to improve the accuracy of analysis is to make sure that the process of analysis is followed faithfully.  Rouse and Owen (1998) suggest asking the following questions about decision analysis to discern if it was done accurately:

1. "Were all realistic strategies included?
2. Was the appropriate type of model employed?
3. Were all important outcomes considered?
4. Was an explicit and sensible process used to identify, select and combine the evidence into probabilities
5. Were utilities assigned to outcomes plausible and were they obtained in a methodologically acceptable manner
6. Was the potential impact of any uncertainty in the probability and utility estimates thoroughly and systematically evaluated"

These authors also point out four serious limitations to decision analysis which are important to keep in mind:

1. "Decision analysis may oversimplify problems to the point that they do not reflect the real concerns of the patient nor accurately represent the perspective from which the analysis is being conducted.
2. Available data simply may be inadequate to support the analysis.
3. Utility assessment, in particular, assessment of quality of life may be problematic.  "Measuring quality of life, while conceptually appealing and logical, has proven methodologically problematic and philosophically controversial.
4. Outcomes of decision analyses are not amenable to traditional statistical analysis. Strictly, by the tenets of decision analysis, the preferred strategy or treatment is the one that yields the greatest utility (or maximizes the occurrence of favorable outcomes) no matter how narrow the margin of improvement."

In the end, the value of decision analysis (with all of its limitations) is in the eye of the beholder.  If the decision maker can understand and have new insights into a problem, if the problem and suggested course of action can be documented and communicated to others more easily, a decision maker may judge decision analysis, even imperfect analysis, as useful.

What do you know?

1. Who makes the decision? 
2. What actions are possible (list at least two actions)? 
3. What are the possible outcomes? 
4. Besides costs what other values enter these decisions and whose values are considered relevant to the decision?
5. Why are the outcomes uncertain?

Presentations

To assist you in reviewing the material in this lecture, please see the following resources:

1. Listen to Introduction.  See the slides.

Narrated lectures require use of  Flash.

More & References

• References
   

  Detsky AS, Naglie G, Krahn MD, Naimark D, Redelmeier DA . Primer on medical decision analysis: Part 1. Getting started Med Decis Making, 1998  Apr-Jun;18(2):237-8.

  Goodwin P, Wright G.  Decision Analysis for Management Judgment  UK: Wiley, 1998

  Gustafson DH, Holloway DC.  1975. "A Decision Theory Approach to Measuring Severity of Illness." Health Services Research 10: 97‑196.

  Gustafson DH, Fryback DG, Rose JH, Prokop CT, Detmer DE, Rossmeissl JC, Taylor CM, Alemi F, Carnazzo AJ.. 1983. "An Evaluation of Multiple Trauma Severity Indices Created by Different Index Development Strategies." Medical Care 21: 674‑91.

  Gustafson DH, Fryback DG, Rose JH, Yick V, Prokop CT, Detmer DE, Moore J. 1986. "A Decision Theoretic Model for Severity Index Development." Medical Decision Making 6 (1): 27‑35.

  Health Services Research Group. 1975. "Development of the Index of Medical Under service." Health Services Research (Summer).

  Johnson, S., and D. Gustafson. 1989. Final Report of the Psychiatric Emergency Severity Index Project, Maine Health Information Center, Augusta.

  Keefer DL, Kirkwood CW, Corner JL. Perspective on decision analysis applications, 1990-2001 Decision Analysis Vol. 1, No. 1, March 2004, pp. 4–22.

  Rouse DJ, Owen J. Decision analysis. Clin Obstet Gynecol. 1998 Jun;41(2):282-95.

  Soto J. Health economic evaluations using decision analytic modeling. Principles and practices--utilization of a checklist to their development and appraisal. Int J Technol Assess Health Care. 2002 Winter;18(1):94-111.

  Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, Woolacoot N, Glanville J. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol Assess. 2004 Sep;8(36):iii-iv, ix-xi, 1-158.

  Phillips, L. D. 1984. "A Theory of Requisite Decision Models." Acta Psychologica 56: 29‑48.

  Weinstein MC, O'Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, Luce BR; ISPOR Task Force on Good Research Practices--Modeling Studies. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices--Modeling Studies. Value Health. 2003 Jan-Feb;6(1):9-17.

Recommended Reading (logoff after each article, requires library membership)

• Coughlin, Michael T. MA. Angus, Derek C. MD, MPH. Economic evaluation of new therapies in critical illness. Critical Care Medicine. 31(1) Supplement:S7-S16, January 2003.   Accession Number: 00003246-200301001-00002.  (From the abstract) The article reviews the rationale behind use of decision analysis in evaluations of new therapies and the alternative economic approaches available. Elements of a cost-effectiveness analysis are also discussed.
   

• Sachdeva, Ramesh C. MD, PhD, MBA. Measuring the impact of new technology: An outcomes-based approach. Critical Care Medicine. 29(8) Supplement:N190-N195, August 2001.   Accession Number: 00003246-200108001-00008.  (From the abstract) This article examines the evaluation of new technologies by health care organizations and the questions faced by the organizations when they must make the decision to adopt a new technology. The article argues for integrating business and clinical considerations using quantitative and qualitative models which can help facilitate answering such questions.
   

• Paton, Calum R. Evidence-Based Health Care: How to Make Health Policy and Management Decisions. BMJ. 318(7177):201, January 16, 1999. Accession Number: 00002591-199901160-00088.  (From the introduction) The authors review two publications of means of making evidence-based health care decisions, including Grayson’s Evidence-Based Medicine and Muir Gray’s compilation.
   

• Fleisher, Lee A. MD. Mantha, Srinivas MD. Roizen, Michael F. MD. Medical Technology Assessment: An Overview. Anesthesia & Analgesia. 87(6):1271-1282, December 1998.   Accession Number: 00000539-199812000-00012.  (From the introduction) This article presents and overview of the assessment of medical technology including definitions of technology, the process of medical technology assessment, and a focus on recent dramatic changes in the method of medical reimbursement.
   

• For a review of progress in understanding psychological aspects of decision making see:  Decision Analysis in Psychiatry. British Journal of Psychiatry. 166(2):184-190, February 1995. Accession Number: 00002405-199502000-00005.  (From the abstract) This article considers the applicability to psychiatry of a quantitative approach to making difficult decisions. The article provides an example of how decision analysis could be used in psychiatry and provides an outlook for the future of decision analysis in psychiatry.
   

• Frisch, Deborah 1,3. Clemen, Robert T. 2. Beyond Expected Utility: Rethinking Behavioral Decision Research. Psychological Bulletin. 116(1):46-54, July 1994. Accession Number: 00006823-199407000-00005.  (From the abstract) This article evaluates the use of subjective expected utility (SEU) theory in psychology as a means for evaluating the quality of people’s decisions. Contrary to this theory, the authors argue that to evaluate the quality of a decision, researchers need to first identify decision processes that lead to desirable outcomes. The authors go on to argue that a good decision making process must consider how decision makers evaluate potential consequences of decisions, the extent to which they accurately identify consequences and the manner in which they make final choices.
   

• Kleinmuntz, Benjamin 1,2. Why We Still Use Our Heads Instead of Formulas: Toward an Integrative Approach. Psychological Bulletin. 107(3):296-310, May 1990.  Accession Number: 00006823-199005000-00002. (From the abstract) This article discusses of when to use one's head instead of the formula for clinical prediction. The article reviews the contemporary significance of this query for decision analysts and argues for the use of systematic decision models over intuition in clinical decisions.
   

Societies and Associations

• For  Society for Medical Decision Making (SMDM) see  http://www.smdm.org
• For Decision Analysis Society see http://faculty.fuqua.duke.edu/daweb/
• For Institute for Operations Research and the Management Sciences (INFORMS) see  http://www.informs.org/
• For Society for Judgment and Decision-Making (JDM) see  http://www.sjdm.org
• For Society for Mathematical Psychology (SMP)  see http://aris.ss.uci.edu/smp/index.html
• For International Society for Bayesian Analysis (ISBA) see  http://www.bayesian.org/
• For American Statistical Association Section on Risk Analysis see  http://www.amstat.org/  
• For European Association of Decision Making (EADM) see http://www.eadm.org/
• For International Society for Multi-Criteria Decision Making (MCDM) see http://www.terry.uga.edu/mcdm/
• For  Society for Risk Analysis see  http://www.sra.org
• For ASA Section on Bayesian Statistical Science see http://www.amstat.org/sections/SBSS/
• For  Association for Uncertainty in Artificial Intelligence see  http://www.auai.org/
• For American Risk and Insurance Association  see  http://www.aria.org/
• For Public Choice Society see  http://www.pubchoicesoc.org
• For  Decision Sciences Institute see http://dsi.gsu.edu/

Other Sites

• For a glossary of commonly used terms in Medical Decision Making see http://www.smdm.org/Glossary.htm. 

• A tutorial application of concepts of decision analysis to clinical settings  can be found at  http://www.cche.net/usersguides/decision.asp.  �

• Other Decision Analysis courses can be found at  http://www.bus.utexas.edu/~dyerj/DA_syllabi

• Annotated references on decision theory and uncertainty compiled by Peter Wakker can be found at  http://www.fee.uva.nl/creed/wakker/refs/rfrncs.htm.

• Internet resources for geo-information-based decision analysis can be found at  http://publish.uwo.ca/~jmalczew/gimda/intres2.htm.

• For five articles on how to perform a Decision Analysis see  http://umg.umdnj.edu/smdm/default.asp?vol=17&num=02. This tutorials include:

  • Part 1- Primer on Medical Decision Analysis

  • Part 2- Building a Tree
  • Part 3- Estimating Probabilities and Utilities
  • Part 4- Analyzing the Model and Interpreting the Results
  • Part 5- Working with Markov Processes

This page is part of the course on Decision Analysis, the lecture on Introduction to Decision Analysis.  This page was last edited on 01/15/19 by Farrokh Alemi, Ph.D.   ©Copyright protected.  This page is based on a chapter with the same name published by Gustafson DH, Cats-Baril WL, Alemi F. in the book  Systems to Support Health Policy Analysis:  Theory, Model and Uses/a>, Health Administration Press: Ann Arbor, Michigan, 1992.