Georgetown University's Health Systems Administration
Process Improvement
   

Cost Effectiveness of Clinics


 

 

This chapter is based in part on Costs and Benefits of Combining Probation and Substance Abuse Treatment by Alemi F, Taxman F, Vang J, Thanner M, Baghi H. (in review)

This chapter demonstrates the application of decision trees to analysis of cost and effectiveness of clinics.  Health care managers often start clinics and programs.  They need to understand how much to charge for the services offered through their new investment.  They need to justify the effectiveness of clinics to various funding agencies.  They need to identify weak and money losing operations and work on improving them.  Unfortunately, the cost effectiveness of clinics is not always clear as it is difficult to distinguish program costs from other expenditures in the organization.  Many clinicians have multiple roles within the organization and it is not clear how much their time or effort is going into the new program versus other activities within the organization.  It is difficult to understand how personnel in management and other indirect costs affect the viability of the new clinic.  To make things more confusing, new clinics share facilities with existing programs making it difficult to charge them rent or allocate portion of capital expenditures to these clinics.  A rational approach would require us to isolate the cost of the program from other costs and to decide how much of the overhead should be carried by the program. 

In addition, program costs are just part of the picture.  Any program or clinic not only has its own services but also affects services offered by other units.  Offering a new program in cardiology might help identify patients for the existing home health care service of the organization.  If the new clinic is more modern and advance than existing operations (e.g. it has an electronic medical record or it uses new surgical equipment), then it might change the image of the entire organization and affect the referral to all existing clinics.  The new clinic might be the loss leader that attracts patients to other parts of the organization.  In short, any new investment may have unforeseen or planned consequences.  If we are evaluating the cost and effectiveness of a new clinic, it is important to go beyond program operations and costs and look at its impact on other services too.  This chapter shows how healthcare managers can isolate the costs of a clinic/program and trace the consequences of a service through use of decision analytic tools.

            Managers who have to justify the operations of a clinic to outside planning offices or insurance companies might be interested to look at the consequences of the clinic broadly.  Instead of just looking at how the clinic affects their existing services, they might want to look at the impact of the clinic on payers.  They might not want to limit the analysis to specific internal components but also include utilization of services outside of the organization.  In this chapter we show how such a broad analysis of consequences of opening a clinic can be carried out.

To demonstrate our ideas, we show how we applied them to evaluation of the cost effectiveness of co-locating a substance abuse treatment clinic within a probation agency.  Over 4 million adults are under probation or parole supervision.  Supervision failures, often linked to return to drug use, create a cycle back into prison and jails.  If supervision can be enhanced with treatment, then perhaps the cycle can be broken and offenders can return to community, not only restoring their lives but also perhaps saving money associated with crime and return to prison.  In recent years there have been a number of studies of cost effectiveness of substance abuse treatment,[i]-[ii][iii][iv][v][vi][vii][viii][ix][x][xi][xii] some of these studies have focused on cost effectiveness of programs for reducing recidivism.[xiii]-[xiv][xv][xvi][xvii][xviii][xix][xx][xxi]   We used a decision analytic model to evaluate the cost effectiveness of co-locating substance abuse treatment clinic within a probation agency.  For ease of reference, we refer to the clinic coordinated with the probation agency as the seamless probation and compare it to the traditional model of providing probation and substance abuse treatment services independently.

A Decision Analytic Model

Decision models have been used to model cost and benefits of a wide array of services,[xxii][xxiii][xxiv][xxv][xxvi][xxvii][xxviii][xxix][xxx][xxxi][xxxii][xxxiii][xxxiv]-[xxxv] including the cost and benefits of substance abuse prevention.[xxxvi],[xxxvii],[xxxviii],[xxxix]  The typical analysis (see Figure 1) starts with a decision node, contrasting joining the new clinic and the usual or standard care alternative.  Then the various events within the program are indicated (e.g. treatment retention).  In Figure 1 this is shown as a visit.  Since the analysis is done per day, we show this as a day in which a visit has occurred.  Next two costs are reported, the cost of the program and the costs of consequences of the program.  The cost of program refers to the personnel, material, information system, building and other capital and operating costs of delivering the program.  Of course, these costs have to be calculated per day in which there is a visit and a day in which there is no visit so that it corresponds to the unit of analysis in which probabilities are measured. 

Figure 1:  A Typical Decision Tree for Analysis of Cost Effectiveness of Clinics

The cost of consequences refers to subsequent costs that the patient or the organization may incur.  For example, a clinic visit may lead to additional referrals or in some cases to a hospitalization.  The cost of consequences is itself a separate decision tree.  In Figure 1 this is shown as node “C.”  This node is broken into a more detailed tree, in Figure 2.  The tree shows the daily probability of various consequences and the daily cost associated with each.    

Figure 2:  Typical Consequences for Analysis of Cost Effectiveness of Clinics

Note that in the analysis all probabilities and costs are calculated per day.  Thus we talk of daily probability of initiating treatment and daily probability of retaining the client.  For consequences, we talk of daily probability of needing a service and daily cost of various services.  In this fashion, a consistent unit of analysis is kept through out the analysis.  In Figure 2, for example, we are assuming that after the clinic visit the patient may be hospitalized or sent to specialist.  In addition, the figure has a place holder for other major events that might occur as a consequence of the clinic visit.  Finally, to make the list of possible events complete, we need to also include the situation where none of the major events imagined will happen.  Strictly speaking the events depicted must be mutually exclusive and exhaustive.  Some events, for example visiting a specialist and being hospitalized may occur on the same day and thus may not be mutually exclusive.  If this is not occurring often, the analysis depicted in Figure 2 may continue as is and be considered an approximation of the reality. 

Unlike traditional cost benefit analysis, a decision tree explicitly differentiates the probability of an event from its unit costs.  For example, in Figure 2 the probability of a day of hospitalization is separated from the cost of a day of hospitalization.  Because unit costs of services are unlikely to change under different alternatives, a decision analytic model reduces the number of estimates needed.  For example, while the new clinic is expected to affect the probabilities of hospitalization, it is not expected to affect the unit cost of a day of hospital care.  Differentiating the probability of the event from its costs allows us to rely, where necessary, on national estimates of unit costs.  Therefore, the method makes data collection easier and perhaps more accurate by breaking the analysis task into smaller components and using national estimates for components that do not change under various alternatives.

The decision tree in Figure 3 shows the overall structure of the analysis for seamless and traditional probation.  The left section of the tree depicts the decision.  Immediately after this node, the probability of receiving one day of probation or one day of treatment is shown.  Like our previous analysis these indicate utilization of services.  The middle section of the tree depicts treatment and probation outcomes, what we previously called consequences of clinic visits.  Next, the tree assumes that probabilities for a day of homelessness, a day of unemployment, a day of foster care for their children, a day of hospitalization, and a day in prison will vary with client’s probation and treatment status.  The right hand section of the tree depicts various costs incurred by payers in association with the consequences or recidivism or return to drug use.  These costs include the costs of one day of probation or one day of treatment.  Since the analysis relies on daily probabilities of the events, costs are also estimated per day. 


Figure 3:  Decision Tree for Seamless Probation

Perspective

In any analysis, the perspective of analysis dictates whose costs are included.  If one is analyzing costs to the patient, then cost to the hospital are not included.  If one is including costs to the organization only, then costs to external agencies are not included.  A societal perspective may include costs incurred to caregivers and other social components of care.  If the analysis is done for payers, then costs to these organizations are included and other costs are ignored.  In the seamless probation study, the focus of the analysis was costs and benefits to government agencies that were expected to fund the clinic.

Time frame

The impact of most social interventions is experienced over time--the entire life of the client and perhaps into other generations.  If we need to follow the consequences of opening a clinic, we need to decide how far in the future we want to look.  Many of the benefits occur several years later (e.g. reduced hospitalization) while the costs occur during the clinic visits.   It is therefore important to define precisely the timeframe for cost benefit analysis to discount future returns to current values.   For example, the benefit of seamless probation shows years later in the form of reduced crime.  To capture these, a sufficiently long perspective is needed.  The study on seamless probation examined the effect of the substance abuse clinic for 2-3 years after enrollment in the study.  This is long enough to see the impact of short term events but not long enough to capture life time events.   

Estimating Probabilities

This section discusses how the probabilities needed for the analysis are calculated.  There are four ways to assess these probabilities (see Table 1). 

 

Time to event

Frequency counts

Objective

Number of days to event is measured and transferred by formula to daily probabilities of the event

A large cohort of patients are followed and frequencies of various events calculated

Subjective

Experts are asked to estimate the number of days to the event & daily probabilities are calculated from these estimates

Experts are asked to estimate the frequency of events

Table 1:  Four Ways of Assessing Probabilities

The first method is to follow a large cohort of patients (preferably randomly assigned to various alternative arrangements) over time and count the frequency of the events.  This method is objective, requires considerable follow-up time and access to large number of patients.  For example, we might follow 100 patients in our clinic and report the frequency with which they are hospitalized.

The second method is to examine time between re-occurrences of events in the decision tree.  If we can assume the daily probability of the event has a binomial distribution, then time between the occurrences of the event has a Geometric distribution and the daily probability can be calculated by number of days in between the reoccurrence:

Daily probability of event = 1 / (1 + Number of days to reoccurrence of the event)

This method is objective but requires a shorter follow-up time than the first method, especially if the probabilities being assessed are relatively small.    For example, the daily probability of hospitalization might be calculated as number of days before the patient is hospitalized.  If a patient is hospitalized after 80 days, then the daily probability of hospitalization is 1/81. 

The third and fourth methods are same as the first two but based on experts opinions and not objective data.  We might ask about days to the event or the frequency of the event.  We might ask a clinician how many days before a patient is hospitalized; or we might ask an expert about the patient’s prognosis by asking them to specify the likelihood that the patient might be hospitalized within 30 days.

In the seamless probation example, we used the first method.  We recruited 272 offenders with extensive criminal justice histories and randomly assigned them to seamless and traditional probation. Offenders were interviewed at baseline and at three 12 month follow up periods to examine their utilization of various services.  Of the clients, 78.01% of the clients on traditional probation and 77.10% of the clients in the seamless group were available for follow-up interviews.   

When assessing objective frequencies, it is important to keep in mind that the decision tree requires the calculation of daily probability of various events.  These probabilities are calculated by dividing the length of various events by the total number of follow up days.  When calculated in this fashion, the probability of one day of an event is affected by the duration of the event.  For example, client’s length of stay in a treatment alters the probability of a day of treatment.  The longer the treatment program the larger the daily probability.  A client who is in treatment for one year will have a daily probability of one.  A client in treatment for one month in a year will have a daily probability of  8%.  As clients stop and re-start treatment, the daily probability changes.  Calculating probabilities in this fashion allows us to take into account the client’s length of treatment and multiple returns to treatment.  Table 2 provides average length of various events for the clients we followed and Table 3 turns these durations into daily probabilities. 

 

Traditional

Seamless

 

Average for 110 clients

Standard Deviation

Average for 101 clients

Standard deviations

Follow-up days

1001.20

308.60

1006.21

339.03

Treatment days

114.78

212.21

200.00

215.00

Probation days

410.12

195.54

456.81

213.52

Arrests in first year

1.00

1.40

0.86

1.09

Technical violation of probation

0.35

0.48

0.40

0.49

Days in prison

112.15

193.46

140.23

213.38

Days employed

378.65

390.59

391.32

439.38

Days in hospital (mental illness)

0.30

1.49

1.65

12.31

Days in hospital (physical illness)

0.25

1.70

0.12

1.19

Days in shelter

1.22

8.96

6.51

40.44

Table 2:  Average Duration of Various Events Per Client

 Table 2 shows that the clients in the seamless group had more treatment and probation days.  The consequences of these additional treatment and probation were slightly lower number of arrests but also higher frequency of technical violation, where the probation officer issued a warrant for the client to appear in front of the judge.  On the balance, the clients in the seamless group spent more days in prison/jail waiting to appear in front of the judge.  The clients in the seamless group also had higher rate of homelessness.  Across all these variables there were considerable variation and in none of the variables the seamless group had a statistically significant difference with the traditional probation group. 

Table 3 shows the probability of various consequences for each pathway in the decision tree in Figure 3.  This table is calculated by dividing the length of stay reported in Table by the follow-up period of each patient.  With the exception of arrest and technical violations, these probabilities reflect both the incidence of the event and the length of the event.  If seamless probation was more able to reduce adverse outcomes, one would expect lower numbers in rows associated with seamless probation.

Conditions

Probability of Event Given Probation & Treatment Conditions

Type of probation

Probation day

Treatment day

Technical violation

Arrest

Hospital day (mental)

Hospital day (physical)

Day in prison

Day employed

Day homeless

Traditional

No

No

0.0001

0.0005

0.0004

0.0003

0.1221

0.4430

0.0016

Traditional

No

Yes

0.0010

0.0007

0.0000

0.0000

0.3725

0.1250

0.0000

Traditional

Yes

No

0.0011

0.0020

0.0000

0.0001

0.0800

0.3432

0.0000

Traditional

Yes

Yes

0.0003

0.0005

0.0000

0.0000

0.0990

0.1663

0.0000

Seamless

No

No

0.0001

0.0001

0.0036

0.0002

0.1850

0.4631

0.0061

Seamless

No

Yes

0.0000

0.0000

0.0599

0.0000

0.3069

0.2873

0.0208

Seamless

Yes

No

0.0011

0.0016

0.0073

0.0000

0.1029

0.2629

0.0005

Seamless

Yes

Yes

0.0034

0.0012

0.0296

0.0000

0.0898

0.2787

0.0147

Total for traditional probation

0.0003

0.0027

0.0003

0.0002

0.1075

0.3993

0.0009

Total for seamless probation

0.0004

0.0024

0.0021

0.0001

0.1405

0.3824

0.0075

Table 3:  Conditional Probability of Consequences of Lack of Treatment and Probation
Probability of arrest was calculated for 1 year; all other rates calculated for 2.75 years.

 The decision tree in Figure 3 depicts treatment occurring after participation in probation; therefore, we need to show the conditional probability of treatment given probation.  This is calculated by dividing the joint probability of probation and treatment by the marginal probability of being in probation.  For example, using Table 4, the conditional probability of clients in seamless probation seeking treatment while they are in probation was 0.14/0.45 = 0.30.  In contrast, the same conditional probability for clients who were in a traditional probation was 0.05/0.41  

 

110 Traditional Clients

101 Seamless Clients

 

Not a probation day

Probation day

Total

Not a probation day

Probation day

Total

Not a treatment day

0.53

0.36

0.89

0.48

0.32

0.80

Treatment day

0.06

0.05

0.11

0.06

0.14

0.20

Total

0.59

0.41

1.00

0.55

0.45

1.00

Table 4:  Probability of successfully completing a day of probation & treatment

Daily Cost of Clinics

The daily cost of a clinic or program is calculated from dividing the total cost of a program during a year by its yearly census.  Program costs include operating and fixed costs of the program during the year.  Yearly census is the number of days clients were enrolled in the program during last year.  This creates some counter intuitive relationships.  Very expensive programs may have a low daily cost if they also have high census.  Likewise, inexpensive programs may have a high daily costs if the census is low.  As census increases daily costs usually go down.  This section describes how to estimate both the program costs and its census. 

Program costs are estimated from the organization’s budget.  Typically the budget provides cost of personnel, supplies, equipment, buildings and information services.  To these costs one adds the market value of donated buildings, volunteer services and unaccounted retirement costs.  Economic cost of a clinic differs from accounting cost as it includes the value of assets or personal services donated to the program.  For example, accounting procedures usually depreciate building costs.  This distorts real market value of the asset.  To correct for these inaccuracies within the budget, whenever possible the analysis should rely on lease value of major assets such as buildings, office space or information services.   

When a picture of true economic costs is established, then costs are allocated to various programs within the organization.  Sometimes these allocations are clear – as when the budget of the clinic is separate from other operations.    Other times costs of various programs and clinics are mixed together in the same budget and an allocation scheme must be decided upon.  Table 5 provides an allocation scheme for various component of the budget.

Budget Category

Allocation Scheme for Shared Items

Clinical and support personnel

Distribution of personnel’s time

Management personnel

Distribution of clinic and support personnel

Information services

Frequency of requests to the service

Equipment cost

Frequency of use and age of equipment

Building

Square footage of shared use is allocated based on patient census.   

Supplies

Personnel if supplies are used by employee otherwise proportional to census if supplies are used by patients

Utilities & other overheads

Total cost of clinic and other operations (after above allocations)

Table 5:  Allocation Schemes for Calculating Program Cost from Budgets

The cost within many budget categories are allocated to the program using the activity of employees by the following formula:

Cp, c = (Cb,c+Cm,c) Ep/Eb

where:

Cp, c

is the program cost in budget category “c.”

Cb, c

is the cost to the entire organization in budget category “c” – estimated from the budget

Cm, c

is the market value of donated services or unreported capital resources in budget category “c”

Ep

is the number of full time equivalent working in the program

Eb

is the number of full time equivalent working in the entire organization

The key for allocation of organization’s budget to program costs is determination of personnel activities.  Clearly some personnel will have dual roles and it is important to ask the percent of time they allocate to various activities.  This is typically done by a survey of employees.   Because of the focus on employee activities, the approach described above is often called the Activity Based Costing (ABC),

In the seamless probation, two new operations were introduced: a new clinic for providing substance abuse treatment and a new way of doing probation.  The study estimated daily cost of both operations.  The daily cost of providing the probation was calculated from the budget of the probation agency.  Costs not directly available through the accounting system were added in.  Building costs were based on lease value of equivalent office space.  Cost of information services provided by other State agencies, not directly on the budget of the probation agency, was added in.  Table 6 shows the cost of seamless and traditional probation.  The first column shows the total agency budget.  The three columns to the right of it show the allocation of these costs to various probation programs (investigative reporting, seamless supervision and traditional supervision).

 

 

 

June 30 2000 to July 1st 2001 Costs

 

 

 

Agency costs

Investigative reporting++

Seamless supervision++

Traditional supervision++

Personnel services

$1,191,362

$163,182

$79,320

$948,859

Contractual services

$11,984

$1,641

$798

$9,544

Supplies & materials

$9,436

$1,293

$628

$7,516

Building rental

$206,144

$28,236

$13,725

$164,183

Equipment rentals*

$122,083

$16,722

$8,128

$97,233

Information services+

$148,621

$20,357

$9,895

$118,369

Economic cost of volunteers

5,013

$687

$334

$3,993

Total

$1,694,643

$232,117

$112,828

$1,349,697

Cost per work day

$6,009

$823

$400

$4,786

Number of client-days

 

15,792

9,588

206,424

Cost per day per client

 

$15

$12

$7

 *Estimated from market lease value

 ++ Personnel, Contractual, Supplies, Building, Equipment, Information services, and volunteer costs were allocated proportional to activities of probation officers involved in investigative reporting, seamless and traditional supervision

 +Estimated from State and City operating budgets

 

Table 6:  Cost of Probation Per Day and Per Client

This table is based on Alemi F, Taxman F, Doyon V, Thanner M, Baghi H.  Activity Based Costing of Probation with and without Substance Abuse Treatment:  A Case Study.  Journal of Mental Health Economics.  J Ment Health Policy Econ. 2004 Jun;7 (2):51-7.

            A similar procedure was followed for cost of the substance abuse treatment clinic.  Table 7 shows the cost of outpatient treatment received by clients at the clinic.  The first column shows the total budget of the clinic.  This clinic has three programs and the total budget was allocated to these three programs based on personnel activities and use of services.  Of particular interest is the fact that centralized management costs did not show in the budget of the clinic but needed to be added in as a portion of these services were for the clinic.    

Category Total CROP
 program
Outpatient
program
Methadone
program
Personnel 1,266,651 64,425 732,452 469,774
Building lease value 75,435 5,372 48,275 21,788
Equipment lease value 65,420 2,456 26,773 36,191
Operations 47,425 1,259 15,888 30,278
Centralized management 397,259 15,814 240,449 140,996
Total costs 1,852,189 89,325 1,063,836 699,027
Enrollment days estimated from Panel size 9,490 89,790 40,150
Time between discharges 11,043  44,860 57,945
Cost per enrollment day 8.7 15.8 14.25

Table 7:  Cost of a Day of Treatment

Based on Alemi F, Sullivan T.  An example of activity based costing of treatment
programs using time between discharges (in review)

Clinic Census

A key factor in estimation of cost per day of service is the estimation of number of days of enrollment, what we call program census.  It is relatively simple to calculate the cost per visit; one could count the number of visits.  But clients stay in a service even when they do not visit the clinic.  Furthermore, the analysis needed to estimate the cost per enrollment day (i.e. the days from admission to treatment to discharge) and not per visit as daily probabilities were being used.  There are three methods to estimate enrollment days.  If admission and discharge dates are available, enrollment can be measured from the difference.  Sometimes this data is not readily available.  In the second method, the clinicians can be asked to estimate their panel size during last month.  Then enrollment days are calculated as 365 times the panel size.  The third method is to look at time between most recent discharge and admission dates for the clinician.  All three methods are subject to errors as clinicians may overestimate their panel size and historical discharge and admission dates may not reflect recent patterns.  To make a more stable estimate, the enrollment days estimated from the various methods can be averaged.  In addition, the range of the estimates can be used to guide the sensitivity analysis.

Table 7 shows estimates of the enrollment days for three different programs.  There is considerable variability between the estimates.  In two programs the two estimates are close to each other but for the outpatient program the two estimates are considerably different.     

Cost of Consequences

Typically, daily cost of consequences (e.g. day of hospitalization) does not change across alternatives.  The clinic and the standard care will differ in the frequency of occurrences of various consequences but not the daily cost of each occurrence.  Therefore, these daily costs can be estimated from national or regional values available through the literature. 

For example, the decision tree in Figure 3 separates the probability of arrest from its costs.  Because cost of arrest is unlikely to change with the use of seamless probation program, national estimates were used for these costs.   Table 8 shows the estimated cost of arrest and court processing in 2001 and 2004 values. 

 

Number of
cases in millions

Expenditure in
2001 in millions

Cost per case
in 2001

Cost per case
inflated to 2004 prices

Police arrests

13.7

$72,406

$5285.11

$6330.35

Adult judicial cases

92.8

$37,751

$406.80

$487.25

Table 8:  Cost of Arrest & Court Processing

Includes local, State and Federal costs.  Data from Bauer L, Owens SD.  Justice expenditure and Employment in the United States 2001, Bureau of Justice Statistics Bulletin, May 2004.

Table 9 shows the daily cost of various consequences as estimated from published national or regional data.  The cost of a day of employment was the exception to the rule.  This cost was estimated by the tax paid by offenders on legal income they had during probation.

Cost of Consequences

Tech. violation

Arrest

Hospital day (mental)

Hospital day (physical)

Day in prison

Day employed

Day of shelter

$487

$6,818

$1,164

$1,868

$74

($1.50)

$30

Table 9:  Cost Per Occasion or Per Day of a Consequence

The cost of various consequences can be calculated by multiplying the daily probability of the consequence by its daily costs.  Table 10 provides the expected cost for the eight pathways in the decision tree in Figure 3.   

Condition

Cost of Consequences

Cost of programs

Type of probation

Prob. Day

Treat. day

Tech. violation

Arrest

Hospital day (mental)

Hospital day (physical)

Day in prison

Day employed

Day of shelter

Cost of prob.

Cost of treat

Traditional

No

No

0.03

3.27

0.45

0.58

9.04

(0.66)

0.05

0

0

Traditional

No

Yes

0.51

4.53

0.00

0.00

27.56

(0.19)

0.00

0

15.8

Traditional

Yes

No

0.54

13.70

0.05

0.18

5.92

(0.51)

0.00

7

0

Traditional

Yes

Yes

0.16

3.74

0.00

0.00

7.33

(0.25)

0.00

7

15.8

Seamless

No

No

0.04

0.64

4.20

0.33

13.69

(0.69)

0.18

0

0

Seamless

No

Yes

0.00

0.00

69.77

0.00

22.71

(0.43)

0.63

0

15.8

Seamless

Yes

No

0.56

10.96

8.50

0.00

7.61

(0.39)

0.01

12

0

Seamless

Yes

Yes

1.68

8.24

34.49

0.00

6.64

(0.42)

0.45

12

15.8

Cost per day or occasion

$487

$6,818

$1,164

$1,868

$74

($1.50)

$30

 

 

Difference of seamless & traditional expected costs

$0.17

($2.31)

$13.50

($0.20)

$2.08

$0.01

$0.17

$2.58

$1.24

Table 10:  Expected Cost in Different Decision Tree Paths

Expected Costs

As mentioned before, expected cost can be calculated by folding back a tree, where each node is replaced by its expected value.  Another way of calculating the expected cost, a method that is easily implemented within Excel, is to calculate the joint probability of events within each pathway and multiply this probability by the total costs incurred during the pathway.  For example, in Figure 1 there are four pathways, each with a probability of occurring and corresponding program and consequence costs.  The expected value of this tree can be calculated by first calculating the expected cost of consequences by multiplying the cost of each consequence by its probability and summing across all consequences.  Next, the expected cost for each pathway is calculated by multiplying the probability of the path by its total costs (sum of program costs and expected cost of consequences).  The expected cost of each alternative is calculated by summing the expected cost of each pathway that emerges from the alternative.

The expected cost for seamless and traditional probation was calculated in three steps.  First, we calculated the expected cost of consequences by multiplying the probability of each consequence by its costs.  This is shown in columns 4 through 10 in Table 10.  Next for each pathway, the expected cost of the pathway was calculated.  This was done by summing the cost values reported in rows in Table 10 and multiplying the total by the joint probability of events in the pathway (product of probabilities in column one and two).  Finally, the expected cost of seamless probation was calculated by summing the cost of pathways that followed from joining seamless probation (rows five through 8 in Table 10).  The expected cost of traditional probation was calculated by summing the cost associated with the pathways that followed traditional probation (rows 1 through 4).  The expected cost for seamless probation was $38.84 per follow-up day per client.  The expected cost for traditional probation was $21.60 per follow-up day per client.  The net difference was $6,293 per client per year.  As anticipated, seamless probation led to reduced arrest rates; this led to $2.31 reduction in expected cost per client per follow-up day.  This cost savings was not enough to compensate for the increased cost of mental hospitalization ($13.50 per client per follow-up day), increased cost for delivery of seamless probation ($2.58 per client per follow-up day), additional cost due to use of prison/jail ($2.08 per client per follow-up day), and increased cost of providing treatment ($1.24 per client per follow-up day).  Therefore, co-locating the clinic within probation had not led to costs that were less than traditional probation.

Sensitivity Analysis

See a video on how to conduct single parameter sensitivity analysis

Numerous assumptions were made concerning the estimation of costs and probabilities.  Concern over the accuracy of these estimates leads one to conduct sensitivity analysis.  For each parameter in the decision tree, a breakeven point is found:  the parameter is changed until the conclusion is reversed.  The percent of change to reach the breakeven point is reported.  In addition, where alternative estimates were available, the alternative estimates are used to see if conclusions change.   

Decision makers are often concerned how much confidence they should put in the analysis.  Statisticians answer these concerns by measuring statistical significance of differences between new clinic and standard care.  In decision analysis, one way to help decision makers gain confidence in the analysis is to conduct sensitivity analysis.  First, single parameters are changed.  Then two parameters are changed at same time, finally multiple parameters are changed.  If conclusions are insensitive to reasonable changes in the parameters, then decision makers gain confidence in the analysis.

Table 11 shows the sensitivity of the conclusion in the seamless probation case to changes in rates of any one of the consequences.   There was no reduction in rate of any single adverse outcome that could make seamless probation more cost effective than traditional probation (e.g. even when arrest rate of the seamless population was set to zero the traditional probation was still more cost effective).  The breakeven points were examined for simultaneous changes in several variables.  A 54% reduction in all adverse outcome rates would have made seamless probation more cost effective than traditional probation.  The analysis was most sensitive to reductions in arrest, mental hospitalization and incarceration rates.   57% reduction in these three rates would have made seamless probation more cost effective.  When examining the sensitivity of the conclusion to simultaneous changes in two rates, the analysis was most sensitive to changes in mental hospitalization rates and incarceration rates.  A  69% reduction in these two rates would have made seamless probation more cost effective. 

 

 

Initial rate

Breakeven rate

Percent of initial rate

Changes in estimates for seamless probation

Technical violation

0.0004

None

N/A

Arrest rate

0.0010

None

N/A

Mental hospitalization

0.0021

None

N/A

Hospitalization (physical)

0.0001

None

N/A

Incarceration

0.1405

None

N/A

Employment

0.3824

None

N/A

Homeless

0.0075

None

N/A

All adverse outcome rates

1

0.4609

46%

Arrest, mental hosp & incar. rates

1

0.4255

43%

Arrest & mental hosp rates

1

None

N/A

Mental hosp & incarceration rates

1

0.3130

31%

Changes in estimates for traditional probation

Technical violation

0.0003

None

N/A

Arrest rate

0.0027

0.0093

339%

Mental hospitalization

0.0003

0.0193

7042%

Hospitalization (physical)

0.0074

0.0925

1245%

Incarceration

0.1075

0.3145

293%

Employment

0.3993

None

N/A

Homeless

0.0009

0.6597

69877%

Cost of arrest

$6,818

$57,721

847%

Cost of seamless probation

12

None

N/A

Table 11:  Sensitivity of Conclusion to Changes in One Estimate

 The sensitivity of conclusions to changes in any one of the estimated daily costs was examined.  Note that both the traditional and seamless probation have the same daily cost for all adverse events and therefore small changes in these estimates are unlikely to affect the difference between the two groups.  For example, the daily cost of arrest is the same for both seamless and probation.  The cost of arrest had to increase by 8 fold (from $6,818 to $57.721) before the conclusion that traditional probation is more cost effective is reversed.  This analysis suggests that small variations in daily cost estimates of adverse outcomes were unlikely to affect the conclusion.

Conclusions

This chapter has show how cost and effectiveness of a clinic can be examined.  A decision tree allows the calculation of cost effectives to be broken down into several estimates:  assessing daily probability of enrolling in clinic services, daily probability of facing various consequences, daily cost of clinic operations, and daily cost of various consequences.  The later,  is available through the literature and the former variables can be measured through tracking a large cohort of patients, through asking time to events of interest or through subjective estimates of experts familiar with the clinic operations.    

The advantage of decision analytic evaluation of a clinic is that it reduces the number of estimates needed as daily cost of consequences can be obtained from the literature.  In addition, sensitivity analysis could be used to understand how conclusions might depend on various estimates.  When conclusions are sensitive to the estimated model parameters, then additional data should be collected to improve the precision of the estimates.

We showed the application of the concepts to measurement of cost effectiveness of substance abuse clinic coordinated with probation, the so called seamless probation.  The total cost of seamless probation exceeded traditional probation by $6,293 per client per year.  Sensitivity analysis suggested that the analysis was not sensitive to small changes in the estimated parameters. 

References

 

[i]           Shepard, D.S., Reif, S. (2004). The value of vocational rehabilitation in substance user treatment: a cost-effectiveness framework.  Substance Use and Misuse. 39(13&14), 2581-2609, 2004.

[ii]           Kunz, F. M., French, M. T., & Bazargan-Hejazi, S. (2004). Cost-effectiveness analysis of a brief intervention delivered to problem drinkers presenting at an inner-city hospital emergency department. Journal of Studies on Alcohol, 65, 363-370.

[iii]          Kedia, S. & Perry, S. W. (2003). Substance abuse treatment effectiveness of publicly funded clients in Tennessee. Journal of the National Medical Association, 95, 270-277.

[iv]          Kaskutas, L. A., Witbrodt, J., & French, M. T. (2004). Outcomes and costs of day hospital treatment and nonmedical day treatment for chemical dependency. J Stud.Alcohol, 65, 371-382.

[v]           Jofre-Bonet, M., Sindelar, J. L., Petrakis, I. L., Nich, C., Frankforter, T., Rounsaville, B. J., & Carroll, K. M. (2004). Cost effectiveness of disulfiram: treating cocaine use in methadone-maintained patients. J Subst.Abuse Treat., 26, 225-232.

[vi]          French, M.T., Roebuck, M.C., Dennis, M. et al.  (2002).  The economic cost of outpatient marijuana treatment for adolescents:  findings from a multi-site field experiment.  Addiction, 97(suppl 1), 84-97.

[vii]         Doran, C. M., Shanahan, M., Bell, J., & Gibson, A. (2004). A cost-effectiveness analysis of buprenorphine-assisted heroin withdrawal. Drug and Alcohol Review, 23, 171-175.

[viii]        Dismuke, C. E., French, M. T., Salome, H. J., Foss, M. A., Scott, C. K., & Dennis, M. L. (2004). Out of touch or on the money: Do the clinical objectives of addiction treatment coincide with economic evaluation results? J Subst.Abuse Treat., 27, 253-263.

[ix]          Dennis, M., Godley, S. H., Diamond, G., Tims, F. M., Babor, T., Donaldson, J., Liddle, H., Titus, J. C., Kaminer, Y., Webb, C., Hamilton, N., & Funk, R. (2004). The Cannabis Youth Treatment (CYT) Study: Main findings from two randomized trials. J Subst.Abuse Treat., 27, 197-213.

[x]           Daley, M., Love, C.T; Shepard, D.S., Peterson, C.B., White, K.L., Hall, F.B. (2004).  Cost-effectiveness of Connecticut’s in-prison substance abuse treatment.  Journal of Offender Rehabilitation. 39(3), 69-92.

[xi]         Caulkins, J.P., Pacula, R.L., Paddock, S. & Chiesa.  (2004).  What we can—and cannot—expect from school-based drug prevention.  Drug and Alcohol Review.  23, 79-87.

[xii]         Berger, L. M. (2002). Estimating the benefits and costs of a universal substance abuse screening and treatment referral policy for pregnant women. Journal of Social Service Research, 29, 57-84.

[xiii]        McCollister, K. E., French, M. T., Prendergast, M. L., Hall, E., & Sacks, S. (2004). Long-term cost effectiveness of addiction treatment for criminal offenders. Justice Quarterly, 21, 659-679.

[xiv]         McCollister, K. E., French, M. T., Inciardi, J. A., Butzin, C. A., Martin, S. S., & Hooper, R. M. (2003). Post-release substance abuse treatment for criminal offenders: A cost-effectiveness analysis. Journal of Quantitative Criminology, 19, 389-407.

[xv]          Logan, T. K., Hoyt, W. H., McCollister, K. E., French, M. T., Leukefeld, C., & Minton, L. (2004). Economic evaluation of drug court: methodology, results, and policy implications. Evaluation and Program Planning, 27, 381-396.

[xvi]         Fass, S. M. & Pi, C. R. (2002). Getting tough on juvenile crime: An analysis of costs and benefits. Journal of Research in Crime and Delinquency, 39, 363-399.

[xvii]        Aos, S.  (2002).  The Juvenile Justice System in Washington State: Recommendations to Improve Cost-Effectiveness, Washington State Institute for Public Policy, October 

[xviii]       Aos, S., Phipps, P., Barnoski, R., & Lieb, R.  (2001) The Comparative Costs and Benefits of Programs to Reduce Crime, v 4.0, Washington State Institute for Public Policy, May.

[xix]         Aos, S., Lieb, R., Mayfield, J., Miller, M. & Pennucci, A.  (2004) Benefits And Costs Of Prevention And Early Intervention Programs For Youth. Washington State Institute for Public Policy, September.

[xx]          Aos, S. (2003)  The Criminal Justice System in Washington State: Incarceration Rates, Taxpayer Costs, Crime Rates, and Prison Economics, Washington State Institute for Public Policy, January

[xxi]         Aos, S. & Barnoski, R.  (2003).  Washington State's Drug Courts For Adult Defendants: Outcome Evaluation And Cost-Benefit Analysis,  Washington State Institute for Public Policy,  March

[xxii]        Varghese B, Peterman TA, Holtgrave DR. Cost-effectiveness of counseling and testing and partner notification: a decision analysis.  AIDS. 1999 Sep 10;13(13):1745-51. 

[xxiii]       Marley DS. Decision and cost-effectiveness analysis.  J Fam Pract. 1990 Feb;30(2):233, 236.

[xxiv]        Davis JE.  Decision analysis: a prescriptive method for decision and cost-effectiveness research.  J Fam Pract. 1989 Oct;29(4):367-9. 

[xxv]        McNeil BJ.  Changing roles of decision analysis and cost-effectiveness analyses in medicine and radiology.  Eur Radiol. 2000;10 Suppl 3:S340-3.

[xxvi]        Carlos RC, Bree RL, Abrahamse PH, Fendrick AM. Cost-effectiveness of saline-assisted hysterosonography and office hysteroscopy in the evaluation of postmenopausal bleeding: a decision analysis.  Acad Radiol. 2001 Sep; 8(9): 835-44. 

[xxvii]       Palmer CS, Brunner E, Ruiz-Flores LG, Paez-Agraz F, Revicki DA. A cost-effectiveness clinical decision analysis model for treatment of Schizophrenia.  Arch Med Res. 2002 Nov-Dec;33(6):572-80. 

[xxviii]      Post PN, Kievit J, van Bockel JH.  Optimal follow-up strategies after aorto-iliac prosthetic reconstruction: a decision analysis and cost-effectiveness analysis.  Eur J Vasc Endovasc Surg. 2004 Sep;28(3):287-95.

[xxix]        Kocher MS, Henley MB. It is money that matters: decision analysis and cost-effectiveness analysis.  Clin Orthop Relat Res. 2003 Aug;(413):106-16. 

[xxx]        Inadomi JM. Decision analysis and economic modelling: a primer.  Eur J Gastroenterol Hepatol. 2004 Jun;16(6):535-42. Review.

[xxxi]        Sonnenberg A. Decision analysis in clinical gastroenterology.  Am J Gastroenterol. 2004 Jan;99(1):163-9. Erratum in: Am J Gastroenterol. 2004 Feb;99(2): following 398.

[xxxii]       You JH, Chan FW, Wong RS, Cheng G. The potential clinical and economic outcomes of pharmacogenetics-oriented management of warfarin therapy - a decision analysis.  Thromb Haemost. 2004 Sep; 92(3): 590-7. 

[xxxiii]      Targownik LE, Spiegel BM, Sack J, Hines OJ, Dulai GS, Gralnek IM, Farrell JJ. Colonic stent vs. emergency surgery for management of acute left-sided malignant colonic obstruction: a decision analysis.  Gastrointest Endosc. 2004 Dec;60(6):865-74.

[xxxiv]      Culligan PJ, Myers JA, Goldberg RP, Blackwell L, Gohmann SF, Abell TD. Elective cesarean section to prevent anal incontinence and brachial plexus injuries associated with macrosomia--a decision analysis.

[xxxv]       Lejeune C, Al Zahouri K, Woronoff-Lemsi MC, Arveux P, Bernard A, Binquet C, Guillemin F. Use of a decision analysis model to assess the medicoeconomic implications of FDG PET imaging in diagnosing a solitary pulmonary nodule.  Eur J Health Econ 2005 Apr 15. 

[xxxvi]      Jordan TJ, Lewit EM, Montgomery RL, Reichman LB. Isoniazid as preventive therapy in HIV-infected intravenous drug abusers. A decision analysis.  JAMA. 1991 Jun 12;265(22):2987-91.

[xxxvii]     Malow RM, West JA, Sutker PB. Anxiety and pain response changes across treatment: sensory decision analysis.  Pain. 1989 Jul;38(1):35-44. 

[xxxviii]    Zarkin GA, Hubbard RL.  Analytical issues for estimating the benefits and costs of substance abuse prevention. 

[xxxix]      Zarkin GA, French MT, Anderson DW, Bradley CJ.  A conceptual framework for the economic evaluation of substance abuse interventions.  Evaluation and Program Planning, 1994, 17: 4, 409-418.

What do you know?

  1. Using Excel and the data provided in Table 3, 4 and 10, calculate the expected cost of consequences associated with days clients are in probation but not in treatment.  This is done by first reading the data on cost and probabilities into Excel.  See how data can be read from Internet Explorer into Excel Then you need to multiply the probability of incurring a cost by its amount and sum over all possible consequences.  See how expected costs are calculated inside Excel.  Make sure that all calculations are done using relative cell values and not by entering the result by hand.  Make sure that probability of opposite events are calculated as one minus the event.  Make sure that cell values in the table are defined relative to marginal values.  Check that your answers correspond roughly with the answers in the table 10 to make sure that formulas lead to same answers as the reading.
  2. Using Excel and the data provided in Table 3, 4 and 10 of the reading, calculate the expected cost of seamless probation and traditional probation per client per day.  These three tables show either the probability or the cost.  The expected cost can be calculated as the sum of probability of incurring a cost times the dollar amount of the cost.  For seamless and traditional probation, this is done by multiplying probability of four situations (1. In probation and in treatment, 2. In probation but not in treatment, 3. Not in probation but in treatment and 4. Not in probation and not in treatment) by the cost at each of these situations.  The cost at any of these situations is calculated as the cost of consequences (see previous question) plus cost of program (probation or treatment).  See how expected costs are calculated inside Excel.  Make sure that all calculations are done using relative cell values and not by entering the result by hand.  The expected value should be calculated as the probability of combination of probation and treatment times the cost of that combination.  Check that your answers correspond roughly with the answers in the table to make sure that formulas lead to same answers as the reading.
  3. Conduct sensitivity analysis on the data by making single parameter changes in the decision tree in the section on seamless probation.  Before doing so make sure that the probability of opposite events are calculated as one minus the probability of the event.  See a video on how to do this inside Excel.  See a video on how to conduct single parameter sensitivity analysis.  Report the breakeven points for the following parameters:
     
      Name of parameter changed Current value Value at breakeven point Percent of change to reach break even point
    1. Probability of a probation day      
    2 .Probability of treatment day given probation      
    3. Probability of technical violation given probation and treatment day      
    4. Probability of arrest given  probation and treatment day      
    5. Probability of technical violation given probation day and no treatment      
    6. Probability of mental health hospitalization given probation and treatment days      
    7. Probability of employment given probation and treatment days      
    8 Cost of treatment      
    9 Cost or seamless probation      
    10. Cost of arrest      
             
  4. Draw a chart showing the sensitivity of conclusions of the analysis to changes in the probability of arrest.   Put on the X-axis the probability of arrest.  On the Y-axis put the expected cost.  Draw two lines one showing how the expected cost of seamless probation changes when the probability of arrest in the seamless probation changes from 0 to 1.  Draw another line showing how the expected cost for traditional probation changes when the cost of arrest in the seamless probation changes from 0 to 1.   Note the point when the two lines meet.  This is the point at which the conclusion regarding which program is preferred is reversed. 
  5. Conduct a multi-parameter sensitivity analysis by simultaneously allowing following changes:
    • 30% increase in the cost of arrest (from $6,818 up to $8,863),
    • Any change in the probability of arrest in the seamless probation and treatment group (from 0 to 1)
    • Any change in the probability of arrest in the traditional probation and treatment group (from 0 to 1).  

    Report what parameters need to change to arrive at a breakeven point, where current conclusions are reversed.  See a video on how to conduct multi-parameter sensitivity analysis.  In order to accomplish this assignment, instruct the Excel program to set the difference between the expected cost for traditional and seamless probation to zero subject to several constraints.  Include at least the following constraints:
     

    • Cost of arrest <= 8863
    • Cost of arrest >= 6818
    • Probability of arrest in seamless probation & treatment group >= 0
    • Probability of arrest in seamless probation & treatment group <= 1
    • Probability of arrest in traditional probation & treatment group >= 0
    • Probability of arrest in traditional probation & treatment group >= 0

    Report if there is a combination of changes in these estimates that would set the difference of expected value of seamless and traditional probation equal to zero. 

Send your response by email to your instructor.  Include both the question and the answers in your response.  Include your contact information.

Bi-weekly assignment

Please note that this assignment is not currently assigned to students in HSCI 730. 

  1. Estimate the daily and per visit cost of a clinic operation.  The purpose of this assignment is to use the steps described here to analyze cost of operating a clinic.  To accomplish this task, use your own familiarity with the clinic operations to estimate the following:
    • Proportion of employees working in the clinic
    • Proportion of volunteers to employees within the clinic
    • Proportion of patients cared for in the clinic
    • Square footage used by the clinic based on your estimate of the square footage used by the clinic exclusively and the square footage shared among clinics.
    • Number of clients served in the organization and in the clinic in the last year or last month.
    • Panel size of clinicians working in the clinic
    • Time between visits per client

    Follow these steps to accomplish the cost analysis:

    • Select a publicly available operating budget of a health care organization, preferably one in which you work or one in which you have a friend who is interested in your help in analyzing their costs. 
    • Identify the various clinics within the organization and based on proportion of employees who work in the clinic allocate the operating budget to the the cost of the clinic.  Divide the operating budget into personnel and other operating costs.   Increase the personnel expenses of the clinic proportional to the ratio of volunteers to employees within the clinic. 
    • Add to the clinic cost the building capital costs.  Estimate this based on your estimated square footage used by the clinic times market value of medical office space in the zip code of the clinic.    Collect this information from real estate agents in your community or through Internet.
    • If the clinic relies on information systems or medical record provided by other units of the organization and not part of the operating budget you have analyzed, add this cost into the total expense proportional to number of clients served in the clinic. 
    • Estimate the daily clinic census from the panel size of clinicians
    • Estimate the number of visits of an average client (estimate this as 1 divided by 1 plus time between visits for an average client)
    • Estimate the total number of visits during the last year by multiplying the number of visits of the average client by the number of clients.
    • Report the daily cost of operating the clinic and cost per visit.
    • Report which source of data in your analysis needs additional precision and what steps you can take to collect this information.  Include estimate of how much time would be needed to collect this information.
  2. Analyze what are the consequences of purchasing a physician primary care practice on a tertiary hospital system.  Select a clinic and tertiary clinical service group, preferably settings you are familiar with or have access to someone who is familiar with their operations.  Estimate the variables needed based on your knowledge of these organizations and publicly available data.  Follow these steps:
    • Create a decision model that as its first decision node has whether the to purchase or not to purchase the primary care office.  The chance node should indicate the frequency of visits to the primary care setting, frequency of visits to specialists and the probability of hospitalization at tertiary hospital after visit to a specialist.
    • Estimate the probabilities for the model and use publicly available estimates of the cost of clinic visits and hospital visits.  Adjust the cost of hospitalization based on your estimate of differences in case mix in the tertiary hospital and the types of patients needing hospitalization in the primary care setting. 
    • Report the expected increase in revenues if the office is purchased.
    • Conduct single variable sensitivity analysis to see which estimate is most likely to affect the expected increase in revenues.  Indicate how much of the additional revenue comes from direct primary care visits and how much from subsequent referrals.
    • Report on the availability of the data needed to conduct the analysis.  Where would you look for each data item and how long do you think collection of the data will take.

Presentations

  1. Listen to lecture on cost effectiveness
  2. Download the slides for the lecture. 
  3. See a video on how to conduct single parameter sensitivity analysis.
  4. See a video on how to conduct multi-parameter sensitivity analysis. 
  5. See videos on how you can calculate Expected cost within Excel:

More

See Agency for Healthcare Policy and Research's synthesis of evidence regarding cost effectiveness of clinical practices.  Evidence-based Practice Centers (EPC) is a program of the Agency for Healthcare Research and Quality. These centers review all relevant scientific literature on clinical, behavioral, and organization and financing topics. More information is at http://www.ahrq.gov/clinic/epc/
  The site http://www.pitt.edu/~tjs/econtab.html provides information on cost effectiveness of diabetes treatment practices.  Included are a large number of published studies.
Cost Effectiveness and Resource Allocation is an online journal published by BioMed Central.  It can be found at http://www.resource-allocation.com/home/  
  Benjamin Johns , Rob Baltussen and Raymond Hutubessy from Global Programme on Evidence for Health Policy (GPE/EQC) of the World Health Organization have written a paper on methdology of doing cost effectiveness analysis.  The paper "Programme costs in the economic evaluation of health interventions" was published in Cost Effectiveness and Resource Allocation 2003, 1:1 doi:10.1186/1478-7547-1-1.  The electronic version of this article is can be found online at: http://www.resource-allocation.com/content/1/1/1  
For a list of recent articles published on procedures for conducting cost effectiveness, click here
Taghreed Adam , David B Evans and Christopher JL Murray from Global Programme on Evidence for Health Policy (GPE/EQC) of the World Health Organization are gathering information on hispital costs across countries.  Details are available in the article titled: Econometric estimation of country-specific hospital costs.  This article is published in Cost Effectiveness and Resource Allocation 2003, 1:3 doi:10.1186/1478-7547-1-3.  The electronic version of this article can be found online at: http://www.resource-allocation.com/content/1/1/3
  The cost effectiveness of screening has been studied in a number of studies.  An example is the study conducted by Neil Simpson , Rob Anderson , Franco Sassi , Alexandra Pitman , Peter Lewis , Karen Tu and Heather Lannin.  The study was titled: he cost-effectiveness of neonatal screening for Cystic Fibrosis: an analysis of alternative scenarios using a decision model.  It was published in Cost Effectiveness and Resource Allocation 2005, 3:8 doi:10.1186/1478-7547-3-8.  The electronic version of this article can be found online at: http://www.resource-allocation.com/content/3/1/8
There are numerous books published on cost effectiveness methods.  These books generally do not approach the analysis from a decision theoretic point of view.  Nevertheless, they are excellent coverage of methods and issues.  Some examples of books include:
  • Cost-Effectiveness in Health and Medicine by Marthe R. Gold (Editor) (Hardcover - June 1996)
  • Designing and Conducting Cost-Effectiveness Analyses in Medicine and Health Care (Jossey-Bass Health Care Series) by Peter Muennig, Kamran Khan (Hardcover)
  • Cost-Effectiveness Analysis : Methods and Applications (1-Off) by Henry M. Levin, Patrick J. McEwan (Paperback)

For a list of books in this area click here.

 

This page is part of the course on Decision Analysis.  It was last revised on 09/28/2015 by Farrokh Alemi, Ph.D.