Georgetown University's Health Systems Administration
Process Improvement

Rubric for Evaluating the Modeling Uncertainty Project

When you are learning a topic, you learn better by teaching it.  To accomplish this goal, from time to time we ask you to comment on another student's work.  To help you do so we provide a rubric that you need to follow closely.  This rubric applies to the project on Modeling Uncertainty.

Where do I get the student's work?

It is your responsibility to arrange to receive a student's work.  You can find other students currently in class by visiting a password protected page.  You can exchange your work with the same student.  Keep in mind that you are required to make your comments within 48 hours of receiving a copy of the work.  Because voice files are large, mail a compact disk to the student involved; emails will not work well as many service providers have limits on size of attachments to emails. 

Where do I send the comments?

Email responses to the following questions to both the student from whom you received the assignment and to the instructor.  Make sure that each section is properly identified and a comment is made in every section.  Follow the rubric, do not provide free-form comments.  Maintain a copy for yourself.

What should be included?

Administrative

What is the title of the work being reviewed?

What are the initials of the student whose work is being reviewed? 

 

What are your initials?

 

What date you received the work (use format MM/DD/YY)?

What is your email (in the format name@server.com)?

What is the email of the student that provided you with the work you have reviewed?

Presentation format

No Adjectives
Do not use adjectives in your comments.
Instead, describe why you liked a piece
by giving details of what you liked.

Full sentences

Always write full sentences, with a subject,
verb and object.  Always end with a period.

What worked well in the presentation?  Discuss at least one aspect of the project that was presented well (provide a complete sentence ending with a period.  Make sure the sentence has a subject, verb and an object.)

 

Was the voice recording easy to hear?

The volume of the recording changed unnecessarily 
The recording was not loud enough to easily understand the content?

Was the visual presentation appealing? 

The font used in Slides was too small and difficult to read
The numbers reported had more digits than the justified by the precision these estimates were obtained (Maximum of 2 digits after a period)
The narration was verbatim from the slides 
The use of color and graphics did not help the narration and and did not strengthen the central points being made? 

Was the length of the presentation appropriate? 

Too many slides used, typically because the presentation did not focus on the analysis and spent a great deal of time describing the field or making introductory comments. 
It was too short.  5-7 slides were expected and many questions were left unanswered.
Narration of some slides exceeded 2 minutes

Was privacy of sources maintained?

References to name of organizations was not removed
Slide discussing whether the slide show can be posted was not present
Personal information was not stripped from the presentation
Expert was not identified by initials

Content

What worked well in the way the model was constructed?   Please use complete sentences with appropriate punctuations. 

 

Was the target event correctly defined?

Target events were mutually exclusive and exhaustive events
It was clear how a manager will use the predicted probability to improve the organization or care of patients

Were the probabilities correctly estimated? 

The likelihood estimates were incorrectly estimated.  Many students assess likelihood of a clue by asking for the probability of target event given the clue.  This is wrong.  It should be assessed by asking for the prevalence of the clue among situations were target event has occurred. 
Conditional independence was not verified or not verified correctly
The probability of target event was incorrectly calculated from its odds 

Was the model validated?

The face validity of the model was not reported
The scenarios were not correctly generated for at least 15 cases
The correlation between expert's judgment and the model was not correctly calculated
 

Suggestions for Improvements

Contrast the work done with your own effort to model uncertainty and describe what you learned from the work you have reviewed that could help you in the future?  Please use complete sentences with appropriate punctuations. 

 

Suggest what the student might want to do to further improve the assignment.   Please use complete sentences with appropriate punctuations. 

 

Before you submit your work, check that all entries are correct sentences, with a subject, verb and object.  Always end with a period.  Email your work to the student and to the instructor.  You can copy this page into the email.

 


This page is part of the course on Decision Analysis for Healthcare.  It was first created in 2006.  It was last edited on 10/21/2011 by Farrokh Alemi, Ph.D.  © Copyright protected.