Georgetown University
Statistical Process Improvement

How satisfaction surveys should be organized  Slides►  Listen► Paper
Analysis of complaints  More►  

Satisfaction Survey Planning Project

This section describes the Satisfaction Survey Planning Project in the course on Process Improvement. 

Objectives

At the conclusion of this project participants should be able to:

  • Apply quality improvement tools and techniques.
  • Plan efforts to assess patients' satisfaction or health status.
  • Construct a quality improvement storyboard.

The purpose of this project is to help you design and put in place satisfaction surveys.  The purpose is to plan for the surveys but not to do them. 

Steps by Step Guide to Project Completion

Here are the steps you can take to accomplish this project:

  1. Team up with a classmate that is working within an organization and can obtain permission from the organization for you to plan a satisfaction survey to address their needs. 

  2. Obtain a letter of introduction from the instructor and send it to the sponsor.  Download►

  3. Read about how satisfaction surveys are done and review the literature.  Review►  Questionnaires►  Response rate►  H-CAPH► Abstracts►  Full text►  Library►  Google►  

  4. Make an initial plan
     

    • Decide on the survey tool you will use.  We have developed a tool for active solicitation of complaints and praise which we call “Minute Evaluation” to emphasize the short nature of the form.  Short surveys are expected to increase the client's response rate and reduce cost of conducting satisfaction surveys.  The response rate to current satisfaction surveys is relatively low.  It takes repeated call backs and other reminders to improve the response rate.  Naturally, the cost of conducting satisfaction surveys is high.  One way of improving response rate is to make it easier for people to respond.  The following form reduces the number of questions asked from the patients to a bare minimum.



      Note that patients respond to two questions.  The first question asks the patient to rate their experience.  The second question provides the explanation for the rating.  In addition, the provider enters the information on visit code and location code prior to providing the postcard to the patient.  The question on the visit code establishes the sequence of the data collected and is important in analysis of time (number of visits) to dissatisfied customer.  The question on location relates the rating to a specific unit within the health care institution and allows the provider to focus on evaluation of specific units.  The Minute Evaluation of Satisfaction is typically printed on a postcard.  Preferably, patients should drop their response in a box before leaving the organization.  Some, may wish to mail the postcard at a later time.  The postcard contains no personal information and can be mailed without cover. 

      Sometimes, when organizations want to understand more details about the reason behind the patient's complaint or praise, additional survey tools are used to gather more details.  Patients who rate their experience in the two extremes may be contacted to complete a more detailed survey tool.  In this manner, most patients are spared the burden of completing long and detailed survey tools.  Patients find the Minute Evaluation of Satisfaction more convenient than other longer tools, response rate improves and cost of collecting patient's satisfaction with care is reduced.
       

    • Decide on the frequency of the survey.  Discuss sampling strategies (convenient sample, sampling the next n patients, two stage sampling, inverse sampling or sampling until two dissatisfied patients are found, adaptive sample or sampling more intensively when a dissatisfied patient is found.  More 
       

    • Decide on who will do what, when and how.  Decide survey procedures (e.g. list of patients, sampling steps, cover note, consent form, questionnaire distribution, collection of questionnaire, data entry, timing of data analysis, timing of reports, recipients of reports.)   Consider various options for who will hand out the survey (mail, receptionist at start, receptionist exit, etc.)  Give details and discuss these details with the organizations.  Estimate resources needed to carryout the survey.  Estimate the start and end times.  More►
       

    • Do not collect data.  You are expected to only plan for data collection and not actually collect the data.  Use the data provided by the instructor (or make up your own data) to demonstrate your project plans.  Analyze the data and show how the findings will be reported.  In your presentation make sure it is clear that the tables and graphs are not based on real data.   Download


     

    • Describe who will use the data and why.  Specify the improvement team that is waiting for the data or plan the selection and maintenance of the team.

  5. Engage members of the organization in your plans and get their reactions. 

    • Prepare a narrated slide show that provides an over view of your plans.  The intent is to prepare a short narrative that is effective in selling your plan to others within the organization (use customer's voice, use vivid images or videos, use short slogans, give the advantages of the plan, why it is easy, why it is effective, etc.).  See guide on how to post narrated videos to the web.  Video►  SWF  You Tube► 

  6. Prepare a final report using the following headings:

    • Executive summary:  A one page summary of the entire plan

    • Time to dissatisfied customer:  Review the literature on satisfaction surveys (at least 2-3 pages) and report the low rate of response to satisfaction surveys.  Discuss what is known to improve response rate and the importance of examining time to dissatisfied customers. 

    • Minute survey:  Describe different questionnaires available for measuring satisfaction of patients.  Contrast these with the minute evaluation form.

    • Survey timing:  Describe how often patients will be surveyed and why.  Describe the two stage sampling plan and why detailed information is not collected in the first stage.

    • Plan details:  Describe who, what, where, when the survey will be done

    • Pilot test:  Using a sample data, show how the data will be analyzed.   You can obtain sample data from the instructor.  Analyze the data and show the results that could be obtained from the data collection.

    • Plan acceptance:  Describe reaction of people who have reviewed the plan or participated in the planning process

The following resources may be of use: 

  1. Listen to the instructor's view on how satisfaction surveys should be organized.  Slides►  Listen►
  2. Read the paper on analysis of complaints.  Download►   More►
  3. Read paper on advantages of Minute Survey of satisfaction.  More►

Examples

Here are some examples of the work done by a student.  Please note these examples are provided without judgment of quality of the work.  Furthermore, note that the instructions for the project have changed since these assignments were completed.   Example 1►  Example 2►, Example 3►and Example 4►

The following are two examples of narrated slides prepared by students about their plans:



Here is a comic example of media students put together to present their work:

Here is another narrated example on use of Minute Survey.  Note that in this example the use of Time-between Control Chart is not appropriate, as multiple surveys are available after each class.  You Tube 3►   
 

Peer Evaluation

We ask students to evaluate each other’s work because they often have fresh perspectives that are absent when only instructor provides feedback.  In addition, when students teach an idea, they learn it in much more detail.  The exercise ends up helping the evaluator more than the person being evaluated.   When providing peer evaluation of group projects, students can use the following rubric to guide their evaluation: 

1)       Date:  The document should include the date the draft was received and the date evaluation was sent back.

2)       Praise:  Student evaluators should start with what worked well.  They should use many adjectives and be clear about what was done well.

3)       Presentation:  Comment on effectiveness of the presentation.  Students will do their colleagues a service by setting a very high standard for effective presentation as that is what the instructor expects.  There is no specific prescription but asking the following questions may help:

  • Is it too much text and too little visual guides?  

  • Is it written poorly? 

  • Are pages numbered and titled clearly? 

  • Is the document fun?  Are there any cartoons to break the boredom? 

  • Are there images to help make the points more succinctly? 

        No one is required to do any of these elements but these are given as examples of thoughtful presentations.

4)       Improvement:  Student evaluators should describe what needs more improvement within each section (literature review, survey selection, survey timing, plan details, pilot test and plan acceptance).  They should make at least one comment in each section.  They should not use any adjectives in their comments.  They should give details of the improvements and avoid general statements.  They should be honest and clear.

5)       Completeness:  Student evaluators should comment on whether all sections of the report were present, including sections waiting for input or data: 

a)       Executive summary (all reports should have this or have a placeholder for it)

b)       Literature review  (there should be a reasonable review attached)

c)       Survey (there should be several different surveys discussed and contrasted)

d)       Survey timing (two stage sampling or other sampling strategies should be discussed)

e)       Plan details (it should be clear who is doing what and when)

f)        Pilot test. (some data should be analyzed and presented or plans to do this should be clear)

g)       Plan acceptance (plans to solicit and report acceptance should be described)

6)       Student evaluators should finish by describing in detail what they have learned from the review of the draft project.  They should give details of how they plan to change their own project as a consequence of what they have learned.

Students should not provide a grade.  Email the evaluation to all members of the group as well as to the instructor.  Please send one email copied to the relevant people.

 


This page is part of the course on Quality / Process Improvement, the section on "About the course."  It was first created in 1996.  It was last edited on 08/29/2007 by Farrokh Alemi, Ph.D.  Copyright protected.