Skip Navigation


Evaluation of Patient Quality and Safety


Despite many attempts to improve quality and patient safety, projects often are not rigorously evaluated, making it difficult to determine whether improvements were actually made. Among the quality improvement/patient safety (QI/PS) evaluations that are conducted, a fair portion are done post-hoc, without a guiding conceptual framework or theory, and are underfunded. Further, contextual data that can help explain why some sites achieve greater success than others is often not collected or analyzed, constraining the further application of interventions to other settings.

This course prepares participants to evaluate QI/PS projects by developing their competencies in three areas: designing a robust evaluation of a project; conducting a small-scale qualitative study; and critiquing evaluations of projects in the academic literature and elsewhere. The lessons will present examples from both large-scale, multi-site improvement programs, as well as projects at the unit or clinic level.




Registration Details


Virtual participants: $995 per participant

Johns Hopkins Medicine employees: No cost but seats limited.

Please see our Cancellation/No Show & Refund Policy. Beginning January 1, 2021, the No-Show Policy has been reinstated. 



Please join us during live sessions for directly delivered course material and work with peers to develop evaluation projects (as scheduled below). In addition, please go to to access pre-taped course materials. In general, you should watch the pre-taped lectures prior to the live Zoom meetings, as indicated on the agenda below.

If there are helpful readings, the instructors will post references for them on Blackboard. In addition, there will be a chat space on the Blackboard for question and answer and group interactions.

  • View pre-taped talks and review Blackboard site

  • Date/ Time



    11 a.m.-12 p.m. Introductions and Description of the Course

    Jill Marsteller, PhD, MPP, Professor

    12-1 p.m. Why good evaluation is important Jill Marsteller, PhD, MPP, Professor
  • View pre-taped talks 1-7 on Blackboard by various lecturers. 

  • Date/ Time



    1-2:30 p.m. Group work all
    2:30-3 p.m. Report out Jill Marsteller, PhD, MPP, Professor
  • View pre-taped talks 8-14 on Blackboard by various lecturers. 

  • Date/ Time



    3-4 p.m. Group Work All
    4-5 p.m.  Report our and closing Jill Marsteller, PhD, MPP, Professor
    1. Evaluation Designs for Small and Medium PSQ Studies-- Lilly Engineer, MD, DrPH
    2. Literature Reviews for PSQ and Evaluation—Christina Yuan, PhD
    3. Surveys of Clinicians-- Henry Michtalik, MD, MPH
    4. Using a Logframe for Evaluation –Yea-Jen Hsu, PhD, MHA
    5. Using EMR data for your QI projects—Yea-Jen Hsu, PhD, MHA
    6. Interview and focus group methods—Jill Marsteller, PhD, MPP
    7. Observation methods—Jill Marsteller, PhD, MPP
    8. Quantitative Methods: Statistical Process Control for PSQ Evaluation—Yea-Jen Hsu, PhD, MHA
    9. Evaluation Aspects of the TRIP Model for Evaluation—Kristina Weeks DrPH candidate, MPH
    10. Engaging Stakeholders for Evaluation—Kristina Weeks, DrPH candidate, MPH
    11. Data Quality Control—Lisa Lubomski, PhD
    12. Quality Improvement vs. Research/ IRB—Lisa Lubomski, PhD
    13. Survey-based Patient Outcomes--Albert Wu, MD, MPH
    14. Implementation Science in Application--Lilly Engineer, MD, DrPH

Learning Objectives

Upon successfully completing this course, students will be able to:

  • Enumerate and communicate the importance of understanding environmental, organizational, group, provider, task, work system, implementation and patient influences on outcomes
  • Describe and evaluate strengths and weaknesses of designs for testing success of QI/PS interventions
  • Describe operational steps to conducting robust data collection and analysis (quantitative and qualitative)
  • Define and describe remedies for common problems in QI/PS studies

Topics Covered

  • Introduction to QI/PS initiatives and changes
  • Identifying framework and stakeholders
  • Evaluation design
  • Data collection and quality control
  • Quantitative data analysis
  • Using a logical framework in evaluation
  • Qualitative methods

The Participant Experience

In addition to lectures that can be accessed online at your convenience, during live online sessions, participants will frequently break into work groups in which they will apply course concepts to challenges in QI/PS project evaluation. Participants are encouraged, but not required, to come to the course with a project that they are working on or a project idea for group discussion.

The course provides a mix of practical how-to information and research-based evaluation concepts.


Jill Marsteller, Ph.D., M.P.P
Professor of Health Policy and Management
Johns Hopkins Bloomberg School of Public Health


Email or call 410-637-7170.

back to top button