Despite many attempts to improve quality and patient safety, projects often are not rigorously evaluated, making it difficult to determine whether improvements were actually made. Among the quality improvement/patient safety (QI/PS) evaluations that are conducted, a fair portion are done post-hoc, without a guiding conceptual framework or theory, and are underfunded. Further, contextual data that can help explain why some sites achieve greater success than others is often not collected or analyzed, constraining the further application of interventions to other settings.
This course prepares participants to evaluate QI/PS projects by developing their competencies in three areas: designing a robust evaluation of a project; conducting a small-scale qualitative study; and critiquing evaluations of projects in the academic literature and elsewhere. The lessons will present examples from both large-scale, multi-site improvement programs, as well as projects at the unit or clinic level.
Dates to be announced.
Virtual participants: $995 per participant
Johns Hopkins Medicine employees: No cost but seats limited.
Please see our Cancellation/No Show & Refund Policy.
Please join us during live sessions for directly delivered course material and work with peers to develop evaluation projects (as scheduled below). In addition, please go to https://blackboard.jhu.edu/ to access pre-taped course materials. In general, you should watch the pre-taped lectures prior to the live Zoom meetings, as indicated on the agenda below.
If there are helpful readings, the instructors will post references for them on Blackboard. In addition, there will be a chat space on the Blackboard for question and answer and group interactions.
View pre-taped talks and review Blackboard site
11 a.m.-12 p.m. Introductions and Description of the Course
Jill Marsteller, PhD, MPP, Professor
12-1 p.m. Why good evaluation is important Jill Marsteller, PhD, MPP, Professor
View pre-taped talks 1-7 on Blackboard by various lecturers.
1-2:30 p.m. Group work all 2:30-3 p.m. Report out Jill Marsteller, PhD, MPP, Professor
View pre-taped talks 8-14 on Blackboard by various lecturers.
3-4 p.m. Group Work All 4-5 p.m. Report our and closing Jill Marsteller, PhD, MPP, Professor
- Evaluation Designs for Small and Medium PSQ Studies-- Lilly Engineer, MD, DrPH
- Literature Reviews for PSQ and Evaluation—Christina Yuan, PhD
- Surveys of Clinicians-- Henry Michtalik, MD, MPH
- Using a Logframe for Evaluation –Yea-Jen Hsu, PhD, MHA
- Using EMR data for your QI projects—Yea-Jen Hsu, PhD, MHA
- Interview and focus group methods—Jill Marsteller, PhD, MPP
- Observation methods—Jill Marsteller, PhD, MPP
- Quantitative Methods: Statistical Process Control for PSQ Evaluation—Yea-Jen Hsu, PhD, MHA
- Evaluation Aspects of the TRIP Model for Evaluation—Kristina Weeks DrPH candidate, MPH
- Engaging Stakeholders for Evaluation—Kristina Weeks, DrPH candidate, MPH
- Data Quality Control—Lisa Lubomski, PhD
- Quality Improvement vs. Research/ IRB—Lisa Lubomski, PhD
- Survey-based Patient Outcomes--Albert Wu, MD, MPH
- Implementation Science in Application--Lilly Engineer, MD, DrPH
Upon successfully completing this course, students will be able to:
- Enumerate and communicate the importance of understanding environmental, organizational, group, provider, task, work system, implementation and patient influences on outcomes
- Describe and evaluate strengths and weaknesses of designs for testing success of QI/PS interventions
- Describe operational steps to conducting robust data collection and analysis (quantitative and qualitative)
- Define and describe remedies for common problems in QI/PS studies
- Introduction to QI/PS initiatives and changes
- Identifying framework and stakeholders
- Financial impact assessment of interventions
- Evaluation design
- Data collection and quality control
- Quantitative data analysis
- Using a logical framework in evaluation
- Qualitative methods
In addition to lectures that can be accessed online at your convenience, during live online sessions, participants will frequently break into work groups in which they will apply course concepts to challenges in QI/PS project evaluation. Participants are encouraged, but not required, to come to the course with a project that they are working on or a project idea for group discussion.
The course provides a mix of practical how-to information and research-based evaluation concepts.
Jill Marsteller, Ph.D., M.P.P
Professor of Health Policy and Management
Johns Hopkins Bloomberg School of Public Health
Email AILearning@jhmi.edu or call 410-637-7170.