When Technology Fails

Disaster drill helps Johns Hopkins Medicine prepare for a systemwide outage.

Published in Dome - March/April 2017

Maternal-fetal medicine nurse Barbara Kennedy-Kosick realized how much she depended on the technology system of The Johns Hopkins Hospital in October when it went down for several hours. Without a central fetal monitoring system, she says, “you couldn’t see what was going on in the rooms unless you were there.”

Three months later, Kennedy-Kosick and three others in her department were among roughly 250 leaders and information system specialists participating in an enterprisewide simulation of a technology outage.

The Jan. 24 discussion-based exercise included staff members from The Johns Hopkins Hospital, Johns Hopkins Bayview Medical Center, Howard County General Hospital, Suburban Hospital, Sibley Memorial Hospital, Johns Hopkins Home Care Group, remote outpatient clinics and Johns Hopkins Community Physicians, all linked through teleconferencing. The event was led by the Office of Critical Event Preparedness and Response (CEPAR), the Office of Emergency Management, and the Department of Clinical Informatics.

“Since many of our business systems and medical records are now centralized, it was a great opportunity for so many of the entities to video link, respond to a mock disaster in a virtual format and learn from each other,” says Cindy Notobartolo, administrative director for emergency medicine, trauma, safety, security and employee health services at Suburban Hospital.

The exercise was prompted, in part, by a technology outage that affected much of Johns Hopkins Medicine for several hours on Oct. 14, 2016, as well as a ransomware attack on the computer networks of MedStar Health hospitals in March 2016.

Electronic systems manage virtually all aspects of daily operations at Johns Hopkins Medicine. As dependency on technology increases, so does vulnerability to system breaches and failures.  

“After the October outage, we heard feedback that we needed to make improvements in our preparedness,” says Carrie Stein, director of clinical informatics at Johns Hopkins Bayview. “Exercises like this help us practice our business continuity plans and evaluate gaps.”

The exercise imagined a simple scenario with far-reaching impact: An electrical shortage in the Mount Washington data center severed service to technologies throughout the institution, including email, the Epic electronic medical record system and Kronos workforce management software. “Basically, most technology we use at Johns Hopkins Medicine is down, folks,” explained Dianne Whyne, CEPAR’s director of operations, at the start of the exercise. She led participants through the imaginary unfolding scenario, pausing frequently for discussion.

“As much as possible, we want you to behave as if this event is really happening,” she said.

Because emergencies come in many forms, Johns Hopkins locations regularly simulate disasters, such as large-scale motor vehicle accidents, earthquakes and pandemics. Last month’s exercise, which took the better part of a day, was the first of several planned to help Johns Hopkins Medicine better prepare for a temporary loss of technology.

As the exercise came to a close, Kennedy-Kosick praised the experience. “The October outage caught us off guard,” she says. “The simulation helped me break down what happened so I can prepare.”