Checking in on Mass General Brigham two years after the CrowdStrike outage

On July 19, 2024, a software update gone wrong at cybersecurity company CrowdStrike caused widespread outages across the country, shutting down flights, banks, and even some hospital operations.

CrowdStrike’s products and services provide software designed to protect IT infrastructure. But instead, an update triggered an error that cascaded into the outage.

Soon after the incident, we spoke with executives at Mass General Brigham (MGB), the largest health system in Massachusetts, about how it recovered. The outage caused 45,000 computers to get the blue screen of death—or crash—meaning MGB’s staff had to shift from digital to paper documentation. After the outage was resolved, the health system rushed to update electronic medical records (EMRs) and get their computers operational again.

MGB wasn’t the only healthcare organization impacted. A 2025 JAMA study reported that 759 hospitals had network disruptions stemming from the CrowdStrike outage.

In the two years since, MGB execs said they’ve learned some important lessons on how to prepare for future IT challenges.

Here’s how.

Up during downtime. A big part of being prepared involves updating a hospital’s protocols during downtime, when there’s no internet or EMR access. Downtime can be planned as a drill for hospitals, but in the case of CrowdStrike, it wasn’t a drill at all, it was a real outage incident. During downtime, clinicians need to document patient information on physical paper, but the outage revealed that many employees hadn’t really done that before.

“Maybe half of our workforce, this is really sort of a guesstimate, has never really worked on paper,” Paul Biddinger, MGB’s chief preparedness and continuity officer, said.

When writing up an order for a medication, for example, physicians need to input information like the drug’s name and dosage along with information on the provider, including their name and ID number. Biddinger said most clinicians are used to being prompted by the EMR, but when systems are down, that isn’t an option.

Where there used to be a blank spot on the old paper documentation, after the outage, MGB updated the paper medical records with little boxes that prompt clinicians on how to input the drug, dose, the concentration, the time and the hold parameters.

Dealing with downtime during an outage also requires strong logistical plans to move patient information around the hospital. Healthcare relies a lot on diagnostic tests such as blood tests, urine tests, and pregnancy tests as well as imaging, like MRIs and X-rays, Biddinger added. During downtime, lab technicians have to call each physician to deliver results. That is a tried-and-true method, he said, but hospitals have thousands of tests a day, and this process takes a lot of time.

“Human-to-human connection can’t keep up,” Biddinger said, “so we’ve been working a ton on how we get all those results written down somehow on a piece of paper. Usually it’s runners flying around the hospital to try and keep up with the volume of care so we actually are still able to care for patients in a timely manner.”

Device dependance. MGB has also implemented training for staff on how to use business continuity devices. These are computers in each clinical area that are usually connected to the internet but can also operate independently from it if the internet goes down. They periodically download critical data onto the device’s local hard drive “so that if the computer network goes down, we still have access to some of the data on those devices,” Biddinger said.

That mission-critical information includes medical records or order sets, which are lists of medical instructions for specific diagnoses, like labs, imaging, or nursing procedures.

This is helpful because some departments have specialized protocols built into electronic systems, Biddinger said, like for how to write an order set for a patient in labor. Providers put these instructions onto the business continuity devices so they can reference them during downtime and keep care flowing.

Ch-cha-cha-changes. Jason Mafera, field CTO for healthcare at IT company Igel Technology, told us the CrowdStrike outage helped companies realize that automatically updating systems without testing first is not a great idea.

“It highlighted something that no one was focused on, which was: It’s not just ransomware and malware that can cause business-level outages,” he said. “Now you have to start thinking about update outages.”

Since July 2024, MGB has been trying to tighten up its change management processes, Biddinger said.

“Computer systems always need change. There’s always a new version. There’s always an update. But each of those episodes of change represents a real, unfortunate opportunity for things to go badly,” Biddinger said.

Now, before system upgrades or other IT changes, he added, MGB runs tight testing, plans exactly when changes happen, and ensures permissions are in place, meaning the health system can prepare for the update with network access and security controls to avoid crashes.

The legacy of the CrowdStrike outage, Mafera said, is its size—which could help with endurance into the future.

“People will remember this. They will bring it up, and then they will ask themselves: ‘What are we doing to make sure that we’re not going to have an inadvertent update outage?’” Mafera said.

About a year after the incident, CrowdStrike itself released information about a resilience plan, which included new permissions for how customers can control system updates, including their timing and configuration.

The company hired a new chief resilience officer and improved its sensor self-recovery systems, which catch crash loops and automatically trigger safe mode. It also established a new Digital Operations Center to more clearly see how its platforms are performing.