Q&A: Model helps health systems prepare COVID-19 responses

CDC data show that as of April 9, the impact of the COVID-19 pandemic within the United States has been uneven. For example, New York and Texas each have thousands of cases, while Montana and Wyoming have only several hundred cases. Such fluctuations and the rapid spread of the disease are some of the factors that have made it challenging for health care systems to predict how many supplies they will need to help patients.

Earlier this year, when Italy and China were dominating the COVID-19 headlines, researchers at the University of Pennsylvania and Penn Medicine Predictive Healthcare developed a susceptible, infected, removed (SIR) model known as the COVID-19 Hospital Impact Model for Epidemics (CHIME) to predict the course of COVID-19 in the Philadelphia area and ascertain how prepared the University of Pennsylvania Health System was for the imminent wave of cases. Their work was recently published in Annals of Internal Medicine.

Healio Primary Care recently spoke with one of the model’s developers — Gary Weissman, MD, MSHP, an assistant professor in pulmonary and critical care medicine in the department of medicine at the University of Pennsylvania Perelman School of Medicine — about the model’s development, how it has steered his health system’s COVID-19 response, how other health facilities can implement the model at their locations and more. – by Janel Miller

Q: How was the model developed?

 

A: The genesis of the model arose from questions from the executive leadership within Penn Medicine, from service line directors and divisional and departmental leaders who are trying to figure out how best to prepare for the pandemic. They had specific questions like how many beds, how many ICU beds and how many ventilators we will need. They also had other corollary questions like when we should stop [accepting] transfers from other hospitals or when should we stop scheduling elective surgeries or elective procedures for people who may take up hospital beds.

Those questions were brought to the predictive health care team, which is the internal data science team at Penn. Within about 2 days, the team was able to set up this web interface to allow exploring different scenarios of the epidemic to produce numbers that could guide some of these decisions.

Q: How has the model helped guide decision making at Penn regarding COVID-19?

A: The model has guided a lot of our efforts to try to acquire more ventilators, more personal protective equipment and to come up with creative ways to either re-sanitize or re-use personal protective equipment, while also ensuring safety. The model also provided numbers that allowed Penn to make decisions around elective surgeries. Those numbers were really able to help inform those decisions because it became clear that we were going to need so many resources, even in the best-case scenario.

We're still collecting data on that. But as the model evolves every few days, we match it against our predictions from the week before.

Q: What are some of the model’s limitations?

A: The key challenge in building the model was to recognize that there's no perfect epidemic model. And because of the timeframe that this was model was requested, we were only a week or so into getting some published data from places like China and Italy to then try and parameterize models to input into a candidate model.

The SIR model that has been around since the late 1920s underlies the tool. It is a simplified epidemic model that does not capture many of the complexities of spatial relationships of the structure of contact networks [such as] social distancing, or structure of social networks in a way that actually reflects those complexities and realities. The SIR model also assumes a constant doubling time, which of course, isn't true, because as different policies emerge around shutting down schools, canceling events, and those kinds of things, the doubling time will change as a [pandemic] moves through populations. The model ... needs to be reset using updated data depending on how things have changed over time.

But the tradeoff is that early in the epidemic, the model projects an exponential growth pattern that is relatively easy to parameterize because it only has a few variables that we can draw on. The tool also allows the user to very easily inputs [those variables] to explore a range of scenarios. And that's what our analysis tried to convey: the tool does not provide a specific number, but really a range of scenarios that we can then begin to prepare for within the health system.

Q: How can other facilities implement the model in their practice?

A: The interface that was developed by the data science team allows the user to change a lot of the assumptions, including the size of the local population and the proportion of the market share that might be served by a specific hospital or health system. [The interface] also allows the user to input known data on the current number of cases in the region or in the hospital and to change the expected doubling time based on observed local trends. It was built in that way to try and be intuitive and flexible so that hopefully, other regional leaders or health system leaders can use it to make projections. The interface is available to everybody under an open source license.

Q: How should your simulation be considered in the context of other models that COVID-19 has spurred?

A: Most models out there have their merits. A savvy user of a models will understand what the limitations and assumptions are in each model. It would be helpful for a local health system leader or hospital to try to understand two things. One, which model most closely resembles the assumptions and the dynamics of their local situation and then also to explore multiple models. Two, get a sense of the range of forecasts and why they might be different.

I do not advocate any health system leader using only a single model or a single data set, because there's so much uncertainty around what's coming in the future, but rather to try different models simultaneously and compare and contrast the results in the context of whatever is happening locally.

References:

CDC. Coronavirus 2019 (COVID-19). Cases in U.S. https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/cases-in-us.html. Accessed April 9, 2020.

Weissman GE, et al. Ann Intern Med. 2020;doi:10.7326/M20-1260.

Disclosures: Weissman reports no relevant financial disclosures. Please see the study for all other authors’ relevant financial disclosures.