For anyone who has been following the modeling projections for the spread of COVID-19, there was an apparent bombshell in the last few days: The Imperial College model that was released roughly two weeks ago – and which generated massive media and policy maker attention — was updated. Based on more data, the modelers were increasingly confident that their worst case scenario, which they previously characterized as “unlikely,” was even less likely to occur. So they revised their projections for infections, hospitalizations, and deaths substantially downward.
The Wall Street Journal ran an editorial on this yesterday, and it is a must read for anyone interested in understanding the likely trajectory of the pandemic: “Worst-Case Coronavirus Science.” The Journal made four points that warrant elaboration:
We need to update our projections when the situation changes.
In California, our COVID-19 situation is going to change two times in the next three weeks, and everyone needs to recognize this. The simplest way to think about this is as follows: COVID-19 is hitting California in three waves. The first wave corresponds to the roughly unchecked spread of the virus before March 8th, which is when the State and Local governments started issuing escalating public health orders to slow the spread. The second wave corresponds to the period between March 8th and March 19th, when the stay at home order was issued. Although the virus will have been spreading rapidly during this phase, it should have been at a sub-maximum rate because of the public health measures in place. Finally, there is Wave 3, which corresponds to the period after the stay at home order was issued.
The key question is when is each of these waves going to hit? Because the virus has a 14-day incubation period and our testing results are not reported immediately, each wave has a potentially 14-21+ day lifespan after its last calendar day. I have used 19 days in my models. This means that the Wave 1 trajectory could continue to appear in our data on confirmed cases, hospitalizations, etc. as late as March 29th. In other words, there is a decent chance that some of the new cases reported over the weekend originated before we undertook any public health measures. The data from Wave 2 could appear in our data as late as April 8th. This means that the full benefits from the stay at home order will likely not be visible in the data until roughly April 9th, although there will be clues in the data long before then.
If the decentralized public health measures taken before March 19th were effective, we should start seeing that impact reflected in the data this week. Please note that this will not mean a decrease in cases or hospitalizations. The numbers will continue to increase, but at a slower rate of acceleration. That is what we need to look for in the data, especially hospitalizations. We should then see another change when the full impact of the stay at home order is reflected in the data, which I am estimating to occur on April 9th.
“If we hope to neutralize a pandemic we don’t fully understand, we need to encourage a culture in which scientists feel able to adapt and clarify with new evidence. Scientists would help themselves if, in explaining their findings, they would be more candid about the assumptions and variables.”
As someone with a public policy and modeling background, I could not agree more. This is why I explicitly list every assumption and variable upon which my model for California healthcare capacity depends and note that it will be updated as more and better data become available. I expect to release an updated model early in the week.
I would add a key corollary principle: Publishing models for peer review would also improve our ability to create more precise models to guide policy makers. This is why I have repeatedly called on the State to publish the model it is using to predict the spread of COVID-19 in California and the healthcare system capacity required to handle that spread. The initial California predictions were crafted at roughly the same time as the Imperial College model and they likely reflected an equally unlikely worst case scenario. They need to be tested and updated based on actual data, just as the Imperial College model was.
“Instead of a presentation of what we know and don’t, too often the focus [of press reporting] has been . . . sensationalizing.”
Here too, I could not agree more. Most of this alarmist reporting stems from an inability or unwillingness to do the math required to present information about the spread of the virus in the correct context. Here are three examples of how this dynamic manifests itself:
- The “Surge” or “Spike” in Cases Each Day
I prefer to use hospitalizations and fatalities as the best measures of the trajectory of the virus. But the reporting on hospitalizations is limited at this point in time, and the fatality data reflects the spread of the virus 15-25 days previously. The trajectory of a spreading virus can range from an increase in infections of 1x/day (a totally flattened curve) to > 1.3x/day (an unchecked virus – see the early days of Italy’s outbreak). We are nowhere near averaging 1.3x/day in California. This is already clear from the daily fatality data and we have not seen the full impact of the early social distancing measures in California.
We also have not seen a surge in cases. This chart below explains the differences in virus trajectories and compares them to California over the 6 days ending March 26th:
The blue line represents the uncontrolled growth of Covid-19. The red line represents constant growth (i.e., no spike or surge) at a rapid, but sub-maximal rate. The green line represents the growth in daily cases in California over the past 6 days. The purple line represents a curve that is flattening. As you can see, there has not been any surge or spike in our daily confirmed cases, except (perhaps – see below) from Day 5 to Day 6, despite many headlines to the contrary over the past 6 days.
2. But wait a minute. Shouldn’t I be worried about that big jump from Day 5 to Day 6 (March 26th)?
The answer is: “Not yet, and probably not.”
I am unwilling to read too much into the case data at this point because the number of tests performed each day varies, there is a variable lag in reporting test results, and the State has a backlog of test results that have not been reported (and may be disproportionately negative). Under these facts, the cases confirmed today could include tests that were administered 48 hours ago, 120 hours ago, or 168 hours ago (or more, conceivably). And we could get a batch of results today that is much larger or smaller than the day before. So, we cannot compare daily confirmed cases with prior days on a truly apples-to-apples basis to figure out if the situation is worsening. And remember: The number of new daily cases does not always increase. It actually decreased in California on March 15th, 18th, and 20th. But no one declared the spread of the virus to be in decline based on those 3 days, viewed in isolation from the larger context. So why are we declaring “surges” and “spikes” when the opposite occurs on a given day? We shouldn’t.
One way to try to address this lag and some of the variability in the data would be to report cases as a moving 3-, 4-, or 5-day average. This would, in theory, smooth out the curve and produce a more accurate snapshot of the spread of the virus over time. But it would not address some of the other challenges I mentioned, and a moving average could be misleading as well – it could overly-smooth the curve and delay the recognition of important changes. Nonetheless, to illustrate the potential impact of using a moving average, I calculated the moving 3-day average increase in daily cases for California from March 12th to March 26th in order to reduce the impact of dramatic 1 day differences without overly smoothing the curve. Here’s what the curve looks like, including our seeming spike on March 26th:
There are three important characteristics of this curve that everyone should understand: (1) it looks more like an EKG – the data are messy (as we expect given all of the reporting challenges noted above); (2) the “surge” on March 19th wasn’t actually a surge (and the most recent spike may not prove to be either – we’ll know in one more day); and (3) the trend line in this particular curve is fairly constant (a different depiction of the red line from the first chart) (although I would not read to much into this case data at this point in time).
3. What would a flattened curve look like in California ?
In a very simplified form, my March 26th Model projects the following curve for California daily hospital admissions from March 24th to April 16th:
This shows daily hospital admissions rising for several days, then dropping slightly as the full impact of the stay at home order is achieved, and then leveling off as long as the stay at home order remains in place. Note: My curve for daily confirmed new cases would be identical (but with different numbers).
Here is the critical point to understand from this curve: With public health measures in place, we can manage a certain (reasonably flat) level of new COVID-19 hospital admissions each day for an indefinite period of time (this is how the healthcare system functions in normal periods). This is because at a certain point in the controlled spread of the virus, the number of new hospital cases each day will roughly correspond to the number of discharges (and sadly, deaths) each day. The point of the stay at home order is to ensure we are at (or below, ideally) this number of daily hospital admissions (which by my calculations, is well above 588 new admissions/day).
Using the predicted rate of daily growth in hospital admissions, the curve for my March 26th Model would look something like this:
This curve reflects a constant rate of growth before the stay at home order begins to take effect (as noted above). When it does, the rate of growth slows to its lowest level under the order. From that point forward, the rate of growth is flat so long as the stay at home order is maintained. Note: I don’t expect the curve in either chart to be completely flat. Instead, I expect it to look like an EKG hovering very closely around the flattened trend line.
These are the types of curves we should be reporting to inform the public regarding the trajectory of the outbreak, rather than graphs that simply show the inexorable (and seemingly steep) rise in total confirmed cases (the vast majority of which turn into confirmed recoveries).
“In the battle to save lives and address the scourge of COVID-19, good information is paramount.”
I could not agree more. This is why I have called on the government to collect and publish the data we need to accurately model the course of the virus and our healthcare system capacity, assess whether the stay at home order has worked and, if so, whether we have less-disruptive public health measures we could employ to control the growth of the virus over the mid-to-long-term.
Conclusions
As I have said, models are only as good as the assumptions and data on which they are built. I would encourage everyone to examine these issues carefully before accepting a model as reasonable, including the (unpublished) model the State has used to predict the spread of the virus and healthcare system capacity in California.
Please do not be overly-alarmed by the reporting on any particular day. I would suggest looking at the trends based on a moving 3-day average focusing on fatalities (remember the data tell us about the spread of the virus 15-25 days before) and hospitalizations (when they are reported). I may even start reporting the data this way to make it easier for people to determine the actual trajectory of the virus.
Lastly, ignore the count of total cases. It does not tell you what you need to know about the spread of the virus or its likely impact on the healthcare system. Instead, focus on the rate of growth in daily hospital admissions (if we get that data) (using a moving 3-day average). If it is constant or trending downward toward 1x, then we are on the right track. Right now, it looks like we are.
Read critically. Look at sustained trends, not daily headlines. Stay well.
Updated April 6th.
Leave a Reply