D.C.’s COVID-19 Data Problems

I wrote this before yesterday evening’s phone conference with reporters (obviously, I wasn’t on it). DC Health sort of suspended the march to fourteen days and moved it back to eleven days, since they claim, based on this figure, that they had a spike, but it wasn’t large enough:


I get why they might not want to reset the clock to zero over this (there’s still a downward trend). That said, the rule should have taken this into account: a better rule would have been to require twelve out of fourteen days of decline, with no back-to-back increases. Because most reporters simply aren’t trained in this area (to say the least), there is tremendous confusion over this, and at least a couple of ANC board members (along with others) feel the D.C. government is juking the stats, which I think, makes the following post all the more relevant.

And by D.C., I mean the state government, not Wor-Shing-Tun. According to Mayor Bowser and DC Health, here’s the status of COVID-19:

Screen Shot 2020-05-24 at 10.12.49 AM

That’s partly based on this graph, which doesn’t have a clear explanation of what it means (and also seems out of date):

Screen Shot 2020-05-24 at 10.13.08 AM

Here’s the problem. When we look at the data D.C. is releasing, which is daily deaths and new cases, it doesn’t look like things are getting considerably better. New case data does have its problems though (boldface mine):

While the raw positive counts published daily have become a stress-inducing metric for many — Are we going up? Are we going down? — D.C. officials said last week that those numbers aren’t the only thing they’re looking at. Instead, they’re focused on what they say is “community spread” of COVID-19, which is measured in a more nuanced way than simply reporting new positive tests on any given day.

Dr. LaQuandra Nesbitt, the head of the D.C. Department of Health, explained that the daily tally of cases is just an accounting of positive tests. But that’s not wholly helpful in understanding the where, when, and how of COVID-19’s transmission through the city.

“A much better indicator… is when we get those new lab reports, we’re able to see what the actual symptom onset for those individuals are. So if you really want to appreciate the disease patterns or new infections in the community, you can actually look at the symptom onset for the District,” she said during a press conference where Mayor Muriel Bowser announced she was extending the city’s stay-at-home order through June 8.

By the time an infected person has tested positive for the virus, they may have been symptomatic for days. The Centers for Disease Control and Prevention says the average lag between symptom onset and a positive test is four days… There are also issues around when tests happen (fewer on the weekends) and how long results can take to come in.

That’s why knowing the symptom-onset date is critical information for health officials, since they generally know what the virus’s lifespan is.

“My decline in 14 days starts from knowing when people started not to feel well versus when test results were reported,” said Nesbitt during a phone conference with members of the D.C. Council late last week.

Additionally, not all positive cases are created equal. If you’re a resident at a nursing home or an inmate at the D.C. Jail — both hot spots for the virus — testing positive for COVID-19 means a lot for your immediate neighbors, but not necessarily very much for everyone else outside the walls of those institutions.

“We take the rate of decline for people moving around the community versus people who are in confined spaces,” Nesbitt told lawmakers.

So, measuring based on the onset date (versus the date someone tests positive) and controlling for cases in congregate settings gives D.C. officials a better means to understand how the virus is progressing through the city. And that can look very different from the raw number of cases published on a daily basis.

“This is not a new methodology. The Epi Team has been collecting this information and using it to inform the Epi curve all along,” said D.C. Health spokeswoman Alison Reeves in an email, referring to the “epidemic curve,” which has long been used by health officials to measure outbreaks over time.

None of these numbers are public, though, which has led to some confusion around D.C.’s actual decline in cases — or, rather, “community spread.” On the same day that Nesbitt explained this methodology to reporters last week, she also detailed it to councilmembers, who questioned how a then-five-day decline in community spread had been calculated.

When asked today why these more nuanced numbers aren’t made public, D.C. Deputy Mayor John Falcicchio said it’s because they can shift quickly; someone who tests positive one day can remember the next day that they came down with symptoms a week earlier, giving health officials the data they’re most interested in.

Not releasing the data DC Health is using to make its decisions is a really bad idea. If you have two sets of data, one of which seems to support not reopening, the other set which is better and which supports reopening, and you don’t release the latter, people are going to have doubts. Yes, some people might wonder why the numbers shift, but, considering DC Health reported on May 22 almost 9,000 tests were performed in a single day–a nearly seven fold increase over any other daily total–worrying about some small shifts in the “nuanced numbers” seems trivial (and foolish).

Release the numbers you’re using. You’ll look less foolish than using ‘hidden data’ which contradicts the public data you are releasing. It’s basic public health communications.

While we’re here, the other thing is that D.C. Health needs to explain how they’re calculating Rt. Presumably, they’re using the epidemic doubling time method, and not one of the SIR-based or GIS-based methods (I don’t see how they would have those data), but who knows?

To instill confidence, especially as Bowser has publicly mulled the idea of opening earlier than June 8, DC Health should release the data they are using, and how they are calculating Rt. And more journalists need to commit some journalism and ask that they release those data–or FOIA them.

It’s only our lives at stake.

This entry was posted in COVID-19, DC, Statistics. Bookmark the permalink.