Data Nuances, Caveats and Pitfalls

Article by the Wolf

We get a lot of questions about data.

Because we only use official, Florida Department of Health-published data, we are limited by the restrictions they place on the data.

They describe some of those issues here, in a document I wrote while still at DOH that has since made been revised, but still leaves much out (in addition to some information being outdated or incorrect).

I’ve added some warnings about the data to the Florida COVID Action dashboard, and am working on a more complete document in the next few weeks. I’ve emailed the DOH communications line multiple times with questions about data, and still have not received a response.

Here are two of the major warnings regarding more recent changes in data, though we’ll expand upon them in future posts.


All fields with the PUI (Person Under Investigation) prefix:
The PUI definition in DOH’s document is wrong. The state stopped updating the PUI data in late May. As described in their documentation, a Person Under Investigation (PUI) is “any person who has been or is waiting to be tested.”

By that definition, the total PUI’s should, at a minimum, equal the number of people DOH claims have been tested, and the number of “pending” tests from BPHL.

But it’s not even close. There was a period of time after I was fired where you could use the PUI data to calculate the true percent positivity, since PUI’s were individual people (cases) and total tests now included duplicate negatives.

That PUI data stopped getting updated around June 1 (not all coincidentally after I called out DOH about how the “total people tested” figure was equal to total tests, not people).

And it’s not just the total number of people tested that’s now static.

Now the all of the data with the PUI prefix is being updated with small numbers — some days equal only to the number of new cases, other days numbers we can’t figure out where they came from.

That means there’s no data available about the number of people being tested, their age, gender, race, ethnicity OR geography. Those are key elements in studying trends in cases, testing and deaths, as well as the quality of the data being reported.

Because the totals of the demographic groups (so the sum of male, female and unknown gender, or the sum of all age groups) still equals the PUI total field, we know there’s something wrong on the back-end.

DOH has told reporters multiple times since then that the data is right. If so, that means the state has tested half the amount of people they claim to have tested. But given the trajectory of testing (people and tests), it’s more likely someone messed up my script and either doesn’t know how to fix it or is deliberately hiding that data.

I believe it more likely someone messed up and hasn’t fixed it, giving the lack of experience of the people who replaced me in my role as the coder, programmer, analyst, researcher, html and java-writer, etc etc over the dashboard and public data distribution.


Hospitalization-case data
DOH has tried to make clear its limitations on hospitalization data for months. In the data definitions document they provide, they state:

“‘Hospitalizations’ is a count of all laboratory confirmed cases in which an inpatient hospitalization occurred at any time during illness. These people may no longer be hospitalized. This number does not represent the number of COVID-19 positive persons currently hospitalized.”

DOH Data Definitions Document

Because Florida doesn’t track what’s commonly referred to as “probable” cases and deaths, we can’t really come up with a good number for hospitalization data, at least not “active” hospitalizations.

The CDC provided guidance about reporting probable cases back in April 2020. Here’s the information they provide on their website:

probable case or death is defined as:

Here are our warnings from our dashboard:

Warning about ER, In-patient admissions and death information related to cases

There is no active count of COVID-19 positive persons admitted to a hospital ER or as an in-patient in the state of Florida. 

Therefore, any statement regarding the current trend in ER and hospital admissions (and deaths-more on that below) are entirenly non-evidence based. 

The Florida Department of Health (DOH) has never provided data about the number of people currently hospitalized or in the emergency room who are COVID-19 positive. They do not provide dates for when COVID-19 persons were admitted to or discharged from the ER or as in-patients to a hospital. 

The only hospital surveillance data available to the public is the AHCA current bed availability data (see note about that data to the left) and the Florida Metrics data published by DOH every Sunday, which counts the total number of legal Florida residents who visited an ER in the state of Florida with symptoms matching the key terms of influenza and COVID-19 (reported by week).  

Additionally, DOH does not provide the date of death for any case. The chart on the now-disabled DOH dshboard showing resident deaths by date is just that – residents only by date of death for cases where a death certificate has been received and the date of death verified by the state. 

The CDC warns “delays in reporting a person’s death to the state can range from 1 week to 8 weeks or more, depending on the jurisdiction and cause of death.” 

According to internal DOH records, deaths reported by DOH were reported an average 7-21 days after the COVID-19 positive person died (date from March through mid-June_, with some deaths reported as late as 44 days after death.

WARNING ABOUT HOSPITAL ICU BED AVAILABITY DATA:

On June 22, 2020, Governor DeSantis announced that the Agency for Health Care Administration (AHCA) would no longer report the number of occupied ICU beds as it had previously been done.

AHCA provides the hospital data you see on this page. 

Consequently, the number of available beds shown here likely includes beds that are currently occupied, but that the DeSantis administration has determined by an un-disclosed metric could be made available by removing the person occupying it.

We continue to monitor this situation for changes and will post any new information as it becomes available. 

New data additions, site expansion and planned projects

Article by the Eagle

We’ve been busy this Fourth-of-July weekend.

We hope everyone stayed safe and preferably at home, but at a minimum adhering to guidelines about wearing masks and social distancing in public.

We know that laws and ordinances are changing quickly, and we’re trying to keep up by providing the most up-to-date list of local, county and state requirements for masks. Check out our blog for the most recent info about those policies, and feel free to leave comments to update that information as more information becomes available (but be sure to cite your info).

Now to the new stuff.

Dashboard and Website Updates:

  1. #COVIDStorm map and web page launch
  2. Demographics data on the dashboard
  3. Outreach to DOH about missing/inconsistent data

The storms in the Gulf and Atlantic oceans motivated us to get the basic frames of our #COVIDStorm map and dedicated web page up today around noon. A lot is still in development, so be patient with us while we aggregate and verify data to add to this critical resource.

With Rebekah (a published hurricane scientist) at the helm of this project, we want to be a leader in providing Hurricane-COVID-19 information, data, and analysis, so we’re doing a lot of modeling behind the scenes. This would have been Rebekah’s job at the Florida Department of Health (DOH) had she not been fired for refusing to manipulate data during a public health crisis.

We added a demographics page to the dashboard today, too. We want to provide more meaningful statistics about testing, cases and deaths, which is why we’ve requested more information from the Florida Department of Health regarding a number of data issues. We plan on publishing our letters and communication to and with DOH shortly, assuming they respond.

New website for COVID-19 Victims

We launched a new sister site today, FloridaCovidVictims.com.

This is where we will now post about known victims of COVID-19 in Florida. Rebekah works on this site and personally writes each memorial. So far we have identified and named more than 1,800 of the 3,832 persons who have died. That means more 2,000 victims remain nameless, and we’d like your help fixing that.

If you have information about someone who died from COVID-19, or you would like to contribute a photo or obituary link for a loved one, please email us at: Contact@FloridaCovidVictims.com.

We are also working to complete the datasets to release to the public to use for analysis and research regarding those who have died. There are hundreds of deaths missing from DOH’s data that were either previously listed by DOH, or are included in the Florida Department of Law Enforcement reports of cases.

Before we release the names of those persons who have been removed from the case line records, we want to ensure their cases have not been reassigned to other counties, reclassified as another cause of death, or otherwise moved or transferred.

Fundraising goals and plans

We also recently adjusted our fundraising goal.

We went through and calculated how much hardware, software and maintenance we’ll need through the next 6-18 months, how much time we’ll need to dedicate to that work, and the salary of the big boss (Rebekah Jones– who founded this venture) while she works on this project and searches for a permanent position post-COVID-19.

This more-exact calculation allowed us to drop our original fundraising goal by 40%. Our overall goal now is $300,000, and we’re still fundraising through GoFundMePayPal.com, and snail-mail donations (we prefer the snail-mail).

There is a small fee with both GoFundMe (2.9% on all transactions) and PayPal (3.05% if using credit/debit card, 0% if using PayPal funds), but ZERO fees if you send a check!

If sending a check, please address to:

Rebekah Jones
PO Box 13747
Tallahassee, FL 32317

Planned projects in the works

Once a tropical storm threatens our coast, we plan on launching public surveys to monitor and map a number of critical resources, including:

  • Hotel surveillance
    • Public reporting of available rooms, coronavirus rules/policies/protections in place, and more. Hotels are encouraged to participate.
  • Shelter surveillance
    • State policies on coronavirus per shelter, as well as shelter capacity and availability.
    • Special needs shelters, precautions in place, and public reporting of facility standards.

Think of Florida’s price-gouging reporting tool, but imagine that you could submit information about hotels that are fully booked or not following local ordinances regarding masks and social distancing, or the CDC guidelines, or shelters that are over-crowded and under-staffed. That’s what we’re going for. Even during the storm, providing information, whether photos or information about non-compliance by the staff or occupants, it’s all information that’s useful to your community.

That’s all for now, folks. Stay tuned as we ramp up our COVID-hurricane resources and work.

Updated Mask Policy List for July 4 Weekend; Please contribute

Article by the Wolf

I’ve included an updated list of current mask policies/ordinances. If you have information about your county or local policies, please comment below with a link to information verifying your comment, please and thank you!

— Updated July 3, 2020 —

PlaceDateDescriptionCity/County
Sanibel7/3Face masks required at any public building, government facility, library, or restaurantCity
Apalachicola7/2Masks must be worn inside establishments within city limitsCity
Lakeland7/2Face masks required in certain public indoor locations when not maintaining social distancingCity
Fort Myers Beach7/1Face masks required at indoor businesses, but not at parks or outdoorsCity
Seminole7/1Order that goes into effect on July 1 requires those living, working and visiting the county to wear a face covering at all businesses, places of assembly and places open to the publicCounty
Hialeah6/30Reported requirementsCity
Holmes Beach6/30Reported requirementsCity
Duval6/29Face masks required in all public and indoor spacesCounty
Jacksonville6/29Face masks required in all public and indoor spacesCity
Sarasota6/29Face masks required whenever individuals are indoors, as well as when outdoors where social distancing is not possibleCity
Milton6/28Face masks required for employees and customers of businessesCity
Anna Maria6/27Face coverings required indoors except in individuals homesCity
Gulf Breeze6/27Face masks required for all customers and employees of businessesCity
Indian River6/26Face masks required for employees of businesses open to the public. Order also applies to anyone visiting facilities where social distancing is not possibleCounty
Miami Springs6/26Face masks required at all open establishments, centers and meeting places within the city, including parking lotsCity
Pensacola6/26Face masks required for anyone inside a business within city limitsCity
St. Augustine6/26Face masks required at indoor locations, besides homes, when unable to maintain social distanceCity
Leon6/25County order requires individuals inside businesses to wear a face covering while inside buildingsCounty
Manatee6/24Face masks “recommended” – Voted down mask requirementsCounty
Pinellas6/24All people will be required to wear face coverings while in any indoor establishmentCounty
Charlotte6/23Voted down County mandate requiring masksCounty
Collier6/23Voted down County mandate requiring masksCounty
Palm Beach6/23Order requires all residents and visitors to wear facial coverings in buildings where the public is welcomeCounty
Pasco6/23Executive order requiring masks while indoors at all public businessesCounty
Hillsborough6/22Face masks mandatory in all businessesCounty
Miami6/22“Mask in public” rule within the cityCity
Orange County6/20People in public not eating or drinking must wear masksCounty
Martin6/19Required for eployees of restaurants, grocery stores and food serivce establishments. Martin County’s EO only requires face coverings for workers interacting with the public and strongly recommends them for the public.County
St. Petersburg6/19Employees in all businesses reuired to wear masks. 22-Jun Mayor issued EO mandating residents wear face covering in all public spaces and spaces where people are face-to-face.City
Tampa6/19Mandatory mask at all indoor locations where 6 foot social distancing can not be maintainedCity
Daytona Beach6/18Face masks required in all city buildingsCity
Monroe County6/17Owner, manager, employee, customer or patron of a business establishment must wear a face covering while in that business establishmentCounty

DOH to include hospital data; COVID Action Data Updates and More

Article by both the Wolf and the Badger

When we first launched this website, we knew hospitalization data would be a critical data element that had to be included. We include hospitalization data on the home page of the dashboard, and on the “Health and Hospitals” page.

Since then, hospital data has become a controversial subject.

When Rebekah worked as the COVID-19 data and surveillance manager at the Department of Health, she lobbied DOH for the inclusion of hospital reporting data on public-facing data products, like the DOH dashboard she built and the data portal that provided tabular data to the public. Hospital data was already provided by the Agency for Health Care Administration (AHCA) which monitors hospital capacity across the state for about 300 hospitals.

The AHCA data was always included during hurricane planning and response as a critical source of information, and epidemiologists across the world said hospital capacity was one of the most important indicators of COVID activity and the health system’s ability to adsorb another large influx of cases.

Not two full weeks after Florida COVID Action launched, the Governor decided to personally intervene in how hospital data was being reported.

Now, DOH has announced that it will be adding some hospital data to its “publicly available data on cases.”

Bloomberg news reported yesterday:

Florida will start reporting current Covid-19 hospitalization for the first time in the “next few days,” potentially adding an important new data set as the state faces a dramatic resurgence of the virus.

In an emailed response to questions, Florida Department of Health spokesman Alberto Moscoso said the information will be published alongside other publicly available data on cases.

Florida has been criticized for not publishing the data sooner. Its daily reports include a cumulative total of Covid-19 related hospitalizations, but not the number of coronavirus patients admitted currently in the state’s health-care system.

Read the article here.

Here’s the problem with that: DOH doesn’t have that data.

The data you see on the state’s and on the Florida COVID Action dashboard is the number of total hospitalized COVID-19 persons to date. It is not an active count. DOH may not find out that a person has been hospitalized (or even that they died) until months later. And the ICU data by AHCA is a count of the number of beds available — not how many are being occupied by COVID patients.

Florida does not (and has never) tracked the number of currently hospitalized COVID-19 patients. That’s because many people don’t even get tested until they’re so sick that they require hospitalization, so there’s no way to know how many are hospitalized at any given time. If forced to report this data as described by Moscoso, the number will likely be incredibly low (which is what I imagine the state wants).

We’ll keep an eye for this change, and update you with any information we get.

Still, we’re glad to see DOH adding a selection of hospital data, and we’ll remain optimistic that it will be reported fairly and accurately.

There are still a number of items that have yet to be addressed by DOH that we’d like to see added. Those include:

  1. Testing, cases and deaths in county/local jails
  2. Testing and cases in state prisons (only deaths are currently reported)
  3. Probable cases and deaths (as recommended by the CDC)
  4. Recovery data (as recommended by the CDC)
  5. Demographic data about persons tested (was provided and updated daily until Rebekah left in May)
  6. Testing and deaths by zip code (by day) for all resident types (currently only resident cases (cumulative) are provided)
  7. Statistics and data on contact tracing

MORE: UPDATES AND PENDING WORK

Important data notes:

  1. DOH switched how it runs updates for its reports/data last week.

Since March 13, each day’s case data (both the PDF and AGO APIs) were updated from data current as of 8 AM that morning. That meant the report actually included any cases, deaths, tests, etc., that came in between 8 AM yesterday and 8 AM today.

Now, DOH is pulling/updating data based on a 24-hour calendar date. So today’s report is the full report for yesterday only, and doesn’t include any data from today (June 30); So tomorrow’s report will only include cases/tests/etc from today (July 1).

It could be reasonably argued that case data should have been reported this way from the beginning. I’d agree that it makes the most sense to report by calendar date. That the change came right smack in the middle of the largest spike in cases to date is a little suspicious, though.

2. Starting yesterday (June 30), DOH split the giant, state-wide case line PDF file on the main Florida COVID website into two different files.

There is now a summary file that includes testing statistics, the number of cases by Florida cities, and the labs processing results. The line list of cases is now in a separate PDF report.

Data updates:

Long-term care facility (LTCF) cases and deaths tables have been updated
Note: DOH currently only publishes the provider name and county. This makes it difficult for us to put this data on a map, as there are multiple facilities with the same name in a single county. We are working to make sure we’re attributing the right case data to the correct facility, and will add it to the map once we’ve done that.
Also, the cases data and the deaths data are provided in two separate files, and without unique ID’s for each facility, we can’t compare cases and deaths for most facilities.

Daily county data from DOH for May 1- July 1
Visit the Data Library to see all our historical data. Click here to download the county data for May 1 – July 1 (downloads as an excel workbook).

Going live later this week: Testing by number of people tested, total tests (labs) processed, and cases by day, based on daily DOH data.

Please visit our Data Library for historical data and links to APIs or tables, and check out our Open Data Hub (working on cleaning this up a bit) for AGO web layers.

-30-

NPR Morning Edition: Scientist Says She Was Fired For Not Manipulating COVID-19 Data

Article by the Wolf

Click here to listen to the story on NPR.com or play audio below.

NPR’s Rachel Martin talks to Rebekah Jones, a scientist who was fired from Florida’s health department, who is now publishing a coronavirus dashboard of her own to track the state’s COVID-19 cases.

DeSantis Intervenes in Hospital Reporting Data, Cases Surge in Florida, Lead Epidemiologist Quits

Article by the Wolf

On Monday, June 22, Governor Ron DeSantis announced that he was personally intervening with how the Agency for Healthcare Administration (AHCA) reports current ICU capacity and bed availability.

Stating that he wanted only those in ICU (which stands for “Intensive Care Unit“) requiring “an intensive level of care” to be counted, the dramatic announcement has been met with a wave of public backlash.

When Homestead Hospital reached full capacity on Tuesday, June 23, the AHCA data showed they had at least eight available beds. Just a few days earlier, DeSantis publicly stated that all South Florida had plenty of hospital beds available for newly diagnosed COVID patients, despite what the state’s own data indicated — that major hospitals in the area were near capacity.

The political manipulation of the hospital data comes one week after we published the Florida COVID Action website, which prominently features hospital data, specifically ICU capacity, on both the home page and the “Health and Hospitals” tab of the dashboard.

To be fair, the move seems more related to AHCA’s publication of hospital capacity data that is published here (billed as a dashbaord, but is really just a table). It’s the same data that AHCA publishes as an API feed through the Florida Department of Emergency Management.

This move also comes as Florida continues to see its highest number of cases per day for nearly a week straight, totaling more than 130,000 positive persons as of 11:00 AM June 26 and more than 3,400 deaths.

We will continue to report the total number of ICU beds available in the state as AHCA reports it because they are the state authority for such data. However, we have added the following note to the dashboard so that viewers are aware of the changes and how it may impact our access to the true information.

Notice: On June 22, 2020, Governor DeSantis announced that the state would no longer report the number of occupied ICU beds as it had previously done. Consequently, the number of available beds shown here likely includes beds that are currently occupied, but that the DeSantis administration has determined by an un-disclosed metric could be made available by removing the person occupying it. We continue to monitor this situation for changes and will post any new information as it becomes available.”

From the Florida COVID Action dashboard, “Health and Hospitals” page

Florida “leadership” and its role in bucking against the science is nothing new.

In May, Florida fired the lead scientist responsible for reporting and tracking cases and deaths to the public (yours truly).Three weeks later, the head of the Bureau of Epidemiology and Interim Director of Infectious Disease abruptly quit when DeSantis announced schools would be reopening at full capacity in August. Several other key staff at DOH have also quit, resigned or been fired in recent weeks as pressure mounted for further manipulation of data within the department.

Florida never followed CDC guidance on reporting the number of probable cases and deaths, or the number of recovered persons, making Florida’s past, present and future state of coronavirus impossible to analyze.

It is no surprise that a Governor more concerned with vilifying his own scientists than protecting his people would end up causing a massive surge in cases — higher than any other state in the country.