Beware of COVID-19 projections based on flawed global comparisons

Continuing on the data question, found this to be a good explainer given the variances in how data is collected across jurisdictions:

As the COVID-19 pandemic unfolds, every day we are bombarded with numbers. Never before has the public been exposed to so much statistical information. You have been told that “shelter in place” measures are needed to flatten the curve of infections so that local healthcare systems have the capacity to deal with them. On the other hand, you hear that available statistics will not show if and when the curve of infections is flattening, and that existing projections are unreliable because input data are unsuitable for forecasting. Meanwhile, the issue of data and the pandemic fuels a debate in Canada over the release of federal and provincial forecasts of a COVID-19 death toll.

Should we then lose faith in the numbers altogether? The answer is no, but it is important to understand what statistics are available, what they measure, and which ones we should be looking at as the virus continues to spread around the world. One of the key areas where we need to exercise caution is especially when we compare ourselves with the situation in other countries.

As overwhelming as the flow of daily pandemic statistics might seem, data on COVID-19 around the world come from one source: health facilities’ administrative reporting about the number of positive cases, hospitalizations, intensive therapies, deaths, and recoveries. Most countries including Canada follow the guidelines of the World Health Organization and only test individuals with fever, cough, and/or difficulty breathing. Reported data on COVID-19 thus generally refer to symptomatic individuals who have presented themselves at health facilities and have met the established testing criteria.

One of the main indicators derived from these data is the overall case-fatality rate (CFR), which is the ratio between the total number of COVID-19-related deaths and the total number of confirmed positive cases. The CFR is an important indicator in an emerging pandemic because it measures the severity of the disease (how many infected people die from it). As of March 24, the CFR varied substantially across countries, ranging from 0.4 percent in Germany to 7.7 percent in Italy. In Canada and Quebec, it stands at 1.3 percent and 0.7 percent respectively.

It is well understood that different testing strategies for COVID-19 are responsible for a good part of the observed differences in the overall case-fatality rate across countries. For instance, South Korea, Germanyand Iceland adopted a large-scale testing strategy since the beginning of the outbreak, focusing on individuals in the wider population regardless of whether they were high risk or showing symptoms of COVID-19. Most other countries including Canada are following the recommendations of the World Health Organization to test only for COVID-19 symptomatic individuals.

These different testing strategies have a direct impact on the overall CFR because its value is smaller if asymptomatic individuals are included in the calculation, since the total number of positive cases (the denominator) increases. This is the first reason why the CFR is not immediately comparable across countries and should not be used as a measure of whether certain healthcare systems are dealing better with COVID-19 than others.

The second reason is that different testing strategies across countries also matter for the demographics of confirmed positive cases. As it can be seen in the figure below, because of widespread testing in Iceland, the age distribution of COVID-19 positive cases is much younger than in the Netherlands. This does not mean that younger people in Iceland are not respecting social distancing measures, or that the Netherlands has been more effective than Iceland in identifying infections among vulnerable elderly people. On the contrary, countries like Iceland that have effectively tested for COVID-19 early on have been able to identify and isolate clusters of potential infections before they spread to the more vulnerable segments of the population. By doing so, they have limited the number of COVID-19-related deaths and thus reduced the numerator in the calculation of the overall CFR. This is why the demographics of positive cases needs to be considered in the calculation of the overall case-fatality rate to make appropriate comparisons across countries.

The different demographics of COVID-19 positive cases underscore the importance of comparable data that are disaggregated by the patients’ most basic characteristics, notably age and sex. However, these data are only available for a handful of countries, because national health agencies release mainly aggregate figures on the total number of cases, hospitalizations, deaths and recoveries.

We all want to know how the COVID-19 pandemic will evolve. Considering the deep economic implications of the current worldwide standstill, there is a strong pressure to produce projections of the course of the pandemic and its human toll. Yet our efforts will continue to be misguided if we do not coordinate efforts to improve our understanding of where it is across countries through comparable statistics. This could be easily achieved by tracing the evolution not just of the total number of infections and the overall CFR, but also across age groups and for men and women separately.

National health agencies have been disseminating data and indicators about COVID-19 as they see fit because there is no global coordination about how to do so. The World Health Organization has not fulfilled its mandate to facilitate this coordination. Canada, thanks to its longstanding tradition of excellence in statistical reporting, is ideally placed to fill this gap and lead countries around the world to coordinate their monitoring efforts of the pandemic through comparable statistics. This may be one of the crucial steps to win the war against COVID-19.

Source: Beware of COVID-19 projections based on flawed global comparisons

About Andrew
Andrew blogs and tweets public policy issues, particularly the relationship between the political and bureaucratic levels, citizenship and multiculturalism. His latest book, Policy Arrogance or Innocent Bias, recounts his experience as a senior public servant in this area.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: