How Not To Analyse COVID-19 Data
Mumbai & Guwahati: On June 11, 2020, as India’s reported COVID-19 tally neared 290,000 cases, the health ministry said that India’s cases per 100,000 population is amongst the lowest in the world.
#India's cases per Lakh Population amongst the Lowest in the World: DG @ICMRDELHI in the media briefing pic.twitter.com/zueFS48s6z
— PIB India (@PIB_India) June 11, 2020
Such comparisons are misleading because of many confounding factors including the stage of the epidemic, the extent of testing and various socio-economic factors, experts told IndiaSpend.
The main problem with such a comparison is that the number of cases is compared against the total population when different proportions of the population in these countries have been tested, said Jacob John, former professor of virology at Christian Medical College, Vellore. “All information in India is from a very small ‘country’ of 4 million people. That country is known as ‘total tested’.”
As of 9 a.m. on June 12, 2020, India had tested 5.4 million people--0.4% of the total 1,353 million population. This is the lowest testing per capita among the worst affected countries, as per Our World In Data.
The extent of the spread in the remaining 99.6% of the population is not known. This is the result of continuously denying any "community transmission", John said, because the "community" is outside the 0.4% tested.
Initially, the purpose of testing was to detect importations (since most infections came with people from overseas) and try to stop local transmissions from them, added John. When community transmission began, the importance of testing for contact-tracing faded but the need for testing to confirm clinically diagnosed cases grew in priority and importance.
While other countries are testing for lab confirmation of clinical COVID-19 cases, India is still testing for contact-tracing, John said, “Hence, the ability for meaningful comparisons is gone.”
(On June 11, the Indian Council of Medical Research (ICMR) denied any community transmission in the country, after it released the results of the first serological survey.)
Also, each country is at a different stage of its epidemic. “After China, each country has imported the initial cases through foreign travellers at different times,” said Tanmay Mahapatra, an epidemiologist with CARE India, an NGO, “Thus the spread and upsurge varied over time.”
Besides, disparities in socio-economic factors, access to healthcare, and the quality and accuracy of tests used in the respective countries also affect the way cases are reported.
Why it is difficult to follow trends
The number of samples being tested is influenced by the testing strategy adopted. Since March 9, 2020, when the ICMR issued the first testing strategy, there have been five revisions.
“Different strategies influence data differently over time intervals, making trends a mix of those and thus difficult to follow,” said Mahapatra. For example, a change from testing only symptomatic patients with severe acute respiratory illness (SARI) to those with travel history or contact irrespective of symptoms will change the proportions of asymptomatic cases getting identified and included in the reported case pools.
“Most countries shifted strategy once,” said John, “The second strategy was testing to confirm clinical diagnosis of COVID. That is what India should be doing. If the testing strategy gets repeatedly changed, the results cannot be used for understanding time trends.”
While comparing trends on mortality or recovery, the denominator plays an important role. If India’s mortality ratio is calculated by including only those cases where the outcome is known, the ratio would be 5.5%, 2.7 percentage points higher than the government’s estimate of 2.8%, IndiaSpend reported on June 7, 2020. The lower estimate by the government is because it includes all the active cases in the denominator.
How cases and deaths are defined matters
How a COVID-19 case or death is defined also affects the data. “In epidemic management, there is always a "case definition",” said John, adding that the central government has not provided the doctors with any case definition, forcing them to use influenza-like illness (ILI) as a surrogate.
“ILI is to diagnose another respiratorily-transmitted virus disease called influenza. We are dealing with COVID-19 or Coronavirus disease and doctors must look for COVID-like illness and not influenza-like illness,” said John.
How deaths are reported in countries or states and whether COVID-19 is recorded as a cause of death or not, is another big question, said Madhukar Pai, Canada Research Chair in epidemiology and global health and director of the McGill International TB Centre in Montreal, Canada.
Within India, different states have been reporting deaths differently, making data incomparable.
West Bengal, for instance, reported on April 24, 2020 that it had a total of 57 “COVID-linked deaths” but only 18 fatalities “directly related to the coronavirus”, with the state government claiming that the remaining 39 had died due to comorbidities.
On the following day, it was reported that the Kolkata Medical College and Hospital was issuing death certificates for “strongly suspected” COVID-19 cases by attributing the cause of death to the underlying health condition or comorbidity.
Similarly, when the Municipal Corporation of Greater Mumbai revised its recording protocols on April 15, 2020, it began auditing each COVID death, thereby increasing the possibility that those with comorbidities would be de-classified as a COVID-19 fatality.
In Delhi, discrepancy was found between the death tolls provided by dedicated COVID-19 hospitals in the city, and those reported by the health department. Even as the state government did not report any new fatality between May 3 and May 6, 2020--when there were 64 deaths according to the central health ministry--two major government hospitals reported 20 deaths attributable to COVID-19 in the same period.
On May 10, 2020, the ICMR issued guidelines for the appropriate recording of COVID-19-related deaths, in which it stated that deaths with inconclusive test results, but having known symptoms of COVID-19, should be recorded as “probable COVID-19” deaths.
The World Health Organization (WHO), on April 16, 2020, defined death due to COVID-19 as a death resulting from a clinically compatible illness, in a probable or confirmed COVID-19 case, unless there is a clear alternative cause of death that cannot be related to COVID (e.g. trauma).
A death due to COVID-19 may not be attributed to another disease (e.g. cancer) and should be counted independently of preexisting conditions that are suspected of triggering a severe course of COVID-19, said the WHO guidelines.
Disparities in state-level data
In addition to the way COVID-19 deaths are reported, states are also testing differently. The latest testing strategy, released by the ICMR on May 18, 2020, mandated the testing of “asymptomatic direct and high-risk contacts of a confirmed case... once between day 5 and day 10 of coming into contact”.
However, the Delhi government issued an order on June 2, 2020 saying it would test such persons only if they are senior citizens (above 60 years) or have severe comorbidities--cancer, diabetes and hypertension.
Further, in a June 4, 2020 update, the health department cracked down on hospitals that had admitted asymptomatic or mildly symptomatic cases without recommending them for ‘home isolation’ first.
On April 12, 2020, Maharashtra had changed its testing strategy and stopped testing asymptomatic contacts of confirmed COVID-19 patients, unless they were 34-weeks pregnant, on dialysis or chemotherapy, or health workers exposed to COVID-19 patients.
Some states have issued conflicting directives regarding private testing. Private laboratories are allowed to test patients only when they have been prescribed by a ‘qualified physician’, according to ICMR guidelines on testing of COVID-19 in private laboratories released on May 21, 2020.
In Mumbai, the city with the highest number of cases at 53,985 as of June 11, 2020, the municipal corporation sent show-cause notices to some doctors who prescribed private lab testing without adequate physical examination, even though the ICMR has made no such recommendation.
Telangana has barred private labs from conducting tests despite the Telangana High Court directing the state government to allow testing in the 11 private facilities approved by the ICMR. The state has reported a total of 4,320 cases as of June 12, 2020, after a 31% increase in positive cases in the last week. It had tested 668 per million as of May 17, 2020--when the state last released data on testing--against India's 1,646 per million.
In Gujarat, private clinics and hospitals had to report the clinical details of suspected patients to a government-approved medical college, based on which the medical superintendent of the college or the chief district medical officer had to prescribe testing in private labs. This requirement was later struck down by the state’s high court and was replaced by the ICMR directive which did not mandate clinical reporting to government hospitals. However, the Gujarat government has since issued a new directive that makes hospitalisation mandatory for those who have been recommended for private tests.
Contact-tracing of the infected patients also varies across states. While Karnataka has traced 47.4 contacts per confirmed case, on an average, Delhi and Maharashtra have traced 2.1 and 2.3 respectively, an ICMR study on data till April 30, 2020 found.
Besides testing, the implementation of the lockdown; extent and effectiveness of isolation, quarantining and reporting; health system response and systemic errors also act as potential confounding factors for state-level data.
(Shreya Raman is a data analyst and Saha is an intern with IndiaSpend.)
