Study that suggests BCG vaccine may protect against Covid-19 is flawed – TB experts

BizNews has carried the most important insights from world medical experts on the possible causes, preventions and cures for Covid-19, with in-depth interviews carried in our Inside Covid-19 section. When news emerged that a compulsory vaccine given to babies to protect against tuberculosis, TB, may have the extra advantage of shielding against Covid-19, BizNews immediately interviewed New York Institute of Technology’s Dr Gonzalo Otazu to help our community better understand the patterns in the data that may give us hope. As is healthy and normal in scientific discovery, other experts have scrutinised Dr Otazu’s data, cautioning against early optimism that the TB vaccine may help prevent the spread of Covid-19. BizNews editor-in-chief Alec Hogg contacted the experts at the McGill International TB Centre in Canada to ascertain whether South Africans have a better chance than many at keeping Covid-19 at bay. As debate rages over the value of the BCG vaccine in containing Covid-19, it’s worth remembering the wise words of Professor Alan Whiteside, a leading South African expert on HIV-Aids and global health policy, who has been analysing the reporting of Covid-19 cases around the world: Numbers are political. – Jackie Cameron

From Alec Hogg, BizNews Editor-in-chief to McGill International TB Centre, Canada

The recent research by NYIT’s Dr Otazu has lifted spirits in South Africa where most of the population has had BCG vaccinations which may be a shield against Covid-19.

However, there is great confusion over the date when people were vaccinated – primarily because your Atlas is being used as the source by academics who have read it to state the date it was introduced here was 1973.

I’m a little older than 47 and have two clear sheaf gun marks, as does my wife and many others of my acquaintance. Medical practitioners tell me this is from the BCG vaccination.

Another published source states vaccinations in SA go back to the 1940s and has been universally applied to infants since the 1960s.

Seems strange that SA, a British colony until 1961, would simply not vaccinate until 1973, more than half a century after the vaccine was in common use elsewhere – especially as TB has been a problem here for a long time.

I am the editor of which has been giving a lot of attention to the BCG vaccination story because of its relevance in the Covid-19 crisis.

Please can you help by clarifying the 1973 date and its source. Was this perhaps the date at which vaccinations became universal – ie for all sectors of the population (with that crazy Apartheid government one never really knows?)

McGill International TB Centre, Canada to BizNews

Our team is skeptical of the ecological studies on BCG and Covid for the reasons in our blog post:

We do not have any additional data from South Africa.  What I can tell you, is that from our data it indicates that the national universal BCG vaccination policy was implemented in 1973, that does not preclude, BCG vaccination among some parts of South Africa or among key populations prior to that date.

Prof Madhukar Pai, MD, PhD, FCAHS

Canada Research Chair in Epidemiology & Global Health

Director, McGill Global Health Programs

Director, McGill International TB Centre

McGill International TB Centre blog post: 

Universal BCG vaccination and protection against Covid-19

Lena Faust, Sophie Huddart, Emily MacLean and Anita Svadzian – PhD students in Epidemiology at the McGill International TB Centre, Montreal, Canada – raise concerns about a recent study that has raised hopes that BCG vaccination protects against Covid-19. BizNews shares the key points made in their blog post on the potential effectiveness of an existing vaccine, the Bacille Calmette-Guérin (BCG) vaccine, against Covid-19. The BCG vaccine is used in some countries for the prevention of tuberculosis (TB), a bacterial infectious disease caused by Mycobacterium tuberculosis (Mtb), which they say accounted for 1.5 million deaths globally in 2018. Prof Madhukar Pai, an expert in epidemiology at the McGill International TB Centre, supports their arguments.

Who has had the BCG vaccine?

  • National BCG vaccination policies vary across the world, with universal BCG vaccination at birth still occurring in high TB-burden countries such as India and Ethiopia, while countries with lower overall TB incidence have largely discontinued universal BCG vaccination in favour of targeted vaccination of specific TB risk groups (such as children from families immigrating from high TB burden countries), says the McGill International TB Centre.
  • The BCG World Atlas, created by the McGill International TB Centre, provides data on BCG policies and practices around the world. Furthermore, their information suggests that BCG vaccination was implemented universally in South Africa from 1973, though others may have received it.

The study by Miller et al – a quick snapshot

The McGill International TB Centre students looked at the following study: Correlation between universal BCG vaccination policy and reduced morbidity and mortality for COVID-19: an epidemiological study. Authors: Aaron MillerMac Josh ReandelarKimberly FasciglioneVioleta RoumenovaYan LiGonzalo H Otazu. 

They highlight the following:

Data of Covid-19 cases and death per country were obtained on March 21, 2020, from The ecological analysis found an association between Covid-19 death rates and universal BCG vaccination policies among middle and high income countries. The authors also observed an association between Covid-19 cases per capita and BCG vaccination policies, with countries with a universal BCG vaccination policy having lower cases per capita and death rates. Based on the analysis, the authors state “The correlation between the beginning of universal BCG vaccination and the protection against Covid-19 suggests that BCG might confer long-lasting protection against the current strain of coronavirus”.

Six reasons to question the findings

1. Flawed study design

“Ecological studies often conflate population level exposures, such as country-wide policies, with exposures at the individual level, such as the effect of the BCG vaccine on a human body. What may hold at the aggregate level, will not necessarily be true when the heterogeneity of individuals is considered,” say the McGill International TB Centre students.

“In this study, low quality evidence observed at the population level is used to make sweeping inferences about BCG’s effectiveness on an individual level. Ecological analysis techniques simply cannot directly inform etiology of exposure/disease relationships; rather, they serve us well as strictly hypothesis-generating queries and should not be stretched beyond this purpose,” they caution.

2. Rapidly changing data

Drawing premature conclusions from rapidly changing data in a pandemic is problematic, say the academics.

“Since the analysis (COVID-19 data were downloaded on March 21, 2020), the pandemic has escalated in many countries, including low and middle income countries (LMICs) where BCG is given at birth. For example, COVID-19 cases in India have increased from 195 on March 21  to 1071 on March 31. In South Africa, cases have increased from 205 on March 21 to 1326 on March 31. Therefore, the paper’s findings could look very different as countries progress in their epidemic timelines (and expand diagnostic capacity),” they note.

3. Coronavirus cases are underestimated worldwide

There are shortages of diagnostic tests and the number of tests performed per capita varies wildly by country, point out PhD students Faust, Huddart, MacLean and Svadzian.

“Underestimation of the number of cases in lower income countries could entirely explain the authors’ observed results. India, for example, has one of the lowest Covid-19 testing rates in the world. Additionally, countries have each applied their own algorithm to determine who gets tested. Thus, the number and composition of known cases is heavily influenced by testing strategy,” they argue.

Although Miller and associates acknowledge the concerns surrounding current estimates of Covid-19 cases and also use Covid-19 deaths as a measure of the severity of each countries’ outbreak, “unfortunately, even current Covid-19 death data are unreliable in some parts of the world”.

“In wealthy countries, it is true that most deaths due to Covid-19 will occur in a hospital after having received a diagnosis of Covid-19. However, we know that in LMICs, even before this current pandemic, most deaths occurred at home without medical assistance. People dying at home in LMICs are unlikely to receive a confirmed diagnosis of Covid-19 and may not have their death accounted for in government records. Coronavirus deaths may also be missed in high-income countries that have been overwhelmed by the virus, like Italy,” say the McGill International TB Centre students.

4. Correlation is not causation

A critical flaw in the study is reading too much into the pattern in the statistics. The “perceived relationship between an exposure and an outcome does not mean one causes the other – a concept often expressed through the well-known adage ‘correlation does not imply causation’.”

“Specifically, in the case of the relationship between universal BCG vaccination and the apparent lower Covid-19 case numbers and Covid-19-associated deaths, a multitude of underlying factors are at play. The crucial take-away here is that if these factors could be appropriately accounted for, we would very likely not see the association between BCG vaccination and protection against Covid-19 suggested by this paper,” say the Canadian university students.

“The concerns about data quality now become a potential source of confounding in the relationship between BCG vaccination and lower Covid-19 case numbers. It is likely that under-funded health systems in (LMICs) have low Covid-19 testing capacity, and at the same time, are more likely to have universal BCG vaccination policies still in place, given that TB burden is highest in LMICs. Therefore, the study actually does not demonstrate that populations in countries with universal BCG vaccination policies are somehow more protected against Covid-19, because lower case numbers and deaths may be more attributable to low testing capacity than to any supposed protectiveness of BCG.”

“Statements in the paper that imply causality, such as the statement that ‘BCG vaccination also reduced the number of reported Covid-19 cases in a country’ are dangerously misleading, particularly in a context where reported case numbers are in large part a function of testing capacity and testing strategy, rather than actual relative burden,” they emphasise.

5. Countries are too different to make them comparable

It is difficult to draw causal conclusions across a broad range of countries with varying underlying demographic characteristics, such as differences in the age distribution of the populations of interest – and this is particularly true of diseases for which severity differs by age, say the McGill International TB Centre students.

“As shown in a recent study from Wuhan, China, older individuals are at higher risk for severe Covid-19, so differences in the age distribution of populations may significantly influence death rates observed across countries (for example, the WHO regions of South-East Asia and Africa have a median age of 27.0 and 18.7 years, respectively, whilst the population in the European region is much older (median 38.6 years), which may in part contribute to the higher death rates observed in Europe).”

6. No clear evidence TB vaccine protects against anything other than TB

Lastly, the students turn to biological plausibility and warn that some of the “common sense” appeal of this paper’s claim is based on a bit of a false equivalence.

“The authors twice mention that BCG protects broadly against respiratory diseases. The idea that a vaccine for an infectious disease of the lungs could also protect against other infectious diseases of the lungs seems somewhat consistent and thus palatable. This hits a snag though, when we look at the type of TB against which BCG most effectively protects,” they point out.

“Due to an unfortunate natural experiment wherein there was a global shortage of the BCG vaccine from 2013 to 2015, retrospective data show there was a surge in cases of TB meningitis, that is, TB of the central nervous system. The tidy idea of a lung disease vaccine protecting against another lung disease, i.e. COVID-19, is not quite so straight-forward, since BCG seems to offer most significant protection against TB meningitis.

“Additionally, it is well-established in the TB field that the BCG vaccine only confers protection against TB in young children, and that protection almost completely wanes by age 12. Recent studies have even examined whether applying a booster vaccination to older adolescents or adults would help extend this anti-TB protection. The important take-away is that BCG’s protective effect fades over time. So it is not clear how a vaccine that offers protection early in an individual’s life would protect individuals against Covid-19 once they are of an advanced age,” the McGill International TB Centre students note.

But don’t give up hope; BCG trials may save the world

The students warn that giving people hope that BCG will protect against Covid-19 could lead to a false sense of security and lead to inaction, ultimately causing harm. Nevertheless, they say that ” with further research, it may emerge that the BCG vaccine does confer protection against Covid-19″.

“An Australian trial is set to start imminently, aiming to investigate whether BCG vaccination protects against Covid-19 or reduces severity of Covid-19 in Australian healthcare workers. If, in a well-performed trial, we do in fact see  a benefit in re-purposing this vaccine, the evidence would be much more convincing,” they add.

(Visited 27,516 times, 15 visits today)