• Health

Comparing SARS-CoV-2 infection rate of vaccinated and unvaccinated populations doesn’t reflect the real COVID-19 vaccine effectiveness

Posted on:  2022-04-13

Key takeaway

Vaccinated and unvaccinated populations may differ in many characteristics, such as age, population size, social behavior or health seeking behavior. These differences must be taken into consideration when comparing the SARS-CoV-2 infection rates between vaccinated and unvaccinated populations. Failure to do so may lead to biased conclusions. Studies accounting for these differences showed that COVID-19 vaccines effectively reduce the risk of getting sick.

Reviewed content

Verdict:

Lacks context

Claim:

The SARS-CoV-2 infection rate is higher among vaccinated than unvaccinated in the UK

Source: Facebook, Gregg Schmedes, 2022-04-06

Verdict detail

Lack of context: The comparison of the COVID-19 infection rate between vaccinated and unvaccinated populations didn’t take into consideration possible differences between those groups, such as social interactions or health seeking behavior. Such differences change the risk for individuals to get infected and thus needs to be accounted for to avoid biasing the analysis.

See our method of evaluation

Full Claim

The SARS-CoV-2 infection rate is higher among vaccinated people than unvaccinated people in the UK; vaccines don’t protect from disease

Review

The United Kingdom was the first country to authorize COVID-19 vaccines and to initiate the vaccine roll out in December 2020. Since then, 86% of the UK population above 12 has received at least two vaccine doses, as of 7 April 2022.

Although clinical trials and real world studies found that COVID-19 were effective at preventing illness and death, the impact of the nation’s large vaccination campaign has been a recurrent subject of misinformation. Some used biased comparison of mortality rates between vaccinated and unvaccinated people to promote the inaccurate claims that COVID-19 vaccines weaken the immune system or aren’t effective at preventing COVID-19 death. Health Feedback reviewed these claims on several occasions.

However, claims like this still continue to crop up. On 6 April, U.S. State Senator for New Mexico Gregg Schmedes published a Facebook post in which he claimed that “Unvaxxed have lowest infection rates in UK”, showing a graph of infection rates in the UK between 6 to 27 March 2022. A similar comparison of COVID-19 cases and deaths between vaccinated and unvaccinated people led cardiologist Peter mcCullough in an article in The Epoch Times to claim that COVID-19 vaccines were “having a negligible effect in populations”.

As we will explain below, while these comparisons used official data, they are scientifically meaningless as they didn’t take into account other important factors that set vaccinated and unvaccinated people apart, besides vaccination status.

When comparing the rate of cases or deaths between two groups of people, for example between vaccinated and unvaccinated people, it is important to account for any other differences between them apart from vaccination status that could also affect their risk of getting sick. A failure to account for these differences could lead to a biased result and an incorrect conclusion. Health Feedback previously explained the pitfalls of such comparisons and the measures that epidemiological or clinical studies take to avoid them.

The claim under review is based on the weekly COVID-19 report of the UK Health Security Agency (UKHSA) of 31 March 2022. It used data from Table 14 of that report, which presented the rate of newly reported COVID-19 cases among vaccinated and unvaccinated groups between 6 March and 27 March 2022.

However, UKHSA clearly warned in its weekly report that a crude comparison without additional adjustment of the figures cannot provide meaningful information about vaccine effectiveness:

“The vaccination status of cases, inpatients and deaths should not be used to assess vaccine effectiveness because of differences in risk, behavior and testing in the vaccinated and unvaccinated populations.“

In a blog post, UKHSA listed some of the factors that can bias the results of a crude comparison of infection rates. One of them is the fact that the vaccination campaign began by prioritizing people with a higher risk of exposure to COVID-19, such as healthcare workers. This means that people who are more likely to develop COVID-19 are overrepresented among the vaccinated compared to the unvaccinated. This might lead to an increase in the number of cases among vaccinated people that is unrelated to vaccine effectiveness.

Another factor to consider is the potential behavioral difference between vaccinated and unvaccinated individuals. The UKHSA blog post cited two possible behavioral differences. Firstly, unvaccinated and vaccinated individuals may behave differently with respect to social interactions. For instance, vaccinated people may become less compliant about physical distancing measures which may increase the risk of infection, knowing that they benefit from vaccine-induced immunity. The increased risk of infection in this case is unrelated to vaccine effectiveness[1-3].

Second, unvaccinated and vaccinated individuals may differ in their healthcare-seeking behavior. If one group tends to seek medical attention and get tested more often, then the likelihood of detecting COVID-19 in that group will be higher. Therefore, they may be overrepresented among new COVID-19 cases. This is not because that group is more vulnerable to infection, but because the group gets tested more often.

A preprint—a scientific study that hasn’t yet been peer-reviewed by other scientists—reported that vaccinated people were more likely to seek out testing than unvaccinated individuals[4]. Therefore, such a difference in behavior can affect the number of new COVID-19 cases detected in vaccinated and unvaccinated people.

Taken together, these factors indicate that one cannot directly compare the infection rates without additional steps to limit the risk of bias. In particular, differences in healthcare-seeking behavior can be accounted for by using a specific study design called a test-negative study[5,6]. Even when focusing on a specific population subgroup, such as children, behavioral differences or medical differences between vaccinated and unvaccinated children must be taken into consideration before drawing conclusions on vaccine effectiveness.

In fact, the UKHSA stated that this method was employed to assess the effectiveness of COVID-19 vaccines while accounting for various biases, like the ones described in this review:

“These factors are all accounted for in our published analyses of vaccine effectiveness which uses the test-negative case control approach. This is a recommended method of assessing vaccine effectiveness that compares the vaccination status of people who test positive for COVID-19, with those who test negative.

This method helps to control for different propensity to have a test and we are able to exclude those known to have been previously infected with COVID-19. We also control for important factors including geography, time period, ethnicity, clinical risk group, living in a care home and being a health or social care worker.”

UKHSA listed in Table 5 of their report several vaccine effectiveness studies and preprints using the test-negative design. They reported that COVID-19 vaccines were effective at preventing illness, with effectiveness varying depending on the SARS-CoV-2 variant, the use of booster dose, and the elapsed time since vaccination[7-11].

These analyses showed that, when taking into consideration potential factors that may lead to bias, vaccines proved effective at protecting against COVID-19, with differences depending on the SARS-CoV-2 variant and the elapsed time since vaccination. Curiously, the Facebook post by Schmedes made no mention of this analysis. This also shows that McCullough’s claims are unsubstantiated as they rely on crude comparisons that don’t take into consideration the aforementioned bias.

In summary, a crude comparison of the infection rate between vaccinated and unvaccinated populations leads to misleading conclusions. While the data presented in Table 14 of the UKHSA weekly report took into account certain factors, such as age and the difference in population size between the vaccinated and the unvaccinated, it didn’t consider other potential biases that could be produced by differences in social interaction or healthcare-seeking behaviors. More carefully designed studies of vaccine effectiveness, which account for differences between the vaccinated and the unvaccinated apart from vaccination status, showed that vaccines are effective at preventing symptomatic infections.

Update (2022-05-19):

This article was updated to include a discussion of vaccine effectiveness in children and address a claim made by Peter McCullough.

REFERENCES

Survey

Science Feedback is a non-partisan, non-profit organization dedicated to science education. Our reviews are crowdsourced directly from a community of scientists with relevant expertise. We strive to explain whether and why information is or is not consistent with the science and to help readers know which news to trust.
Please get in touch if you have any comment or think there is an important claim or article that would need to be reviewed.

Get in touch

Tags:

,

Published on:

Editor:

Pablo Rougerie

Related Articles