Contrary to viral Instagram videos, COVID-19 vaccines remain highly effective against severe disease

In December 2022, social media users on Instagram began sharing a video that showed a rapid series of headlines referring to the COVID-19 vaccines, suggesting that their effectiveness has fallen significantly.

Some users posted variations of the video containing additional comments, such as “Slowly but surely the truth comes out”; “They’d rather say that the ‘science changed’ instead of admitting that they have been lying the whole time”; “‘Highly effective!’ 🤣🤣🤣📉📉📉”; and “From 100% effective to under 50% effective to 20% effective” (see Figure 1).

Figure 1. A screenshot of one variation of the video shared on Instagram, showing the comments added.

The dozens of headlines shown in the video, mostly taken from news stories, started with reports of vaccine effectiveness at 100% and then falling to 20% as the video progressed. In each clip, the percentage figure was highlighted.

The implication from the video and the comments about the video by social media users is that the original vaccine effectiveness figures were incorrect, and that over time the true effectiveness, which is lower, was revealed.

In addition, the video employed several potentially misleading approaches in the way it presented information. For example, the order that the headlines were presented leaves viewers with the impression that they are in chronological order, and that vaccine effectiveness fell over time.

In many cases, the sources and dates are cropped from the images, leaving viewers with no context for each headline, even though the date of publication is a relevant piece of information for understanding changes in vaccine effectiveness, as we will explain later. Based on our online searches for the corresponding articles, the headlines appear to be from articles published between November 2020 and October 2021.

However, the headlines weren’t ordered by date. For example, one of the first headlines was “Pfizer/BioNTech says its Covid-19 vaccine is 100% effective and well tolerated in adolescents” from an article published 31 March 2021. This was followed by another headline “Moderna vaccine is more effective against COVID-19 infection at 96% compared to Pfizer shot at 89% another study finds” from an article published 23 September 2021, and then another headline stating “Updated Pfizer Data Shows Vaccine Is 95% Effective”, which was published 18 November 2020. Much later in the video, the headline “One dose of vaccine ‘only 33% effective’ leaving over-60s vulnerable, Israeli experts claim” appeared, which was published 20 January 2021.

This ordering, plus text saying, “Slowly but surely the truth comes out,” creates a misleading impression that these are in chronological order and that the reported effectiveness of the COVID-19 vaccines was changed from 100% to 20%.

However, as this review will explain, the numbers shown aren’t comparable to each other. In fact, the variation in reported vaccine effectiveness is expected as studies took place under different conditions, with different populations, and measured different outcomes.

The vaccine effectiveness figures reported in the headlines refer to different forms of protection against infection, severe disease, and death

The COVID-19 vaccines work by preparing the immune system to fight a SARS-CoV-2 infection, although they don’t provide 100% protection against infection Where the vaccines’ greatest strength lies is in reducing disease severity, so it has the biggest effect on protecting against death from COVID-19^[1]. The vaccines are less effective in preventing any COVID-19 symptoms, particularly more so since the spread of newer, more infectious virus variants like Omicron. So, if two studies looked at different outcomes of patients, then they aren’t comparable. However, this nuance is lost in the way that the video is presented.

One headline presented in the video originated from a Times of Israel article in May 2021, which reads, “Pfizer vaccine 96.7% effective at preventing COVID deaths, Israeli data shows”. This figure came from a peer-reviewed study published in The Lancet^[2].

However, the 96.7% figure was soon followed by another headline stating, “The Pfizer and Moderna vaccines are 94 percent effective at preventing hospitalization in older adults, a study finds”. This headline came from a New York Times article published in April 2021 reporting on a small study that looked at hospitalization rate, not deaths^[3]. Therefore, comparing these two figures with each other is like a comparison of apples and oranges.

The video switches between clinical trials and real-world studies

The headlines presented in the video are taken from a mixture of articles that reported on clinical trials and studies of real-world data. The figures taken from these two types of research are not comparable as they come from different groups of people.

For instance, the headline “Moderna vaccine is more effective against COVID-19 infection at 96% compared to Pfizer shot at 89%, another study finds” comes from an article by the Mail Online in September 2021, which reported on a study of real-world data. Meanwhile the headline “Pfizer/BioNTech says its Covid-19 vaccine is 100% effective and well tolerated in adolescents” came from a CNN article in March 2021, which reported on a clinical trial.

Clinical trials are typically done in selective groups, often healthy adults. For instance, the original clinical trial of the Pfizer vaccine included only “Adults 16 years of age or older who were healthy or had stable chronic medical conditions” ^[4]. This trial excluded people with compromised immune systems and pregnant women, who did receive the vaccine when it was rolled out.

Scientists use different terms to discuss how well a vaccine works depending on the setting. In clinical trials, we refer to the efficacy of the vaccine, while in the real world, it is known as the effectiveness.

In an email to Health Feedback, Julie Bettinger, a vaccine safety scientist at the Vaccine Evaluation Center based at the British Columbia Children’s Hospital in Canada, explained that:

“Efficacy is ALWAYS going to be higher than effectiveness because in the real world we give the vaccine to people where we know it likely won’t work (e.g. cancer patients) but we don’t do this in a clinical trial. So then, when you are counting who it worked for and who it didn’t, you include all those people that you expect it wouldn’t work in, and the estimate is lower. The headlines are comparing efficacy and effectiveness estimates which is not accurate and shouldn’t be done.”

The vaccine effectiveness figures originate from studies of different SARS-CoV-2 variants

The vaccines used in national rollouts were developed to protect against the initial SARS-CoV-2 virus that was first identified in Wuhan, China in December 2019. As the virus spread around the world, new variants evolved that changed how the virus behaves, for instance, making it more transmissible or more likely to evade the immune system.

The Instagram video included headlines that actually referred to several different variants over the course of the pandemic:

• “Updated Pfizer Data Shows Vaccine Is 95% Effective” – from WebMD in November 2020, referring to clinical trials testing against the original variant
• “Johnson & Johnson 1-dose vaccine tests only 57% effective against South African COVID-19 variant” – appears to be from Cleveland.com in January 2021, referring to preliminary findings by the manufacturer from a study in South Africa, where the Beta variant had spread widely. The 57% figure is for effectiveness against moderate-to-severe COVID-19
• “Pfizer’s Covid vaccine is only 42% effective against Indian ‘Delta’ variant while Moderna’s jab is 76% effective, Mayo Clinic Study suggests” – from Mail Online in August 2021, referring to a study in Minnesota that had not been peer-reviewed yet at the time of the Mail article’s publication. The 42% figure refers to vaccine effectiveness against infection. A later peer-reviewed version of the study with more data revised this figure to 63.5% effectiveness for the Pfizer vaccine^[5].

As we can see from the above, the video didn’t make it clear that the figures were from a variety of studies about different vaccines against different variants.

Bettinger explained that the mismatch between the strain targeted by the vaccine and the strains currently circulating is important knowledge when it comes to interpreting vaccine effectiveness figures:

“The effectiveness has not changed against the vaccine variant. Unfortunately, the vaccine variant is not the main variant currently circulating. The ‘match’ between what is in circulation and what is in the vaccine is not 100% so the vaccine does not work as well against infection with a new variant. It is still highly effective against severe disease as measured by hospitalization and death, even with a non-vaccine variant.”

The Beta variant, which was widespread in South Africa in 2021, caused particular concern among scientists as it had mutations in a key part of the virus that the vaccines were designed to target. This meant the vaccines were less protective against the Beta variant, however it’s no longer circulating and is not an ongoing concern.

Based on surveillance data collected by the U.S. Centers for Disease Control and Prevention, the Omicron variant has since overtaken all other variants to become the dominant variant in the U.S. and has proven to be more infectious than previously identified variants.

The evolution of new variants is common among viruses. For instance, flu vaccines are updated each year to adapt to the new strains that are circulating. COVID-19 vaccine manufacturers have since developed new versions that provide greater protection against several variants.

The vaccine effectiveness figures come from studies of different vaccines

The effectiveness figures presented in the video come from several different COVID-19 vaccines, including those produced by Pfizer, Moderna, Johnson & Johnson, AstraZeneca, Novavax, and Cadila Healthcare, an Indian company that produced a DNA vaccine.

We know that different COVID-19 vaccines confer differing levels of protection against the disease. Many of these vaccines take different approaches to induce immunity.

Researchers reported that the Johnson & Johnson vaccine is less effective than the mRNA COVID-19 vaccines at preventing symptomatic infection. Consequently, the vaccine is used less frequently compared to the other two, accounting for less than 3% of doses given in the U.S. and EU.

And as we explained in the previous section, studies also found differences in vaccine effectiveness depending on the type of variant circulating at the time.

The way that the vaccine effectiveness figures were rapidly shown in the video made it difficult for viewers to understand that some of the figures differed because, rather than compare like with like, the video presented figures for completely different vaccines.

The headlines are from different periods after vaccination

The immune response to all types of vaccines decreases over time. Some vaccines are given in two or three doses, separated by weeks or months, to help maintain the protection conferred by vaccination.

This is also true of the COVID-19 vaccines. For example, the primary vaccination series for mRNA COVID-19 vaccines consists of two doses, but researchers later discovered that protection against infection began to diminish over the following months. This is why booster vaccination campaigns were later launched.

The Instagram video selected effectiveness figures taken from studies at different time points. Many of the figures quoted are from clinical trials or studies that examined the weeks after vaccination. Examples of such headlines are “Updated Pfizer Data Shows Vaccine Is 95% Effective” and “Pfizer/BioNTech says its Covid-19 vaccine is 100% effective and well tolerated in adolescents,” which both referred to announcements from Phase 3 clinical trials on the vaccine’s efficacy in preventing cases of COVID-19.

However, one of the headlines that appeared later in the video read, “Pfizer’s Covid vaccine efficacy against infection plunges to just 20% after six months – but protection against severe illness barely dips, study concludes.” This was taken from an article published by the Mail Online in October 2021 that reported on a study of real-world data in Qatar^[6].

As the headline states, this figure only relates to protection against SARS-CoV-2 infection, whereas the protection against severe disease remains high.

As previously explained, it is normal for vaccine effectiveness to wane over time. Therefore, a comparison between vaccine effectiveness figures shortly after vaccination and six months post-vaccination would need to make this time gap clear to prevent misinterpretation of the results, which the video didn’t do.

Summary

Our review illustrates the importance of placing studies of vaccine effectiveness in the context of what they examined. Differences in vaccine effectiveness can be due to newer, more infectious variants, different vaccines, or the immune response waning over time. In addition, effectiveness can refer to different outcomes, like death, severe disease, symptomatic disease, or transmission.

The video’s simple, linear narrative about diminishing vaccine effectiveness belies the fact that the figures were presented out of chronological order, without regard for differences in the type of vaccine, the type of variant, and the amount of time that had elapsed after vaccination. This created the misleading impression that estimates of vaccine effectiveness fell from 100% to 20%. Some users also seized on the video to imply that authorities lied about COVID-19 vaccine effectiveness, when in fact, the changes in vaccine effectiveness are due to the previously mentioned factors.

Many studies have shown that the COVID-19 vaccines have become less effective over time, particularly against infection, due to waning immunity and newer, more infectious variants^[7,8]. However, they still retain a high level of effectiveness against severe disease and death.

I’m not sure it is ever accurate to look just at headlines. First of all the clinical trials referred to here would measure efficacy not effectiveness. Efficacy is how well a vaccine or drug works in an ideal setting (e.g. a randomized controlled clinical trial) while effectiveness measures how well it works in the real world. Efficacy is ALWAYS going to be higher than effectiveness because in the real world we give the vaccine to people where we know it likely won’t work (e.g. cancer patients), but we don’t do this in a clinical trial. So then, when you are counting who it worked for and who it didn’t, you include all those people that you expect it wouldn’t work in and the estimate is lower. The headlines are comparing efficacy and effectiveness estimates which is not accurate and shouldn’t be done.

Additionally, the original vaccine and the original clinical trials were measuring how well the vaccine worked against an early strain/variant of the virus, which I will call the vaccine variant. We know there have since been multiple variants that are different than what is in the vaccine as we would expect the effectiveness to decrease because of this. The effectiveness has not changed against the vaccine variant. Unfortunately, the vaccine variant is not the main variant currently circulating. The “match” between what is in circulation and what is in the vaccine is not 100% so the vaccine does not work as well against infection with a new variant. It is still highly effective against severe disease as measured by hospitalization and death, even with a non-vaccine variant.

REFERENCES

• 1 – Macchia et al. (2021) Evaluation of a COVID-19 Vaccine Campaign and SARS-CoV-2 Infection and Mortality Among Adults Aged 60 Years And Older in a Middle-Income Country. JAMA Network Open
• 2 – Haas et al. (2021) Impact and effectiveness of mRNA BNT162b2 vaccine against SARS-CoV-2 infections and COVID-19 cases, hospitalisations, and deaths following a nationwide vaccination campaign in Israel: an observational study using national surveillance data. The Lancet
• 3 – Tenforde et al (2021) Effectiveness of Pfizer-BioNTech and Moderna Vaccines Against COVID-19 Among Hospitalized Adults Aged ≥65 Years — United States, January–March 2021. Morbidity and Mortality Weekly Report
• 4 – Polack et al. (2020) Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. New England Journal of Medicine.
• 5 – Puranik et al. (2022) Comparative effectiveness of mRNA-1273 and BNT162b2 against symptomatic SARS-CoV-2 infection. Med
• 6 – Chemaitelly et al. (2021) Waning of BNT162b2 Vaccine Protection against SARS-CoV-2 Infection in Qatar. New England Journal of Medicine
• 7 – Lopez Bernal et al. (2021) Effectiveness of Covid-19 Vaccines against the B.1.617.2 (Delta) Variant. New England Journal of Medicine
• 8 – Ferdinands et al. (2022) Waning of vaccine effectiveness against moderate and severe covid-19 among adults in the US from the VISION network: test negative, case-control study. British Medical Journal