Understanding causality in adverse events after vaccination

Introduction

Whether you’re watching the news or scrolling through your social media feeds, you’re certain to come across content about COVID-19 vaccines. This is unsurprising—vaccine safety, and in particular adverse events occurring after COVID-19 vaccination, have been the subject of scrutiny from regulatory agencies, scientists, and the public.

While it’s critical to be vigilant and informed about potential safety concerns with the vaccines, communicating this subject is fraught with the potential of fuelling vaccine misinformation. After all, the weaponizing of vaccine-related communications to amplify vaccine misinformation, employed by multiple parties including the anti-vaccine movement, has been documented by researchers^[1].

Discussing risks in a manner that isn’t misleading has proved to be a challenge, even for public health experts, let alone the media. While some media outlets are careful not to attribute post-vaccination adverse events to the vaccine until scientific evidence supports it, other outlets persistently place such events in the spotlight, sometimes suggesting that these events are caused by the vaccines.

The idea that an event was caused by an earlier event isn’t an unexpected train of thought—after all, one key indicator of causality is temporality. This means that causes precede effects in time. It’s even a key concept that underpins public health authorities’ efforts to collect information on adverse events occurring after vaccinations. 

Providing everyone with an avenue for reporting such events helps authorities detect potential safety concerns, for example when reports of a particular adverse event, such as anaphylaxis or blood clots, are unusually high after receipt of a certain vaccine.

Patterns of unusually high numbers of adverse events alert authorities to investigate these cases in detail to determine if the vaccine was indeed the cause. References to vaccine adverse events databases can be commonly found on the Internet. Some popular ones are the Vaccine Adverse Events Reporting System (VAERS) in the U.S., EudraVigilance, which covers most of Europe, and the Yellow Card Scheme in the U.K.

Before we go any further, it’s important to define the term “adverse event”, which is commonly misunderstood. Some use the terms “adverse events”, “adverse reactions”, and “side effects” interchangeably, but these terms have significantly different meanings in the context of public health.

The term “adverse event” describes all health problems that occur after vaccination, regardless of whether it’s caused by the vaccine. For example, a broken leg after a car accident that happened to someone who just received a vaccine would be considered an adverse event. In this case, the cause of the broken leg is obviously not the vaccine, but the car accident.

The terms “adverse reaction” and “side effect” are only used to refer to adverse events known to be caused by the vaccine. For example, certain types of egg proteins in the flu vaccine can trigger an allergic reaction in people with egg allergy. These allergic reactions are considered a side effect, since they are causally linked to the egg protein in the vaccine.

Returning to our original question: How do we know if an adverse event was caused by a vaccine? Given that health problems, like heart inflammation and blood clotting disorders, existed long before COVID-19 vaccines were developed, how can researchers distinguish between a true side effect and an adverse event that occurred due to coincidence and/or bad luck? This review explains how.

Contents

• Not all evidence is created equal: why adverse event reports aren’t enough
• Don’t put the cart before the horse: Establishing association first
• Are there no adverse events occurring after vaccination that were shown to be causally related to the vaccine?
• Spike in reports of vaccine adverse events after authorization of COVID-19 vaccines: the result of broader adverse event reporting requirements due to increased scrutiny
• Conclusion

Not all evidence is created equal: why adverse event reports aren’t enough

As stated previously, temporality is essential to causality. Cause must precede effect. The problem arises when we rely solely on temporality to establish causality. Indeed, if we used temporality as the only determinant of causality, we might well conclude that it is the crowing of the rooster that causes the sun to rise, since roosters generally crow before sunrise. (Here you can see humorous examples illustrating this fallacy, also known in Latin as post hoc ergo propter hoc, meaning “after this, therefore because of this”.)

XKCD explains the fallacy in three panels.

Many would quickly dismiss the idea that roosters are the cause of sunrise, but that statement illustrates the same line of reasoning that leads some people to assert that the COVID-19 vaccines are dangerous on the basis of VAERS reports, as Health Feedback covered in previous reviews here, here, and here.

The misuse of VAERS isn’t a new trend; parties opposed to vaccines have been using this tactic for a long time, as noted by Vaxopedia. But the COVID-19 pandemic has turned databases like VAERS into “a breeding ground for misinformation”, as described in an article published by the Poynter Institute. Reports in these databases, as well as unverified anecdotal claims of adverse events after vaccination circulating on social media, have formed the basis of much vaccine misinformation. Disinformation campaigns, originating from China and Russia, also leveraged on such reports to build narratives aimed at casting doubt over vaccines developed in the West, as documented by EUvsDisinfo.

So if temporality alone doesn’t work, then how can we approach the question of causality?

Scientists have asked themselves the same question. In 1965, Austin Bradford Hill, a British epidemiologist, dealt with the subject in his address to the Royal Society of Medicine, titled “The Environment and Disease: Association or Causation?”^[2].

In that article, he laid out a framework consisting of nine aspects, which he called “viewpoints”, to consider when attempting to infer causality. These nine are:

• Strength of association
• Consistency
• Specificity
• Temporality
• Biological gradient
• Plausibility
• Coherence
• Experiment
• Analogy

Bradford Hill thought that one couldn’t have “hard-and-fast rules of evidence that must be obeyed before we can accept cause and effect”. But he believed that there were various angles from which one could approach the question of causality, through which one could answer with varying degrees of certainty:

“None of my nine viewpoints can bring indisputable evidence for or against the cause-and-effect hypothesis and none can be required as a sine qua non. What they can do, with greater or less strength, is to help us to make up our minds on the fundamental question – is there any other way of explaining the set of facts before us, is there any other answer equally, or more, likely than cause and effect?”

His viewpoints are now generally accepted by scientists as useful guidelines to consider when inferring causality. To show how this relates to vaccines, we can turn to the World Health Organization (WHO) guide that applies the criteria to the analysis of adverse events following vaccination.

“Several criteria are relevant to establishing causality but only the first criterion is absolutely essential:

• Temporal relationship: The vaccine exposure must precede the occurrence of the event.
• Strength of association: The association should meet statistical significance to demonstrate that it was not simply a chance occurrence.
• Dose−response relationship: Evidence that increasing exposure increases the risk of the event supports the suggestion of a causal relationship. However, one should keep in mind that, in the case of vaccines, dose and frequency tend to be fixed.
• Consistency of evidence: Similar or the same results generated by studies using different methods in different settings support a causal relationship.
• Specificity: The vaccine is the only cause of the event that can be shown.
• Biological plausibility and coherence: The association between the vaccine and the adverse event should be plausible and should be consistent with current knowledge of the biology of the vaccine and the adverse event.”

[Back to top]

Don’t put the cart before the horse: Establishing association first

Most aspects of the criteria assume that there is already an association between the exposure and the effect. For this reason, it is important to begin by determining, through statistical means, whether a vaccine is indeed associated with an effect, before attempting to infer causality. 

We say that there is a statistical association between two variables, for instance, between the number of vaccinations and the number of an adverse event, when both appear to be related to each other in a specific pattern that is unlikely to be due to chance (see Figure 1 below).

It’s important to keep in mind that statistical associations aren’t always causal in nature. And observing that there is an increasing number of deaths with a growing number of vaccinated people alone isn’t sufficient to state that there is a causal association between the two.

Why is this so? First, all else being equal, the larger the size of a population, the more number of deaths we will observe. Second, even unvaccinated people die for a variety of reasons. As medicinal chemist Derek Lowe illustrated in this commentary in Science Translational Medicine:

“We’re talking about treating very, very large populations, which means that you’re going to see the usual run of mortality and morbidity that you see across large samples. Specifically, if you take 10 million people and just wave your hand back and forth over their upper arms, in the next two months you would expect to see about 4,000 heart attacks. About 4,000 strokes. Over 9,000 new diagnoses of cancer. And about 14,000 of that ten million will die, out of usual all-causes mortality. No one would notice. That’s how many people die and get sick anyway.

But if you took those ten million people and gave them a new vaccine instead, there’s a real danger that those heart attacks, cancer diagnoses, and deaths will be attributed to the vaccine. I mean, if you reach a large enough population, you are literally going to have cases where someone gets the vaccine and drops dead the next day (just as they would have if they *didn’t* get the vaccine). It could prove difficult to convince that person’s friends and relatives of that lack of connection, though. Post hoc ergo propter hoc is one of the most powerful fallacies of human logic, and we’re not going to get rid of it any time soon.”

Also explained in a STAT News article:

“Every single day, people die unexpectedly. They have strokes and heart attacks and seizures. On an average day, 110 people in this country may develop Bell’s palsy, a temporary facial paralysis, and another 274 will develop Guillain-Barré syndrome, a form of paralysis that usually resolves over time. The trigger for these medical events often isn’t known. But when they happen shortly after someone gets a vaccine — especially a new one — well, conclusions will be drawn. […]

Heart attacks occur most commonly in the morning, yet we don’t blame breakfast for causing them. A heart attack on the morning after a Covid-19 vaccine, though? That might be another matter.”

And in the WHO’s user manual “Causality assessment of an adverse event following immunization (AEFI)”:

“It is important to consider the background rates for the occurrence of an event of interest and then after a population has received vaccine, determine if the observed rate of that event is in excess of the background rates.”

In other words, there is already a baseline or background rate of such medical events occurring in people who didn’t receive the vaccine. 

This is why common claims that vaccines are unsafe (see these examples here and here reviewed by Health Feedback) based solely on reports of deaths post-vaccination are misleading.

One example comes from Fox News host Tucker Carlson, who claimed in early May 2021 that more than 3,000 people died after receiving a COVID-19 vaccine, implying that the COVID-19 vaccines are unsafe. That figure, numbering in the thousands, can come across as worrisome—until we consider the fact that at that time, more than 157 million people in the U.S. had received at least one dose of a COVID-19 vaccine. Placed in that context, that figure alone no longer suggests that the vaccines are unsafe or that they’re the cause of those deaths.

In order to suggest a causal link between a vaccine and an adverse event, it isn’t enough to simply point to the number of events that have occurred. What scientists would need to do is:

1. Determine the proportion of such events occurring among all the people who received the vaccine. Using the proportion, as opposed to the absolute number, is critical, since the larger the population size is, the larger the number of incidental illnesses we expect. As explained in our earlier example from Tucker Carlson, 3,000 deaths sound like a lot, until we account for the fact that these occurred among more than 157 million vaccinated people.
2. Determine if this proportion exceeds that of the baseline or background rate in unvaccinated people.

Indeed, researchers and public health authorities routinely conduct such assessments when investigating serious adverse events like death. In the case of deaths occurring during clinical trials of the Pfizer-BioNTech and Moderna COVID-19 vaccines, these assessments didn’t show that the proportion of deaths in the vaccinated group exceeded that of the unvaccinated group.

The FDA Briefing Document for the Pfizer-BioNTech COVID-19 vaccine stated:

“A total of six (2 vaccine, 4 placebo) of 43,448 enrolled participants (0.01%) died during the reporting period from April 29, 2020 (first participant, first visit) to November 14, 2020 (cutoff date) […] All deaths represent events that occur in the general population of the age groups where they occurred, at a similar rate.” (emphasis added)

Likewise for the Moderna vaccine, the FDA Briefing Document stated:

“As of December 3, 2020, 13 deaths were reported (6 vaccine, 7 placebo) […] These deaths represent events and rates that occur in the general population of individuals in these age groups.” (emphasis added)

Deaths among nursing home residents, who were among the first people to receive the COVID-19 vaccine due to their higher vulnerability to the disease, made headlines. Concern over these deaths prompted the Global Advisory Committee on Vaccine Safety (GACVS) COVID-19 Vaccine Safety subcommittee to review these reports. The GACVS is “a scientific and clinical advisory body to WHO”, which “aims to provide a reliable and independent scientific assessment of vaccine safety issues”. The GACVS is not itself part of the WHO.

On 22 January 2021, the GACVS subcommittee released the following statement after completing the review:

“Based on a careful scientific review of the information made available, the subcommittee came to the following conclusions:
The current reports do not suggest any unexpected or untoward increase in fatalities in frail, elderly individuals or any unusual characteristics of adverse events following administration of BNT162b2. Reports are in line with the expected, all-cause mortality rates and causes of death in the sub-population of frail, elderly individuals, and the available information does not confirm a contributory role for the vaccine in the reported fatal events.” (emphasis added)

The bottom line being that these reports of deaths are occurring at rates that we would expect in an unvaccinated population. If the claim that the COVID-19 vaccines are causing widespread death was true, we would have seen the rate of death in vaccinated populations exceed that of unvaccinated populations. Since this isn’t the case, these reports don’t provide a basis to suggest that vaccines caused these deaths.

[Back to top]

Are there no adverse events occurring after vaccination that were shown to be causally related to the vaccine?

Not at all. We have counter-examples demonstrating that the above approach can and is used to identify adverse events that are now considered to be caused or potentially caused by a vaccine.

One prominent example is a rare blood clotting disorder after receipt of viral vector COVID-19 vaccines, specifically the AstraZeneca-Oxford and Johnson & Johnson vaccines. Multiple such cases were recorded in both the U.S. and Europe. Scientists coined the name thrombosis with thrombocytopenia syndrome (TTS), to describe the condition. Thrombosis refers to blood clotting, and thrombocytopenia means that there is an abnormally low number of platelets in the blood. Platelets (thrombocytes) are a type of blood cell that helps blood to clot.

Contrary to post-vaccine adverse event reports of deaths, reviews by health authorities to date concluded that a causal relationship between the vaccines and TTS is plausible, although not yet proven.

For example, the European Medicines Agency stated that there is a “possible link to very rare cases of unusual blood clots with low blood platelets” with the AstraZeneca-Oxford vaccine in this statement released on 7 April 2021.

And an analysis of the rate of these incidents between unvaccinated and vaccinated people shows why. An article by Raina McIntyre, a professor at the University of New South Wales who specializes in emerging infectious diseases^[3], discussed such events following receipt of the AstraZeneca-Oxford vaccine, specifically incidents of cerebral venous sinus thrombosis (CVST), which is one manner in which TTS can manifest:

“The rate of CVST reported from Germany was 31 from 2.7 million vaccinations, which is a rate of 11.5 per million, compared to a reported general community incidence of 3 to 5 per million. CVST appears to be 2.2 to 3.8 times higher than the reported community incidence.”

On 19 May 2021, the GACVS committee released a statement on the Johnson & Johnson vaccine along the same lines:

“Current evidence suggests a plausible causal association between the J&J COVID-19 vaccine and TTS.”

And the U.S. Centers for Disease Control and Prevention (CDC) also stated:

“[R]ecent reports indicate a plausible causal relationship between the J&J/Janssen COVID-19 Vaccine and a rare and serious adverse event—blood clots with low platelets—which has caused deaths.”

This statement is supported by investigations by the CDC. One such supporting piece of evidence can be seen in this 14 April 2021 report from the CDC COVID-19 Vaccine Task Force:

“Observed cases following Janssen COVID-19 vaccines appear to exceed expected based on background rates of CVST among women aged 20–50 years (3-fold or greater)”

In other words, we can observe an association between a higher rate of this rare blood clotting disorder in vaccinated people, relative to unvaccinated people. This clues investigators into a possible causal link.

McIntyre’s article, mentioned earlier, placed these events within the framework of the Bradford Hill criteria^[3]. She proposed that what we know of TTS so far meets multiple aspects of the criteria, such as strength of association, consistency, specificity, temporality, and biological plausibility. This led her to conclude that: “The application of the modified Bradford-Hill criteria to [vaccine-induced thrombotic thrombocytopenia] following CHADOX1 [NCOV-19] vaccine strongly supports a causal relationship.”

While the amount of evidence supporting the inference of a causal relationship between TTS and viral vector COVID-19 vaccines grows, there is still more work that scientists need to undertake. For example, we still don’t know exactly how the vaccines cause TTS. Apart from providing further evidence to link the vaccines and TTS, this knowledge would help us to better prevent or treat it in the future, thereby minimizing the risks of the vaccines. Indeed, this is exemplified by a statement from Anne Schuchat, the principal deputy director of the CDC, who explained that the pause in the Johnson & Johnson vaccine was enacted in part to “prepare the health care system to recognize and treat patients appropriately”.

[Back to top]

Spike in reports of vaccine adverse events after authorization of COVID-19 vaccines: the result of broader adverse event reporting requirements due to increased scrutiny

Another trend associated with misinformation using VAERS are social media posts highlighting a spike in vaccine adverse event reports in 2021 and 2022, and comparing this to the relatively lower number of reports in pre-pandemic years. The types of adverse events highlighted in these posts include miscarriages, stillbirths, and deaths.

The implication of this comparison is commonly left unsaid, like in this tweet by David Wolfe, who has spread false claims about vaccines and cancer cures, and this Instagram post by podcaster Ginny Robinson.

But the message is clear, based on the comments left by users on these posts: COVID-19 vaccines are more dangerous than past approved vaccines.

Others are less subtle, such as this tweet by Turning Point USA activist Adam Dommeyer, and this Epoch Times interview with internal medicine physician Meryl Nass.

However, all of these posts leave out any mention of one critical difference between COVID-19 vaccines and past vaccines: the broader adverse event reporting criteria for COVID-19 vaccines. As the U.S. CDC states on its website, the FDA stipulated reporting of many serious adverse events after COVID-19 vaccination, regardless of whether the healthcare provider thinks the adverse event was caused by the vaccine. This requirement is the same for both authorized and approved COVID-19 vaccines (see Figure 2 below).

Figure 3 – Types of adverse events that are mandatory to report following COVID-19 vaccination. Source: VAERS.

In contrast, approved non-COVID-19 vaccines aren’t subjected to the same requirement and there are fewer adverse events that are mandatory to report (see the VAERS Table of Reportable Events Following Vaccination). For example, deaths only need to be reported if they are associated with known side effects of vaccination.

Figure 4 – Adverse events that are mandatory to report with non-COVID-19 vaccines. The VAERS Table of Reportable Events Following Vaccination can be found here. Source: VAERS.

This difference in adverse event reporting requirement translates to a gap in the number of reports submitted for COVID-19 vaccines related to past vaccines. And if we consider that to date, more than 266 million people in the U.S. have received at least one dose of COVID-19 vaccines (see Figure 5 below), the number of adverse event reports for deaths and miscarriages may not necessarily be beyond the expected number in an unvaccinated population.

Indeed, large-scale studies in pregnant women and vaccinated people in general haven’t shown an increased risk of negative pregnancy outcomes or death associated with COVID-19 vaccines, as previous Health Feedback reviews reported.

Overall, the difference in reporting rate for COVID-19 vaccines compared to non-COVID-19 vaccines—as a result of the broader reporting criteria imposed on COVID-19 vaccines—means that directly comparing the number of reports between the two doesn’t provide us with meaningful information. Nor does it mean that COVID-19 vaccination is causing more serious side effects than previous vaccines.

[Back to top]

Conclusion

Knowing about adverse events that occur after vaccination is critical, as they help public health experts recognize that there is a potential problem with a vaccine and trigger investigations to determine what the true cause of these events is. Regardless of whether the investigations do link the adverse event to the vaccine, they can yield information that helps to improve vaccine safety and safeguard the health of the public.

Causality cannot be inferred only from temporality. For this reason, it is important to consider multiple aspects when inferring causality. There is no doubt that temporality is essential for inferring a causal link between a vaccine and an adverse event. However, when used as the sole criteria for causality, it gives rise to logical missteps, thereby forming the basis for common forms of vaccine misinformation. 

Ensuring that one evaluates data by taking into account multiple pieces of contextual information, such as the baseline rate of a certain health problem and the size of the population under consideration, can help one to understand if there are indeed grounds for concern, or if claims about unsafe vaccines are unfounded.

Our understanding of the world is constantly changing as we gather new information, and vaccines are no exception. It is possible that we may yet uncover more information about how vaccines interact with our bodies, and that there may be unwanted effects discovered later. But that isn’t a reason to disregard the considerable weight of evidence we do have at a given time, demonstrating that the benefits of vaccines outweigh their risks, nor is it a reason not to act in accordance with this knowledge. As Bradford Hill wrote:

“All scientific work is incomplete – whether it be observational or experimental. All scientific work is liable to be upset or modified by advancing knowledge. That does not confer upon us a freedom to ignore the knowledge we already have, or to postpone the action that it appears to demand at a given time.”

[Back to top]

UPDATE (8 November 2022):

This article was updated to include a new section “Spikes in reports of vaccine adverse events after authorization of COVID-19 vaccines”, in order to address viral claims regarding the difference in the number of VAERS reports for COVID-19 vaccines and past vaccines.

REFERENCES

• 1 – Broniatowski et al. (2018) Weaponized Health Communication: Twitter Bots and Russian Trolls Amplify the Vaccine Debate. American Journal of Public Health.
• 2 – Bradford Hill (1965) The Environment and Disease: Association or Causation? Proceedings of the Royal Society of Medicine.
• 3 – MacIntyre C. (2021) Using the Bradford-Hill criteria to assess causality in the association between CHADOX1 NCOV-19 vaccine and thrombotic immune thrombocytopenia. Global Biosecurity.