Effectiveness of a Covid mRNA Vaccine Against Two Early Variants: A Reanalysis of Published Data.

introductionTwo strong biases, immortal time and healthy vaccinee, have not received the attention they deserve in observational studies of Covid vaccines. The former arises from excluding early events in the vaccinated, and the latter from residual confounding by health-related attributes. Both biases lead to overestimation of vaccine effectiveness.

methodsPublished data from a test-negative case-control study of an mRNA vaccine were reanalyzed. The original study estimated the effect on infection by two early variants, Alpha and Beta, and the effect on a severe outcome (severe, critical, or fatal infection) by either. Immortal time bias was removed by considering all events and estimating built-up immunity by 27 days after the first dose and by 13 days after the second. Healthy vaccinee bias was reduced by applying a bias factor correction derived from the pseudo-effect on non-Covid death (negative controls).

resultsAll estimates were strikingly different from the original results. Effectiveness against infection by either variant might have been negative in the first two weeks after the first dose and has reached only 41% and 15% against Alpha and Beta, respectively, by the end of the fourth week. At the time of full immunity, it was around 60% against Alpha and below 50% against Beta, not over 95% as originally computed for each variant. Accounting for immortal time bias alone, effectiveness against a severe outcome changed from 100% to about 50%. Then, a rudimentary correction for healthy vaccinee bias not only eliminated meaningful benefit but also raised the possibility of negative effectiveness in the first month or so.

conclusionImmortal time bias and healthy vaccinee bias have severely distorted estimates of Covid vaccine effectiveness. This analysis should be replicated in other observational studies to corroborate or refute the findings.