The test-negative design for estimating influenza vaccine effectiveness

Objective

The test-negative design has emerged in recent years as the preferred method for estimating influenza vaccine effectiveness (VE) in observational studies. However, the methodologic basis of this design has not been formally developed.

Methods

In this paper we develop the rationale and underlying assumptions of the test-negative study. Under the test-negative design for influenza VE, study subjects are all persons who seek care for an acute respiratory illness (ARI). All subjects are tested for influenza infection. Influenza VE is estimated from the ratio of the odds of vaccination among subjects testing positive for influenza to the odds of vaccination among subjects testing negative.

Results

With the assumptions that (a) the distribution of non-influenza causes of ARI does not vary by influenza vaccination status, and (b) VE does not vary by health care-seeking behavior, the VE estimate from the sample can generalized to the full source population that gave rise to the study sample. Based on our derivation of this design, we show that test-negative studies of influenza VE can produce biased VE estimates if they include persons seeking care for ARI when influenza is not circulating or do not adjust for calendar time.

Conclusions

The test-negative design is less susceptible to bias due to misclassification of infection and to confounding by health care-seeking behavior, relative to traditional case-control or cohort studies. The cost of the test-negative design is the additional, difficult-to-test assumptions that incidence of non-influenza respiratory infections is similar between vaccinated and unvaccinated groups within any stratum of care-seeking behavior, and that influenza VE does not vary across care-seeking strata.     

The impact of selection bias on vaccine effectiveness estimates from test-negative studies

IntroductionEstimates of vaccine effectiveness (VE) from test-negative studies may be subject to selection bias. In the context of influenza VE, we used simulations to identify situations in which meaningful selection bias can occur. We also analyzed observational study data for evidence of selection bias.MethodsFor the simulation study, we defined a hypothetical population whose members are at risk for acute respiratory illness (ARI) due to influenza and other pathogens. An unmeasured “healthcare seeking proclivity” affects both probability of vaccination and probability of seeking care for an ARI. We varied the direction and magnitude of these effects and identified situations where meaningful bias occurred. For the observational study, we reanalyzed data from the United States Influenza VE Network, an ongoing test-negative study. We compared “bias-naïve” VE estimates to bias-adjusted estimates, which used data from the source populations to correct for sampling bias.ResultsIn the simulation study, an unmeasured care-seeking proclivity could create selection bias if persons with influenza ARI were more (or less) likely to seek care than persons with non-influenza ARI. However, selection bias was only meaningful when rates of care seeking between influenza ARI and non-influenza ARI were very different. In the observational study, the bias-naïve VE estimate of 55% (95% CI, 47-–62%) was trivially different from the bias-adjusted VE estimate of 57% (95% CI, 49-–63%).ConclusionsIn combination, these studies suggest that while selection bias is possible in test-negative VE studies, this bias in unlikely to be meaningful under conditions likely to be encountered in practice. Researchers and public health officials can continue to rely on VE estimates from test-negative studies.     

On the bias of estimates of influenza vaccine effectiveness from test–negative studies

Estimates of the effectiveness of influenza vaccines are commonly obtained from a test-negative design (TND) study, where cases and controls are patients seeking care for an acute respiratory illness who test positive and negative, respectively, for influenza infection. Vaccine effectiveness (VE) estimates from TND studies are usually interpreted as vaccine effectiveness against medically-attended influenza (MAI). However, it is also important to estimate VE against any influenza illness (symptomatic influenza (SI)) as individuals with SI are still a public health burden even if they do not seek medical care. We present a numerical method to evaluate the bias of TND-based estimates of influenza VE with respect to MAI and SI. We consider two sources of bias: (a) confounding bias due to a (possibly unobserved) covariate that is associated with both vaccination and the probability of the outcome of interest and (b) bias resulting from the effect of vaccination on the probability of seeking care. Our results indicate that (a) VE estimates may suffer from substantial confounding bias when a confounder has a different effect on the probabilities of influenza and non-influenza ARI, and (b) when vaccination reduces the probability of seeking care against influenza ARI, then estimates of VE against MAI may be unbiased while estimates of VE against SI may be have a substantial positive bias.     

Basic principles of test-negative design in evaluating influenza vaccine effectiveness

Based on the unique characteristics of influenza, the concept of “monitoring” influenza vaccine effectiveness (VE) across the seasons using the same observational study design has been developed. In recent years, there has been a growing number of influenza VE reports using the test-negative design, which can minimize both misclassification of diseases and confounding by health care-seeking behavior. Although the test-negative designs offer considerable advantages, there are some concerns that widespread use of the test-negative design without knowledge of the basic principles of epidemiology could produce invalid findings. In this article, we briefly review the basic concepts of the test-negative design with respect to classic study design such as cohort studies or case-control studies. We also mention selection bias, which may be of concern in some countries where rapid diagnostic testing is frequently used in routine clinical practices, as in Japan.     

Comparison of test-negative and syndrome-negative controls in SARS-CoV-2 vaccine effectiveness evaluations for preventing COVID-19 hospitalizations in the United States

BackgroundTest-negative design (TND) studies have produced validated estimates of vaccine effectiveness (VE) for influenza vaccine studies. However, syndrome-negative controls have been proposed for differentiating bias and true estimates in VE evaluations for COVID-19. To understand the use of alternative control groups, we compared characteristics and VE estimates of syndrome-negative and test-negative VE controls.MethodsAdults hospitalized at 21 medical centers in 18 states March 11–August 31, 2021 were eligible for analysis. Case patients had symptomatic acute respiratory infection (ARI) and tested positive for SARS-CoV-2. Control groups were test-negative patients with ARI but negative SARS-CoV-2 testing, and syndrome-negative controls were without ARI and negative SARS-CoV-2 testing. Chi square and Wilcoxon rank sum tests were used to detect differences in baseline characteristics. VE against COVID-19 hospitalization was calculated using logistic regression comparing adjusted odds of prior mRNA vaccination between cases hospitalized with COVID-19 and each control group.Results5811 adults (2726 cases, 1696 test-negative controls, and 1389 syndrome-negative controls) were included. Control groups differed across characteristics including age, race/ethnicity, employment, previous hospitalizations, medical conditions, and immunosuppression. However, control-group-specific VE estimates were very similar. Among immunocompetent patients aged 18–64 years, VE was 93 % (95 % CI: 90–94) using syndrome-negative controls and 91 % (95 % CI: 88–93) using test-negative controls.ConclusionsDespite demographic and clinical differences between control groups, the use of either control group produced similar VE estimates across age groups and immunosuppression status. These findings support the use of test-negative controls and increase confidence in COVID-19 VE estimates produced by test-negative design studies.     

A comparison of the test-negative and traditional case-control study designs with respect to the bias of estimates of rotavirus vaccine effectiveness

Estimation of the effectiveness of rotavirus vaccines via the test-negative control study design has gained popularity over the past few years. In this study design, children with severe diarrhea who test positive for rotavirus infection are considered as cases, while children who test negative serve as controls. We use a simple probability model to evaluate and compare the test-negative control and the traditional case-control designs with respect to the bias of resulting estimates of rotavirus vaccine effectiveness (VE). Comparisons are performed under two scenarios, corresponding to studies performed in high-income and low-income countries. We consider two potential sources of bias: (a) misclassification bias resulting from imperfect sensitivity and specificity of the test used to diagnose rotavirus infection, and (b) selection bias associated with possible effect of rotavirus vaccination on the probability of contracting severe non-rotavirus diarrhea.Our results suggest that both sources of bias may produce VE estimates with substantial bias. Particularly, lack of perfect specificity is associated with severe negative bias. For example, if the specificity of the diagnostic test is 90% then VE estimates from both types of case-control studies may under-estimate the true VE by more than 20%. If the vaccine protects children against non-rotavirus diarrhea then VE estimates from test-negative control studies may be close to zero even though the true VE is 50%. However, the sensitivity and specificity of the enzyme immunoassay test currently used to diagnose rotavirus infections are both over 99%, and there is no solid evidence that the existing rotavirus vaccines affect the rates of non-rotavirus diarrhea. We therefore conclude that the test-negative control study design is a convenient and reliable alternative for estimation of rotavirus VE.     

Bias of influenza vaccine effectiveness estimates from test-negative studies conducted during an influenza pandemic

Test-negative (TN) studies have become the most widely used study design for the estimation of influenza vaccine effectiveness (VE) and are easily incorporated into existing influenza surveillance networks. We seek to determine the bias of TN-based VE estimates during a pandemic using a dynamic probability model. The model is used to evaluate and compare the bias of VE estimates under various sources of bias when vaccination occurs after the beginning of an outbreak, such as during a pandemic. The model includes two covariates (health status and health awareness), which may affect the probabilities of vaccination, developing influenza and non-influenza acute respiratory illness (ARI), and seeking medical care. Specifically, we evaluate the bias of VE estimates when (1) vaccination affects the probability of developing a non-influenza ARI; (2) vaccination affects the probability of seeking medical care; (3) a covariate (e.g. health status) is related to both the probabilities of vaccination and developing an ARI; and (4) a covariate (e.g. health awareness) is related to both the probabilities of vaccination and of seeking medical care. We considered two outcomes against which the vaccine is supposed to protect: symptomatic influenza and medically-attended influenza.When vaccination begins during an outbreak, we found that the effect of delayed onset of vaccination is unpredictable. VE estimates from TN studies were biased regardless of the source of bias present. However, if the core assumption of the TN design is satisfied, that is, if vaccination does not affect the probability of non-influenza ARI, then TN-based VE estimates against medically-attended influenza will only suffer from small (<0.05) to moderate bias (≥0.05 and <0.10). These results suggest that if sources of bias listed above are ruled out, TN studies are a valid study design for the estimation of VE during a pandemic.     

Effectiveness of trivalent influenza vaccines against hospitalizations due to laboratory-confirmed influenza a in the elderly: Comparison of test-negative design with register-based designs

IntroductionMeasuring influenza vaccine effectiveness (IVE) seasonally is important and has been conducted utilizing several observational study designs. The active test-negative design has been most widely used and the validity of passive register-based studies has been debated. We aimed to explore the potential differences, advantages, and weaknesses of different study designs in estimating influenza vaccine effectiveness.MethodsWe compared three study designs in estimating IVE against hospitalization in the elderly aged 65 years or more over three influenza seasons 2015/16, 2016/17 and 2017/18. Designs compared were active test-negative design (TND), register-based cohort design and register-based case-control design with different selection criteria for cases and controls.ResultsAdjusted IVE estimates for the three consecutive seasons 2015–18 in active test-negative design were 82% (95% confidence interval 26, 96), 21% (-179, 77), 15% (-113, 66). For case-control design, estimates from different analyses ranged in 2015/16 from 47% (-16, 76) to 52% (-48, 84), in 2016/17 from 10% (-42, 43) to 29% (-20, 58), and in 2017/18 from ?27% (-91, 15) to 1% (-40, 30). In the cohort design, the adjusted IVE estimates were 48% (-9, 75), 29% (1, 49), 13% (-21, 37) for the three seasons.ConclusionsThe register-based cohort design produced results more concordant with the active test-negative design than the case-control design. Furthermore, the register-based cohort design yielded most precise estimates with narrower confidence intervals. In Finland with the availability of near real-time nationwide register data, the register-based cohort design is the method of choice to continue the annual surveillance of influenza vaccine effectiveness.     

Methodologic issues regarding the use of three observational study designs to assess influenza vaccine effectiveness

BACKGROUND: Influenza causes substantial morbidity and annual vaccination is the most important prevention strategy. Accurately measuring vaccine effectiveness (VE) is difficult. The clinical syndrome most closely associated with influenza virus infection, influenza-like illness (ILI), is not specific. In addition, laboratory confirmation is infrequently done, and available rapid diagnostic tests are imperfect. The objective of this study was to estimate the joint impact of rapid diagnostic test sensitivity and specificity on VE for three types of study designs: a cohort study, a traditional case-control study, and a case-control study that used as controls individuals with ILI who tested negative for influenza virus infection. METHODS: We developed a mathematical model with five input parameters: true VE, attack rates (ARs) of influenza-ILI and non-influenza-ILI and the sensitivity and specificity of the diagnostic test. RESULTS: With imperfect specificity, estimates from all three designs tended to underestimate true VE, but were similar except if fairly extreme inputs were used. Only if test specificity was 95% or more or if influenza attack rates doubled that of background illness did the case-control method slightly overestimate VE. The case-control method usually produced the highest and most accurate estimates, followed by the test-negative design. The bias toward underestimating true VE introduced by low test specificity increased as the AR of influenza- relative to non-influenza-ILI decreases and, to a lesser degree, with lower test sensitivity. CONCLUSIONS: Demonstration of a high influenza VE using tests with imperfect sensitivity and specificity should provide reassurance that the program has been effective in reducing influenza illnesses, assuming adequate control of confounding factors.     

Influenza vaccine effectiveness by test-negative design – Comparison of inpatient and outpatient settings

BackgroundObservational studies of influenza vaccine effectiveness (VE) are increasingly using the test-negative design. Studies are typically based in outpatient or inpatient settings, but these two approaches are rarely compared directly. The aim of our study was to assess whether influenza VE estimates differ between inpatient and outpatient settings.MethodsWe searched the literature from Medline, PubMed and Web of Science using a combination of keywords to identify published studies of influenza VE using the test-negative design. Studies assessing any type of influenza vaccine among any population in any setting were considered, while interim studies or re-analyses were excluded. Retrieved articles were reviewed, screened and categorized based on study setting, location and influenza season. We searched for parallel studies in inpatient and outpatient settings that were done in the same influenza season, in the same location, and in the same or similar age groups. For each of the pairs identified, we estimated the difference in VE estimates between settings, and we tested whether the average difference was significant using a paired t-test.ResultsIn total 25 pairs of estimates were identified that permitted comparisons between VE estimates in inpatient and outpatient study settings. Within pairs, the prevalence of influenza was generally higher among patients enrolled in the outpatient studies, while influenza vaccination coverage among the test-negative control groups was generally higher in the inpatient studies. There was no heterogeneity in the paired differences in VE, and the pooled difference in VE between inpatient and outpatient studies was −2% (95% confidence interval: −12%, 10%).ConclusionsWe found no differences in VE estimates between inpatient and outpatient settings by studies using the test-negative design. Further research involving direct comparisons of VE estimates from the two settings in the same populations and years would be valuable.     

The case test-negative design for studies of the effectiveness of influenza vaccine

Background

A modification to the case–control study design has become popular to assess vaccine effectiveness (VE) against viral infections. Subjects with symptomatic illness seeking medical care are tested by a highly specific polymerase chain reaction (PCR) assay for the detection of the infection of interest. Cases are subjects testing positive for the virus; those testing negative represent the comparison group. Influenza and rotavirus VE studies using this design are often termed “test-negative case-control” studies, but this design has not been formally described or evaluated. We explicitly state several assumptions of the design and examine the conditions under which VE estimates derived with it are valid and unbiased.

Methods

We derived mathematical expressions for VE estimators obtained using this design and examined their statistical properties. We used simulation methods to test the validity of the estimators and illustrate their performance using an influenza VE study as an example.

Results

Because the marginal ratio of cases to non-cases is unknown during enrollment, this design is not a traditional case-control study; we suggest the name “case test-negative” design. Under sets of increasingly general assumptions, we found that the case test-negative design can provide unbiased VE estimates. However, differences in health care-seeking behavior among cases and non-cases by vaccine status, strong viral interference, or modification of the probability of symptomatic illness by vaccine status can bias VE estimates.

Conclusions

Vaccine effectiveness estimates derived from case test-negative studies are valid and unbiased under a wide range of assumptions. However, if vaccinated cases are less severely ill and seek care less frequently than unvaccinated cases, then an appropriate adjustment for illness severity is required to avoid bias in effectiveness estimates. Viral interference will lead to a non-trivial bias in the vaccine effectiveness estimate from case test-negative studies only when incidence of influenza is extremely high and duration of transient non-specific immunity is long.     

Comparison of two control groups for estimation of oral cholera vaccine effectiveness using a case-control study design

BackgroundCase-control studies to quantify oral cholera vaccine effectiveness (VE) often rely on neighbors without diarrhea as community controls. Test-negative controls can be easily recruited and may minimize bias due to differential health-seeking behavior and recall. We compared VE estimates derived from community and test-negative controls and conducted bias-indicator analyses to assess potential bias with community controls.MethodsFrom October 2012 through November 2016, patients with acute watery diarrhea were recruited from cholera treatment centers in rural Haiti. Cholera cases had a positive stool culture. Non-cholera diarrhea cases (test-negative controls and non-cholera diarrhea cases for bias-indicator analyses) had a negative culture and rapid test. Up to four community controls were matched to diarrhea cases by age group, time, and neighborhood.ResultsPrimary analyses included 181 cholera cases, 157 non-cholera diarrhea cases, 716 VE community controls and 625 bias-indicator community controls. VE for self-reported vaccination with two doses was consistent across the two control groups, with statistically significant VE estimates ranging from 72 to 74%. Sensitivity analyses revealed similar, though somewhat attenuated estimates for self-reported two dose VE. Bias-indicator estimates were consistently less than one, with VE estimates ranging from 19 to 43%, some of which were statistically significant.ConclusionsOCV estimates from case-control analyses using community and test-negative controls were similar. While bias-indicator analyses suggested possible over-estimation of VE estimates using community controls, test-negative analyses suggested this bias, if present, was minimal. Test-negative controls can be a valid low-cost and time-efficient alternative to community controls for OCV effectiveness estimation and may be especially relevant in emergency situations.     

Estimates of 2012/13 influenza vaccine effectiveness using the case test-negative control design with different influenza negative control groups

Background

In recent years several reports of influenza vaccine effectiveness (VE) have been made early for public health decision. The majority of these studies use the case test-negative control design (TND), which has been showed to provide, under certain conditions, unbiased estimates of influenza VE. Nevertheless, discussions have been taken on the best influenza negative control group to use. The present study aims to contribute to the knowledge on this field by comparing influenza VE estimates using three test-negative controls: all influenza negative, non-influenza respiratory virus and pan-negative.

Methods

Incident ILI patients were prospectively selected and swabbed by a sample of general practitioners. Cases were ILI patients tested positive for influenza and controls ILI patients tested negative for influenza. The influenza negative control group was divided into non-influenza virus control group and pan-negative control group. Data were collected on vaccination status and confounding factors. Influenza VE was estimated as one minus the odds ratio of been vaccinated in cases versus controls adjusted for confounding effect by logistic regression.

Results

Confounder adjusted influenza VE against medically attended laboratory-confirmed influenza was 68.4% (95% CI: 20.7–87.4%) using all influenza negatives controls, 82.1% (95% CI: 47.6–93.9%) using non-influenza controls and 49.4% (95% CI: −44.7% to 82.3%) using pan-negative controls.

Conclusions

Influenza VE estimates differed according to the influenza negative control group used. These results are in accordance with the expected under the hypothesis of differential viral interference between influenza vaccinated and unvaccinated individuals. Given the wide importance of TND study further studies should be conducted in order to clarify the observed differences.     

Interim influenza vaccine effectiveness: A good proxy for final estimates in Spain in the seasons 2010–2014

IntroductionThe agreement between interim and final influenza vaccine effectiveness (VE) estimates would support the use of interim assessments as a proxy for final VE results to guide health authorities in influenza prevention. We aimed to compare interim/final VE estimates in Spain.MethodsWe used a test-negative case-control study (cycEVA) for 2010/11–2013/14 seasons. Sensitivity analyses were carried out by type/subtype of influenza virus and by target groups for vaccination.ResultsIn general, interim estimates were higher compared to end-season estimates. Interim and final VE differences were higher for the target groups compared to all population. Subtype-specific interim/final VE estimates showed greater concordance (3–13%) than for any virus (7–24%).ConclusionIn Spain, interim influenza VE estimates over 2010–2014 were a good proxy of the final protection of the vaccine. Interim and final estimates showed greater concordance for all population and if performed subtype-specific.     

Comparison of local influenza vaccine effectiveness using two methods

BackgroundIn some settings, research methods to determine influenza vaccine effectiveness (VE) may not be appropriate because of cost, time constraints, or other factors. Administrative database analysis of viral testing results and vaccination history may be a viable alternative. This study compared VE estimates from outpatient research and administrative databases.MethodsUsing the test-negative, case-control design, data for 2017–2018 and 2018–2019 influenza seasons were collected using: 1) consent, specimen collection, RT-PCR testing and vaccine verification using multiple methods; and 2) an administrative database of outpatients with a clinical respiratory viral panel combined with electronic immunization records. Odds ratios for likelihood of influenza infection by vaccination status were calculated using multivariable logistic regression. VE = (1 ? aOR) × 100.ResultsResearch participants were significantly younger (P < 0.001), more often white (69% vs. 59%; P < 0.001) than non-white and less frequently enrolled through the emergency department (35% vs. 72%; P < 0.001) than administrative database participants. VE was significant against all influenza and influenza A in each season and both seasons combined (37–49%). Point estimate differences between methods were evident, with higher VE in the research database, but insignificant due to low sample sizes. When enrollment sites were separately analyzed, there were significant differences in VE estimates for all influenza (66% research vs. 46% administrative P < 0.001) and influenza A (67% research vs. 49% administrative; P < 0.001) in the emergency department.Conclusions:The selection of the appropriate method for determining influenza vaccine effectiveness depends on many factors, including sample size, subgroups of interest, etc., suggesting that research estimates may be more generalizable. Other advantages of research databases for VE estimates include lack of clinician-related selection bias for testing and less misclassification of vaccination status. The advantages of the administrative databases are potentially shorter time to VE results and lower cost.     

Three-season effectiveness of inactivated influenza vaccine in preventing influenza illness and hospitalization in children in Japan, 2013–2016

ObjectivesWe assessed the vaccine effectiveness (VE) of inactivated influenza vaccine (IIV) in children 6 months to 15 years of age in 2015/16 season. In addition, based on the data obtained during the three seasons from 2013 to 2016, we estimated the three-season VE in preventing influenza illness and hospitalization.MethodsOur study was conducted according to a test-negative case-control design (TNCC) and as a case-control study based on influenza rapid diagnostic test results.ResultsDuring 2015/16 season, the quadrivalent IIV was first used in Japan. The adjusted VE in preventing influenza illness was 49% (95% confidence interval [CI]: 42–55%) against any type of influenza, 57% (95% CI: 50–63%) against influenza A and 34% (95% CI: 23–44%) against influenza B. The 3-season adjusted VE was 45% (95% CI: 41–49%) against influenza virus infection overall (N = 12,888), 51% (95% CI: 47–55%) against influenza A (N = 10,410), and 32% (95% CI: 24–38%) against influenza B (N = 9232). An analysis by age groups showed low or no significant VE in infants or adolescents. By contrast, VE was highest in the young group (1–5 years old) and declined with age thereafter. The 3-season adjusted VE in preventing hospitalization as determined in a case-control study was 52% (95% CI: 42–60%) for influenza A and 28% (95% CI: 4–46%) for influenza B, and by TNCC design, it was 54% (95% CI: 41–65%) for influenza A and 34% (95% CI: 6–54%) for influenza B.ConclusionWe demonstrated not only VE in preventing illness, but also VE in preventing hospitalization based on much larger numbers of children than previous studies.     

Can routinely collected laboratory and health administrative data be used to assess influenza vaccine effectiveness? Assessing the validity of the Flu and Other Respiratory Viruses Research (FOREVER) Cohort

BackgroundLinking data on laboratory specimens collected during clinical practice with health administrative data permits highly powered vaccine effectiveness (VE) studies to be conducted at relatively low cost, but bias from using convenience samples is a concern. We evaluated the validity of using such data for estimating VE.MethodsWe created the Flu and Other Respiratory Viruses Research (FOREVER) Cohort by linking individual-level data on respiratory virus laboratory tests, hospitalizations, emergency department visits, and physician services. For community-dwelling adults aged > 65 years, we assessed the presence and magnitude of information and selection biases, generated VE estimates under various conditions, and compared our VE estimates with those from other studies.ResultsWe included 65,648 unique testing episodes obtained from 54,434 individuals during the 2010–11 to 2015–16 influenza seasons. To examine information bias, we found the proportion testing positive for influenza for patients with unknown interval from illness onset to specimen collection was more similar to patients for whom illness onset date was ≤ 7 days before specimen collection than to patients for whom illness onset was > 7 days before specimen collection. To assess the presence of selection bias, we found the likelihood of influenza testing was comparable between vaccinated and unvaccinated individuals, although the adjusted odds ratios were significantly greater than 1 for some healthcare settings and during some influenza seasons. Over 6 seasons, VE estimates ranged between 36% (95%CI, 27–44%) in 2010–11 and 5% (95%CI, –2, 11%) in 2014–15. VE estimates were similar under a range of conditions, but were consistently higher when accounting for misclassification of vaccination status through a quantitative sensitivity analysis. VE estimates from the FOREVER Cohort were comparable to those from other studies.ConclusionsRoutinely collected laboratory and health administrative data contained in the FOREVER Cohort can be used to estimate influenza VE in community-dwelling older adults.     

Assessment of temporally-related acute respiratory illness following influenza vaccination

BackgroundA barrier to influenza vaccination is the misperception that the inactivated vaccine can cause influenza. Previous studies have investigated the risk of acute respiratory illness (ARI) after influenza vaccination with conflicting results. We assessed whether there is an increased rate of laboratory-confirmed ARI in post-influenza vaccination periods.MethodsWe conducted a cohort sub-analysis of children and adults in the MoSAIC community surveillance study from 2013 to 2016. Influenza vaccination was confirmed through city or hospital registries. Cases of ARI were ascertained by twice-weekly text messages to household to identify members with ARI symptoms. Nasal swabs were obtained from ill participants and analyzed for respiratory pathogens using multiplex PCR. The primary outcome measure was the hazard ratio of laboratory-confirmed ARI in individuals post-vaccination compared to other time periods during three influenza seasons.ResultsOf the 999 participants, 68.8% were children, 30.2% were adults. Each study season, approximately half received influenza vaccine and one third experienced ≥1 ARI. The hazard of influenza in individuals during the 14-day post-vaccination period was similar to unvaccinated individuals during the same period (HR 0.96, 95% CI [0.60, 1.52]). The hazard of non-influenza respiratory pathogens was higher during the same period (HR 1.65, 95% CI [1.14, 2.38]); when stratified by age the hazard remained higher for children (HR 1·71, 95% CI [1.16, 2.53]) but not for adults (HR 0.88, 95% CI [0.21, 3.69]).ConclusionAmong children there was an increase in the hazard of ARI caused by non-influenza respiratory pathogens post-influenza vaccination compared to unvaccinated children during the same period. Potential mechanisms for this association warrant further investigation. Future research could investigate whether medical decision-making surrounding influenza vaccination may be improved by acknowledging patient experiences, counseling regarding different types of ARI, and correcting the misperception that all ARI occurring after vaccination are caused by influenza.     

Effect of propensity of seeking medical care on the bias of the estimated effectiveness of rotavirus vaccines from studies using a test-negative case-control design

BackgroundRotavirus is the leading cause of severe diarrhea among children worldwide, and vaccines can reduce morbidity and mortality by 50–98%. The test-negative control (TNC) study design is increasingly used for evaluating the effectiveness of vaccines against rotavirus and other vaccine-preventable diseases. In this study design, symptomatic patients who seek medical care are tested for the pathogen of interest. Those who test positive (negative) are classified as cases (controls).MethodsWe use a probability model to evaluate the bias of estimates of rotavirus vaccine effectiveness (VE) against rotavirus diarrhea resulting in hospitalization in the presence of possible confounding and selection biases due to differences in the propensity of seeking medical care (PSMC) between vaccinated and unvaccinated children.ResultsThe TNC-based VE estimate corrects for confounding bias when the confounder’s effects on the probabilities of rotavirus and non-rotavirus related hospitalizations are equal. If this condition is not met, then the estimated VE may be substantially biased. The bias is more severe in low-income countries, where VE is known to be lower. Under our model, differences in PSMC between vaccinated and unvaccinated children do not result in selection bias when the TNC study design is used.ConclusionsIn practice, one can expect the association of PSMC (or other potential confounders) with the probabilities of rotavirus and non-rotavirus related hospitalization to be similar, in which case the confounding effects will only result in small bias in the VE estimate from TNC studies. The results of this work, along with those of our previous paper, confirm the TNC design can be expected to provide reliable estimates of rotavirus VE in both high- and low-income countries.     

Evaluation of the 2018–2019 vaccine effectiveness against medically attended influenza-like illness using medical records and claims data

BackgroundHealthcare administrative databases are a rich source of information that could be leveraged to estimate real-world influenza vaccine effectiveness (VE). We aimed to evaluate the VE of standard egg-based influenza vaccines and determine if administrative healthcare data provide accurate VE estimates compared to the US CDC data.MethodsThis retrospective cohort study was conducted during the 2018–2019 influenza season. Individuals who had at least one relevant record per year between 2015 and 2019 in their electronic medical record were included. Individuals were considered protected 14 days after receiving an influenza vaccine. The outcome was the occurrence of medically attended influenza-like illness (MA-ILI) defined by clinical diagnostic codes. Adjusted odds ratios (aORs) were derived from multivariate logistic regression and adjusted VE (aVEs) were calculated using 100 × (1-aORs).ResultsA total of 5,066,980 individuals were included in the analysis with 1,307,702 (25.8%) considered vaccinated. Overall, the median age was 54 (IQR, 32–66) and 58.1% were female. Vaccine protection against MA-ILI was moderate in children and low in adults. All estimates were lower than VEs reported by the CDC for the 2018–2019 influenza season. Our results were robust to potential loss to follow up, but misclassification bias and residual confounding led to underestimation of the 2018–2019 aVE. When stratified by the number of primary care visits, aVE estimates and vaccination coverage increased with the number of primary care visits, reaching estimates similar to those obtained by the US CDC and US national vaccination coverage among those with ≥ 6 primary care visits, resulting in significant positive vaccine protection in frequent healthcare users.ConclusionsModerate and low aVEs were observed during the 2018–2019 season using administrative healthcare data, which was likely due to detection and misclassification biases, correlated with healthcare seeking behaviour, leading to an underestimation of the 2018–2019 influenza VE.