Basic principles of test-negative design in evaluating influenza vaccine effectiveness

Based on the unique characteristics of influenza, the concept of “monitoring” influenza vaccine effectiveness (VE) across the seasons using the same observational study design has been developed. In recent years, there has been a growing number of influenza VE reports using the test-negative design, which can minimize both misclassification of diseases and confounding by health care-seeking behavior. Although the test-negative designs offer considerable advantages, there are some concerns that widespread use of the test-negative design without knowledge of the basic principles of epidemiology could produce invalid findings. In this article, we briefly review the basic concepts of the test-negative design with respect to classic study design such as cohort studies or case-control studies. We also mention selection bias, which may be of concern in some countries where rapid diagnostic testing is frequently used in routine clinical practices, as in Japan.     

Interim influenza vaccine effectiveness: A good proxy for final estimates in Spain in the seasons 2010–2014

IntroductionThe agreement between interim and final influenza vaccine effectiveness (VE) estimates would support the use of interim assessments as a proxy for final VE results to guide health authorities in influenza prevention. We aimed to compare interim/final VE estimates in Spain.MethodsWe used a test-negative case-control study (cycEVA) for 2010/11–2013/14 seasons. Sensitivity analyses were carried out by type/subtype of influenza virus and by target groups for vaccination.ResultsIn general, interim estimates were higher compared to end-season estimates. Interim and final VE differences were higher for the target groups compared to all population. Subtype-specific interim/final VE estimates showed greater concordance (3–13%) than for any virus (7–24%).ConclusionIn Spain, interim influenza VE estimates over 2010–2014 were a good proxy of the final protection of the vaccine. Interim and final estimates showed greater concordance for all population and if performed subtype-specific.     

Lineage-matched versus mismatched influenza B vaccine effectiveness following seasons of marginal influenza B circulation

BackgroundSeveral countries have recently transitioned from the trivalent inactivated influenza vaccine (TIV) to the quadrivalent inactivated influenza vaccine (QIV) in order to outweigh influenza B vaccine-mismatch. However, few studies thus far evaluated its benefits versus the TIV in a systematic manner. Our objective was to compare the QIV VE with lineage-mismatched TIV VE.MethodsWe estimated the 2015–2016, 2017–2018, 2019–2020 end-of season influenza B VE against laboratory-confirmed influenza-like illness (ILI) among community patients, using the test-negative design. VE was estimated for pre-determined age groups and for moving age intervals of 15 years.ResultsSince 2011–2012 season, alternate seasons in Israel were dominated by influenza B circulation. Compared with the lineage-mismatched TIV used during the 2015–2016 and 2017–2018 seasons, the 2019–2020 QIV showed the highest all-ages VE, with VE estimates of 56.9 (95% CI 30.1 to 73.4), 16.5 (95% CI –22.5 to 43.1) and ?25.8 (95% CI ?85.3 to 14.6) for the 2019–2020, 2017–2018 and 2015–2016 seasons, respectively. The 2019–2020 VE point estimated were the highest for the 0.5–4, 5–17 and 18–44 years age groups and for more 15-year age intervals as compared to the other seasons.ConclusionsOur results support the rapid transition from the TIV to the QIV.     

Vaccine effectiveness against laboratory-confirmed influenza in Europe – Results from the DRIVE network during season 2018/19

The DRIVE project aims to establish a sustainable network to estimate brand-specific influenza vaccine effectiveness (IVE) annually. DRIVE is a public–private partnership launched in response to EMA guidance that requires effectiveness evaluation from manufacturers for all individual influenza vaccine brands every season. IVE studies are conducted by public partners in DRIVE. Private partners (vaccine manufacturers from the European Federation of Pharmaceutical Industries and Association (EFPIA)) provide written feedback moderated by an independent scientific committee.Test-negative design (TND) case-control studies (4 in primary care and five in hospital) were conducted in six countries in Europe during the 2018/19 season. Site-specific confounder-adjusted vaccine effectiveness (VE) estimates for any vaccine exposure were calculated by age group (<18 years (y), 18-64y and 65 + y) and pooled by setting (primary care, hospital) through random effects meta-analysis. In addition, one population-based cohort study was conducted in Finland.TND studies included 3339 cases and 6012 controls; seven vaccine brands were reported. For ages 65 + y, pooled VE against any influenza strain was estimated at 27% (95%CI 6–44) in hospital setting. Sample size was insufficient for meaningful IVE estimates in other age groups, in the primary care setting, or by vaccine brand.The population-based cohort study included 274,077 vaccinated and 494,337 unvaccinated person-years, two vaccine brands were reported. Brand-specific IVE was estimated for Fluenz Tetra (36% [95%CI 24–45]) for ages 2-6y, Vaxigrip Tetra (54% [43–62]) for ages 6 months to 6y, and Vaxigrip Tetra (30% [25–35]) for ages 65 + y.The results presented are from the second influenza season covered by the DRIVE network. While sample size from the pooled TND studies was still too low for precise (brand-specific) IVE estimates, the network has approximately doubled in size compared to the pilot season. Taking measures to increase sample size is an important focus of DRIVE for the coming years.     

Vaccine effectiveness in preventing influenza hospitalizations in Navarre, Spain, 2010-2011: cohort and case-control study

We evaluated the 2010-2011 seasonal influenza vaccine effectiveness in preventing hospitalizations. Using healthcare databases we defined the target population for vaccination in Navarre, Spain, consisting of 217,320 people with major chronic conditions or aged 60 years and older. All hospitalized patients with influenza-like illness (ILI) were swabbed for influenza testing. A total of 269 patients with ILI were hospitalized and 61 of them were found positive for influenza virus: 58 for A(H1N1)2009 and 3 for B virus. The incidence rates of hospitalization with laboratory-confirmed influenza were compared by vaccination status. In the Cox regression model adjusted for sex, age, children in the household, urban/rural residence, comorbidity, pandemic vaccination, pneumococcal vaccination, outpatient visits and hospitalization in the previous year, the seasonal vaccine effectiveness was 58% (95% CI: 16-79%). The nested test-negative case-control analysis gave an adjusted estimate of 59% (95% CI: 4-83%). These results suggest a moderate effect of the 2010-2011 seasonal influenza vaccine in preventing hospitalization in a risk population. The close estimates obtained in the cohort and the test-negative case-control analyses suggest good control of biases.     

Sample size considerations for mid-season estimates from a large influenza vaccine effectiveness network in the United States

IntroductionMid-season influenza vaccine effectiveness (VE) estimates are a useful tool to help guide annual influenza vaccine strain selection, vaccine policy, and public health messaging. We propose using a sample size-driven approach with data-driven inputs for publication of mid-season influenza VE.MethodsWe used pooled inputs for VE by (sub)type and average vaccine coverage by age groups using data from eight seasons of the US Influenza VE Network to calculate sample sizes needed to estimate mid-season VE.ResultsWe estimate that 135 influenza-positive cases would be needed to detect an overall VE of 40% with 55% vaccine coverage among test-negative controls. Larger sample sizes would be required to produce reliable estimates specifically against influenza A/H3N2 and for older age groups.ConclusionUsing an existing network, most of the recent influenza seasons in the US would facilitate valid mid-season VE estimates using the proposed sample sizes for broad age groupings.     

Influenza vaccine effectiveness among adult patients in a University of Lyon hospital (2004-2009)

The aim of this study was to estimate influenza vaccine effectiveness (IVE) against laboratory-confirmed influenza among hospitalized patients. A case-control investigation was based on the prospective surveillance of influenza-like illness (ILI) during five flu seasons. We compared influenza-positive cases and influenza-negative controls. Unadjusted overall IVE was 62% (95% confidence interval 24% to 81%). We found that IVE was lower during the 2004-05 flu season (11%; 95% CI −232% to 76%) when the vaccine and circulating viruses were mismatched. Expansion of the study to other hospitals could provide IVE estimates earlier in the season, for different age groups and emerging virus strains.     

Repeated seasonal influenza vaccination among elderly in Europe: Effects on laboratory confirmed hospitalised influenza

In Europe, annual influenza vaccination is recommended to elderly. From 2011 to 2014 and in 2015–16, we conducted a multicentre test negative case control study in hospitals of 11 European countries to measure influenza vaccine effectiveness (IVE) against laboratory confirmed hospitalised influenza among people aged ≥65 years. We pooled four seasons data to measure IVE by past exposures to influenza vaccination.We swabbed patients admitted for clinical conditions related to influenza with onset of severe acute respiratory infection ≤7 days before admission. Cases were patients RT-PCR positive for influenza virus and controls those negative for any influenza virus. We documented seasonal vaccination status for the current season and the two previous seasons.We recruited 5295 patients over the four seasons, including 465A(H1N1)pdm09, 642A(H3N2), 278 B case-patients and 3910 controls. Among patients unvaccinated in both previous two seasons, current seasonal IVE (pooled across seasons) was 30% (95%CI: −35 to 64), 8% (95%CI: −94 to 56) and 33% (95%CI: −43 to 68) against influenza A(H1N1)pdm09, A(H3N2) and B respectively. Among patients vaccinated in both previous seasons, current seasonal IVE (pooled across seasons) was −1% (95%CI: −80 to 43), 37% (95%CI: 7–57) and 43% (95%CI: 1–68) against influenza A(H1N1)pdm09, A(H3N2) and B respectively.Our results suggest that, regardless of patients’ recent vaccination history, current seasonal vaccine conferred some protection to vaccinated patients against hospitalisation with influenza A(H3N2) and B. Vaccination of patients already vaccinated in both the past two seasons did not seem to be effective against A(H1N1)pdm09. To better understand the effect of repeated vaccination, engaging in large cohort studies documenting exposures to vaccine and natural infection is needed.     

Effectiveness of seasonal influenza vaccine in Australia, 2015: An epidemiological,antigenic and phylogenetic assessment

BackgroundA record number of laboratory-confirmed influenza cases were notified in Australia in 2015, during which type A(H3) and type B Victoria and Yamagata lineages co-circulated. We estimated effectiveness of the 2015 inactivated seasonal influenza vaccine against specific virus lineages and clades.MethodsThree sentinel general practitioner networks conduct surveillance for laboratory-confirmed influenza amongst patients presenting with influenza-like illness in Australia. Data from the networks were pooled to estimate vaccine effectiveness (VE) for seasonal trivalent influenza vaccine in Australia in 2015 using the case test-negative study design.ResultsThere were 2443 eligible patients included in the study, of which 857 (35%) were influenza-positive. Thirty-three and 19% of controls and cases respectively were reported as vaccinated. Adjusted VE against all influenza was 54% (95% CI: 42, 63). Antigenic characterisation data suggested good match between vaccine and circulating strains of A(H3); however VE for A(H3) was low at 44% (95% CI: 21, 60). Phylogenetic analysis indicated most circulating viruses were from clade 3C.2a, rather than the clade included in the vaccine (3C.3a). VE point estimates were higher against B/Yamagata lineage influenza (71%; 95% CI: 57, 80) than B/Victoria (42%, 95% CI: 13, 61), and in younger people.ConclusionsOverall seasonal vaccine was protective against influenza infection in Australia in 2015. Higher VE against the B/Yamagata lineage included in the trivalent vaccine suggests that more widespread use of quadrivalent vaccine could have improved overall effectiveness of influenza vaccine. Genetic characterisation suggested lower VE against A(H3) influenza was due to clade mismatch of vaccine and circulating viruses.     

Methodologic issues regarding the use of three observational study designs to assess influenza vaccine effectiveness

BACKGROUND: Influenza causes substantial morbidity and annual vaccination is the most important prevention strategy. Accurately measuring vaccine effectiveness (VE) is difficult. The clinical syndrome most closely associated with influenza virus infection, influenza-like illness (ILI), is not specific. In addition, laboratory confirmation is infrequently done, and available rapid diagnostic tests are imperfect. The objective of this study was to estimate the joint impact of rapid diagnostic test sensitivity and specificity on VE for three types of study designs: a cohort study, a traditional case-control study, and a case-control study that used as controls individuals with ILI who tested negative for influenza virus infection. METHODS: We developed a mathematical model with five input parameters: true VE, attack rates (ARs) of influenza-ILI and non-influenza-ILI and the sensitivity and specificity of the diagnostic test. RESULTS: With imperfect specificity, estimates from all three designs tended to underestimate true VE, but were similar except if fairly extreme inputs were used. Only if test specificity was 95% or more or if influenza attack rates doubled that of background illness did the case-control method slightly overestimate VE. The case-control method usually produced the highest and most accurate estimates, followed by the test-negative design. The bias toward underestimating true VE introduced by low test specificity increased as the AR of influenza- relative to non-influenza-ILI decreases and, to a lesser degree, with lower test sensitivity. CONCLUSIONS: Demonstration of a high influenza VE using tests with imperfect sensitivity and specificity should provide reassurance that the program has been effective in reducing influenza illnesses, assuming adequate control of confounding factors.     

Trends in effectiveness of inactivated influenza vaccine in children by age groups in seven seasons immediately before the COVID-19 era

BackgroundWe have reported the vaccine effectiveness of inactivated influenza vaccine in children aged 6 months to 15 years between the 2013/14 and 2018/19 seasons. Younger (6–11 months) and older (6–15 years old) children tended to have lower vaccine effectiveness. The purpose of this study is to investigate whether the recent vaccine can be recommended to all age groups.MethodsThe overall adjusted vaccine effectiveness was assessed from the 2013/14 until the 2020/21 season using a test-negative case-control design based on rapid influenza diagnostic test results. Vaccine effectiveness was calculated by influenza type and by age group (6–11 months, 1–2, 3–5, 6–12, and 13–15 years old) with adjustments including influenza seasons.ResultsA total of 29,400 children (9347, 4435, and 15,618 for influenza A and B, and test-negatives, respectively) were enrolled. The overall vaccine effectiveness against influenza A, A(H1N1)pdm09, and B was significant (44% [95% confidence interval (CI), 41–47], 63% [95 %CI, 51–72], and 37% [95 %CI, 32–42], respectively). The vaccine was significantly effective against influenza A and B, except among children 6 to 11 months against influenza B. The age group with the highest vaccine effectiveness was 1 to 2 years old with both influenza A and B (60% [95 %CI, 55–65] and 52% [95 %CI, 41–61], respectively). Analysis for the 2020/21 season was not performed because no cases were reported.ConclusionsThis is the first report showing influenza vaccine effectiveness by age group in children for several seasons, including immediately before the coronavirus disease (COVID-19) era. The fact that significant vaccine effectiveness was observed in nearly every age group and every season shows that the recent vaccine can still be recommended to children for the upcoming influenza seasons, during and after the COVID-19 era.     

Effectiveness of the current and prior influenza vaccinations in Northern Spain, 2018–2019

BackgroundThe population targeted for influenza vaccination can be repeatedly vaccinated over successive seasons, and vaccines received in previous seasons may retain preventive effect. This study aims to estimate the effectiveness of inactivated influenza vaccines received in the current and prior seasons in the 2018–2019 season.MethodsInfluenza-like illness patients attended by sentinel general practitioners or admitted to hospitals in Navarre, Spain, were tested for influenza. Vaccination status in the current and three prior seasons was obtained from the vaccination registry. The test-negative design was used to estimate the vaccine effectiveness.ResultsA total of 381 influenza A(H1N1)pdm09 cases, 341 A(H3N2) cases and 1222 controls were analysed. As compared to individuals unvaccinated in the current and three prior seasons, the influenza vaccine effectiveness against A(H1N1)pdm09 was 57% (95% confidence interval [CI]: 40%, 70%) for current season vaccination regardless of prior doses and 48% (95%CI: 14%, 68%) for vaccination in prior seasons but not in the current season. These estimates were 12% (95%CI: −23%, 37%) and 27% (95%CI: −22%, 56%), respectively, against influenza A(H3N2). Individuals vaccinated with the two A(H1N1)pdm09 strains in influenza vaccines since 2009, A/Michigan/45/2015 and A/California/07/2009, had higher protection (68%; 95%CI: 53%, 77%) than those vaccinated with A/Michigan/45/2015 only (29%, p = 0.020) or with A/California/07/2009 only (34%, p = 0.005).ConclusionThese results suggest moderate effectiveness of influenza vaccination against A(H1N1)pdm09 and low effectiveness against A(H3N2) influenza in the 2018–2019 season. Vaccination in prior seasons maintained a notable protective effect. Strains included in previous vaccines were as effective as the current vaccine strain, and both added their effects against influenza A(H1N1)pdm09.     

A nested case-control study of influenza vaccination was a cost-effective alternative to a full cohort analysis

OBJECTIVE: In the absence of trial results that are applicable to the target population, nested case-control studies might be an alternative to full-cohort analysis. We compared relative and absolute estimates of associations in an influenza vaccine study using both designs. STUDY DESIGN AND SETTING: Data from elderly persons enrolled during six consecutive influenza seasons were used (147,551 person-periods). The endpoints "hospitalization for pneumonia or influenza" (P&I) or "death" were used combined and separately to define three types of cases. Controls for the case-control samples were randomly selected from the remainder of the total cohort at different ratios (1:1 to 1:4). Logistic regression analysis was used to assess adjusted vaccine effectiveness (VE). Sampling fractions were used to determine the number needed to treat to prevent one outcome. Receiver-operator-curve analysis was conducted to estimate the area under the curve (AUC) as a measure of discriminative capacity of the prognostic model. RESULTS: In all, 978 P and I hospitalizations and 1,339 deaths were observed. The adjusted estimates of relative estimates (VE, AUC) and their corresponding 95% confidence intervals were virtually the same using both study designs, notably when the case-control ratio was high (1:4). CONCLUSION: A nested case-control design can provide valid and precise estimates of associations and is a cost-effective alternative for full-cohort analysis.     

Brand-specific influenza vaccine effectiveness estimates during 2019/20 season in Europe – Results from the DRIVE EU study platform

DRIVE (Development of Robust and Innovative Vaccine Effectiveness) is an IMI funded public–private platform that aims to annually estimate brand-specific influenza vaccine effectiveness (IVE), for public health and regulatory purposes. IVE analyses and reporting are conducted by public partners in the consortium.In 2019/20, four primary care-based test-negative design (TND) studies (Austria, England, Italy (n = 2)), eight hospital-based TND studies (Finland, France, Italy, Romania, Spain (n = 4)), and one population-based cohort study (Finland) were conducted. The COVID-19 pandemic affected influenza surveillance in all participating study sites, therefore the study period was truncated on February 29, 2020. Age-stratified (6 m-17y, 18-64y, ≥65y), confounder-adjusted, site-specific adjusted IVE estimates were calculated and pooled through meta-analysis. Parsimonious confounder-adjustment was performed, adjusting the estimates for age, sex and calendar time.TND studies included 3531 cases (351 vaccinated) and 5546 controls (1415 vaccinated) of all ages. IVE estimates were available for 8/11 brands marketed in Europe in 2019.Most children and adults < 64y were captured in primary care setting and the most frequently observed vaccine brand was Vaxigrip Tetra. The estimate against any influenza for Vaxigrip Tetra in primary care setting was 61% (95%CI 38–77) in children and 32% (95%CI −13–59) in adults up to 64y. Most adults ≥ 65y were captured in hospital setting and the most frequently observed brand was Fluad, with an estimate of 52% (95%CI 27–68).The population-based cohort covered 511,854 person-years and two vaccine brands. In children aged 2-6y, the IVE against any influenza was 68% (95%CI 58–75) for Fluenz Tetra and 71% (56–80) for Vaxigrip Tetra. In adults ≥ 65y, IVE against any influenza was 29% (20–36) for Vaxigrip Tetra.DRIVE is a growing platform. Public health institutes with surveillance data and hospitals in countries with high influenza vaccine coverage are encouraged to join DRIVE.     

Evaluating the case-positive,control test-negative study design for influenza vaccine effectiveness for the frailty bias

IntroductionPrevious influenza vaccine effectiveness studies were criticized for their failure to control for frailty. This study was designed to see if the test-negative study design overcomes this bias.MethodsAdults ≥ 50 years of age with respiratory symptoms were enrolled from November 2006 through May 2012 during the influenza season (excluding the 2009–2010 H1N1 pandemic season) to perform yearly test-negative control influenza vaccine effectiveness studies in Nashville, TN. At enrollment, both a nasal and throat swab sample were obtained and tested for influenza by RT-PCR. Frailty was calculated using a modified Rockwood Index that included 60 variables ascertained in a retrospective chart review giving a score of 0 to 1. Subjects were divided into three strata: non frail (≤0.08), pre-frail (>0.08 to <0.25), and frail (≥0.25). Vaccine effectiveness was calculated using the formula [1-adjusted odds ratio (OR)] × 100%. Adjusted ORs for individual years and all years combined were estimated by penalized multivariable logistic regression.ResultsOf 1023 hospitalized adults enrolled, 866 (84.7%) participants had complete immunization status, molecular influenza testing and covariates to calculate frailty. There were 83 influenza-positive cases and 783 test-negative controls overall, who were 74% white, 25% black, and 59% female. The median frailty index was 0.167 (Interquartile: 0.117, 0.267). The frailty index was 0.167 (0.100, 0.233) for the influenza positive cases compared to 0.183 (0.133, 0.267) for influenza negative controls (p = 0.07). Vaccine effectiveness estimates were 55.2% (95%CI: 30.5, 74.2), 60.4% (95%CI: 29.5, 74.4), and 54.3% (95%CI: 28.8, 74.0) without the frailty variable, including frailty as a continuous variable, and including frailty as a categorical variable, respectively.ConclusionsUsing the case positive test negative study design to assess vaccine effectiveness, our measure of frailty was not a significant confounder as inclusion of this measure did not significantly change vaccine effectiveness estimates.     

Effectiveness of subunit influenza vaccination in the 2014–2015 season and residual effect of split vaccination in previous seasons

BackgroundIn Navarra, Spain, subunit vaccine was first used in the 2014–2015 season, whereas trivalent split-virion influenza vaccines had been used in previous seasons. We estimate the effectiveness of the subunit vaccine in the current season and split vaccine in the two previous seasons against laboratory-confirmed influenza in the 2014–2015 season.MethodsPatients with influenza-like illness hospitalized or attended by sentinel general practitioners were swabbed for influenza testing. The previous and current vaccine status of laboratory-confirmed cases was compared to test-negative controls.ResultsAmong 1213 patients tested, 619 (51%) were confirmed for influenza virus: 52% influenza A(H3N2), 46% influenza B, and 2% A(H1N1)pdm09. The overall effectiveness for subunit vaccination in the current season was 19% (95% confidence interval [CI]: −13 to 42), 2% (95%CI: −47 to 35) against influenza A(H3N2) and 32% (95%CI: −4 to 56) against influenza B. The effectiveness against any influenza was 67% (95%CI: 17–87) for 2012–2013 and 2013–2014 vaccination only, 42% (95%CI: −31 to 74) for 2014–2015 vaccination only, and 38% (95%CI: 8–58) for vaccination in the 2012–2013, 2013–2014 and 2014–2015 seasons. The same estimates against influenza A(H3N2) were 47% (95%CI: −60 to 82), −54% (95%CI: −274 to 37) and 28% (95%CI: −17 to 56), and against influenza B were 82% (95%CI: 19–96), 93% (95%CI: 45–99) and 43% (95%CI: 5–66), respectively.ConclusionThese results suggest a considerable residual protection of split vaccination in previous seasons, low overall effectiveness of current season subunit vaccination, and possible interference between current subunit and previous split vaccines.