Impact of state weights on national vaccination coverage estimates from household surveys in Nigeria

National vaccination coverage estimates from household surveys are widely used in monitoring and planning of immunization programs. In Nigeria, survey-reported national coverage estimates have shown large fluctuations in the past few years. In this paper, we examine the impact of state-level survey weighting on Nigeria’s national vaccination coverage estimation. In particular, we focus three vaccination-related outcomes among children aged 12–23 months: the coverage of the third dose of diphtheria, pertussis, and tetanus vaccine (DPT3); the coverage of the first dose of measles-containing vaccine (MCV1); and the availability rate of home-based vaccination record (HBR). We compare the sample selection and weight assignment of three major survey programs in Nigeria, and show that considerable portions of the changes in survey-reported national coverage estimates can be explained by shifts in state-level weights. Our analysis demonstrates the importance of state weighting method in estimating aggregated national coverage figures and provides important context for interpreting changes in coverage estimates between surveys in the future.     

High resolution age-structured mapping of childhood vaccination coverage in low and middle income countries

BackgroundThe expansion of childhood vaccination programs in low and middle income countries has been a substantial public health success story. Indicators of the performance of intervention programmes such as coverage levels and numbers covered are typically measured through national statistics or at the scale of large regions due to survey design, administrative convenience or operational limitations. These mask heterogeneities and ‘coldspots’ of low coverage that may allow diseases to persist, even if overall coverage is high. Hence, to decrease inequities and accelerate progress towards disease elimination goals, fine-scale variation in coverage should be better characterized.MethodsUsing measles as an example, cluster-level Demographic and Health Surveys (DHS) data were used to map vaccination coverage at 1 km spatial resolution in Cambodia, Mozambique and Nigeria for varying age-group categories of children under five years, using Bayesian geostatistical techniques built on a suite of publicly available geospatial covariates and implemented via Markov Chain Monte Carlo (MCMC) methods.ResultsMeasles vaccination coverage was found to be strongly predicted by just 4–5 covariates in geostatistical models, with remoteness consistently selected as a key variable. The output 1 × 1 km maps revealed significant heterogeneities within the three countries that were not captured using province-level summaries. Integration with population data showed that at the time of the surveys, few districts attained the 80% coverage, that is one component of the WHO Global Vaccine Action Plan 2020 targets.ConclusionThe elimination of vaccine-preventable diseases requires a strong evidence base to guide strategies and inform efficient use of limited resources. The approaches outlined here provide a route to moving beyond large area summaries of vaccination coverage that mask epidemiologically-important heterogeneities to detailed maps that capture subnational vulnerabilities. The output datasets are built on open data and methods, and in flexible format that can be aggregated to more operationally-relevant administrative unit levels.     

Mapping the distribution of zero-dose children to assess the performance of vaccine delivery strategies and their relationships with measles incidence in Nigeria

Geographically precise identification and targeting of populations at risk of vaccine-preventable diseases has gained renewed attention within the global health community over the last few years. District level estimates of vaccination coverage and corresponding zero-dose prevalence constitute a potentially useful evidence base to evaluate the performance of vaccination strategies. These estimates are also valuable for identifying missed communities, hence enabling targeted interventions and better resource allocation. Here, we fit Bayesian geostatistical models to map the routine coverage of the first doses of diphtheria-tetanus-pertussis vaccine (DTP1) and measles-containing vaccine (MCV1) and corresponding zero-dose estimates in Nigeria at 1x1 km resolution and the district level using geospatial data sets. We also map MCV1 coverage before and after the 2019 measles vaccination campaign in the northern states to further explore variations in routine vaccine coverage and to evaluate the effectiveness of both routine immunization (RI) and campaigns in reaching zero-dose children. Additionally, we map the spatial distributions of reported measles cases during 2018 to 2020 and explore their relationships with MCV zero-dose prevalence to highlight the public health implications of varying performance of vaccination strategies across the country. Our analysis revealed strong similarities between the spatial distributions of DTP and MCV zero dose prevalence, with districts with the highest prevalence concentrated mostly in the northwest and the northeast, but also in other areas such as Lagos state and the Federal Capital Territory. Although the 2019 campaign reduced MCV zero-dose prevalence substantially in the north, pockets of vulnerabilities remained in areas that had among the highest prevalence prior to the campaign. Importantly, we found strong correlations between measles case counts and MCV RI zero-dose estimates, which provides a strong indication that measles incidence in the country is mostly affected by RI coverage. Our analyses reveal an urgent and highly significant need to strengthen the country’s RI program as a longer-term measure for disease control, whilst ensuring effective campaigns in the short term.     

Validity of maternal recall for estimating childhood vaccination coverage – Evidence from Nigeria

BackgroundVaccination coverage surveys in low- and middle-income countries typically estimate vaccination coverage using data from vaccination cards, parental recall, or a combination of the two. However, these surveys are often complicated by the pervasive absence of vaccination cards, forcing researchers to rely on parental recall. We assessed the validity of mothers’ recall against home-based vaccination cards using data from a community-based household survey in Nigeria.MethodsA cross-sectional survey of 1,254 mothers of children aged 12–23 months was performed in Enugu State, Nigeria in July 2020. Data on vaccination status for BCG, OPV, DPT, Measles, Yellow fever, and Vitamin A supplement were collected using two data sources: home-based vaccination cards and mothers’ recall. We evaluated the level of agreement between the two data sources; estimated the sensitivity and specificity of mothers’ recalls; and computed multivariable regression models to identify socio-demographic factors associated with mothers’ recall bias.ResultsOut of 1,254 mothers interviewed, 578 (46.1%) mothers with vaccination cards were included in this analysis. Vaccination coverage levels were generally similar across data sources, though recall-based data generally underestimated the coverage. The level of agreement between the two data sources was high (≥91.0% for all vaccine types) with recall bias due to under-reporting generally higher than recall bias due to over-reporting. The sensitivity of parental recalls was high for all vaccine types, while the specificity was low across vaccine types. Across all vaccines, mothers recall bias was significantly associated with the rural residence and not receiving postnatal care.ConclusionIn the absence of vaccination cards, mothers’ recall of their children’ vaccination status for BCG, OPV, DPT, Measles, Yellow fever and Vitamin A is a valid instrument for estimating childhood vaccination coverage in this setting in Nigeria. However, additional research is needed to confirm these findings at higher sub-national and national levels.     

Children vaccination coverage surveys: Impact of multiple sources of information and multiple contact attempts

BackgroundMonitoring vaccination coverage is an essential component of vaccination program evaluation. In Québec (Canada), children vaccination coverage surveys are conducted every two years since 2006. The objectives of this study were to evaluate the impact of supplementing data based on vaccination booklets with data from vaccine providers, on the final estimated vaccination coverage and to compare vaccination coverage between respondents to each survey contact attempt.MethodsData from six cross-sectional surveys were used, which included 3508 children aged 2 years. Parents were invited to transcribe the information available in their child’s vaccination booklet on the questionnaire received by mail. The survey included a maximum of 4 contact attempts to obtain a response. Data were completed among vaccine providers identified by parents. The main outcome was a complete vaccination status by 24 months of age.ResultsThe addition of data from vaccine providers to those present in vaccination booklets increased the proportion of children fully vaccinated from 5.5% to 23.7% depending on the survey year. The proportion of children fully vaccinated by 24 months of age estimated among respondents to contact 1 was only 2.1% higher than the estimates among all respondents.ConclusionsWithout validation among vaccine providers for children with missing doses according to vaccination booklets, results underestimated vaccination coverage in the target population. Conducting multiple contact attempts increased the response rate but had limited impact on the validity of estimates. It would be useful in future surveys to present the coverage obtain from respondents to each contact attempt.     

Geospatial variation in measles vaccine coverage through routine and campaign strategies in Nigeria: Analysis of recent household surveys

Measles vaccination campaigns are conducted regularly in many low- and middle-income countries to boost measles control efforts and accelerate progress towards elimination. National and sometimes first-level administrative division campaign coverage may be estimated through post-campaign coverage surveys (PCCS). However, these large-area estimates mask significant geographic inequities in coverage at more granular levels. Here, we undertake a geospatial analysis of the Nigeria 2017–18 PCCS data to produce coverage estimates at 1 × 1 km resolution and the district level using binomial spatial regression models built on a suite of geospatial covariates and implemented in a Bayesian framework via the INLA-SPDE approach. We investigate the individual and combined performance of the campaign and routine immunization (RI) by mapping various indicators of coverage for children aged 9–59 months. Additionally, we compare estimated coverage before the campaign at 1 × 1 km and the district level with predicted coverage maps produced using other surveys conducted in 2013 and 2016–17. Coverage during the campaign was generally higher and more homogeneous than RI coverage but geospatial differences in the campaign’s reach of previously unvaccinated children are shown. Persistent areas of low coverage highlight the need for improved RI performance. The results can help to guide the conduct of future campaigns, improve vaccination monitoring and measles elimination efforts. Moreover, the approaches used here can be readily extended to other countries.     

Estimating vaccination coverage: validity of household-retained vaccination cards and parental recall

Public health programs rely on household-survey estimates of vaccination coverage as a basis of programmatic and policy decisions; however, the validity of estimates derived from household-retained vaccination cards and parental recall has not been thoroughly evaluated. Using data from a vaccination coverage survey conducted in the Western Pacific's Northern Mariana Islands, we compared results from household data sources to medical record sources for the same children. We calculated the percentage of children aged 1, 2, and 6 years who received all vaccines recommended by age 12 months, 24 months, and for school entry, respectively. Coverage estimates based on vaccination cards ranged from 14% to 30% in the three age groups compared to 78-91% for the same children based on medical records. When cards were supplemented by parental recall, estimates were 51-53%. Concordance, sensitivity, specificity, positive and negative predictive values, and kappa statistics generally indicated poor agreement between household and medical record sources. Household-retained vaccination cards and parental recall were insufficient sources of information for estimating vaccination coverage in this population. This study emphasizes the importance of identifying reliable sources of vaccination history information and reinforces the need for awareness of the potential limitations of vaccination coverage estimated from surveys that rely on household-retained cards and/or parental recall.     

Assessing vaccination coverage in infants,survey studies versus the Flemish immunisation register: Achieving the best of both worlds

Infant immunisation coverage in Flanders, Belgium, is monitored through repeated coverage surveys. With the increased use of Vaccinnet, the web-based ordering system for vaccines in Flanders set up in 2004 and linked to an immunisation register, this database could become an alternative to quickly estimate vaccination coverage. To evaluate its current accuracy, coverage estimates generated from Vaccinnet alone were compared with estimates from the most recent survey (2012) that combined interview data with data from Vaccinnet and medical files.     

Issues and considerations in the use of serologic biomarkers for classifying vaccination history in household surveys

Accurate estimates of vaccination coverage are crucial for assessing routine immunization program performance. Community based household surveys are frequently used to assess coverage within a country. In household surveys to assess routine immunization coverage, a child's vaccination history is classified on the basis of observation of the immunization card, parental recall of receipt of vaccination, or both; each of these methods has been shown to commonly be inaccurate. The use of serologic data as a biomarker of vaccination history is a potential additional approach to improve accuracy in classifying vaccination history. However, potential challenges, including the accuracy of serologic methods in classifying vaccination history, varying vaccine types and dosing schedules, and logistical and financial implications must be considered. We provide historic and scientific context for the potential use of serologic data to assess vaccination history and discuss in detail key areas of importance for consideration in the context of using serologic data for classifying vaccination history in household surveys. Further studies are needed to directly evaluate the performance of serologic data compared with use of immunization cards or parental recall for classification of vaccination history in household surveys, as well assess the impact of age at the time of sample collection on serologic titers, the predictive value of serology to identify a fully vaccinated child for multi-dose vaccines, and the cost impact and logistical issues on outcomes associated with different types of biological samples for serologic testing.     

Vaccination coverage in India: A small area estimation approach

Information on population health indicators in India come from a number of surveys that vary in periodicity, scope and detail. In the case of immunization, the most recent coverage indicators are derived from the first round of Annual Health Survey (AHS-1, 2010-11), but these were conducted only in 9 of 35 states and union territories. The most recent national surveys of immunization coverage were conducted in 2009 (Coverage Evaluation Survey) by UNICEF. Therefore, reliable immunization coverage data for the entire country since 2009 is lacking. We used an established approach of small area estimation to predict coverage rates of several vaccinations for the remaining 26 states (not covered by AHS-1) in 2011. In our method, we considered a linear mixed model that combines data from five cross sectional surveys representing five different time points. Our model encompasses sampling error of the survey estimates, area specific random effects, autocorrelated area by time random effects and hence, borrows strength across areas and time points both. Model-based estimates for 2011 are almost identical to the AHS-1 estimates for the nine states, suggesting that our model provides reliable prediction of vaccination coverage as AHS-1 estimates are highly precise because of their large sample size. Results indicate that coverage inequality between rural and urban areas has been reduced significantly for most states in India. The National Rural Health Mission has had both supply side and demand side effects on the immunization programme in rural India. In combination, these effects may have contributed to the reduction of vaccination coverage gaps between urban and rural areas.     

Monitoring vaccination coverage: Defining the role of surveys

Vaccination coverage is a widely used indicator of programme performance, measured by registries, routine administrative reports or household surveys. Because the population denominator and the reported number of vaccinations used in administrative estimates are often inaccurate, survey data are often considered to be more reliable. Many countries obtain survey data on vaccination coverage every 3–5 years from large-scale multi-purpose survey programs. Additional surveys may be needed to evaluate coverage in Supplemental Immunization Activities such as measles or polio campaigns, or after major changes have occurred in the vaccination programme or its context.When a coverage survey is undertaken, rigorous statistical principles and field protocols should be followed to avoid selection bias and information bias. This requires substantial time, expertise and resources hence the role of vaccination coverage surveys in programme monitoring needs to be carefully defined. At times, programmatic monitoring may be more appropriate and provides data to guide program improvement. Practical field methods such as health facility-based assessments can evaluate multiple aspects of service provision, costs, coverage (among clinic attendees) and data quality. Similarly, purposeful sampling or censuses of specific populations can help local health workers evaluate their own performance and understand community attitudes, without trying to claim that the results are representative of the entire population. Administrative reports enable programme managers to do real-time monitoring, investigate potential problems and take timely remedial action, thus improvement of administrative estimates is of high priority. Most importantly, investment in collecting data needs to be complemented by investment in acting on results to improve performance.     

Quantifying bias in a health survey: modeling total survey error in the national immunization survey

Random-digit-dial telephone surveys are experiencing both declining response rates and increasing under-coverage due to the prevalence of households that substitute a wireless telephone for their residential landline telephone. These changes increase the potential for bias in survey estimates and heighten the need for survey researchers to evaluate the sources and magnitudes of potential bias. We apply a Monte Carlo simulation-based approach to assess bias in the NIS, a land-line telephone survey of 19-35 month-old children used to obtain national vaccination coverage estimates. We develop a model describing the survey stages at which component nonsampling error may be introduced due to nonresponse and under-coverage. We use that model and components of error estimated in special studies to quantify the extent to which noncoverage and nonresponse may bias the vaccination coverage estimates obtained from the NIS and present a distribution of the total survey error. Results indicated that the total error followed a normal distribution with mean of 1.72 per cent(95 per cent CI: 1.71, 1.74 per cent) and final adjusted survey weights corrected for this error. Although small, the largest contributor to error in terms of magnitude was nonresponse of immunization providers. The total error was most sensitive to declines in coverage due to cell phone only households. These results indicate that, while response rates and coverage may be declining, total survey error is quite small. Since response rates have historically been used to proxy for total survey error, the finding that these rates do not accurately reflect bias is important for evaluation of survey data. Published in 2011 by John Wiley & Sons, Ltd.     

Assessment of vaccination coverage, vaccination scar rates, and smallpox scarring in five areas of West Africa

In 1966, nineteen countries of West and Central Africa began a regional smallpox eradication and measles control programme in cooperation with the World Health Organization. This paper summarizes sample survey data collected to assess the results of the programme in Northern Nigeria (Sokoto and Katsina Provinces), Western Nigeria, Niger, Dahomey, and Togo. These data indicate that the programme, which used mass vaccination campaigns based on a collecting-point strategy, was generally successful in reaching a high proportion of the population. Analysis of vaccination coverage and vaccination scar rates by age underlined the importance to the programme of newborn children who accumulate rapidly following the mass campaign. Of all persons without vaccination scars at the time of the surveys, 34.4% were under 5 years of age; in the absence of a maintenance programme, this figure would rise to 40% after 1 year.     

Coverage and timeliness of vaccination and the validity of routine estimates: Insights from a vaccine registry in Kenya

BackgroundThe benefits of childhood vaccines are critically dependent on vaccination coverage. We used a vaccine registry (as gold standard) in Kenya to quantify errors in routine coverage methods (surveys and administrative reports), to estimate the magnitude of survivor bias, contrast coverage with timeliness and use both measures to estimate population immunity.MethodsVaccination records of children in the Kilifi Health and Demographic Surveillance System (KHDSS), Kenya were combined with births, deaths, migration and residence data from 2010 to 17. Using inverse survival curves, we estimated up-to-date and age-appropriate vaccination coverage, calculated mean vaccination coverage in infancy as the area under the inverse survival curves, and estimated the proportion of fully immunised children (FIC). Results were compared with published coverage estimates. Risk factors for vaccination were assessed using Cox regression models.ResultsWe analysed data for 49,090 infants and 48,025 children aged 12–23 months in 6 birth cohorts and 6 cross-sectional surveys respectively, and found 2nd year of life surveys overestimated coverage by 2% compared to birth cohorts. Compared to mean coverage in infants, static coverage at 12 months was exaggerated by 7–8% for third doses of oral polio, pentavalent (Penta3) and pneumococcal conjugate vaccines, and by 24% for the measles vaccine. Surveys and administrative coverage also underestimated the proportion of the fully immunised child by 10–14%. For BCG, Penta3 and measles, timeliness was 23–44% higher in children born in a health facility but 20–37% lower in those who first attended during vaccine stock outs.ConclusionsStandard coverage surveys in 12–23 month old children overestimate protection by ignoring timeliness, and survivor and recall biases. Where delayed vaccination is common, up-to-date coverage will give biased estimates of population immunity. Surveys and administrative methods also underestimate FIC prevalence. Better measurement of coverage and more sophisticated analyses are required to control vaccine preventable diseases.     

Meeting the Need for State-Level Estimates of Health Insurance Coverage: Use of State and Federal Survey Data

Objective. Critically review estimates of health insurance coverage available from different sources, including the federal government, state survey initiatives, and foundation-sponsored surveys for use in state policy research.
Study Setting and Design. We review the surveys in an attempt to flesh out the current weaknesses of survey data for state policy uses. The main data sources assessed in this analysis are federal government surveys (such as the Current Population Survey's Annual Social and Economic Supplement, and the National Health Interview Survey), foundation-supported surveys (National Survey of America's Families, and the Community Tracking Survey), and state-sponsored surveys.
Principal Findings. Despite information on estimates of health insurance coverage from six federal surveys, states find the data lacking for state policy purposes. We document the need for state representative data on the uninsured and the recent history of state data collection efforts spurred in part by the Health Resources Services Administration State Planning Grant program. We assess the state estimates of uninsurance from the Current Population Survey and make recommendations for a new consolidated federal survey with better state representative data.
Conclusions. We think there are several options to consider for coordinating a federal and state data collection strategy to inform state and national policy on coverage and access.     

Modelling the geographical distribution of co-infection risk from single-disease surveys

Background: The need to deliver interventions targeting multiple diseases in a cost‐effective manner calls for integrated disease control efforts. Consequently, maps are required that show where the risk of co‐infection is particularly high. Co‐infection risk is preferably estimated via Bayesian geostatistical multinomial modelling, using data from surveys screening for multiple infections simultaneously. However, only few surveys have collected this type of data. Methods: Bayesian geostatistical shared component models (allowing for covariates, disease‐specific and shared spatial and non‐spatial random effects) are proposed to model the geographical distribution and burden of co‐infection risk from single‐disease surveys. The ability of the models to capture co‐infection risk is assessed on simulated data sets based on multinomial distributions assuming light‐ and heavy‐dependent diseases, and a real data set of Schistosoma mansoni –hookworm co‐infection in the region of Man, Côte d'Ivoire. The data were restructured as if obtained from single‐disease surveys. The estimated results of co‐infection risk, together with independent and multinomial model results, were compared via different validation techniques. Results: The results showed that shared component models result in more accurate estimates of co‐infection risk than models assuming independence in settings of heavy‐dependent diseases. The shared spatial random effects are similar to the spatial co‐infection random effects of the multinomial model for heavy‐dependent data. Conclusions: In the absence of true co‐infection data geostatistical shared component models are able to estimate the spatial patterns and burden of co‐infection risk from single‐disease survey data, especially in settings of heavy‐dependent diseases. Copyright © 2011 John Wiley & Sons, Ltd.     

Combining cluster surveys to estimate vaccination coverage: Experiences from Nigeria’s multiple indicator cluster survey / national immunization coverage survey (MICS/NICS), 2016–17

In 2015 immunization stakeholders in Nigeria were proceeding with plans that would have fielded two nationally representative surveys to estimate vaccination coverage at the same time. Rather than duplicate efforts and generate either conflicting or redundant results, the stakeholders collaborated to conduct a combined Multiple Indicator Cluster Survey (MICS) / National Immunization Coverage Survey (NICS) with MICS focusing on core sampling clusters and NICS adding supplementary clusters in 20 states, to improve precision of outcomes there. This paper describes the organizational and technical aspects of that collaboration, including details on design of the sample supplement and analysis of the pooled dataset. While complicated, the collaboration was successful; it yielded a unified set of relevant coverage estimates and fostered some novel sub-national results dissemination work.     

Performance of small cluster surveys and the clustered LQAS design to estimate local-level vaccination coverage in Mali

Background

Estimation of vaccination coverage at the local level is essential to identify communities that may require additional support. Cluster surveys can be used in resource-poor settings, when population figures are inaccurate. To be feasible, cluster samples need to be small, without losing robustness of results. The clustered LQAS (CLQAS) approach has been proposed as an alternative, as smaller sample sizes are required.

Methods

We explored (i) the efficiency of cluster surveys of decreasing sample size through bootstrapping analysis and (ii) the performance of CLQAS under three alternative sampling plans to classify local VC, using data from a survey carried out in Mali after mass vaccination against meningococcal meningitis group A.

Results

VC estimates provided by a 10 × 15 cluster survey design were reasonably robust. We used them to classify health areas in three categories and guide mop-up activities: i) health areas not requiring supplemental activities; ii) health areas requiring additional vaccination; iii) health areas requiring further evaluation. As sample size decreased (from 10 × 15 to 10 × 3), standard error of VC and ICC estimates were increasingly unstable. Results of CLQAS simulations were not accurate for most health areas, with an overall risk of misclassification greater than 0.25 in one health area out of three. It was greater than 0.50 in one health area out of two under two of the three sampling plans.

Conclusions

Small sample cluster surveys (10 × 15) are acceptably robust for classification of VC at local level. We do not recommend the CLQAS method as currently formulated for evaluating vaccination programmes.     

National Immunization Survey: the methodology of a vaccination surveillance system

The National Immunization Survey (NIS) was designed to measure vaccination coverage estimates for the US, the 50 states, and selected urban areas for children ages 19-35 months. The NIS includes a random-digit-dialed telephone survey and a provider record check study. Data are weighted to account for the sample design and to reduce nonresponse and non-coverage biases in order to improve vaccination coverage estimates. Adjustments are made for biases resulting from nonresponse and nontelephone households, and estimation procedures are used to reduce measurement bias. The NIS coverage estimates represent all US children, not just children living in households with telephones. NIS estimates are highly comparable to vaccination estimates derived from the National Health Interview Survey. The NIS allows comparisons between states and urban areas over time and is used to evaluate current and new vaccination strategies.