Reference set for performance testing of pediatric vaccine safety signal detection methods and systems

BackgroundSafety signal detection in spontaneous reporting system databases and electronic healthcare records is key to detection of previously unknown adverse events following immunization. Various statistical methods for signal detection in these different datasources have been developed, however none are geared to the pediatric population and none specifically to vaccines. A reference set comprising pediatric vaccine-adverse event pairs is required for reliable performance testing of statistical methods within and across data sources.MethodsThe study was conducted within the context of the Global Research in Paediatrics (GRiP) project, as part of the seventh framework programme (FP7) of the European Commission. Criteria for the selection of vaccines considered in the reference set were routine and global use in the pediatric population. Adverse events were primarily selected based on importance. Outcome based systematic literature searches were performed for all identified vaccine-adverse event pairs and complemented by expert committee reports, evidence based decision support systems (e.g. Micromedex), and summaries of product characteristics. Classification into positive (PC) and negative control (NC) pairs was performed by two independent reviewers according to a pre-defined algorithm and discussed for consensus in case of disagreement.ResultsWe selected 13 vaccines and 14 adverse events to be included in the reference set. From a total of 182 vaccine-adverse event pairs, we classified 18 as PC, 113 as NC and 51 as unclassifiable. Most classifications (91) were based on literature review, 45 were based on expert committee reports, and for 46 vaccine-adverse event pairs, an underlying pathomechanism was not plausible classifying the association as NC.ConclusionA reference set of vaccine-adverse event pairs was developed. We propose its use for comparing signal detection methods and systems in the pediatric population.     

Efficacy of m-Health for the detection of adverse events following immunization – The stimulated telephone assisted rapid safety surveillance (STARSS) randomised control trial

IntroductionPassive surveillance is recommended globally for the detection of adverse events following immunisation (AEFI) but this has significant challenges. Use of Mobile health for vaccine safety surveillance enables a consumer-centred approach to reporting. The Stimulated Telephone Assisted Rapid Safety Surveillance (STARSS) a randomised control trial (RCT) sought to evaluate the efficacy and acceptability of SMS for AEFI surveillance.MethodsMulti-centre RCT, participants were adult vaccinees or parents of children receiving any vaccine at a trial site. At enrolment randomisation occurred to one of two SMS groups or a control group. Prompts on days 2, 7 and 14 post-immunisation, were sent to the SMS group, to ascertain if a medical event following immunisation (MEFI) had occurred. No SMS’s were sent to the control participants. Those in the SMS who notified an MEFI were pre-randomised to complete a computer assisted telephone interview or a web based report to determine if an AEFI had occurred whilst an AEFI in the controls was determined by a search for passive reports. The primary outcome was the AEFI detection rate in the SMS group compared to controls.ResultsWe enrolled 6,338 participants, who were equally distributed across groups and who received 11,675 vaccines. The SMS group (4,225) received 12,675 surveillance prompts with 9.8% being non-compliant and not responding. In those that responded 90% indicated that no MEFI had been experienced and 184 had a verified AEFI. 6 control subjects had a reported AEFI. The AEFI detection rate was 13 fold greater in the SMS group when compared with controls (4.3 vs 0.3%).ConclusionWe have demonstrated that the STARSS methodology improves AEFI detection. Our findings should inform the wider use of SMS-based surveillance which is particularly relevant since establishing robust and novel pharmacovigilance systems is critical to monitoring novel vaccines which includes potential COVID vaccines.