A Review of Evaluations of Electronic Event-based Biosurveillance Systems

Objective

To assess evaluations of electronic event-based biosurveillance systems (EEBS’s) and define priorities for EEBS evaluations.

Introduction

EEBS’s that use near real-time information from the Internet are an increasingly important source of intelligence for public health organizations (1, 2). However, there has not been a systematic assessment of EEBS evaluations, which could identify uncertainties about current systems and guide EEBS development to effectively exploit digital information for surveillance.

Methods

We searched PubMed and consulted EEBS experts to identify EEBS’s that met the following criteria: uses publicly-available Internet info sources, includes events that impact humans, and has global scope. We constructed a list of 17 key evaluation variables using guidelines for evaluating health surveillance systems, and identified the key variables included in evaluations per EEBS, as well as the number of EEBS’s evaluated for each key variable (3,4).

Results

We identified 10 EEBS’s and 17 evaluations (

Selecting Targeted Symptoms/Syndromes for Syndromic Surveillance in Rural China

Objective

To select the potential targeted symptoms/syndromes as early warning indicators for epidemics or outbreaks detection in rural China.

Introduction

Patients’ chief complaints (CCs) as a common data source, has been widely used in syndromic surveillance due to its timeliness, accuracy and availability (1). For automated syndromic surveillance, CCs always classified into predefined syndromic categories to facilitate subsequent data aggregation and analysis. However, in rural China, most outpatient doctors recorded the information of patients (e.g. CCs) into clinic logs manually rather than computers. Thus, more convenient surveillance method is needed in the syndromic surveillance project (ISSC). And the first and important thing is to select the targeted symptoms/syndromes.

Methods

Epidemiological analysis was conducted on data from case report system in Jingmen City (one study site in ISSC) from 2004 to 2009. Initial symptoms/syndromes were selected by literature reviews. And finally expert consultation meetings, workshops and field investigation were held to confirm the targeted symptoms/syndromes.

Results

10 kinds of infectious diseases, 6 categories of emergencies, and 4 bioterrorism events (i.e. plague, anthrax, botulism and hemorrhagic fever) were chose as specific diseases/events for monitoring (Respiratory casesGastrointestinal casesEmergenciesName%Name%NameEvents (No.)^*Pulmonary tuberculosis82.38^†Hand-foot-mouth diseases41.73^†A(H1N1)10^†Mumps9.14^†Bacillary dysentery28.56^†Mumps5^†Measles3.35^†Hepatitis A15.36^†Hand-foot-mouth diseases1^†Varicella2.00^† Infectious diarrhea6.58^†Bacillary dysentery1^†Influenza/A(H1N1)1.79^†Hepatitis E4.30^†Food poisoning2^†Rubella0.72Typhoid3.03^†Unknown reason dermatitis1Scarlet fever0.44Paratyphoid0.22Pertussis0.15Amebic dysentery0.22Meningococcal meningitis0.03Total100.00Total100.00Total20Open in a separate window^*Chronic infectious diseases (excluded).^†Selected specific diseases (top 5) or events (non-infectious excluded).

Table 2

List of symptoms/syndromes

Use of Syndromic Data to Determine Oral Health Visit Burden on Emergency Departments

Objective

The objective was to use syndromic surveillance data from the North Carolina Disease Event Tracking and Epidemiologic Collection Tool NCDETECT and from BioSense to quantify the burden on North Carolina (NC) emergency departments of oral health-related visits more appropriate for care in a dental office (ED). Calculations were sought in terms of the Medicaid-covered visit rate relative to the Medicaid-eligible population by age group and by county.

Introduction

Concern over oral health-related ED visits stems from the increasing number of unemployed and uninsured, the cost burden of these visits, and the unavailability of indicated dental care in EDs [1]. Of particular interest to NC state public health planners are Medicaid-covered visits. Syndromic data in biosurveillance systems offer a means to quantify these visits overall and by county and age group.

Methods

Using BioSense data received from NCDETECT, 60.8 million records from 12.9 million ED visits were collected, covering all NC visits for state fiscal years (SFY) 2008–2010. Roughly 4% of visits were dropped because of patient residence zip codes missing or outside NC. A careful multi-step procedure involving both dentist consultants and data analysis was used to derive classification criteria for visits whose main reason was a nontraumatic oral health problem [2]. This procedure yielded 243,970 visits by ∼174,600 patients based on hospital-specific patient identifiers. Nontraumatic oral health-related visits were collected in a study set with added fields for method of payment, patient residence county, and age group. Based on previous studies, consultant preferences, and NC Medicaid eligibility guidelines, selected age groups were 0–14, 15–19, 20–29, 30–49, 50+ years. Stratified counts of Medicaid-eligibles were obtained from the NC Dental Director by study year. Using these tables and the ED visit study set, rates of nontraumatic oral health-related Medicaid visits per 10,000 eligibles were tabulated by county and age group for each study year. Demographics of multiple-visit patients were also studied.

Results

Rates of ED oral health-related visits were substantially higher for young adults than for other age groups. From statewide rates in Visits per 10,000 EligiblesSFY 2008SFY 2009SFY2010All Age9.59.99.20–14 yrs1.91.81.815–19 yrs8.49.07.920–29 yrs42.643.439.630–49 yrs22.924.222.550+ yrs9.52.52.4Open in a separate windowCounty-level rates showed the same age pattern to varying degrees. Detailed analysis showed problem areas, with rates in 21 of 100 counties exceeding 60 per 10,000 eligibles for the 20–29 year age group. Plots and tables complemented understanding of the ED oral health visit burden by age and county. The state total ED burden for oral health problems was ∼2% (0.2% – 9.7% by county).

Conclusions

Judicious use of syndromic data with external information, such as the detailed Medicaid denominators and the Method of Payment codes for each visit above, can give quantified estimates for policy-related public health issues. In the current study, the derived oral health visit rates gave numerical detail to concerns about the use of NC EDs for nontraumatic oral health problems by low-income persons affected by the economic recession. Results also show rate variation by county and can be combined with access-to-care data to inform planning of effective local measures to improve access to dental services and thus reduce the ED visit burden.     

Can We Use Syndromic Surveillance Data to Identify Primary Care Visits to NYC EDs?

Objective

To develop a syndrome classification based on patient chief complaint to (1) estimate the proportion of primary care-related emergency department (ED) visits in New York City (NYC) hospitals and (2) explore predictors of such visits.

Introduction

NYC EDs saw nearly 4 million visits in 2011. Studies have demonstrated that non-urgent visits can account for more than 50% of visits to EDs [1,2]. Designed to provide rapid diagnosis and first-line treatment of serious illness, EDs often function as a primary care site due to their accessibility. Unfortunately, use of EDs for primary care may affect their ability to meet the needs of severely ill patients.

Methods

We examined syndromic surveillance data from 45 hospitals in NYC for 2011 and classified visits into a primary care syndrome based on the chief complaint field. Data from 4 hospitals were omitted due to data quality issues, as were records from non-NYC residents. Primary care (PC) syndrome visits included visits recorded as referrals, screenings, suture removal/dressing changes, or medication refills; records with a blank or non-informative (e.g. “X”) chief complaint field were omitted from analysis. Using unique patient IDs, we identified patients who visited the same ED multiple times in the previous 12 months. A hierarchical generalized linear mixed effects model with hospital-level random effects was used to explore patient characteristics associated with PC syndrome visits. The model included a random intercept for hospital and the following covariates: duplicate visit, patient gender and age group (ages 0–4, 5–17, 18–64, and 65+), and time of visit (midnight to 8 AM, 8 AM to midnight). Covariates for month and day of week were included to control for temporal trends in ED visits. Model parameters were estimated by maximum likelihood. Estimation was performed in SAS version 9.2 [3] using the GLIMMIX procedure.

Results

Citywide, 7.5% (N=190,431) of visits to EDs during 2011 were classified as PC syndrome visits, but varied by hospital with a median of 4.6% (IQR: 3% to 9%) across hospitals. The average proportion of PC syndrome visits varied by hospital. Of the 45 hospitals included in the analysis, 18 had a lower baseline, 13 were the same, and 14 had a higher baseline than the city mean. Hospitals with a larger census had a larger proportion of PC syndrome visits.Age had a significant effect on the odds of a PC syndrome visit; ages 0–4 had the greatest odds of a PC syndrome visit relative to the 65+ age group. Visits from patients ages 5–17 and 18–64 were also more likely to be primary care visits. Patients with repeat visits were more likely to have PC syndrome visits. Female gender and early morning visits (12A–8A) were associated with lower odds of a PC syndrome visit.

Conclusions

With limited detail on patient visits, our syndrome likely under-counts primary care visits to EDs. However, the relationships between our explanatory variables—age, time of day, and duplicate visits—and PC syndrome visits are consistent with the literature on ED usage for primary care. Gender is an exception [1], but earlier findings may be confounded by the fact that females seek health care more frequently in general. The variation in PC syndrome visits among NYC EDs is significant and may be explained by hospital or community measures not captured in our model, such as clinic wait times, ED capacity, or insurance coverage. In fact, disparities in such predictors of PC syndrome visits could be targets for interventions. Our ability to replicate previous findings on the use of EDs for primary care visits suggests that syndromic data may be a near real-time data source for following trends in such visits.Predictors of PC Syndrome Visits

Computerized Text Analysis to Enhance Automated Pneumonia Detection

Objective

To improve the surveillance for pneumonia using the free-text of electronic medical records (EMR).

Introduction

Information about disease severity could help with both detection and situational awareness during outbreaks of acute respiratory infections (ARI). In this work, we use data from the EMR to identify patients with pneumonia, a key landmark of ARI severity. We asked if computerized analysis of the free-text of clinical notes or imaging reports could complement structured EMR data to uncover pneumonia cases.

Methods

A previously validated ARI case-detection algorithm (CDA) (sensitivity, 99%; PPV, 14%) [1] flagged VAMHCS outpatient visits with associated chest imaging (n = 2737). Manually categorized imaging reports (Non-Negative if they could support the diagnosis of pneumonia, Negative otherwise; kappa = 0.88), served as a reference for the development of an automated report classifier through machine-learning [2]. EMR entries related to visits with Non-Negative chest imaging were manually reviewed to identify cases with Possible Pneumonia (new symptom(s) of cough, sputum, fever/chills/night sweats, dyspnea, pleuritic chest pain) or with Pneumonia-in-Plan (pneumonia listed as one of two most likely diagnoses in a physician’s note). These cases were used as reference for the development of the EMR-based CDAs. CDA components included ICD-9 codes for the full spectrum of ARI [1] or for the pneumonia subset, text analysis aimed at non-negated ARI symptoms in the clinical note [1] and the above-mentioned imaging report text classifier.

Results

The manual review identified 370 reference cases with Possible Pneumonia and 250 with Pneumonia-in-Plan. Statistical performance for illustrative CDAs that combined structured EMR parameters with or without text analyses are shown in the ConclusionsAutomated text analysis of chest imaging reports can improve our ability to separate outpatients with pneumonia from those with a milder form of ARI.

CDA Number	1	2	3	4	5	6	7	8	9	10	11	12
	Possible Pneumonia						Pneumonia-in-Plan
CDA Components
(Pneumonia ICD-9 Codes	•	•					•	•
(ARI ICD-9 Codes			•	•	•	•			•	•	•	•
OR Text of Clinical Notes)					•	•					•	•
AND Chest Imaging Obtained	•	•	•	•	•	•	•	•	•	•	•	•
AND Text of Imaging Reports		•		•		•		•		•		•
Sensitivity (%)	36.8	28.4	85.9	58.4	99.7	66.2	52	40.8	93.6	68.8	100	74.8
Specificity (%)	95.4	99.7	29.8	98.5	2.2	98	95.4	99.6	29.8	96.8	2.3	95.7
PPV (%)	55.3	93.8	16	86.1	13.7	83.3	52.8	91.1	12	68.5	9.3	63.6
NPV (%)	91	90	93.2	93.8	98.1	95	95.2	94.4	98	97	100	97.4
F-Measure	44.2	43.6	27	69.6	24.1	73.8	52.4	56.4	21	68.6	17	68.7

Open in a separate window     

Early Detection of Influenza Activity Using Syndromic Surveillance in Missouri

Objective

To assess how weekly percent of influenza-like illness (ILI) reported via Early Notification of Community-based Epidemics (ESSENCE) tracked weekly counts of laboratory confirmed influenza cases in five influenza seasons in order to evaluate the early warning potential of ILI in ESSENCE and improve ongoing influenza surveillance efforts in Missouri.

Introduction

Syndromic surveillance is used routinely to detect outbreaks of disease earlier than traditional methods due to its ability to automatically acquire data in near real-time. Missouri has used emergency department (ED) visits to monitor and track seasonal influenza activity since 2006.

Methods

The Missouri ESSENCE system utilizes data from 84 hospitals, which represents up to 90 percent of all ED visits occurring in Missouri statewide each day. The influenza season is defined as starting during Centers for Disease Control and Prevention (CDC) week number 40 (around the first of October) and ending on CDC week 20 of the following year, which is usually at the end of May.A confirmed influenza case is laboratory confirmed by viral culture, rapid diagnostic tests, or a four-fold rise in antibody titer between acute and convalescent serum samples. Laboratory results are reported on a weekly basis. To assess the severity of influenza activity, all flu seasons were compared with the 2008–09 season, which experienced the lowest influenza activity based on laboratory data. Analysis of variance (ANOVA) was applied for this analysis using Statistical Analysis Software (SAS) (version 9.2).The standard ESSENCE ILI subsyndrome includes ED chief complaints that contain keywords such as “flu”, “flulike”, “influenza” or “fever plus cough” or “fever plus sore throat”. The ESSENCE ILI weekly percent is the number of ILI visits divided by total ED visits.Time series of weekly percent of ILI in ESSENCE were compared to weekly counts of laboratory confirmed influenza cases. Spearman correlation coefficients were calculated using SAS. The baseline refers to the mean of three flu seasons with low influenza activity (2006–07, 2008–09 and 2010–11 seasons). The threshold was calculated as this baseline plus three standard deviations.The early warning potential of the ESSENCE weekly ILI percent was evaluated for five consecutive influenza seasons, beginning in 2006. This was accomplished by calculating the time lag between the first ESSENCE ILI warning versus the first lab confirmed influenza warning. A warning was identified if either lab confirmed case counts or weekly percent of ILI crossed over their respective baselines.

Results

For each influenza season evaluated, weekly ILI rates reported via ESSENCE were significantly correlated with weekly counts of laboratory-confirmed influenza cases (Figure 1 shows that two influenza seasons (2007–08 and 2009–10) were more severe than others examined based on the ESSENCE percent ILI threshold analysis, this result is consistent with the examination of severity of influenza activity based on lab confirmed influenza data (p<0.05).Open in a separate windowFigure 1Number and baseline of lab confirmed influenza cases, ESSENCE weekly ILI percent and baseline, and ESSENCE ILI threshold for five consecutive influenza seasons, 2006–2011.

Table 1

Correlation between laboratory confrmed influenza cases and ESSENCE ILI weekly percent in five influenza seasons, 2006–2011

Mental Illness and Co-morbid Conditions: BioSense 2008 – 2011

Objective

The purpose of this paper was to analyze the associated burden of mental illness and medical comorbidity using BioSense data 2008–2011.

Introduction

Understanding the relationship between mental illness and medical comorbidity is an important aspect of public health surveillance. In 2004, an estimated one fourth of the US adults reported having a mental illness in the previous year (1). Studies showed that mental illness exacerbates multiple chronic diseases like cardiovascular diseases, diabetes and asthma (2). BioSense is a national electronic public health surveillance system developed by the Centers for Disease Control and Prevention (CDC) that receives, analyzes and visualizes electronic health data from civilian hospital emergency departments (EDs), outpatient and inpatient facilities, Veteran Administration (VA) and Department of Defense (DoD) healthcare facilities. Although the system is designed for early detection and rapid assessment of all-hazards health events, BioSense can also be used to examine patterns of healthcare utilization.

Methods

We used 4 years (2008 – 2011) of BioSense civilian hospitals’ EDs visit data to perform the analysis. We searched final diagnoses for ICD-9 CM codes related to mental illness (290 – 312), schizophrenia (295), major depressive disorder (296.2 – 296.3), mood disorder (296, 300.4 and 311) and anxiety, stress & adjustment disorders (300.0, 300.2, 300.3, 308, and 309). We used BioSense syndromes/sub-syndromes based on chief complaints and final diagnoses for comorbidity. For the purpose of this study, comorbidity was defined broadly as the co-occurrence of mental and physical illness in the same person regardless of the chronological order. The proportion was calculated as the number of mental health visits associated with comorbidity divided by the total number of mental illness relevant visits. We ranked the top 10 proportions of comorbidity for adult mental illness by year.

Results

From 2008–2011, there were 4.6 million visits where mental illness was reported in the EDs visits. Average age of those reported mental illness was 44 years, 55% were women and 45% were men. More women were reported with anxiety (67%), mood (66%), and major depressive disorders (59%) than men; while men were reported more with schizophrenia (56%) than women (44%). The most common comorbid condition was hypertension, followed by chest pain, abdominal pain, diabetes, nausea & vomiting and dyspnea (Rank20082009201020111HypertensionHypertensionHypertensionHypertension2Chest PainChest PainChest PainChest Pain3Abdominal painAbdominal painAbdominal painAbdominal pain4Diabetes mellitusDiabetes mellitusDiabetes mellitusDiabetes mellitus5Nausea & vomitingNausea &. vomitingNausea & vomitingNausea & vomiting6DyspneaDyspneaDyspneaDyspnea7Injury, NOSTailshallshalls8FallsAsthmaHeadacheHeadache9HeadacheHeadacheAsthmaInjury, NOS10AsthmaInjury, NOSInjury, NOSAsthmaOpen in a separate window

Conclusions

This study supports prior findings that adult mental illness is associated with substantial medical burden. We identified 10 most common comorbid condition associated with mental illness. The major limitation of this work was that electronic data does not allow determination of the causal pathway between mental illness and some medical comorbidity. In addition, data represents only those who have access to healthcare or those with health seeking behaviors. Familiarity with comorbid conditions affecting persons with adult mental illness may assist programs aimed at providing medical care for the mentally ill.     

Duration of Immunity to Norovirus Gastroenteritis

The duration of immunity to norovirus (NoV) gastroenteritis has been believed to be from 6 months to 2 years. However, several observations are inconsistent with this short period. To gain better estimates of the duration of immunity to NoV, we developed a mathematical model of community NoV transmission. The model was parameterized from the literature and also fit to age-specific incidence data from England and Wales by using maximum likelihood. We developed several scenarios to determine the effect of unknowns regarding transmission and immunity on estimates of the duration of immunity. In the various models, duration of immunity to NoV gastroenteritis was estimated at 4.1 (95% CI 3.2–5.1) to 8.7 (95% CI 6.8–11.3) years. Moreover, we calculated that children (<5 years) are much more infectious than older children and adults. If a vaccine can achieve protection for duration of natural immunity indicated by our results, its potential health and economic benefits could be substantial.Key words: Norovirus, modeling, mathematical model, immunity, incidence, vaccination, vaccine development, viruses, enteric infections, acute gastroenteritisNoroviruses (NoVs) are the most common cause of acute gastroenteritis (AGE) in industrialized countries. In the United States, NoV causes an estimated 21 million cases of AGE (1), 1.7 million outpatient visits (2), 400,000 emergency care visits, 70,000 hospitalizations (3), and 800 deaths annually across all age groups (4). Although the highest rates of disease are in young children, infection and disease occur throughout life (5), despite an antibody seroprevalence >50%, and infection rates approach 100% in older adults (6,7).Frequently cited estimates of the duration of immunity to NoV are based on human challenge studies conducted in the 1970s. In the first, Parrino et al. challenged volunteers with Norwalk virus (the prototype NoV strain) inoculum multiple times. Results suggested that the immunity to Norwalk AGE lasts from ≈2 months to 2 years (8). A subsequent study with a shorter challenge interval suggested that immunity to Norwalk virus lasts for at least 6 months (9). In addition, the collection of volunteer studies together demonstrate that antibodies against NoV may not confer protection and that protection from infection (serologic response or viral shedding) is harder to achieve than protection from disease (defined as AGE symptoms) (10–14). That said, most recent studies have reported some protection from illness and infection in association with antibodies that block binding of virus-like particles to histo-blood group antigen (HBGA) (13,14). Other studies have also associated genetic resistance to NoV infections with mutations in the 1,2-fucosyltransferase (FUT2) gene (or “secretor” gene) (15). Persons with a nonsecretor gene (FUT2−/−) represent as much as 20% of the European population. Challenge studies have also shown that recently infected volunteers are susceptible to heterologous strains sooner than to homotypic challenge, indicating limited cross-protection (11).One of many concerns with all classic challenge studies is that the virus dose given to volunteers was several thousand–fold greater than the small amount of virus capable of causing human illness (estimated as 18–1,000 virus particles) (16). Thus, immunity to a lower challenge dose, similar to what might be encountered in the community, might be more robust and broadly protective than the protection against artificial doses encountered in these volunteer studies. Indeed, Teunis et al. have clearly demonstrated a dose-response relationship whereby persons challenged with a higher NoV dose have substantially greater illness risk (16).Furthermore, in contrast with results of early challenge studies, several observations can be made that, when taken together, are inconsistent with a duration of immunity on the scale of months. First, the incidence of NoV in the general population has been estimated in several countries as ≈5% per year, with substantially higher rates in children (5). Second, Norwalk virus (GI.1) volunteer studies conducted over 3 decades, indicate that approximately one third of genetically susceptible persons (i.e., secretor-positive persons with a functional FUT2 gene) are immune (18,20,22). The point prevalence of immunity in the population (i.e., population immunity) can be approximated by the incidence of infection (or exposure) multiplied by the duration of immunity. If duration of immunity is truly <1 year and incidence is 5%, <5% of the population should have acquired immunity at any given time. However, challenge studies show population immunity levels on the order of 30%–45%, suggesting that our understanding of the duration of immunity is incomplete (8,11,17,18). HBGA–mediated lack of susceptibility may play a key role, but given the high seroprevalence of NoV antibodies and broad diversity of human HBGAs and NoV, HBGA–mediated lack of susceptibility cannot solely explain the discrepancy between estimates of duration of immunity and observed NoV incidence. Moreover, population immunity levels may be driven through the acquisition of immunity of fully susceptible persons or through boosting of immunity among those previously exposed.

Table 1

Summary of literature review of Norwalk virus volunteer challenge studies*

Evaluating Biosurveillance System Components using Multi-Criteria Decision Analysis

Objective

The use of Multi-Criteria Decision Analysis (MCDA) has traditionally been limited to the field of operations research, however many of the tools and methods developed for MCDA can also be applied to biosurveillance. Our project demonstrates the utility of MCDA for this purpose by applying it to the evaluation of data streams for use in an integrated, global biosurveillance system.

Introduction

The evaluation of biosurveillance system components is a complex, multi-objective decision that requires consideration of a variety of factors. Multi-Criteria Decision Analysis provides a methodology to assist in the objective analysis of these types of evaluation by creating a mathematical model that can simulate decisions. This model can utilize many types of data, both quantitative and qualitative, that can accurately describe components. The decision-maker can use this model to determine which of the system components best accomplish the goals being evaluated. Before MCDA can be utilized effectively, an evaluation framework needs to be developed. We built a robust framework that identified unique metrics, surveillance goals, and priorities for metrics. Using this framework, we were able to use MCDA to assist in the evaluation of data streams and to determine which types would be of most use within a global biosurveillance system.

Methods

MCDA was implemented using the Logical Decisions® software. The construction of the evaluation framework was carried out in several steps: identification and definition of data streams, metrics and surveillance goals, and the determination of the relative importance of each metric to the respective surveillance goal being evaluated. Sixteen data streams types were defined and identified for evaluation from a survey we conducted that collected over 200 surveillance products. A subject matter expert (SME) panel was assembled to help identify the biosurveillance goals and metrics in which to evaluate the data streams. To assign values for the metrics, we referenced properties of data streams used in currently operational systems.

Results

Our survey identified sixteen different classes of data streams: Ambulance Records, Clinic/Health Care Provider Records, ED/Hospital Records, Employment/School Records, Established Databases, Financial Records, Help Lines, Internet Search Queries, Laboraotry Records, News Aggregators, Official Reports, Police/Fire Department Records, Personal Communication, Prediction Markets, Sales, and Social Media.Four biosurveillance goals were identified: Early Warning of Health Threats, Early Detection of Health Events, Situational Awareness, and Consequence Management.Eleven metrics were identified: Accessibility, Cost, Credibility, Flexibility, Integrability, Geographic/Population Coverage, Granularity, Specificity of Detection, Sustainability, Time to Indication, and Timeliness.Using the framework, it was possible to use MCDA to rank the utility of each data stream for each goal.

Conclusions

The results suggest that a “one size fits all” approach does not work and that there is no ideal data stream that is most useful for each goal. Data streams that scored more highly for speed tended to rank more highly when the biosurveillance goal is early warning or early detection, whereas data streams that scored more highly for data credibility and geographic/population coverage ranked highly when the goal was situational awareness or consequence management. However, there are several data streams that rank consistently within the top 5 for each goal: Internet Search Queries, News Aggregators, Clinic/Health Care Provider records, ED/Hospital Records, and Laboratory Records and may be considered useful for integrated, global biosurveillance for infectious disease.

Table 1

Evaluation of Cholera and Other Diarrheal Disease Surveillance System,Niger State,Nigeria-2012

Objective

To determine how the cholera and other diarrheal disease surveillance system in Niger state is meeting its surveillance objectives, to evaluate its performance and attributes and to describe its operation to make recommendations for improvement.

Introduction

Cholera causes frequent outbreaks in Nigeria, resulting in mortality. In 2010 and 2011, 41,936 cases (case fatality rate [CFR]-4.1%) and 23,366 cases (CFR-3.2%) were reported (1). Reported cases in Nigeria by week 26, 2012 was 309 (CFR-1.29%) involving 20 Local Government Areas in 6 States. In Nigeria, there are currently eleven (11) States including Niger state at high risk for cholera/bloodless diarrhea outbreaks.In 2011, Niger state had 2472 cholera cases (CFR-2%) and 45,111 other diarrhea diseases cases, recorded in more than half of state Purpose of surveillance system is to ensure early detection of cholera and other diarrheal cases and to monitor trends towards evidence-based decision for management, prevention and control.

Methods

We conducted evaluation in July, 2012. We used CDC guideline on surveillance system evaluation (2001) as guide to assess operation, performance and attributes (2). We conducted key informant/in-depth interviews with stakeholders. We examined cholera action plans for preparedness and response, conducted laboratory assessment, extracted and analyzed cholera surveillance (2005–2012) for frequencies/proportions using Microsoft Excel. Thematic analysis was done for qualitative data. We shared findings with stakeholders at all levels.

Results

Surveillance system was setup for early detection and monitoring towards evidence-based decision. State government funds system. Case definition used is highly sensitive and is any patient aged 5 years or more who develops acute watery diarrhea, with/without vomiting. Though simple case definition, laboratory confirmation makes surveillance complex. A passive system, active during outbreaks; has formal and informal sources of information and part of Integrated Disease Surveillance and Response (IDSR) system and flow(fig.1). It takes 24–48 hours between outbreaks onset, confirmation and response.Line list showed undefined/poorly labeled outcomes. Of 2472 cases in 2011 1320 (49%) were found in line list. 2011 monthly data completeness was 75%. So far in 2012, 5(0.02%) of all diarrhea cases were cholera. System captures only age as sociodemographics.Of 11 suspected cholera cases tested during 2011 epidemic, 7 confirmed as cholera (PPV-63%). Of 3 rumours of cholera outbreaks (January 2011-July 2012), one (PPV-33%) was true. Acceptability of system is high among all stakeholders interviewed. Timeliness of monthly reporting was 68.7% (Performance attributesExcellent (>90%)Very good (80–89%)Good (70–79%)Average (60–69%)Fair (50–59%)Poor (<50%)Simplicity•Flexibility•Data Quality•Acceptability•Sensitivity•Positive Predictive Value•Representativeness•Timeliness•Stability•Open in a separate windowLaboratory can isolate Vibro cholerae isolation but has no Cary Blair transport medium and cholera rapid test kits.

Conclusions

Evaluation revealed that surveillance system is meeting its objectives by early detection and response to cholera outbreaks. System is simple, stable, flexible, sensitive with poor data quality, low PPV, fair laboratory capacity and moderate timeliness. We recommended electronic and internet-based reporting for timeliness and data quality improvement; and provision of laboratory consumables.Open in a separate window     

Nontraumatic Oral Health Classification for Alternative Use of Syndromic Data

Objective

To develop a nontraumatic oral health classification that could estimate the burden of oral health-related visits in North Carolina (NC) Emergency Departments (EDs) using syndromic surveillance system data.

Introduction

Lack of access to regular dental care often results in costly, oral health visits to EDs that could otherwise have been prevented or managed by a dentist (1). Most studies on oral health-related visits to EDs have used a wide range of classifications from different databases, but none have used syndromic surveillance data. The volume, frequency, and included details of syndromic data enabled timely burden estimates of nontraumatic oral health visits for NC EDs.

Methods

Literature review, input by subject matter experts (SMEs), and analysis of syndromic data was used to create the nontraumatic oral health classification. BioSense, a near real-time, national-level, electronic health surveillance system was the source of the NC ED syndromic data. Visits with at least one oral health-related ICD-9-CM code were extracted for NC fiscal years 2008–2010. Univariate analyses of chief complaint (CC) and final diagnosis data along with SME consultation were used to determine the CC substrings and ‘white list’ of ICD-9-CM codes used as inclusion criteria to classify visits as oral health-related. These analyses and consultations also determined the trauma-related codes and substrings used to exclude visits.

Results

Oral Health-Related ICD-9-CM CodesWhite List ICD-9-CM CodesOral Health-Related CC Substrings521.x780.60388.70Tooth and ache522.x305.1682.0Tooth and abscess523.x401.9786.2Tooth and pain525.x784.0478.19Tooth and abcess^**528.x^*784.2780.6Dental526.9Open in a separate windowx = includes all numbers under this ICD-9-CM subheading^*Except 528.3 and 528.5^**Most common misspelling of abscessIn summary, an ED visit had a nontraumatic oral health classification if it contained 1) an oral health-related CC substring with no trauma-related ICD-9-CM codes or CC substrings or 2) an oral health-related ICD-9 code accompanied by no oral health-related or trauma-related CC substrings and with no other diagnosis codes except for those on the whitelist.

Conclusions

There is increasing demand to determine ways to use syndromic surveillance data in an alternative way for population health surveillance. This use of BioSense data provided a practical classification of patient records for the tracking of nontraumatic oral health-related visits to NC EDs. Visit estimates created using this classification in combination with other pertinent information could prove useful to policymakers when deciding upon resource allocation aimed at reducing this unnecessary burden on the NC ED system. The large volume of records in syndromic surveillance systems offers substantial weight of evidence for alternative use in epidemiological studies; however, accurate classification of records is required to select cases of interest. While data volume precludes validation of every included record, a combination of human expertise and data analysis can provide credible classification criteria.     

Potential use of multiple surveillance data in the forecast of hospital admissions

Objective

This paper describes the potential use of multiple influenza surveillance data to forecast hospital admissions for respiratory diseases.

Introduction

A sudden surge in hospital admissions in public hospital during influenza peak season has been a challenge to healthcare and manpower planning. In Hong Kong, the timing of influenza peak seasons are variable and early short-term indication of possible surge may facilitate preparedness which could be translated into strategies such as early discharge or reallocation of extra hospital beds. In this study we explore the potential use of multiple routinely collected syndromic data in the forecast of hospital admissions.

Methods

A multivariate dynamic linear time series model was fitted to multiple syndromic data including influenza-like illness (ILI) rates among networks of public and private general practitioners (GP), and school absenteeism rates, plus drop-in fever count data from designated flu clinics (DFC) that were created during the pandemic. The latent process derived from the model has been used as a measure of the influenza activity [1]. We compare the cross-correlations between estimated influenza level based on multiple surveillance data and GP ILI data, versus accident and emergency hospital admissions with principal diagnoses of respiratory diseases and pneumonia & influenza (P&I).

Results

The estimated influenza activity has higher cross-correlation with respiratory and P&I admissions (ρ=0.66 and 0.73 respectively) compared to that of GP ILI rates (Surveillance dataCross correlations / lags^*−2−1012Respiratory disease admissionsGP ILI0.480.530.550.500.45Estimated influenza activity0.590.660.660.580.46P&I admissionsGP ILI0.570.620.650.590.49Estimated influenza activity0.660.730.730.650.53Open in a separate window^*negative lags refer to correlations between lagged surveillance data and hospital admissions

Conclusions

The use of a multivariate method to integrate information from multiple sources of influenza surveillance data may have the potential to improve forecasting of admission surge of respiratory diseases.     

Improving ILI Surveillance using Hospital Staff Influenza-like Absence (ILA)

Objective

To address the feasibility and efficiency of a novel syndromic surveillance method, monitoring influenza-like absence (ILA) among hospital staff, to improve national ILI surveillance and inform local hospital preparedness.

Introduction

Surveillance of influenza in the US, UK and other countries is based primarily on measures of influenza-like illness (ILI), through a combination of syndromic surveillance systems, however, this method may not capture the full spectrum of illness or the total burden of disease. Care seeking behaviour may change due to public beliefs, for example more people in the UK sought care for pH1N1 in the summer of 2009 than the winters of 2009/2010 and 2010/2011, resulting in potential inaccurate estimates from ILI (1). There may also be underreporting of or delays in reporting ILI in the community, for example in the UK those with mild illness are less likely to see a GP (2), and visits generally occur two or more days after onset of symptoms (3). Work absences, if the reason is known, could fill these gaps in detection.

Methods

Weekly counts and rates of hospital staff ILA (attributed to colds or influenza) were compared to GP ILI consultation rates (Royal College of General Practitioners Weekly Returns Service)(4) for 15–64 year olds, and positive influenza A test results (PITR) for all inpatients hospitalised in the three London hospitals for which staff data were collected using both retrospective time series and prospective outbreak detection methods implemented in the surveillance package in R (5)

Results

Rates of ILA were about six times higher than rates of ILI. Data on hospital staff ILA demonstrated seasonal trends as defined by ILI. Compared to the ILI rates, ILA demonstrated a more realistic estimate of the relative burden of pandemic H1N1 during July 2009 (1) (Figure). ILA provides potentially earlier warnings than GP ILI as indicated by its ability to predict ILI data for the local region (p < 0.001), as well as its potential for daily ‘real time’ updates. Using outbreak detection methods and examining peak weeks, alarms and thresholds, ILA alarmed, reached threshold rates and peaked consistently earlier or in the same week as ILI and PITR, with the exception of the July 2009, suggesting that it may be predictive of both community and patient cases of influenza (Open in a separate windowFigure:Weekly counts of ILA among hospital staff (blue), PITR among hospital patients (orange), and ILI in the community (red) from April 2008 to March 2011 and prospective alarms for elevated counts (circles) using a Bayesian subsystem algorithm, using the previous six weeks as the reference for prediction. Data plotted by counts rather than rates for clarity.

Table:

Week of the year that alarms commenced and peaks were reached for each of the four official influenza events from March 2008 to April 2011.

Serum Zinc Concentration and Acute Diarrhea in Children from Different Regions of Uzbekistan

Objective

To study the blood serum zinc concentration in children with acute diarrhea (AD) in in-patient facilities before and after therapy.

Introduction

There are several reports of zinc deficiency in pathogenesis of acute and chronic diarrhea. The literature review showed children with diarrhea and chronic gastroduodenitis performed zinc deficiency in majority of cases (1). The normal values of zinc in blood serum are 12.8–27.8 μmol/l (2). There is a threshold of 13μmol/l zinc concentration for zinc deficiency diagnosis. The zinc level 8.2 μmol/l and below is poor prognostic criteria (3).

Methods

Totally 102 children (1–14 years old) with AD in in-patient facility from different regions were studied for serum zinc concentration before and after treatment. Termez and Saraosie cities are located in south of Uzbekistan, in the region with high negative impact from the nearly Tajikistan located aluminum producing plant. The serum zinc level measured by neutron-activation method in the Institute of Nuclear Research (INR).

Results

The zinc concentration in serum significantly varied by the region (CitynZinc concentration, μmol/l (mean ± SD)Before treatmentAfter treatmentTashkent city (captial)3613.8±1.512.5±1.3Termez city409.1±0.087.47±0.01Saraosie city267.9±0.37.5±0.8Open in a separate windowThe level of zinc in children from Tashkent estimated at lower normal limit with reduction below normal values after treatment. Children from Termez during admission to the in-patient facilities were zinc deficient with further reduction to the poor prognostic level. Children in Saraosie admitted to the in-patient with significant zinc deficiency that remained on poor prognostic level after treatment.

Conclusions

The study results may indicate the treatment of AD in children do not replenish the zinc to the appropriate level. Though some confounding factors may contribute the observed zinc disorders the results may indicate environmental factors, such as pollution by aluminum producing plant emission to contribute the difference in zinc concentration and should be considered for the correction and treatment of AD in children.     

Time of Arrival Analysis in NC DETECT to Find Clusters of Interest from Unclassified Patient Visit Records

Objective

To describe a collaboration with the Johns Hopkins Applied Physics Laboratory (JHU APL), the North Carolina Division of Public Health (NC DPH), and the UNC Department of Emergency Medicine Carolina Center for Health Informatics (CCHI) to implement time-of-arrival analysis (TOA) for hospital emergency department (ED) data in NC DETECT to identify clusters of ED visits for which there is no pre-defined syndrome or sub-syndrome.

Introduction

TOA identifies clusters of patients arriving to a hospital ED within a short temporal interval. Past implementations have been restricted to records of patients with a specific type of complaint. The Florida Department of Health uses TOA at the county level for multiple sub-syndromes (1). In 2011, NC DPH, CCHI and CDC collaborated to enhance and evaluate this capability for NC DETECT, using NC DETECT data in BioSense 1.0 (2). After this successful evaluation based on exposure complaints, discussions were held to determine the best approach to implement this new algorithm into the production environment for NC DETECT. NC DPH was particularly interested in determining if TOA could be used for identifying clusters of ED visits not filtered by any syndrome or sub-syndrome. In other words, can TOA detect a cluster of ED visits relating to a public health event, even if symptoms from that event are not characterized by a predefined syndrome grouping? Syndromes are continuously added to NC DETECT but a syndrome cannot be created for every potential event of public health concern. This TOA approach is the first attempt to address this issue in NC DETECT. The initial goal is to identify clusters of related ED visits whose keywords, signs and/or symptoms are NOT all expressed by a traditional syndrome, e.g. rash, gastrointestinal, and flu-like illnesses. The goal instead is to identify clusters resulting from specific events or exposures regardless of how patients present – event concepts that are too numerous to pre-classify.

Methods

In late 2011, NC DPH and JHU APL signed a Software License Agreement and soon thereafter CCHI received the TOA software package. In May 2012, the TOA controller was adapted and set up to run against ED visit data for all NC DETECT hospitals. The TOA looks for clusters in all ED visits by hospital based solely on arrival time in both 30-minute and 60-minute intervals. There is no pre-classification of the chief complaints or triage notes into syndromes. TOA alerts are viewable on the NC DETECT Web application and, as of August 2012, users are able to document any actions taken on these alerts.

Results

From April 15, 2012 to July 31, 2012, TOA generated 173 alerts across all 115 hospitals reporting to NC DETECT. The TOA identified a group of scabies-related ED visits that was not captured in another syndrome. The TOA also identified clusters identified by hospitals as disaster-related which included misspellings that had not been previously identified, e.g. “diaster” and “disater,” as well as events involving out-of-town groups that will not be identified spatially (Cluster Event TypeNumber of ED Visits in TOA alert / Number of visits related to the specific event clusterSample chief complaints in clusterDisaster-related13/9Disaster abd pain, diaster flu, disaster phyc, diaster anxity, diaster blackScabies16/8Scabies 7FER, Scaobes 7FER, 5M FOLLOW UPSame out of town location17/7Cough, laceration(s)Open in a separate window

Conclusions

Our preliminary review of TOA shows that this algorithm approach can be helpful for identifying clusters of ED visits that are not captured by existing syndromes and can be used to identify hospital coding schemes for disaster events. The TOA will continue to be monitored in our production environment and evaluated for additional effectiveness. We will also explore tools that will display counts of terms within a TOA alert to assist in signal investigation.     

Evaluating the Variation on Public Health’s Perceived Field Need of Communicable Disease Reports

Objective

To assess communicable disease report fields required by public health practitioners and evaluate the variation in the perceived utility of these fields.

Introduction

Communicable disease surveillance is a core Public Health function. Many diseases must be reported to state and federal agencies (1). To manage and adjudicate such cases, public health stakeholders gather various data elements. Since cases are identified in various healthcare settings, not all information sought by public health is available (2) resulting in varied field completeness, which affects the measured and perceived data quality. To better understand this variation, we evaluated public health practitioners’ perceived value of these fields to initiate or complete communicable disease reports.

Methods

We chose four diseases: Histoplasmosis, acute hepatitis B, hepatitis C and salmonella. We asked public health practitioners from Mar-ion County Health Department (MCHD) of Indianapolis to list the fields they felt were necessary when submitting a communicable disease report. We then asked them to evaluate those fields using the following criteria:Required – A critical case attribute, when missing or unknown, would make the task of initiating and/or closing a case impossible or exceedingly difficult.Desired – A case attribute allowing more complete epidemiologic profiles to be developed but, if missing, would not prohibit initiating and/or closing a case.Not applicable – A case attribute that is not usually collected to initiate and/or close a case for the particular condition.To quantify the need for the fields, we assigned a number to each response as follows:0 - Not applicable 1 - Desired  2- RequiredWe summed the numbers for each field for each disease and created a table for the perceived need of that field (The perceived need for the fieldFUNGALHEPATITISENTERICHistoplasmosisHepatitis B-acuteHepatitis CSalmonellaField NameInitiateCloseInitiateCloseInitiateCloseInitiateCloseSumDisease8888888864First Name8888888864Middle Initial4545454536Last Name8888888864Parent – First Name3434343428Parent – Middle Initial111111118Parent – Last Name3434343428Phone Number7778787859Street Number7878787860Street Name7878787860City7878787860Zip Code5757575748Country7878787860Date of Birth8888888864Age3434343428Sex6868686856Pregnant3478453438Race4848484848Ethnicity4747474744Health Care Worker45333357Food Service WorkerSchool (student/staff)Day care (attendee/staff)33Name of School/Day Care4511114724Part of and Outbreak4544344634Etiologic Agent7878787860Site of Infection6800006828Date of Diagnosis (m, d, y)5767676751Symptoms associated with infection5868575852If yes to Symptom: Onset Date (m, d, y)5768575750If yes to Symptoms: Pertinent Symp/Signs5858575750If yes to Symptoms: Died? (y/n)6868677856Lab tests(s) and results(s)8888888864Lab tests(s)/results(s) Date7878787860Treatment (name of antibiotic)4811115728Dosage4511114421Date initiated4511114623Antibiotic Resistance (y, n, nd)1100004511If yes, what antibiotic?000000347Reporting Facility Code2323232320If Hospital, Name Hospital7878787860Name of Physician and Address7878787860Record Number5656565644Person Reporting (other than physician)5656565644Telephone Number7777777756Telephone Number (2)2222222216Date of Report6666666648TOTAL (max possible 360)223273210248204241233285Open in a separate window

Results

The perceived needs table showed a difference between the fields needed to initiate or close a case. Moreover the perceived need for fields varied by disease as well. To assess the difference in perceived needs, we calculated the standard deviation of the fields (The Disagreement for the fieldFUNGALHEPATITISENTERICHistoplasmosisHepatitis B-acuteHepatitis CSalmonellaField NameInitiateCloseInitiateCloseInitiateCloseInitiateCloseSumDisease0.000.000.000.000.000.000.000.000.00First Name0.000.000.000.000.000.000.000.000.00Middle Initial0.000.430.000.430.000.430.000.431.73Last Name0.000.000.000.000.000.000.000.000.00Parent – First Name0.500.000.500.000.500.000.500.002.00Parent – Middle Initial0.500.500.500.500.500.500.500.504.00Parent – Last Name0.500.000.500.000.500.000.500.002.00Phone Number0.430.430.430.000.430.000.430.002.17Street Number0.430.430.430.000.430.000.430.001.73Street Name0.430.000.430.000.430.000.430.001.73City0.430.000.430.000.430.000.430.001.73Zip Code0.430.430.430.430.430.430.430.433.46County0.430.000.430.000.430.000.430.001.73Date of Birth0.000.000.000.000.000.000.000.000.00Age0.430.710.430.710.430.710.430.714.56Sex0.500.000.500.000.500.000.500.002.00Pregnant0.430.710.430.000.000.430.430.713.15Race0.000.000.000.000.000.000.000.000.00Ethnicity0.000.430.000.430.000.430.000.431.73Health Care Worker0.000.430.430.430.430.430.430.433.03Food Service WorkerSchool (student/staff)Day Care (attendee/staff)Name of School/Day Care0.000.430.430.430.430.430.000.432.60Part of an Outbreak0.000.430.000.000.430.710.000.502.07Etiologic Agent0.430.000.430.000.430.000.430.001.73Site of Infection0.500.000.000.000.000.000.500.001.00Date of Diagnosis (m, d, y)0.430.430.500.430.500.430.500.433.67Symptoms associated with infection0.430.000.500.000.430.430.430.002.23If yes to Symptoms: Onset Date (m, d, y)0.430.430.500.000.430.430.430.433.10If yes to Symptoms: Pertinent Symp/Signs0.430.000.430.000.430.430.430.432.60If yes to Symptoms: Died? (y/n)0.500.000.500.000.500.430.430.002.37Lab test(s) and result(s)0.000.000.000.000.000.000.000.000.00Lab test(s)/result(s) Date0.430.000.430.000.430.000.430.001.73Treatment (name of antibiotic)0.000.000.430.430.430.430.430.432.60Dosage0.000.430.430.430.430.430.000.002.17Date initiated0.000.430.430.430.430.430.000.502.67Antibiotic Resistance (y, n, nd)0.430.430.000.000.000.000.000.431.30If yes, what antibiotic?0.000.000.000.000.000.000.000.470.47Reporting Facility Code0.470.820.470.820.470.820.470.825.15If Hospital, Name Hospital0.430.000.430.000.430.000.430.001.73Name of Physician and Address0.430.000.430.000.430.000.430.001.73Record Number0.430.500.430.500.430.500.430.503.73Person Reporting (other than physician)0.430.500.430.500.430.500.430.503.73Telephone Number0.430.430.430.430.430.430.430.433.46Telephone Number (2)0.000.000.000.000.000.000.000.000.00Date of Report0.500.500.500.500.500.500.500.504.00Open in a separate window

Conclusions

Data quality is essential, not only for research but to support routine public health practice as well. Many factors affect data quality; one of them is perceived need of the information by Public Health Practitioners. Despite working with public health stakeholders from the same organization we observed variation in their perceived needs for these fields to initiate or close a communicable case. These results highlight another source of the problem regarding health information quality and its goodness of fit issues.     

A System for Surveillance Directly from the EMR

Objective

Our objective was to conduct surveillance of nosocomial infections directly from multiple EMR data streams in a large multi-location Canadian health care facility. The system developed automatically triggers bed-day-level-location-aware reports and detects and tracks the incidents of nosocomial infections in hospital by ward.

Introduction

Hospital acquired infections are a major cause of morbidity, mortality and increased resource utilization. CDC estimates that in the US alone, over 2 million patients are affected by nosocomial infections costing approximately $34.7 billion to $45 billion annually (1). The existing process of detection and reporting relies on time consuming manual processing of records and generation of alerts based on disparate definitions that are not comparable across institutions or even physicians.

Methods

A multi-stakeholder team consisting of experts from medicine, infection control, epidemiology, privacy, computing, artificial intelligence, data fusion and public health conducted a proof of concept from four complete years of admission records of all patients at the University of Ottawa Heart Institute. Figure 1 lists the data elements investigated. Our system uses an open source enterprise bus ‘Mirth Connect’ to receive and store data in HL7 format. The processing of information is handled by individual components and alerts are pushed back to respective locations. The free text components were classified using natural language processing. Negation detection was performed using NegEx (2). Data-fusion algorithms were used to merge information to make it meaningful and allow complex syndrome definitions to be mapped onto the data.

Results

The system monitors: Ventilator Associated Pneumonia (VAP), Central Line Infections (CLI), Methicillin Resistant Staph Aureus (MRSA), Clostridium difficile (C. Diff) and Vancomycin resistant Enterococcus (VRE).21452 hospital admissions occurred in 17670 unique patients over four years. There were 41720 CXRs performed in total, of which 10546 were classified as having an infiltrate. 4575 admissions were associated with at least one CXR showing an infiltrate, 2266 of which were hospital-acquired. Hospital acquired infiltrates were associated with an increased hospital mortality (6.3% vs 2.6%)* and length of stay (19.5 days vs 6.5 days)*. 253 patients had at least one positive blood culture. This was also associated with an increased hospital mortality (23,3% vs. 2.8%)* and length of stay (10.8 vs 40.9 days)*. (* all p values < 0.00001)

Conclusions

This proof of concept system demonstrates the capability of monitoring and analyzing multiple available data streams to automatically detect and track infections without the need for manual data capture and entry. It acquires directly from the EMR data to identify and classify health care events, which can be used to improve health outcomes and costs. The standardization of definitions used for detection will allow for generalization across institutions.

Applying Zero-inflated Mixed Model to School Absenteeism Surveillance in Rural China

Objective

To describe and explore the spatial and temporal variability via ZIMM for absenteeism surveillance in primary school for early detection of infectious disease outbreak in rural China.

Introduction

Absenteeism has great advantages in promoting the early detection of epidemics1. Since August 2011, an integrated syndromic surveillance project (ISSC) has been implemented in China2. Distribution of the absenteeism generally are asymmetry, zero inflation, truncation and non-independence3. For handling these encumbrances, we should apply the Zero-inflated Mixed Model (ZIMM).

Methods

Data for this study was obtained from the web-based data of ISSC in 62 primary schools in two counties of Jiangxi province, China from April 1th, 2012 to June 30st, 2012. The ZIMM was used to explore: 1)the temporal and spatial variability regarding occurrence and intensity of absenteeism simultaneously, and 2) the heterogeneity among the reporting primary schools by introducing random effects into the intercepts. The analyse was processed in the SAS procedure NLMIXED4.

Results

The total 4914 absenteeism events were reported in the 62 primary schools in the study period. The rate of zero report was 49.88% (Fig. 1). According to ZIMM, there are fixed and random effect parameters in this model (Open in a separate windowFig. 1Absenteeism from Apr. 1st to Jun. 30th 2012

Table 1

Fixed parameters and variance components estimates for the absenteeism using ZIMM

Detection of Patients with Influenza Syndrome Using Machine-Learning Models Learned from Emergency Department Reports

Objective

Compare 7 machine learning algorithms with an expert constructed Bayesian network on detection of patients with influenza syndrome.

Introduction

Early detection of influenza outbreaks is critical to public health officials. Case detection is the foundation for outbreak detection. Previous study by Elkin el al. demonstrated that using individual emergency department (ED) reports can better detect influenza cases than using chief complaints [1]. Our recent study using ED reports processed by Bayesian networks (using expert constructed network structure) showed high detection accuracy on detection of influenza cases [2].

Methods

The dataset used in this study includes 182 ED reports with confirmed PCR influenza tests (Jan 1, 2007–Dec 31, 2009) and 40853 ED reports as control cases from 8 EDs in UPMC (Jul 1, 2010–Aug 31, 2010). All ED reports were deidentified by De-ID software with IRB approval.An NLP system, Topaz, was used to extract relevant findings and symptoms from the reports and encoded them with the UMLS concept unique identifier codes [2]. Two subsets were created: DS1-train (67% of cases) and DS1-test (remaining 33%).The algorithms used for training the models are: Naïve Bayes Classifier, Efficient Bayesian Multivariate Classification (EBMC) [3], Bayesian Network with K2 algorithm, Logistic Regression (LR), Support Vector Machine (SVM), Artificial Neural Networks (ANN) and Random Forest (RF).The predictive performance of each method was evaluated using the area under the receiver operator characteristic (AUROC) and the Hosmer-Lemeshow (HL) statistical significance testing, that describes the lack-of-fit of the model to the dataset.

Results

The evaluation results of all the models using DS1-test, including the AUROC, its confidence interval, p-value (between each algorithm and the expert) and the calibration with HL are shown in ConclusionsAll models achieved high AUROC values. The pairwise comparison of p-values in Figure 1.One limitation of the study is that the test dataset has low influenza prevalence, which may bias the detection algorithm performance. We are in the process of testing the algorithms using higher prevalence rate.The same process could also be applied to other diseases to further research the generalizability of our method.Predictive performance and Calibration