Assessing the Validity and Reliability of Three Indicators Self-Reported on the Pregnancy Risk Assessment Monitoring System Survey

Objectives

We investigated the reliability and validity of three self-reported indicators from the Pregnancy Risk Assessment Monitoring System (PRAMS) survey.

Methods

We used 2008 PRAMS (n=15,646) data from 12 states that had implemented the 2003 revised U.S. Certificate of Live Birth. We estimated reliability by kappa coefficient and validity by sensitivity and specificity using the birth certificate data as the reference for the following: prenatal participation in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC); Medicaid payment for delivery; and breastfeeding initiation. These indicators were examined across several demographic subgroups.

Results

The reliability was high for all three measures: 0.81 for WIC participation, 0.67 for Medicaid payment of delivery, and 0.72 for breastfeeding initiation. The validity of PRAMS indicators was also high: WIC participation (sensitivity = 90.8%, specificity = 90.6%), Medicaid payment for delivery (sensitivity = 82.4%, specificity = 85.6%), and breastfeeding initiation (sensitivity = 94.3%, specificity = 76.0%). The prevalence estimates were higher on PRAMS than the birth certificate for each of the indicators except Medicaid-paid delivery among non-Hispanic black women. Kappa values within most subgroups remained in the moderate range (0.40–0.80). Sensitivity and specificity values were lower for Hispanic women who responded to the PRAMS survey in Spanish and for breastfeeding initiation among women who delivered very low birthweight and very preterm infants.

Conclusion

The validity and reliability of the PRAMS data for measures assessed were high. Our findings support the use of PRAMS data for epidemiological surveillance, research, and planning.The Pregnancy Risk Assessment Monitoring System (PRAMS) is one of the largest state-based surveillance systems of women with live births documenting experiences before, during, and after pregnancy. The data from PRAMS surveys are linked to the birth certificate information. The birth certificate is an important source of data for examining infant health at the state and national levels and is used widely.^1–3 Many studies have examined the reliability and validity of data from the U.S. birth certificate and other data sources such as postpartum surveys, medical records, and registry data.^1–19 Some indicators on the birth certificate are found to be more reliable than others, such as maternal demographics, insurance, and birthweight.^{4,5,7,13,15,19} Birth certificate data have been reported to have lower sensitivity for tobacco and alcohol use;^6,10 birth defects;²⁰ prenatal care;¹¹ and maternal weight gain, medical risks, and obstetric complications,^2,8,9,12,13 although the sensitivity varies by subgroups of women and birth outcomes for many items.^16,17Because PRAMS data are self-reported, it is important to examine the reliability and validity with other population-based data-collection systems such as the birth certificate, which is an established system mandated to assess the health of mothers and babies in the U.S. According to state laws, each live birth must be registered and a birth certificate must be filed. Federal law mandates the national collection and publication of birth certificate data.²¹ A new electronic birth certificate reporting standard was implemented in 2003, replacing the version that had been in place since 1989.²² The 2003 version updated the birth certificate by revising or dropping items and adding new elements with detailed guidance provided on coding data.²³States participating in the Centers for Disease Control and Prevention''s (CDC''s) PRAMS project sample women with recent live-born deliveries using the state birth certificate files. Selected data from the state birth certificate file are linked to the PRAMS survey of women with live-born infants. Some of the measures on the new birth certificate reporting standard are the same as or similar to those on the PRAMS survey. Several studies have compared the validity of measures on PRAMS with birth certificate data, and this study examines additional indicators that had not been assessed previously.^7,18Little has been available on the validity and reliability of information on participation in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) during pregnancy; Medicaid payment for delivery; and breastfeeding initiation in the early postpartum period, comparing PRAMS data with other data sources. WIC and Medicaid are important sources of services for low-income women and, because these data are used to examine research and programming questions, we investigated the reliability and validity of these indicators. The PRAMS data were compared with the birth certificate as the standard because the comparison of PRAMS data might provide insight about improving the accuracy and quality of PRAMS data. In addition, this investigation may provide insight into using both sources of data in concert to maximize the accuracy of information on women and children. Two research questions guided this investigation: (1) What degree of agreement/reliability exists between PRAMS survey responses and the birth certificate on participation in prenatal WIC services, Medicaid payment for delivery, and breastfeeding initiation? and (2) Do the reliability and validity vary by selected maternal characteristics and birth outcomes as suggested by previous research?¹⁷