Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials

Integrative Data Analysis (IDA) encompasses a collection of methods for data synthesis that pools participant-level data across multiple studies. Compared with single-study analyses, IDA provides larger sample sizes, better representation of participant characteristics, and often increased statistical power. Many of the methods currently available for IDA have focused on examining developmental changes using longitudinal observational studies employing different measures across time and study. However, IDA can also be useful in synthesizing across multiple randomized clinical trials to improve our understanding of the comprehensive effectiveness of interventions, as well as mediators and moderators of those effects. The pooling of data from randomized clinical trials presents a number of methodological challenges, and we discuss ways to examine potential threats to internal and external validity. Using as an illustration a synthesis of 19 randomized clinical trials on the prevention of adolescent depression, we articulate IDA methods that can be used to minimize threats to internal validity, including (1) heterogeneity in the outcome measures across trials, (2) heterogeneity in the follow-up assessments across trials, (3) heterogeneity in the sample characteristics across trials, (4) heterogeneity in the comparison conditions across trials, and (5) heterogeneity in the impact trajectories. We also demonstrate a technique for minimizing threats to external validity in synthesis analysis that may result from non-availability of some trial datasets. The proposed methods rely heavily on latent variable modeling extensions of the latent growth curve model, as well as missing data procedures. The goal is to provide strategies for researchers considering IDA.

     

Family Functioning in Hispanic Parents of Adolescents: Who Benefits Most from a Family-Based HIV and Substance Use Preventive Intervention?

Prevention Science - To understand which families are likely to benefit most from resource-intensive family-based, evidence-based interventions (EBIs), we must examine the key, modifiable...     

Reducing Internalizing Symptoms Among High-Risk,Hispanic Adolescents: Mediators of a Preventive Family Intervention

Familias Unidas is a family-focused preventive intervention that has been found to reduce drug use and sexual risk behaviors among Hispanic adolescents. In some trials, Familias Unidas has also been found to be efficacious in reducing adolescent internalizing symptoms (i.e., depressive and anxiety symptoms), even though the intervention did not specifically target internalizing symptoms. This study examines potential mediators or mechanisms by which Familias Unidas influences internalizing symptoms, specifically the role of intervention-targeted improvements in parent-adolescent communication and reductions in youth externalizing behaviors. A total of 213 Hispanic eighth grade students with a history of externalizing behavior problems and their primary caregivers were recruited from the public school system. Participants, with a mean age of 13.8 years, were randomized into the Familias Unidas intervention or community practice control condition and assessed at baseline, 6, 18, and 30 months post-baseline. A cascading mediation model was tested in which the Familias Unidas intervention was hypothesized to decrease adolescent internalizing symptoms through two mediators: improvements in parent-adolescent communication leading to decreases in externalizing behaviors. Findings show that the intervention had significant direct effects on youth internalizing symptoms at 30 months post-baseline. In addition, the cascading mediation model was supported in which the Familias Unidas intervention predicted significant improvements in parent-adolescent communication at 6 months, subsequently decreasing externalizing behaviors at 18 months, and ultimately reducing youth internalizing symptoms at 30 months post-baseline. Implications for prevention interventions are discussed.     

Optimizing the US-AUDIT for alcohol screening in U.S. college students

This study examined cutoff scores on the new (2014) US-AUDIT (Alcohol Use Disorders Identification Test), adapted for U.S. standard drinks. No studies have examined optimal cutoff scores on the US-AUDIT for college students. 250 undergraduates (65% men) completed the US-AUDIT. At-risk drinkers reported at least four binge drinking episodes per week. Likely alcohol use disorder was assessed with a self-report diagnostic measure. Using the Youden method, the ideal cutoff to identify at-risk drinkers for the US-AUDIT was 5 for men (sensitivity = .93, specificity = .96) and 6 for women (sensitivity = .77, specificity = .86); and to identify likely alcohol use disorder was 13 for men (sensitivity = .69, specificity = .81) and 8 for women (sensitivity = .83, specificity = .80). Cutoffs were lower than the original AUDIT. Different US-AUDIT cutoffs for men and women should be used for likely alcohol use disorder, which may reflect differences in drinking quantity and frequency. Empirical guidelines for alcohol screening with the new US-AUDIT may be used to enhance research or identification of at-risk drinkers in college settings, or for college students in primary care or other health care settings.     

Two-Year Impact of Prevention Programs on Adolescent Depression: an Integrative Data Analysis Approach

This paper presents the first findings of an integrative data analysis of individual-level data from 19 adolescent depression prevention trials (n = 5210) involving nine distinct interventions across 2 years post-randomization. In separate papers, several interventions have been found to decrease the risk of depressive disorders or elevated depressive/internalizing symptoms among youth. One type of intervention specifically targets youth without a depressive disorder who are at risk due to elevated depressive symptoms and/or having a parent with a depressive disorder. A second type of intervention targets two broad domains: prevention of problem behaviors, which we define as drug use/abuse, sexual risk behaviors, conduct disorder, or other externalizing problems, and general mental health. Most of these latter interventions improve parenting or family factors. We examined the shared and unique effects of these interventions by level of baseline youth depressive symptoms, sociodemographic characteristics of the youth (age, sex, parent education, and family income), type of intervention, and mode of intervention delivery to the youth, parent(s), or both. We harmonized eight different measures of depression utilized across these trials and used growth models to evaluate intervention impact over 2 years. We found a significant overall effect of these interventions on reducing depressive symptoms over 2 years and a stronger impact among those interventions that targeted depression specifically rather than problem behaviors or general mental health, especially when baseline symptoms were high. Implications for improving population-level impact are discussed.

     

Patterns of Substance Use and Arrest Histories Among Hospitalized HIV Drug Users: A Latent Class Analysis

Using baseline data from the NIDA Clinical Trials Network 0049 study (Project HOPE), we performed latent class analyses (LCA) to identify discrete classes, or clusters, of people living with HIV (PLWH) based on their past year substance use behaviors and lifetime arrest history. We also performed multinomial logistic regressions to identify key characteristics associated with class membership. We identified 5 classes of substance users (minimal drug users, cocaine users, substantial cocaine/hazardous alcohol users, problem polysubstance users, substantial cocaine/heroin users) and 3 classes of arrest history (minimal arrests, non-drug arrests, drug-related arrests). While several demographic variables such as age and being Black or Hispanic were associated with class membership for some of the latent classes, participation in substance use treatment was the only covariate that was significantly associated with membership in all classes in both substance use and arrest history LCA models. Our analyses reveal complex patterns of behaviors among substance using PLWH and suggest that HIV intervention strategies may need to take into consideration such nuanced differences to better inform future studies and program implementation.     

Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials

Integrative Data Analysis (IDA) encompasses a collection of methods for data synthesis that pools participant-level data across multiple studies. Compared with single-study analyses, IDA provides larger sample sizes, better representation of participant characteristics, and often increased statistical power. Many of the methods currently available for IDA have focused on examining developmental changes using longitudinal observational studies employing different measures across time and study. However, IDA can also be useful in synthesizing across multiple randomized clinical trials to improve our understanding of the comprehensive effectiveness of interventions, as well as mediators and moderators of those effects. The pooling of data from randomized clinical trials presents a number of methodological challenges, and we discuss ways to examine potential threats to internal and external validity. Using as an illustration a synthesis of 19 randomized clinical trials on the prevention of adolescent depression, we articulate IDA methods that can be used to minimize threats to internal validity, including (1) heterogeneity in the outcome measures across trials, (2) heterogeneity in the follow-up assessments across trials, (3) heterogeneity in the sample characteristics across trials, (4) heterogeneity in the comparison conditions across trials, and (5) heterogeneity in the impact trajectories. We also demonstrate a technique for minimizing threats to external validity in synthesis analysis that may result from non-availability of some trial datasets. The proposed methods rely heavily on latent variable modeling extensions of the latent growth curve model, as well as missing data procedures. The goal is to provide strategies for researchers considering IDA.     

Preventing Youth Internalizing Symptoms Through the Familias Unidas Intervention: Examining Variation in Response

Prevention programs that strengthen parenting and family functioning have been found to reduce poor behavioral outcomes in adolescents, including substance use, HIV risk, externalizing and internalizing problems. However, there is evidence that not all youth benefit similarly from these programs. Familias Unidas is a family-focused intervention designed to prevent substance use and sexual risk among Hispanic youth and has recently demonstrated unanticipated reductions in internalizing symptoms for some youth. This paper examines variation in intervention response for internalizing symptoms using individual-level data pooled across four distinct Familias Unidas trials: (1) 266 eighth grade students recruited from the general school population; (2) 160 ninth grade students from the general school population; (3) 213 adolescents with conduct, aggression, and/or attention problems; and (4) 242 adolescents with a delinquency history. Causal inference growth mixture modeling suggests a three-class model. The two largest classes represent youth with low (60 %) and medium (27 %) internalizing symptoms at baseline, and both intervention and control participants show reductions in internalizing symptoms. The third class (13 %) represents youth with high levels of baseline internalizing symptoms who remain at steady levels of internalizing symptoms when exposed to the intervention, but who experience an increase in symptoms under the control condition. Female gender, low baseline levels of parent–adolescent communication, and older age were associated with membership in the high-risk class. These synthesis analyses involving a large sample of youth with varying initial risk levels represent a further step toward strengthening our knowledge of preventive intervention response and improving preventive interventions.