Program Evaluation Strategies for Substance Abuse Prevention

Substance abuse prevention has grown from a grassroots movement and a topic of marginal academic interest into full-fledged technology. Central to the success of this technology has been the development of evaluation methods for documenting the effectiveness of programs and policies. This paper reviews the state-of-the-art in program evaluation for substance abuse prevention efforts. Included in this review is a discussion of approaches to process, outcome, and impact evaluation. Evaluation designs are reviewed with attention given to topics such as recruitment and retention of sites and participants, defining interventions, data collection, controlling for rival explanations, and data analysis and reporting.     

Prevention of Substance Abuse in the Workplace: Review of Research on the Delivery of Services

With the growing recognition that most illicit drug users and heavy drinkers are members of the work force, the workplace has become an increasingly significant, though still underutilized, vehicle for the delivery of substance abuse prevention services. This paper discusses some of the chief reasons for engaging in substance abuse prevention in the workplace; outlines the foundations of workplace prevention services; and reviews recent research on workplace substance abuse prevention, including the major preventive interventions aimed at the workplace environment and the individual worker. The paper closes with a discussion of future programming and research on the delivery of workplace prevention services.     

Workplace Drug Abuse Prevention Initiatives: A Review

Likely key elements of effective prevention strategies emerging from the literature (and from discussions with experts) are summarized. These include: (1) program content based on proven prevention theory and research; (2) comprehensive approaches that address multiple risk and protective factors, and generalize across settings; (3) material that is relevant during important transitions in an employee's life and career (4) sensitivity to the culture of the workplace and community; (5) sufficient dosage and follow-up; (6) use of interactive teaching techniques; (7) training for prevention program providers; and (8) evaluation to know that the intervention had the desired effect on behavior. Evaluation of workplace substance abuse prevention has been limited; the need for future research is discussed. The paper concludes with a discussion of how to increase interest in substance abuse prevention by employers.     

The Prevention of Substance Abuse Among Construction Workers: A Field Test of a Social-Cognitive Program

A substance abuse prevention program based on social-cognitive principles and cast in a health promotion framework was tested on 374 construction workers from 5 sites. Subjects were randomly assigned to receive the Power Tools program with substance abuse material or without the material. Both self-report and bioassay data were collected before and approximately one year after the program. Data were analyzed mainly by analysis of covariance, analysis of variance, and chi-square analysis. Program effects were not demonstrated on consumption of substances, but were found on stages-of-change measures of smoking and drinking and on the frequency with which parents talk with their children about alcohol and drugs. The results suggest that a program of this type might help construction workers move toward reductions in licit substance use, but not in illicit drug use.     

Implementation and Impact of a Family-Based Substance Abuse Prevention Program in Rural Communities

A family-based alcohol, tobacco, and other drug abuse prevention program was evaluated. The program targeted families with students entering middle or junior high school. The goals of the program were to increase resiliency and protective factors including family cohesion, communication skills, school attachment, peer attachment, and appropriate attitudes about alcohol and tobacco use by adolescents. The Families In Action program is a structured program which includes six 2 1/2 hour sessions, offered once a week for six consecutive weeks to parents and youth. The program was offered to all eligible families in eight rural school districts. Families who chose to participate began the program with lower scores on several protective factors as compared to nonparticipating families. Analysis of covariance controlling for initial differences found several positive effects of program participation at the one year follow-up. The results were strongest for boys. These findings suggest that providing parents and youth with similar communication skills can be an effective approach to substance abuse prevention.     

Characteristics of Effective School-Based Substance Abuse Prevention

Prevention Science - This study summarizes, using meta-analytic techniques, results from 94 studies of school-based prevention activities that examined alcohol or other drug use outcomes. It set...     

Community-Level Collaboration for Substance Abuse Prevention

This paper reviews the literature on the roles of community-wide collaboration in substance abuse prevention. Three broad strategies through which collaboration may have its effects are identified (i.e., building community capacity, increasing service integration, and influencing policy change). Alternative theories of effects, means of measurement, and results and conclusions from studies of collaborative interventions for prevention are discussed. The strength of empirical evidence for the impact of collaboration on substance abuse outcomes varies by strategy, with more support for the logic of policy change. Additional conclusions are offered regarding when and how this approach can work, and what might be useful next steps.     

Advances in Statistical Methods for Substance Abuse Prevention Research

The paper describes advances in statistical methods for prevention research with a particular focus on substance abuse prevention. Standard analysis methods are extended to the typical research designs and characteristics of the data collected in prevention research. Prevention research often includes longitudinal measurement, clustering of data in units such as schools or clinics, missing data, and categorical as well as continuous outcome variables. Statistical methods to handle these features of prevention data are outlined. Developments in mediation, moderation, and implementation analysis allow for the extraction of more detailed information from a prevention study. Advancements in the interpretation of prevention research results include more widespread calculation of effect size and statistical power, the use of confidence intervals as well as hypothesis testing, detailed causal analysis of research findings, and meta-analysis. The increased availability of statistical software has contributed greatly to the use of new methods in prevention research. It is likely that the Internet will continue to stimulate the development and application of new methods.     

Strategies for Ensuring Use of Needs Assessment Findings: Experiences of a Community Substance Abuse Prevention Program

This article offers specific strategies for ensuring utilization of needs assessment findings. It is designed to help prevention specialists, community planners, program evaluators, and involved citizens maximize their efforts in developing community services by tying those services closely to community needs. Program evaluators who completed a broad-based community needs assessment commissioned by their organization, found that the organization had gone ahead in developing programs and saw the needs assessment findings as unrelated to their ongoing activities. Far from being unique, this situation is common in organizations where participants feel an urgency to get programs under way to address widespread community problems.     

Needs for Youth Substance Abuse and Violence Prevention in Schools and Communities

Substance abuse and violence among today's youth are at an all time high. Numerous prevention programs have been implemented to address these issues. Not all are successful. Research has found that when schools and communities are involved in the planning of prevention programs, youth are more cognizant of risk factors and of behaviors that strengthen risk factors. The purpose of this study was to assess the emerging needs for prevention and health education among youth, and to identify effective approaches for prevention program development and service delivery in schools and communities. This study surveyed 312 youth prevention specialists, school and community health educators and counselors, and parents in a regional sample of ten states. Results indicated that substance abuse and violence are the two most critical and priority issues in need of comprehensive prevention. Multiple prevention service delivery strategies appropriate to youth, including training, technical assistance, conferencing and networking, needs assessment and evaluation, and information requests are discussed in detail. Implications for prevention programming emphasized reducing risk factors and strengthening protective factors, reaching and motivating youth participation.     

Creating School and Community Partnerships for Substance Abuse Prevention Programs

The article reviews the scope and scale of the problem, explores a transactional view of etiology, and summarizes the prevailing approaches to prevention, exemplary and promising approaches, and standards for research and practice. The authors stress the importance of addressing the complexity of the problem through creation of comprehensive, multifaceted approaches to reduce substance abuse. Effective intervention frameworks are presented that weave together the resources of school, home, and community.     

Family-Based Approaches to Substance Abuse Prevention

In this paper, the prevalence of child and adolescent substance use is briefly presented. The nature of the co-occurrence of multiple antisocial behaviors, including substance use, during adolescence, and the causal factors which contribute to the early onset and maintenance of youth substance use and substance abuse are discussed. Emphasis is placed on parent and family factors, and children's social competence, which are associated with substance use, and which are the potentially mutable targets of family-based preventive interventions. Family-based preventive interventions were classified as either primarily addressing parent and family skills training, or addressing family therapy and in-home family support models. Distinctions were also made between programs that were universal or selective preventive interventions, or that were indicated preventive interventions directed at high risk individuals who were already showing early signs of being on the trajectory to substance abuse. Intervention research conducted within the past 30 years within these topical areas are summarized within tables, and findings indicate consistent intervention effects on children's problem behaviors, and on potential mediating processes such as parenting behaviors and aspects of family functioning. Recent studies with strong designs are described in greater depth. The paper concludes with discussions of the gaps in current intervention research, of barriers encountered in the implementation and evaluation of family-based prevention programs, and of the implications for future preventive intervention research and for social policy related to family-based preventive interventions.     

Characteristics of Effective Substance Abuse Prevention Programs for High-Risk Youth

The last two decades have witnessed a rapid development of substance abuse prevention programs. Most efforts to evaluate these programs have been limited to single program studies, and nearly all studies involving multiple drug prevention programs have involved school-based programs for general youth populations. In 1995, the Center for Substance Abuse Prevention (CSAP), with the Substance Abuse and Mental Health Administration (SAMHSA), funded the CSAP National Cross-site Evaluation of High Risk Youth Programs, a five-year, multi-site evaluation study involving 46 programs and over 10,500 youth at high risk for substance use (CSAP, 2002(a)). This article reports findings from this evaluation, focusing on program characteristics that help explain reductions in 30-day substance use among program participants. Programs found to be most effective in reducing substance use were those that offered strong behavioral life skills development content, emphasized team-building and interpersonal delivery methods, emphasized introspective learning approaches focusing on self-reflection, were based upon a clearly articulated and coherent program theory, and provided intense contact with youth. Programs utilizing these positive program components produced consistent and lasting reductions in substance use. These findings provide a solid basis for the adoption of positive program characteristics in the development of future prevention programming for high-risk youth.     

Measuring the Effectiveness of the Urban Youth Connection: The Case for Dose-Response Modeling to Demonstrate the Impact of an Adolescent Substance Abuse Prevention Program

Introduction. This paper presents methods and findings from an outcome evaluation of an adolescent drug prevention program serving predominantly Hispanic and African American students in an inner-city public middle school and secondary school in Boston. The program, entitled the Urban Youth Connection, consists of individual, pair and group counseling to high risk students identified by school personnel. The study design and statistical analysis techniques are presented as a model approach to estimating the impact of adolescent drug prevention programs to maximize scientific rigor without compromising the service delivery mission of the program. The objectives of the intervention are to prevent and reduce 30-day use of alcohol, tobacco and other drugs, and to reduce risk and improve resiliency factors associated with healthy behaviors: self-esteem, social coping, depression, interpersonal violence, school involvement and aspirations, school attendance, and academic performance. Methods. A pre-test, post-test comparison group design was used to identify changes over time in behavioral, psychosocial, and academic outcomes. Stratified analysis was used to adjust for baseline differences in outcomes of interest between the treatment and comparison groups. A dose-response relationship was measured between amount of services received and self-reported behavioral outcomes, academic performance, and school attendance, by means of correlational and regression analyses. Self-reported behavioral outcomes were measured through surveys administered in both participating schools in the fall and spring of 1993-1994. Academic performance and school attendance was abstracted from student report cards at the end of the 1993-1994 school year. Service records, self-administered questionnaires, and grade and attendance reports were linked in an integrated computerized management information system. Results. Benefits of increased program exposure were indicated for intermediate outcomes of depression, social coping, school performance, especially for high risk students in the secondary school. Reduction of tobacco, beer and marijuana use among treatment students in the secondary school is suggested from this study. The dose-response model which associated amount of program exposure with self-reported outcomes and academic performance and attendance indicated that the intervention had the greatest effect on academic indicators among high school students who were identified as high risk at baseline. Discussion. The Urban Youth Connection shows promise of having an impact on intermediate outcomes of interest for high risk youth in middle school and high school settings. Behavioral indicators related to substance use or abuse are not impacted by the intervention as measured by a smaller subset of paired pre/post self-reported surveys among treatment and comparison group students. Early findings from the pre-test/post-test comparison are limited by small sample sizes. Trends suggest that program success may be demonstrated with increasing sample size as the program continues. Cohort studies in an action setting can lead to insignificant findings due to lower statistical power as a result of small sample sizes. Dose-response modeling can be a useful technique to make causal inferences to estimate program impact and can have more statistical power since this analytic approach can yield larger sample sizes useful for analysis. Dose-response modeling requires linkage of service information to client outcomes. Either approach requires stratification to adjust for variability within both the treatment group and the comparison group that result from application of quasi-experimental study designs for evaluating outcomes of adolescent drug prevention programs.     

Spreading a Warm Blanket of Prevention over All Children: Guidelines for Selecting Substance Abuse and Related Prevention Curricula for Use in the Schools

There is a need for a set of clear, practical, empirically and theoretically grounded guidelines to select substance abuse and related prevention and competence promotion programs based in schools. This report provides such a framework in the form of recommended key elements of effective school-based prevention programs derived from an extensive literature search. In addition, major programs are summarized and compared in relation to their coverage of these key elements.     

Program Characteristics and the Length of Time Clients are in Substance Abuse Treatment

Is there a relationship between the characteristics of drug addiction treatment programs and an important correlate of better outcomes, the length of time clients are in treatment? Previous research has consistently shown longer periods in treatment and a range of services each have a salutary effect on client outcomes after treatment. Much of this research has examined the characteristics of clients. Program attributes are another important consideration. Multivariate analysis of data collected from a national survey of outpatient drug addiction treatment programs shows offering a range of services along with several other program characteristics are relevant to the duration of treatment. When a range of services are available, this has a positive association with both the number of months programs report clients are in treatment and with the number of counseling sessions programs report clients receive over the course of treatment. Ultimately, this should lead to better outcomes for clients.     

Bridging the Gap Between Family-Focused Research and Substance Abuse Prevention Practice: Preface

This paper introduces the contributions of this special section on connecting family-focused substance abuse prevention research and practice. Other expert reviews and the meta-analyses discussed in this and the following papers have given us a good idea about what works in this area. They have specified a number of family-focused substance abuse prevention approaches and programs effective in decreasing substance abuse in youth. Unfortunately, some practitioners are still primarily implementing untested, ineffective programs. A number of recommendations are made by the authors of this special section to help improve the adoption of evidence-based family programs to prevent substance abuse by youth.     

Parent and Family Skills Training in Targeted Prevention Programs for At-Risk Youth

The first part of this paper provides an overview for targeted preventive interventions using parent and family skills training with high risk children. The typical format and goals of these programs are presented. Empirically-examined parent training programs have either been delivered to the parents alone, to the parents and children in separate group meetings, or to the parents and children together in sessions for at least part of the time. An overview of findings from research on these programs indicates that these programs have consistent and replicated effects on children's behavior, on parents' improved use of effective discipline strategies, and on improved family functioning.The second major part of this paper describes gaps in existing research knowledge about indicated prevention programs are then identified, and the barriers and opportunities involved in the implementation of empirically-supported are discussed. Finally, the paper provides a series of recommendations for future research, knowledge development and policy development.     

NTU: An Africentric Approach to Substance Abuse Prevention Among African American Youth

The primary goal of the NTU project was to reduce risk factors and increase protective factors for ATOD use among African American 5th and 6th graders. The objectives of the NTU program were: 1) to improve knowledge of and increase intolerance of drugs; 2) to improve values; 3) to increase racial identity; 4) to improve self-esteem; 5) to increase knowledge of African culture; 6) to improve family communications; 7) to improve behaviors in school; and 8) to improve problem solving skills. Africentric philosophy and world-view provided the conceptual framework for the development of intervention activities. Intervention components for 5th graders included a rites of passage program, a substance abuse education program, an Africentric education program, a parenting program, and a family therapy program. Sixth graders participated in a booster program designed to reinforce skills and values learned the previous year. Pre and post data were collected from 5th and 6th grade participants in an intervention and a comparison group. Measures of drug knowledge, Africentric values, self-esteem, racial identity, family communication, child behaviors, and problem solving were obtained. The results indicated significant program effects for protective factors including racial identity, knowledge of African culture, self-esteem, and school behaviors.