Treatment Integrity and Therapeutic Change: Issues and Research Recommendations

One of the most important aspects of treatment outcome research is establishing treatment integrity. Integrity of the treatment refers to the degree to which treatment is implemented as intended. Research examining the relationship between treatment integrity and therapeutic change has produced conflicting results. However, assessment, design strategies, and the possible confound of integrity with other variables may explain the inconsistency in findings. This paper elaborates the limitations of existing strategies for evaluating the relationship between treatment integrity and outcome. Recommendations for future research include controlling possible confounding variables, experimentally manipulating treatment integrity, and using novel assessment and evaluation strategies.     

On the Topic of Treatment Integrity

[Clin Psychol Sci Prac 18: 148–153, 2011] Treatment integrity, also known as treatment fidelity, is integral for empirical testing of intervention efficacy, as it allows for unambiguous interpretations of the obtained results. Ensuring treatment integrity is also important for dissemination of evidence‐based practices and quality improvement of services. However, in the examination of the relationship between treatment integrity and treatment outcome, it is important to consider that treatment integrity may be a proxy variable for other variables impacting therapeutic change (e.g., characteristics of intervention, clients, setting, and therapist). Considerations on examining the association between integrity and outcome are discussed. Further, recommendations on the level to which treatment integrity needs to be addressed in psychotherapy research and clinical practice are provided.     

Definitional and Practical Issues in the Assessment of Treatment Integrity

Perepletchikova and Kazdin discuss the issue of establishing treatment integrity in the context of treatment outcome research. They offer important insights into the potential explanations for the conflicting results and suggest directions for future assessment and research designs. This commentary, in the context of Perepletchikova and Kazdin's paper, discusses (a) issues related to the definition and assessment of treatment integrity and its components and (b) the scientific, practical, and clinical applications of assessing treatment integrity.     

Treatment Integrity: Implications for Training

Perepletchikova and Kazdin (this issue) reviewed the extant literature on the relationship between treatment integrity and therapy outcomes. The empirical literature on this relationship is inconsistent, and the authors note existing limitations in current research strategies and provide recommendations for future research efforts. This commentary explores the implications that a focus on treatment integrity has for training. As such, comments are offered on a number of conceptual, methodological, and practical issues relating to treatment integrity, instruction in empirically supported treatments, competence, and the training of future clinical psychologists.     

Treatment Integrity and Differential Treatment Effects

Bhar and Beck (2009) examined the extent to which treatment integrity procedures were implemented in studies comparing psychoanalytic psychotherapies and cognitive-behavioral therapies. Consistent with other reports on attention to treatment integrity in psychotherapy research, the authors noted that most of the evaluated studies did not adequately implement treatment integrity procedures. This highlights methodological neglect of treatment integrity and a need to amend errors in monitoring the independent variables under investigation. This commentary considers how Bhar and Beck's investigation affects the dodo bird verdict that all psychotherapies are presumed to be of equal efficacy. Further, ways to examine the treatment integrity of process-oriented treatments (e.g., humanistic, psychoanalytic) are discussed.     

Using Treatment Integrity Methods to Study the Implementation Process

[Clin Psychol Sci Prac 18: 36–40, 2011] The last decade has witnessed increased interest in the implementation and dissemination of evidence‐based treatments (EBTs) for youth. Nakamura et al. (2011) detail lessons learned over the past decade from the large‐scale implementation of EBTs for children in Hawaii. This commentary discusses how lessons from Hawaii’s initiative can help inform the next generation of implementation research. Specifically, we focus on how treatment integrity models and methods designed to characterize core aspects of treatment delivery can be used to study the implementation process. Using the new interactive online reporting systems developed by Nakamura et al. to collect treatment integrity data offers researchers a way to determine how best to implement EBTs in community‐based service settings with integrity and skill.     

Manual-Based Treatment and Clinical Practice: Hangman of Life or Promising Development?

Wilson (this issue) argues for the advantages of manual-based treatments for enhancing therapeutic outcomes. While largely agreeing with Wilson about the advantages of manuals, I argue here that manuals have been with us since the early 1960s, that their use has done little to enhance patient outcome, and that the field would be best served by tracking patient progress on a routine basis, a task that does not require manual development. Empirical science has much to contribute to positive patient outcome; its greatest direct effect will be felt when clinicians monitor outcome and connect this outcome to the behaviors and attitudes of a particular therapist.     

Reflections on Treatment Integrity in a Dissemination and Implementation Framework

This commentary reflects on issues related to linking the conceptual frame for treatment fidelity presented in the special issue articles to the most current thinking on treatment fidelity within the emerging field of implementation science. The commentary presents several aspects of treatment fidelity in implementation science that generally were not discussed or addressed in the special issue articles, such as the placement of treatment fidelity as only one among several critical implementation outcomes, the relationship of treatment fidelity to design elements such as external validity as well as internal validity, and the importance of considering implementation issues in the measurement of treatment fidelity even in the earlier stages of efficacy and effectiveness research. The excellent in‐depth discussion of treatment fidelity in the special series is a promising start to linking this critical concept to the emerging field of implementation science.     

Neglect of the Therapist and the Human Dimensions of Change: A Commentary

The context for this symposium is outlined, indicating reasons for the emphasis on outcome studies that focus on technique rather than therapist personal qualities. Current trends are critiqued in light of the symposium presentations, and suggestions for the future are provided. Future research needs to take into account humanistic, phenomenological, agentive, and postmodern perspectives as opposed to the present overemphasis on mechanistic/naturalistic conceptions of causality and therapeutic change processes.     

A Major Dilemma in Psychotherapy Outcome Research: Disentangling Therapists From Therapies

In interpreting results of psychotherapy outcome studies, particularly those of comparative studies, it is often difficult to disentangle the effects due to the treatments per se from those that may be due to differential competence and other characteristics of the therapists carrying out the treatments. This article attempts to address this general dilemma in psychotherapy outcome research, drawing on the experience and findings of the NIMH Treatment of Depression Collaborative Research Program. Issues discussed include the nature of the overall therapist sample, variability in therapist efficacy, therapist characteristics, training and supervision of therapists, and adherence and competence in carrying out a treatment. Implications discussed include the need to provide, both in presentations of the results of individual outcome studies and in dissemination of information regarding "empirically validated treatments," critical information about the therapists carrying out the treatment.     

Treatment Integrity of Studies That Compare Short-Term Psychodynamic Psychotherapy With Cognitive-Behavior Therapy

In their meta-analysis, Leichsenring, Rabung, and Leibing (2004) found that short-term psychoanalytic psychotherapy (STPP) and cognitive-behavior therapy (CBT) were equally efficacious. However, they did not examine whether the treatments were delivered as intended. In our review of nine studies comparing STPP and CBT, we found that most did not adequately implement treatment integrity procedures. Leichsenring and colleagues' contention that CBT and STPP are equally efficacious is therefore based on studies without adequate procedures to monitor treatment adherence or therapist competence. The neglect of treatment integrity procedures in these studies leaves open the possibility that the absence of comparative treatment effects may be due to the manner in which the treatments were operationalized.     

Empirically Supported Treatment: A Commentary

Generating and updating lists of empirically supported treatments serves the public interest. Such lists force advocates to test the accuracy of their claims and serve to inform the public regarding the relative benefits of different interventions. Such efforts should cover the full array of types of interventions used to treat a given disorder and be multidisciplinary in nature to guard against bias related to guild identification. I wholly endorse the authors' call for Division 12 to update their list of empirically supported treatments.     

Treatment Adherence Difficulties Among Children with Cancer: The Role of Parenting Style

Investigated the prevalence of treatment adherence problemsamong children undergoing cancer treatment and explored therelationship of sociodemographic factors, child functional status,and parenting Style to these difficulties. Nurse ratings ofadherence difficulties encountered during the cancer treatmentregimen were completed on 77 children and their caregivers.A subset of caregivers also completed a measure of parentingattitudes and practices. Results indicated that certain adherencedifficulties were related to the child's age and functionalstatus, the family's socioeconomic status (SES), and parentingstyle. The child's age was related to adherence to procedure-relatedtasks; younger children had more frequent problems with completingthese tasks. More functional children had a greater number ofadherence problems. Families with a lower SES had more difficultieswith timeliness and cancellations/delays of clinic appointmentsand promptness of reporting of the child's reactions to treatment.Children of caregivers who had a more "supportive" parentingstyle (were more sensitive to children's input, had a less restrictiveattitude towards parenting, and were more nurturant) canceledand delayed fewer appointments, were on time more frequentlyfor appointments, and reported reactions to treatment with lessdelay Implications of these findings for future research ontreatment adherence and for clinical interventions are discussed.     

Some Observations on Studying Therapists Instead of Treatment Packages

Some practical problems are highlighted with regard to studying the outcomes of individual therapists, including the long and inconsistent success in finding therapist variables that explain differential client treatment response. Research on the individual therapist makes it clear that considerable variability in outcome exists at the extreme ends of the normal distribution of therapist effects. Much less is known about the variability of client outcomes within specific therapists' clients. We encourage further research on therapist effects, but concede that such research is not likely to provide actionable information for routine clinical care. In the meantime, client treatment response needs to be monitored if positive outcomes are to be maximized.     

Parent Acceptability and Feasibility of ADHD Interventions: Assessment, Correlates, and Predictive Validity

Examined variables related to parents' stated willingness topursue treatment, and their actual adherence to treatment recommendations,for children with attention deficit-hyperactivity disorder (ADHD).Parents (87 mothers; 63 fathers) of 91 patients seen for evaluationat an outpatient ADHD clinic completed the ADHD Knowledge andOpinions Survey-Revised (AKOS-R). A factor analysis of the AKOS-Rrevealed three relevant factors: Counseling Acceptability, MedicationAcceptability, and Counseling Feasibility. Externalizing problemswere positively related to parents' Counseling Acceptabilityratings, while parents' ADHD knowledge was positively relatedto their Medication Acceptability ratings. However, CounselingAcceptability, Counseling Feasibility, and Medication Acceptabilityscores at intake failed to predict parents' pursual of recommendedcounseling and medication at follow-up. Association betweentreatment acceptability and adherence might be enhanced by assessingtreatment acceptability at multiple points during the assessmentand treatment process.