Using Treatment Integrity Methods to Study the Implementation Process

[Clin Psychol Sci Prac 18: 36–40, 2011] The last decade has witnessed increased interest in the implementation and dissemination of evidence‐based treatments (EBTs) for youth. Nakamura et al. (2011) detail lessons learned over the past decade from the large‐scale implementation of EBTs for children in Hawaii. This commentary discusses how lessons from Hawaii’s initiative can help inform the next generation of implementation research. Specifically, we focus on how treatment integrity models and methods designed to characterize core aspects of treatment delivery can be used to study the implementation process. Using the new interactive online reporting systems developed by Nakamura et al. to collect treatment integrity data offers researchers a way to determine how best to implement EBTs in community‐based service settings with integrity and skill.     

Gaps,Bridges, and the Bumpy Road to Improving Clinic‐Based Therapy for Youth

It has been proposed that one way of strengthening the effects of evidence‐based treatments for youth in service clinics is to improve treatment integrity in these settings. Four assumptions related to this recommendation are evaluated in light of findings from effectiveness studies. The absence of evidence on links between treatment integrity and treatment outcome in effectiveness trials makes it difficult to estimate the potential benefits of many of the proposed quality control strategies. Incremental changes in methods to improve treatment integrity in service clinics must be considered in the context of structural barriers to implementation.     

Evidence‐based implementation practices applied to the intensive treatment of eating disorders: Summary of research and illustration of principles using a case example

Implementation of evidence‐based practices (EBPs) in intensive treatment settings poses a major challenge in the field of psychology. This is particularly true for eating disorder (ED) treatment, where multidisciplinary care is provided to a severe and complex patient population; almost no data exist concerning best practices in these settings. We summarize the research on EBP implementation science organized by existing frameworks and illustrate how these practices may be applied using a case example. We describe the recent successful implementation of EBPs in a community‐based intensive ED treatment network, which recently adapted and implemented transdiagnostic, empirically supported treatment for emotional disorders across its system of residential and day‐hospital programs. The research summary, implementation frameworks, and case example may inform future efforts to implement evidence‐based practice in intensive treatment settings.     

Dissemination and Implementation of Evidence–Based Practices: Training and Consultation as Implementation Strategies

To provide effective treatment for individuals with mental health needs, there is a movement to deploy evidence‐based practices (EBPs) developed in research settings into community settings. Training clinicians in EBPs is often used as the primary implementation strategy in these efforts, despite evidence suggesting that training alone does not change therapist behavior. A promising implementation strategy that can be combined with training is consultation, or ongoing support. This article reviews the literature on consultation following initial training. A model of consultation is presented as well as preliminary findings regarding effective consultation techniques. Future directions are offered.     

Treatment Integrity: Implications for Training

Perepletchikova and Kazdin (this issue) reviewed the extant literature on the relationship between treatment integrity and therapy outcomes. The empirical literature on this relationship is inconsistent, and the authors note existing limitations in current research strategies and provide recommendations for future research efforts. This commentary explores the implications that a focus on treatment integrity has for training. As such, comments are offered on a number of conceptual, methodological, and practical issues relating to treatment integrity, instruction in empirically supported treatments, competence, and the training of future clinical psychologists.     

Treatment integrity in psychotherapy research: analysis of the studies and examination of the associated factors

Treatment integrity refers to the degree to which an intervention is delivered as intended. Two studies evaluated the adequacy of treatment integrity procedures (including establishing, assessing, evaluating, and reporting integrity; therapist treatment adherence; and therapist competence) implemented in psychotherapy research, as well as predictors of their implementation. Randomized controlled trials of psychosocial interventions published in 6 influential psychological and psychiatric journals were reviewed and coded for treatment integrity implementation. Results indicate that investigations that systematically addressed treatment integrity procedures are virtually absent in the literature. Treatment integrity was adequately addressed for only 3.50% of the evaluated psychosocial interventions. Journal of publication and treatment approach predicted integrity implementation. Skill-building treatments (e.g., cognitive-behavioral) as compared with non-skill-building interventions (e.g., psychodynamic, nondirective counseling) were implemented with higher attention to integrity procedures. Guidelines for implementation of treatment integrity procedures need to be reevaluated.     

Parent Management Training‐Oregon Model (PMTO™) in Mexico City: Integrating Cultural Adaptation Activities in an Implementation Model

This article describes the process of cultural adaptation at the start of the implementation of the Parent Management Training intervention‐Oregon model (PMTO) in Mexico City. The implementation process was guided by the model, and the cultural adaptation of PMTO was theoretically guided by the cultural adaptation process (CAP) model. During the process of the adaptation, we uncovered the potential for the CAP to be embedded in the implementation process, taking into account broader training and economic challenges and opportunities. We discuss how cultural adaptation and implementation processes are inextricably linked and iterative and how maintaining a collaborative relationship with the treatment developer has guided our work and has helped expand our research efforts, and how building human capital to implement PMTO in Mexico supported the implementation efforts of PMTO in other places in the United States.     

Eye Movement Desensitization and Reprocessing Treatment for Posttraumatic Stress Disorder

A qualitative review of experimental and quasi-experimental outcome studies of eye movement desensitization and reprocessing (EMDR) treatment for persons with posttraumatic stress disorder (PTSD) suggests that the treatment is effective for civilian but not combat PTSD. The current data indicate that additional research into EMDR's efficacy for PTSD is warranted. Further studies should include comparisons to placebo control procedures and existing validated treatments for PTSD, an adequate treatment dose, systematic efforts to establish and assess treatment integrity and quality, and long-term follow-up data. The therapeutic mechanisms underlying EMDR's observed benefits remain elusive. Whether the eye movement or some other type of stimulation is essential to EMDR's effects cannot be determined from the current data.     

A management model for pediatric somatization

Medically unexplained physical symptoms are common in pediatric settings, though little systematic research is available to guide the development of treatment efforts for pediatric somatization and somatoform disorders. This paper presents a management model for pediatric somatization based on principles distilled from the available pediatric and adult literature. Careful assessment, frank presentation of the diagnosis, and a cognitive-behavioral and rehabilitative approach are emphasized, along with aggressive psychiatric treatment of comorbid psychopathology. Well-designed empirical studies of intervention are needed that should examine efficacy as well as the relationship between symptomatic improvement, functional improvement, and comorbid anxiety and depressive symptoms.     

Developing Cultural Competency in Clinical Practice: Treatment Considerations for Chinese Cultural Groups in the United States

The incorporation of specific cultural beliefs and behaviors into standard clinical treatments is sorely needed to improve our ability to provide services to people from diverse settings. Findings from a critical review of empirical research provides the basis for the modification of an integrative model of differential treatment selection. The relative influence of cultural beliefs and behaviors on patient characteristics, relationship variables, and treatment context are discussed. This article suggests ways to include cultural considerations at various steps within the model. Specific concerns, such as immigration, culture change, and conceptualizations of mental health, are discussed for one cultural group, the Chinese in the United States. A series of clinical practice decisions are suggested to guide clinicians in their efforts to incorporate specific cultural beliefs, including an analysis of the match between cultures and an assessment of the client's perceived opportunity within the culture of settlement. Types of interventions are suggested based on these decisions. A case study of a short-term treatment of a Chinese woman provides an illustration of the ways in which strategies and techniques in the model are modified.     

Treatment Integrity and Therapeutic Change: Commentary on Perepletchikova and Kazdin

Perepletchikova and Kazdin's (this issue) analysis of the treatment integrity literature should guide future research and practice on this vital issue facing clinical research and practice. This commentary considers the construct of treatment integrity and those factors that moderate the relationship between treatment integrity and treatment outcome. Specifically, this paper examines the three components of treatment integrity specified by Perepletchikova and Kazdin (treatment adherence, therapist competence, treatment differentiation), the relationship between treatment integrity and treatment outcome, and measurement and validation issues involving treatment integrity.     

Treatment Integrity and Differential Treatment Effects

Bhar and Beck (2009) examined the extent to which treatment integrity procedures were implemented in studies comparing psychoanalytic psychotherapies and cognitive-behavioral therapies. Consistent with other reports on attention to treatment integrity in psychotherapy research, the authors noted that most of the evaluated studies did not adequately implement treatment integrity procedures. This highlights methodological neglect of treatment integrity and a need to amend errors in monitoring the independent variables under investigation. This commentary considers how Bhar and Beck's investigation affects the dodo bird verdict that all psychotherapies are presumed to be of equal efficacy. Further, ways to examine the treatment integrity of process-oriented treatments (e.g., humanistic, psychoanalytic) are discussed.     

Assessment and Psychological Treatment of Depression in Older Adults with Terminal or Life-Threatening Illness

Depression decreases the quality of life and hinders efforts to palliate symptoms of adults with terminal or life-threatening illness. Nevertheless, depression often may go undetected and untreated in palliative care and hospice settings due to a number of factors, including the overlap of depressive symptoms with those of serious medical illness and concern that frail elderly patients cannot tolerate psychotherapy or antidepressant treatment. In this paper we review the available research regarding assessment and treatment of depression in older adults with terminal or life-threatening illness, focusing on patients who are seen in palliative care, cancer treatment, or hospice settings. Although the prevalence of depression is relatively high in these settings in mixed-age adult samples, studies focused exclusively on older adults are rare and there appear to be no randomized controlled trials of psychotherapy conducted to date that specifically address their needs. There are, however, promising psychological approaches featured in case reports and pilot studies that are consistent with empirically supported therapies for the general treatment of depression in older adults. Based on these preliminary findings and reports, we offer tentative recommendations for the assessment and treatment of depression in terminally ill older adults. We conclude that controlled research on psychotherapy for late-life depression is both feasible and urgently needed in palliative care, cancer care, and hospice settings.     

A Public Health Perspective on the Transport of Evidence-Based Practices

A public health perspective on the dissemination of empirically supported treatments is articulated, and the implications of this perspective for the foci of dissemination efforts and research are discussed. Strategies targeting multiple levels of the practice context, including payers and policy makers, will likely be needed to cultivate and sustain the demand for evidence-based treatments and to support the implementation of such treatments with fidelity. A continuous quality improvement system is recommended to document and improve the implementation and outcomes of evidence-based treatments delivered in community practice settings.     

The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings: A systematic review

BackgroundIt is estimated that one million people infected with HIV initiate anti-retroviral therapy (ART) in resource-constrained countries annually. This occurs against a background of overburdened health workers with limited skills to handle rapidly changing treatment standards and guidelines hence compromising quality of care. Electronic medical record (EMR)-based clinical decision support systems (CDSS) are considered a solution to improve quality of care. Little evidence, however, exists on the effectiveness of EMR-based CDSS on quality of HIV care and treatment in resource-constrained settings.ObjectiveThe aim of this systematic review was to identify original studies on EMR-based CDSS describing process and outcome measures as well as reported barriers to their implementation in resource-constrained settings. We characterized the studies by guideline adherence, data and process, and barriers to CDSS implementation.MethodsTwo reviewers independently assessed original articles from a search of the MEDLINE, EMBASE, CINAHL and Global Health Library databases until January 2012. The included articles were those that evaluated or described the implementation of EMR-based CDSS that were used in HIV care in low-income countries.ResultsA total of 12 studies met the inclusion criteria, 10 of which were conducted in sub-Saharan Africa and 2 in the Caribbean. None of the papers described a strong (randomized controlled) evaluation design. Guideline adherence: One study showed that ordering rates for CD4 tests were significantly higher when reminders were used. Data and process: Studies reported reduction in data errors, reduction in missed appointments, reduction in missed CD4 results and reduction in patient waiting time. Two studies showed a significant increase in time spent by clinicians on direct patient care. Barriers to CDSS implementation: Technical infrastructure problems such as unreliable electric power and erratic Internet connectivity, clinicians’ limited computer skills and failure by providers to comply with the reminders are key impediments to the implementation and effective use of CDSS.ConclusionThe limited number of evaluation studies, the basic and heterogeneous study designs, and varied outcome measures make it difficult to meaningfully conclude on the effectiveness of CDSS on quality of HIV care and treatment in resource-limited settings. High quality evaluation studies are needed. Factors specific to implementation of EMR-based CDSS in resource-limited setting should be addressed before such countries can demonstrate its full benefits. More work needs to be done to overcome the barriers to EMR and CDSS implementation in developing countries such as technical infrastructure and care providers’ computer illiteracy. However, simultaneously evaluating and describing CDSS implementation strategies that work can further guide wise investments in their wider rollout.     

Point-of-care testing for sexually transmitted infections in low-resource settings

BackgroundBoth the global incidence and the prevalence of sexually transmitted infections (STIs) continue to increase, affecting hundreds of millions of individuals, particularly in low-to middle-income countries. Although a definitive diagnosis is desirable to inform STI treatment, syndromic management is the most widely used strategy in resource-limited settings. With the development of point-of-care (POC) tests, it is important to discuss how laboratories will need to adapt to new training and supervisory roles in support of testing, which will largely be performed by peripheral clinical staff.ObjectivesTo discuss potential applications of STI POC tests, how they could improve existing STI control strategies and the role of clinical and reference laboratories in support of initiatives to improve STI management and control activities.SourcesNarrative literature review and expert opinion.ContentThe paper outlines the current status of the STI epidemic worldwide and discusses the problems associated with current approaches to control these infections, particularly in low-resource settings. The roles of clinical and reference laboratories will need to change to provide support for POC and near-patient STI testing as these technologies are introduced into clinical as well as laboratory settings.ImplicationsLaboratories will be expected to play a leading role in the introduction and implementation of POC and near-patient STI testing. They will be required to facilitate training and provide technical and supervisory support to clinical staff on the use of these technologies to augment existing STI management and surveillance programmes. In order to provide quality service, they will need to develop, introduce and maintain sustainable local quality control and external quality assurance systems. Evidence from implementation research for introduction and scale up of STI POC tests in different STI epidemic and laboratory infrastructure settings is required.     

Enhancing quality and integrity in biomedical research in Africa: an international call for greater focus,investment and standardisation in capacity strengthening for frontline staff

The integrity of biomedical research depends heavily on the quality of research data collected. In turn, data quality depends on processes of data collection, a task undertaken by frontline research staff in many research programmes in Africa and elsewhere. These frontline research staff often have additional responsibilities including translating and communicating research in local languages, seeking informed consent for study participation and maintaining supportive relationships between research institutions and study participants and wider communities. The level of skills that fieldworkers need to undertake these responsibilities clearly affects the quality of data collected, the ethics of research ‘on the ground’ and the short and long term acceptability of research.We organised an international workshop in Kenya in July 2014 to discuss the role of frontline staff in scientific research. A total of 25 field managers from 9 African countries and the UK met for 2.5 days to discuss the relationship between data quality and institutional performance management systems and how they affect career progression and supportive supervision policies of research frontline staff.From this workshop, and supporting an expanding literature on the role of fieldworkers in international health research, participants agreed that fieldworkers’ roles present them with practical and ethical challenges that their routine training does not adequately prepare them for. We argue that the common and complex challenges facing fieldworkers should in part be addressed through increased investment and collaborative agreements across types of research institutions in Africa. We call for standardization of core elements of training for this critically important cadre of research staff who perform similar roles and encounter similar challenges in many African settings. Although many valuable training elements are offered in institutions, there is a need to develop broader, more grounded and innovative strategies to address complex realities for fieldworkers, and support the integrity and ethics of health research in these settings.     

On the Topic of Treatment Integrity

[Clin Psychol Sci Prac 18: 148–153, 2011] Treatment integrity, also known as treatment fidelity, is integral for empirical testing of intervention efficacy, as it allows for unambiguous interpretations of the obtained results. Ensuring treatment integrity is also important for dissemination of evidence‐based practices and quality improvement of services. However, in the examination of the relationship between treatment integrity and treatment outcome, it is important to consider that treatment integrity may be a proxy variable for other variables impacting therapeutic change (e.g., characteristics of intervention, clients, setting, and therapist). Considerations on examining the association between integrity and outcome are discussed. Further, recommendations on the level to which treatment integrity needs to be addressed in psychotherapy research and clinical practice are provided.