Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity

Background  

Controlled clinical trials of health care interventions are either explanatory or pragmatic. Explanatory trials test whether an intervention is efficacious; that is, whether it can have a beneficial effect in an ideal situation. Pragmatic trials measure effectiveness; they measure the degree of beneficial effect in real clinical practice. In pragmatic trials, a balance between external validity (generalizability of the results) and internal validity (reliability or accuracy of the results) needs to be achieved. The explanatory trial seeks to maximize the internal validity by assuring rigorous control of all variables other than the intervention. The pragmatic trial seeks to maximize external validity to ensure that the results can be generalized. However the danger of pragmatic trials is that internal validity may be overly compromised in the effort to ensure generalizability. We are conducting two pragmatic randomized controlled trials on interventions in the management of hypertension in primary care. We describe the design of the trials and the steps taken to deal with the competing demands of external and internal validity.     

The Sexunzipped Trial: Young People’s Views of Participating in an Online Randomized Controlled Trial

Background

Incidence of sexually transmitted infections (STIs) among young people in the United Kingdom is increasing. The Internet can be a suitable medium for delivery of sexual health information and sexual health promotion, given its high usage among young people, its potential for creating a sense of anonymity, and ease of access. Online randomized controlled trials (RCTs) are increasingly being used to evaluate online interventions, but while there are many advantages to online methodologies, they can be associated with a number of problems, including poor engagement with online interventions, poor trial retention, and concerns about the validity of data collected through self-report online. We conducted an online feasibility trial that tested the effects of the Sexunzipped website for sexual health compared to an information-only website. This study reports on a qualitative evaluation of the trial procedures, describing participants’ experiences and views of the Sexunzipped online trial including methods of recruitment, incentives, methods of contact, and sexual health outcome measurement.

Objective

Our goal was to determine participants’ views of the acceptability and validity of the online trial methodology used in the pilot RCT of the Sexunzipped intervention.

Methods

We used three qualitative data sources to assess the acceptability and validity of the online pilot RCT methodology: (1) individual interviews with 22 participants from the pilot RCT, (2) 133 emails received by the trial coordinator from trial participants, and (3) 217 free-text comments from the baseline and follow-up questionnaires. Interviews were audio-recorded and transcribed verbatim. An iterative, thematic analysis of all three data sources was conducted to identify common themes related to the acceptability and feasibility of the online trial methodology.

Results

Interview participants found the trial design, including online recruitment via Facebook, online registration, email communication with the researchers, and online completion of sexual health questionnaires to be highly acceptable and preferable to traditional methods. Incentives might assist in recruiting those who would not otherwise participate. Participants generally enjoyed taking part in sexual health research online and found the questionnaire itself thought-provoking. Completing the sexual health questionnaires online encouraged honesty in responding that might not be achieved with other methods. The majority of interview participants also thought that receiving and returning a urine sample for chlamydia testing via post was acceptable.

Conclusions

These findings provide strong support for the use of online research methods for sexual health research, emphasizing the importance of careful planning and execution of all trial procedures including recruitment, respondent validation, trial related communication, and methods to maximize follow-up. Our findings suggest that sexual health outcome measurement might encourage reflection on current behavior, sometimes leading to behavior change.

Trial Registration

International Standard Randomized Controlled Trial Number (ISRCTN): 55651027; http://www.controlled-trials.com/isrctn/pf/55651027 (Archived by WebCite at http://www.webcitation.org/6LbkxdPKf).     

Developing and Implementing Randomized Effectiveness Trials in General Medical Settings

This article discusses the development of two mental health services research programs: one targeting depression treatment in primary care and the other early intervention for posttraumatic stress disorders in acute care. Both programs have used the randomized effectiveness trial to assess the delivery of empirically supported psychotherapeutic and psychopharmacologic interventions in these general medical settings. This article explores the fellowship training, clinical experiences, and conceptual frameworks that have informed the progression of the two research programs. Specific modifications to the traditional randomized clinical trial design employed in the effectiveness trials are discussed. This article concludes with reflection on intervention development trajectories and interdisciplinary team compositions that may facilitate the development of mental health interventions that both derive from the best scientific evidence and can be feasibly delivered in real-world treatment settings.     

Adapting Behavioral Interventions for Social Media Delivery

Patients are increasingly using online social networks (ie, social media) to connect with other patients and health care professionals—a trend called peer-to-peer health care. Because online social networks provide a means for health care professionals to communicate with patients, and for patients to communicate with each other, an opportunity exists to use social media as a modality to deliver behavioral interventions. Social media-delivered behavioral interventions have the potential to reduce the expense of behavioral interventions by eliminating visits, as well as increase our access to patients by becoming embedded in their social media feeds. Trials of online social network-delivered behavioral interventions have shown promise, but much is unknown about intervention development and methodology. In this paper, we discuss the process by which investigators can translate behavioral interventions for social media delivery. We present a model that describes the steps and decision points in this process, including the necessary training and reporting requirements. We also discuss issues pertinent to social media-delivered interventions, including cost, scalability, and privacy. Finally, we identify areas of research that are needed to optimize this emerging behavioral intervention modality.     

Defining, designing, implementing, and evaluating phase 4 HIV prevention effectiveness trials for vulnerable populations

The efficacy of behavioral HIV prevention interventions has been convincingly demonstrated in a large number of randomized controlled phase 3 research outcome trials. Little research attention has been directed toward studying the effectiveness of the same interventions when delivered by providers to their own clients or community members, however. This article argues for the need to conduct phase 4 effectiveness trials of HIV prevention interventions that have been found efficacious in the research arena. Such trials can provide important information concerning the impact of interventions when applied in heterogeneous "real-world" circumstances. This article raises design issues and methodologic questions that need to be addressed in the conduct of phase 4 trials of behavioral interventions. These issues include the selection and training of service providers engaged in such trials, maintenance of fidelity to intervention protocol in provider-delivered interventions, determination of intervention core elements versus aspects that require tailoring, selection of relevant phase 4 study outcomes, interpretation of findings indicative of field effectiveness, sustainability, and other aspects of phase 4 trial design.     

Factors associated with emergency department use in asthma: acute care interventions improving chronic disease outcomes.

OBJECTIVE: To provide a literature review of the factors associated with childhood asthma-related emergency department (ED) visits and to identify elements of effective ED interventions that reduce the frequency of childhood ED visits while increasing primary health care utilization. DATA SOURCE: English Medline articles from 1990 that cross-referenced with the terms asthma, emergency, intervention, pediatric, and/or acute care. Experts in the field of allergy and asthma were also consulted. STUDY SELECTION: Childhood asthma interventions in the ED. RESULTS: Factors associated with childhood asthma-related ED visits include being impoverished, being exposed to allergens, receiving Medicaid or lacking insurance, being noncompliant with self-management skills, and having an African-American heritage. Other minorities may also be at risk, but further investigation is required to determine the extent. Attempts to link the patient to primary health care by the ED staff resulted in increased adherence to followup care. CONCLUSIONS: The ED provides an opportunity to help patients and families deal with asthma to improve their quality of life. Further, current studies demonstrate that the ED is an appropriate setting for an intervention that links the patient back to the primary health care provider. More research is needed on the appropriate educational messages to be delivered in ED. Also, barriers to followup care and regular use of a primary health care provider need to be identified so that future intervention designs can address these issues.     

Motivational interviewing to promote interconception health: A scoping review of evidence from clinical trials

BackgroundPromoting interconception health can improve birth outcomes and long-term women’s health. Motivational Interviewing (MI) is an evidence-based behavior change strategy that can address interconception health behaviors and health care engagement.ObjectiveThis scoping review assessed the evidence for using MI to promote interconception health and assessed features of successful MI interventions.MethodsWe searched PubMed, CHINAL, and Cochrane databases for clinical trials that involved an MI intervention and at least one comparison group published by 8/31/2021. Interventions occurred during pregnancy or within three months postpartum and outcomes were measured between birth and one year postpartum. We abstracted data on trial characteristics including outcome, population, interventionist training, MI fidelity monitoring, intervention dose, and comparison condition. We examined whether trials that demonstrated statistically significant improvement in outcomes had common features.ResultsThere were 37 included studies. Interventions addressed breastfeeding, teen contraception, tobacco, alcohol, or substance use, vaccine acceptance, nutrition, physical activity, and depression. No trials addressed more than one topic. Nineteen studies demonstrated improved outcomes. Interventions during the perinatal or postnatal periods were more likely to demonstrate improved interconception outcomes than interventions in the prenatal period. No other trial characteristics were consistently associated with demonstrating improved outcomes.DiscussionMI has been applied to a variety of interconception health behaviors, with some promising results, particularly for interventions in the perinatal or postpartum period. Outcomes were not clearly attributable to any other differences in intervention or study design. Further exploring context or implementation may help maximize the potential of MI in interconception health promotion.Practical valueMI may be implemented across a range of clinical settings, patient groups, and time points around pregnancy. Interventions on health topics relevant to the interconception period should incorporate perinatal or postpartum components.     

Review of 39 years of randomized controlled trials in the British Journal of General Practice.

BACKGROUND: Randomized controlled trials are being used increasingly to evaluate the effectiveness of health care interventions, including those in primary care. AIM. A study was undertaken to document the topic areas addressed by randomized controlled trials conducted in primary care and the methodological characteristics of these, based on a historical review of all trials published in the British Journal of General Practice (formerly the Research Newsletter and Journal of the Royal College of General Practitioners) between 1953 and 1991 inclusive. METHOD. Trials were identified by systematically hand searching all back issues. RESULTS. A total of 90 randomized controlled trials were identified, 78% of which were undertaken in general practice. The trials covered a wide range of topics; 62% examined pharmacological interventions, 21% non-pharmacological interventions, and 16% an intervention related to provision of an aspect of health service. Among the randomized controlled trials published there was a significant trend towards more reports on non-pharmacological trials in recent years (P < 0.001). The quality of trials, judged by the extent to which bias was controlled, varied considerably. Trials of pharmacological interventions generally controlled for selection bias at entry and at assessment of outcome more effectively than trials of non-pharmacological interventions. CONCLUSION. Some imaginative solutions to the logistic difficulties of conducting randomized trials in general practice were noted. These may serve as an incentive to those undertaking such studies in the future.     

Behavioral Health Services in Primary Care: A New Perspective for Treating Depression

This article attempts to provide information, inspiration, and practical guidelines for mental health professionals who want to work with primary care providers. Depression is an ideal initial focus in starting primary care behavioral services because (1) the majority of currently available efficacy studies on primary care behavioral hearth interventions concern depression and (2) depression is highly prevalent in the primary care setting. Several large clinical trials indicate that on-site mental health services help primary care providers improve quality of care to depressed primary care patients. This article examines five key questions and suggests two service delivery models to help mental health providers plan initiatives that Integrate medical and behavioral health care in the primary care setting.     

Gender differences in computer-mediated communication: A systematic literature review of online health-related support groups

Objective

Previous research has contended that the unique characteristics of the Internet might remove some of the gender differences that exist in face-to-face healthcare. The aims of the present study were to systematically review studies that have examined gender differences in communication within online health communities.

Methods

A literature search was conducted to identify studies addressing gender differences in messages posted to online health-related support groups. Out of the 1186 articles identified, twelve were retrieved for review.

Results

Half of the studies examined gender differences by comparing male and female cancer discussion boards. The literature review revealed that some gender differences were observed in these studies. However, for studies that analysed mixed-gender communities, gender differences were less evident.

Conclusion

Results seemed to reveal gender differences in communications in single-sex online health support groups, and similarities in communication patterns in mixed-sex online health support groups. However, findings should be treated with caution due to the diversity in studies and methodological issues highlighted in the present review.

Practice implications

There is a need for health care professionals to take into account a range of situational and contextual factors that may affect how men and women use online health support groups. However, more robust research is needed before concrete guidelines can be developed to help health care professionals develop effective online support interventions.     

How pragmatic or explanatory is the randomized, controlled trial? The application and enhancement of the PRECIS tool to the evaluation of a smoking cessation trial

ABSTRACT: BACKGROUND: Numerous explanatory randomized trials support the efficacy of chronic disease interventions, including smoking cessation treatments. However, there is often inadequate adoption of these interventions for various reasons, one being the limitation of generalizability of the explanatory studies in real-world settings. Randomized controlled trials can be rated as more explanatory versus pragmatic along 10 dimensions. Pragmatic randomized clinical trials generate more realistic estimates of effectiveness with greater relevance to clinical practice and for health resource allocation decisions. However, there is no clear method to scale each dimension during the trial design phase to ensure that the design matches the intended purpose of the study. METHODS: We designed a pragmatic, randomized, controlled study to maximize external validity by addressing several barriers to smoking cessation therapy in ambulatory care. We analyzed our design and methods using the recently published 'Pragmatic-Explanatory Continuum Indicatory Summary (PRECIS)' tool, a qualitative method to assess trial design across 10 domains. We added a 20-point numerical rating scale and a modified Delphi process to improve consensus in rating these domains. RESULTS: After two rounds of review, there was consensus on all 10 domains of study design. No single domain was scored as either fully pragmatic or fully explanatory; but overall, the study scored high on pragmatism. CONCLUSIONS: This addition to the PRECIS tool may assist other trial designers working with interdisciplinary co-investigators to rate their study design while building consensus.     

Asthma biologics: Comparing trial designs,patient cohorts and study results

ObjectiveFive biologic therapies have FDA-approved indications for difficult-to-control asthma. The clinical trials that proved the efficacy and safety of these biologics were often similar in their inclusion criteria, study designs, and endpoints. Many of these trials have been reanalyzed post hoc to identify subsets of subjects considered to be enhanced responders. As a result, keeping up with the literature and deciding on the most appropriate biologic for our patients has become increasingly difficult. This review summarizes and compares trial designs, patient cohorts, and study results of the major trials involving these therapies.Data SourcesIncluded are basic science articles, online Food and Drug Administration (FDA) applications, and all the published reports of phase II and phase III clinical trials for FDA-approved asthma biologics.Study SelectionsIncluded are the major phase II and phase III clinical trials of 5 asthma biologics.ResultsBecause of variations in inclusion criteria and natural variations in enrolled cohorts, the baseline clinical traits and severity of study populations in asthma biologic trials differed significantly, which is important because baseline annualized exacerbation rates and blood eosinophilia are both strong predictors of a biologic's success. Notwithstanding, the trial results, when considered together, can help guide care providers in choosing the most appropriate biologic for our patients.ConclusionUnderstanding the details and differences in asthma biologic trial designs, patient cohorts, and in study results will help care providers make more informed decisions when choosing a biologic. We are hopeful this review will serve as a reference to care providers for this purpose.     

Assessing the Internal and External Validity of Mobile Health Physical Activity Promotion Interventions: A Systematic Literature Review Using the RE-AIM Framework

Background

Mobile health (mHealth) interventions are effective in promoting physical activity (PA); however, the degree to which external validity indicators are reported is unclear.

Objective

The purpose of this systematic review was to use the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework to determine the extent to which mHealth intervention research for promoting PA reports on factors that inform generalizability across settings and populations and to provide recommendations for investigators planning to conduct this type of research.

Methods

Twenty articles reflecting 15 trials published between 2000 and 2012 were identified through a systematic review process (ie, queries of three online databases and reference lists of eligible articles) and met inclusion criteria (ie, implementation of mobile technologies, target physical activity, and provide original data). Two researchers coded each article using a validated RE-AIM data extraction tool (reach, efficacy/effectiveness, adoption, implementation, maintenance). Two members of the study team independently abstracted information from each article (inter-rater reliability >90%) and group meetings were used to gain consensus on discrepancies.

Results

The majority of studies were randomized controlled trials (n=14). The average reporting across RE-AIM indicators varied by dimension (reach=53.3%, 2.67/5; effectiveness/efficacy=60.0%, 2.4/4; adoption=11.1%, 0.7/6; implementation=24.4%, 0.7/3; maintenance=0%, 0/3). While most studies described changes in the primary outcome (effectiveness), few addressed the representativeness of participants (reach) or settings (adoption) and few reported on issues related to maintenance and degree of implementation fidelity.

Conclusions

This review suggests that more focus is needed on research designs that highlight and report on both internal and external validity indicators. Specific recommendations are provided to encourage future mHealth interventionists and investigators to report on representativeness, settings, delivery agents for planned interventions, the extent to which protocol is delivered as intended, and maintenance of effects at the individual or organizational level.     

The promise of home visitation: Results of two randomized trials

Two randomized trials of prenatal and infancy home visitation conducted in semirural and urban areas are described and their results presented. The studies provide consistent findings that nurse home visitors can improve women's health related behaviors, qualities of infant caregiving, and can help women improve their own life-course development (reflected in behaviors such as rates of subsequent pregnancies and births, and receipt of welfare). The effects of the program on reducing the rates of dysfunctional care (reflected in rates of child maltreatment and health care encounters for injuries) were concentrated in women with few psychological resources. Reviews of other randomized trials indicate that to produce comparable effects, interventions must include those program ingredients embodied in the model tested in these two trials. © 1998 John Wiley & Sons, Inc.     

Mental health intervention programs in primary care: their scientific basis.

This study examines the scientific basis for mental health intervention programs in primary care. The validity of five underlying assumptions is evaluated, using the results of a naturalistic study covering a representative sample of 25 Dutch family practices and data from the literature. Our findings corroborate the validity of the assumptions. Firstly, our study indicates that mental disorders are indeed very prevalent in primary care settings. Secondly, we find that a substantial proportion of mental disorders is not recognized by the general practitioner (GP). Thirdly, our data show that mental disorders in primary care are not transient or self-limiting. Fourthly, it is shown that only half of the GP attenders with a mental disorder receive some form of mental health treatment in the 14 months after their index consultation. Finally, our data suggest that mental disorders, when identified, can be treated effectively in primary care. These findings are in general agreement with the literature. In the discussion we underscore the need for public health intervention programs targeted at primary care providers. Training programs for general physicians must be directed at improving recognition and diagnosis and at enhancing the availability and quality of mental health interventions. The effectiveness of these programs has to be tested in randomized trials.     

Effects of health coaching on behavioral modification among adults with cardiovascular risk factors: Systematic review and meta-analysis

ObjectivesThis meta-analysis examined effects of health coaching on physical activities, dietary behaviors, health responsibility, stress management, and smoking behaviors among populations with cardiovascular risk factors.MethodsMultiple electronic databases were searched for randomized controlled trials utilizing health coaching for people with cardiovascular risk factors to lead behavioral changes. The included studies were pooled to estimate the effect size for health coaching interventions on each of the health behaviors.ResultsThis meta-analysis included 15 randomized trials. Motivational interviewing and education sessions were common coaching interventions with telephone calls or face-to-face contacts as the main contact methods. Health coaching for health behaviors showed small but significant effect sizes on physical activities, dietary behaviors, health responsibility, and stress management except for smoking behaviors.ConclusionThe study findings support that health coaching can induce positive behavioral changes among individuals with cardiovascular risk factors. Health coaching delivered by either expert or peer coaches would be easy to apply in clinical settings.Practical implicationsHealth care professionals should be aware that health coaching could provide effective motivation strategies to improve compliance of those who need to initiate and maintain their health behaviors. Health coaching could be easily delivered via telephone calls, text messages, or short-term face-to-face coaching.     

The clinical effectiveness of guided self-help versus waiting-list control in the management of anxiety and depression: a randomized controlled trial

BACKGROUND: There are significant barriers to accessing effective psychological therapy in primary care resulting from a lack of suitably trained therapists to meet current demand. More efficient service delivery using minimal interventions (such as bibliotherapy) provided by paraprofessional therapists may be one method of overcoming these problems, and is the subject of attention in the UK and elsewhere. A randomized trial was conducted to test the clinical effectiveness of this model. Assistant psychologists delivered a guided self-help intervention to patients with anxiety and depression who were currently waiting for psychological therapy. METHOD: A total of 114 patients were randomized either to guided self-help or a waiting-list control group. All patients were followed up 3 months later, prior to starting conventional psychological therapy. Measures included self-reported adherence to the intervention, anxiety and depressive symptoms, social functioning and patient satisfaction. RESULTS: Adherence to the guided self-help intervention was acceptable and patients reported satisfaction with the intervention. However, there were no statistically significant differences between groups in anxiety and depression symptoms at 3 months. CONCLUSIONS: The results demonstrate that this model of guided self-help did not provide additional benefit to patients on a waiting list for psychological therapy. The results are considered in the context of possible internal and external validity threats, and compared with previous trials of minimal interventions. The implications of the results for the design of future minimal interventions are considered.     

Clinical trials on the pharmacological treatment of long COVID: A systematic review

The postacute sequelae of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection (PASC), also known as post-acute coronavirus disease 19 (COVID-19) or the long COVID syndrome (long COVID) is an emerging public health concern. A substantial proportion of individuals may remain symptomatic months after initial recovery. An updated review of published and ongoing trials focusing on managing long COVID will help identify gaps and address the unmet needs of patients suffering from this potentially debilitating syndrome. A comprehensive literature search was conducted on the international databases and clinical trial registries from inception to 31 July 2022. This review included 6 published trials and 54 trial registration records. There is significant heterogeneity in the characterization of long COVID and ascertainment of primary outcomes. Most of the trials are focused on individual symptoms of long COVID or isolated organ dysfunction, classified according to cardiovascular, respiratory and functional capacity, neurological and psychological, fatigue, and olfactory dysfunction. Most of the interventions are related to the mechanisms causing the individual symptoms. Although the six published trials showed significant improvement in the symptoms or organ dysfunction studied, these initial studies lack internal and external validity limiting the generalizability. This review provides an update of the pharmacological agents that could be used to treat long COVID. Further standardization of the diagnostic criteria, inclusion of participants with concomitant chronic cardiometabolic diseases and standardization of outcomes will be essential in future clinical trials.