Exploring treatment preferences facilitated recruitment to randomized controlled trials

Objective

To explore how patients' treatment preferences were expressed and justified during recruitment to a randomized controlled trial (RCT) and how they influenced participation and treatment decisions.

Study Design and Setting

Qualitative analysis of audio recordings of recruitment appointments with 93 participants aged 51-70 years in a UK multicenter RCT of localized prostate cancer treatments.

Results

Treatment preferences at recruitment were more complex and dynamic than previously assumed. Most participants expressed views about treatments early in appointments, ranging on a continuum from hesitant to well-formed opinions. As recruiters elicited men’s views and provided detailed evidence-based treatment and study information, some opted for their preference, but many became uncertain and open to RCT recruitment, often accepting a different treatment from their original “preference.” Discussion of treatment preferences did not act as the expected barrier to recruitment but actively enabled many to express their concerns and reach an informed decision that often included RCT participation.

Conclusion

Exploring treatment preferences and providing evidence-based information can improve levels of informed decision making and facilitate RCT participation. Treatment preferences should be reconceptualized from a barrier to recruitment to an integral part of the information exchange necessary for informed decision making about treatments and RCT participation.     

PRECIS研究进展

随机对照试验(RCT)可分为评估效力的解释性试验和评估临床效果的实用性临床试验(PCT)。PCT反映了临床实际条件下的干预效应,具有一定的外推性,但其内部真实性相对较差。与此相反,解释性试验是在理想条件下进行,其内部真实性较好,而外推性较差。但在RCT实际设计中,PCT和解释性试验并不是截然分离的两个极端,有许多RCT同时兼有两种设计的属性。PRECIS通过评价RCT设计的解释和实用两方面,指导如何实行干预和试验设计,使RCT在内部和外部真实性之间达到平衡。目前国内对PRECIS介绍甚少,更未见应用的报道,基于此以下介绍PRECIS基本原理、特点及应用,以期为临床试验设计提供参考。     

Meta-analyses of small numbers of trials often agree with longer-term results

Objective

Many systematic reviews include only a few studies. It is unclear whether recommendations based on these will be correct in the longer term; hence, this article explores whether meta-analyses give reliable results after only a few studies.

Study Design and Setting

Cumulative meta-analysis of data from 65 meta-analyses from 18 Cochrane systematic reviews was carried out. Various measures of closeness to the pooled estimate from all trials after three and five trials were included. Changes during the accumulation of evidence were noted.

Results

The 95% confidence interval included the final estimate in 72% of meta-analyses after three studies and in 83% after five studies. It took a median of four (interquartile range: 1.25-6) studies to get within 10% of the final point estimate. Agreement between the results at three and five studies and the final estimate was not predicted by the number of participants, the number of events, τ², or I². Estimates could still change substantially after many trials were included.

Conclusion

Many of the conclusions drawn from systematic reviews with small numbers of included studies will be correct in the long run, but it is not possible to predict which ones.     

Sensitive Clinical Queries retrieved relevant systematic reviews as well as primary studies: an analytic survey

Objective

To determine how well the previously validated broad and narrow Clinical Queries for treatment, diagnosis, prognosis, and etiology studies, retrieve not only primary studies but also relevant systematic reviews.

Study Design and Setting

Using the Clinical Hedges Database housed at McMaster University, we tested the retrieval performance of the Clinical Queries.

Results

For most purpose categories (therapy, diagnosis, prognosis, and etiology) and most databases (MEDLINE, EMBASE, CINAHL, and PsycINFO), the sensitive (broad) Clinical Queries search terms had sensitivities higher than 90% for retrieving relevant systematic reviews as well as primary studies. When testing specific (narrow) Clinical Queries, in 8 of 12 cases, specificity was 94% or higher, but sensitivity dropped below 50%. For all purpose categories and all databases, performance was improved when combining the sensitive or specific Clinical Queries with our existing sensitive or specific systematic review search filter using the Boolean OR; sensitivities ranged from 90.7% to 99.7% and specificities ranged from 92.4% to 98.0% with sensitivities higher than 50%.

Conclusion

The sensitive Clinical Queries for therapy, diagnosis, prognosis, and etiology perform well in retrieving not only primary studies but also systematic reviews. Search performance can be improved by combining the Clinical Queries with our sensitive or specific systematic review filter.     

The efficacy and cost-effectiveness of a community weight management intervention: a randomized controlled trial of the health weight management demonstration

Purpose

The study investigated the efficacy and cost-effectiveness of a cognitive-behavioral weight management program, complemented by an interactive Web site and brief telephone/e-mail coaching.

Methods

In 2006-2007, 1755 overweight, non-active-duty TRICARE beneficiaries were randomized to one of three conditions with increasing intervention intensity: written materials and basic Web access (RCT1), plus an interactive Web site (RCT2), plus brief telephone/e-mail coaching support (RCT3). The study assessed changes in weight, blood pressure, and physical activity from baseline to 6, 12, and 15-18 months. (Study retention was 31% at 12 months.) Average and incremental cost-effectiveness and cost-offset analyses were conducted.

Results

Participants experienced significant weight loss (− 4.0%, − 4.0%, and − 5.3%, respectively, in each RCT group after 12 months and − 3.5%, − 3.8%, and − 5.1%, respectively, after 15 to 18 months), increased physical activity, and decreased blood pressure. Cost-effectiveness ratios were $900 to $1100/quality-adjusted life year (QALY) for RCT1 and RCT2 and $1900/QALY for RCT3. The cost recovery period to the government was 3 years for RCTs 1 and 2 and 6 years for RCT3.

Conclusion

A relatively inexpensive cognitive-behavioral weight management intervention improved patient outcomes. Extrapolation of savings for the entire TRICARE population would significantly reduce direct medical costs.     

Departures from community equipoise may lead to incorrect inference in randomized trials

Objective

To assess the impact of selective enrollment on the results of randomized controlled trials (RCTs).

Study Design and Setting

We simulated an RCT of arthroscopic partial meniscectomy vs. nonoperative therapy in patients with meniscal tear and osteoarthritis (OA). We estimated efficacy with the risk ratio (RR) comparing the likelihood of clinically important improvement after surgery with that after nonoperative therapy. We assumed that efficacy differs by extent of OA. We simulated four scenarios: (1) nonselective enrollment; (2) higher likelihood of enrolling subjects with mild OA; (3) higher likelihood of enrolling subjects with severe OA; (4) much higher likelihood of enrolling subjects with severe OA. For each scenario, we simulated 100 trials with sample size 340.

Results

With nonselective enrollment, reflecting community equipoise, the results in 100 trials were consistent with those in the underlying population (mean RR: 1.87; 95% confidence interval [95% CI]: 1.57, 2.14). Selective enrollment of subjects with much higher likelihood of severe OA resulted in 28% lower efficacy of surgery (mean RR: 1.34; 95% CI: 0.93, 2.15), with 95% CI containing the true efficacy in just 25% of trials and empirical power of 44%.

Conclusion

Selective enrollment with respect to factors associated with efficacy may affect trial results and lead to inaccurate conclusions.     

Are randomized trials conducted in China or India biased? A comparative empirical analysis

Context

China and India are two emerging forces in undertaking randomized clinical trials. The quality of trials from these countries may affect not just their substantial populations but also their contribution to health policy throughout the world.

Objective

The objectives of this study were to describe and contrast the quality and biases in reports of trials conducted in China and India with a set of “gold standard” trials reported in leading European and North American journals.

Method

A systematic review and comparative empirical analysis of randomized controlled trial reports published in selected Chinese, Indian, and European or North American medical journals were performed. Quality was assessed against a subset of criteria from the CONSORT statement. We compared the rate of reporting of positive outcomes in clinical trials to describe potential bias.

Result

In total, 307 Chinese papers, 117 Indian papers, and 304 Western papers were included. Reports of Indian trials were slightly better than Chinese papers on the trial reporting quality indicators and much better than Chinese papers on reporting patients' ethical issues. However, the gold standard Western trial reports scored considerably higher on all quality criteria. Chinese papers were substantially more likely to report statistically significant results (odds ratio [OR] = 2.96, 95% confidence interval [CI] = 2.23-3.94; P < 0.0001). Indian trials reported a similar rate of positive results to Western papers (OR = 0.92, 95% CI = 0.69-1.24; P = 0.59).

Conclusion

Reporting of trials in major Chinese and Indian journals falls short of that achieved in the gold standard Western journals we appraised and may reflect underlying inadequacies in the design and conduct of these trials. Chinese trials appear biased and may selectively report positive outcomes while ignoring neutral or negative outcomes. Trialists and journal editors in China and India should adopt the CONSORT reporting guidelines, should ensure that a primary outcome is prespecified and reported, and should ensure that analysis is conducted according to the intention-to-treat principle. Ethical questions in the conduct of trials in China must be addressed.     

Issues in the design, analysis and interpretation of condom functionality studies

Background

Male condom functionality studies are typically crossover trials in which enrolled couples use both experimental and latex control condoms for sexual intercourse. Noninferiority of the experimental type is assessed using confidence intervals for differences in breakage and slippage probabilities. Seemingly straightforward, the design, analysis and interpretation of functionality studies are complicated by the choice of noninferiority criterion, study population and the potential for learning effects.

Methods

Power calculations, secondary data analyses and simulations were used to illustrate concerns and make recommendations.

Results

The probability of failure can be too low to draw meaningful conclusions in certain population subgroups. Learning effects among inexperienced users can exaggerate differences in performance and undermine power. A product which is, on average, inferior to latex may still be a viable prophylactic for a large percentage of couples.

Conclusions

Heterogeneity of failure probabilities, combined with small acceptable differences in performance, requires care when selecting study participants. Pilot data, adequate training on condom use and reasonable expectations regarding performance of a new condom type are essential to maximizing the chance of identifying a noninferior product.     

Survival distributions impact the power of randomized placebo-phase design and parallel groups randomized clinical trials

Objectives

The study evaluated the power of the randomized placebo-phase design (RPPD)—a new design of randomized clinical trials (RCTs), compared with the traditional parallel groups design, assuming various response time distributions. In the RPPD, at some point, all subjects receive the experimental therapy, and the exposure to placebo is for only a short fixed period of time.

Study Design and Setting

For the study, an object-oriented simulation program was written in R. The power of the simulated trials was evaluated using six scenarios, where the treatment response times followed the exponential, Weibull, or lognormal distributions. The median response time was assumed to be 355 days for the placebo and 42 days for the experimental drug.

Results

Based on the simulation results, the sample size requirements to achieve the same level of power were different under different response time to treatment distributions. The scenario where the response times followed the exponential distribution had the highest sample size requirement. In most scenarios, the parallel groups RCT had higher power compared with the RPPD.

Conclusion

The sample size requirement varies depending on the underlying hazard distribution. The RPPD requires more subjects to achieve a similar power to the parallel groups design.     

Selecting participants that raise a clinical trial's population attributable fraction can increase the treatment effect within the trial and reduce the required sample size

Background

Detection of modest but worthwhile treatment effects in randomized controlled trials (RCTs) demands trials of large sample size. Approaches to decreasing required size of RCTs while maintaining power are needed.

Objective

The epidemiological concept of population attributable fraction (AF_p) was applied to the population selected for an RCT to assess its role in determining the size of treatment effect and the required sample size. The additional effect of efficacy of treatment specifically among participants at risk for attributable target events (relative risk reduction_{at risk} [RRR_{at risk}]) was also examined.

Results

A model is described which accounts for size of treatment effect in an RCT based on AF_p and RRR_{at risk}: RRR_trial = (AF_p) (RRR_at risk). The increase in RRR_trial resulting from raising AF_p exceeds that possible under the traditional high risk/high response approach to trial design and allows a reduction in required trial sample size. AF_p can be estimated from studies of causation that determine both risk and attributable risk (AR) associated with specific risk factors.

Conclusion

Larger treatment effects within RCTs are enabled by choosing a target outcome having a specific cause and selecting participants at specific risk for that outcome. Using information about phenotypic and genetic predictors of AR may increase our capacity to select trial populations having high AF_p.     

Uptake of methods to deal with publication bias in systematic reviews has increased over time, but there is still much scope for improvement

Objective

To evaluate the measures taken to deal with publication bias across different categories of systematic reviews published in 2006 and to compare these with reviews published in 1996.

Study Design and Setting

PubMed was searched for systematic reviews published in 2006; 100 treatment effect, 50 diagnostic accuracy, 100 risk factor, and 50 gene-disease association reviews were randomly selected.

Results

The use of MEDLINE increased from 74% to 95%; checking references increased from 42% to 73%; use of Cochrane Library increased from 5% to 58%; and use of CINAHL increased from 8% in 1996 to 24% in treatment reviews, 20% in diagnostic reviews, 18% in risk factor reviews, and 0% in genetic reviews published in 2006. A 20% increase was observed for explicit searching of non-English-language studies in all reviews published in 2006. Efforts to search for unpublished studies increased to 61% from 35% in treatment reviews published in 1996. Twenty-six percent of the reviews used funnel plots or related methods to test for publication bias compared with less than 6% in earlier reviews.

Conclusion

Recent reviews show a significant improvement in the measures taken to prevent publication bias. However, few methods exist to deal with publication bias in the nonquantitative findings of systematic reviews.     

Randomized controlled trials of herbal interventions underreport important details of the intervention

Objective

To examine the quality of reporting and predictors of reporting in randomized clinical trials (RCTs) of herbal medicine interventions.

Study Design and Setting

We searched Medical Literature Analysis and Retrieval System Online, Excerpta Medica Database, and Academy of Microscope Enhanced Dentistry up to December 2007 for any English language RCT of 11 commonly used herbal medicine interventions. Two individuals separately and independently assessed all trials using the Consolidated Standards of Reporting Trials (CONSORT) checklist for herbal medicines interventions. We randomly selected 100 of these trials, extracted a set of potential predictor variables identified through a literature search and consultation with experts, and performed a conceptually driven stepwise elimination regression analyses for predictor variables.

Results

The 406 trials reported on average 38% of the information suggested in the checklist. Regression analyses revealed better overall reporting in trials with a participant flow diagram (P = 0.008), those of Panax quinquefolius (P = 0.018), and those published in more recent years (P = 0.02).

Conclusion

Our results indicate that RCTs of herbal medicine interventions frequently do not report important characteristics of the intervention. Trialists should refer to the CONSORT for herbal medicines when reporting their trials.     

Flexible mifepristone and misoprostol administration interval for first-trimester medical termination

Background

The administration interval between mifepristone and misoprostol is usually about 36-48 h, which might affect a woman's choice of method of termination. Unwanted outcomes such as uterine bleeding, painful cramps and psychosocial issues which may occur during this long interval can be altered by a shorter administration interval. A shorter interval will be cost-effective as it saves both women's and clinician's time and other resources. If the waiting time interval between therapeutic interventions could be reduced without compromising efficacy, it will potentially improve compliance, patient acceptability and quality of care.

Study design

A systematic review of randomized controlled trials published from 1999 to 2008 was conducted to assess the evidence for a shorter mifepristone and misoprostol administration interval at first trimester medical termination. Searching strategy included MEDLINE, EMBASE, CLINAHL and Cochrane Library. The primary outcome measure was complete abortion without the need for a surgical procedure.

Results

Five randomized controlled trials (RCT) compared the efficacy of mifepristone and misoprostol administration intervals between 0 and 72 h in 5139 participants. The complete abortion rates varied between 90% and 98%. Although the meta-analysis of pooled data of all RCTs shows no statistically significant difference in efficacy between the shorter and longer dosing intervals, there is a trend toward slightly lower success rates with administration intervals earlier than 8 h.

Conclusions

Overall efficacy of complete abortion is not statistically different between the longer and shorter administration intervals. This might encourage the clinician to adopt a ‘flexible policy’ with fully informed consent and consideration of all circumstances.     

Treatments effects from randomized trials and propensity score analyses were similar in similar populations in an example from cardiac surgery

Objective

Analyses comparing randomized to nonrandomized clinical trials suffer from the fact that the study populations are usually different. We aimed for a comparison of randomized clinical trials (RCTs) and propensity score (PS) analyses in similar populations.

Study Design and Setting

In a systematic review, we “meta-matched” RCTs and PS analyses that compared the off- and the on-pump technique in coronary artery bypass grafting. “Meta-confounders” were summarized in a “meta-propensity score” and were used for “meta-matching.” We compared treatment effects between RCTs and PS analyses for 10 previously defined binary clinical outcomes in this “meta-matched” population as differences in “meta-odds ratios.”

Results

For all clinical outcomes, the estimated differences in “meta-odds ratios” were below an absolute value of 0.15, all confidence intervals included the null.

Conclusions

In our example, treatment effects of off-pump versus on-pump surgery from RCTs and PS analyses were very similar in a “meta-matched” population of studies, indicating that only a small remaining bias is present in PS analyses.     

Reporting of noninferiority trials was incomplete in trial registries

Objective

To examine the registration of noninferiority trials, with a focus on the reporting of study design and noninferiority margins.

Study Design and Setting

Cross-sectional study of registry records of noninferiority trials published from 2005 to 2009 and records of noninferiority trials in the International Standard Randomized Controlled Trial Number (ISRCTN) or ClinicalTrials.gov trial registries. The main outcome was the proportion of records that reported the noninferiority design and margin.

Results

We analyzed 87 registry records of published noninferiority trials and 149 registry records describing noninferiority trials. Thirty-five (40%) of 87 records from published trials described the trial as a noninferiority trial; only two (2%) reported the noninferiority margin. Reporting of the noninferiority design was more frequent in the ISRCTN registry (13 of 18 records, 72%) compared with ClinicalTrials.gov (22 of 69 records, 32%; P = 0.002). Among the 149 records identified in the registries, 13 (9%) reported the noninferiority margin. Only one of the industry-sponsored trial compared with 11 of the publicly funded trials reported the margin (P = 0.001).

Conclusion

Most registry records of noninferiority trials do not mention the noninferiority design and do not include the noninferiority margin. The registration of noninferiority trials is unsatisfactory and must be improved.     

Statistical methods can be improved within Cochrane pregnancy and childbirth reviews

Objectives

To assess statistical methods within systematic reviews of the Cochrane Pregnancy and Childbirth Group (CPCG).

Study Design and Setting

We extracted details about statistical methods within 75 reviews containing at least 10 studies.

Results

The median number of forest plots per review was 52 (min = 5; max = 409). Seven of the 75 reviews assessed publication bias or explained why not. Forty-four of the 75 reviews performed random-effects meta-analyses; just 1 of these justified the approach clinically and none interpreted its pooled result correctly. Of 31 reviews not using random-effects, 26 assumed a fixed-effect given potentially moderate or large heterogeneity (I² > 25%). In their Methods section, 25 (33%) of the 75 reviews said I² was used to decide between fixed-effect and random-effects; however, in 12 of these (48%) reviews, this was not carried out in their Results section. Of 72 reviews with moderate or large heterogeneity, 47 (65%) did not explore the causes of heterogeneity or justify why not.

Conclusion

Within CPCG reviews, publication bias is rarely addressed; heterogeneity is often not appropriately considered, and random-effects analyses are incorrectly interpreted. How these shortcomings impact existing review conclusions needs further investigation, but regardless of this, we recomment the Cochrane Collaboration increase “hands-on” statistical support.     

Electronic reminders did not improve postal questionnaire response rates or response times: a randomized controlled trial

Objective

We aim to evaluate the effectiveness of electronic reminders (ERs) to improve the response rates and time to response of postal questionnaires in a health research setting.

Study Design and Setting

This pragmatic randomized controlled trial (RCT) was nested within a multicenter RCT of yoga for lower back pain. Participants who provided an electronic mail address and/or mobile phone number were randomized to receive an ER or no reminder (controls) on the day they were due to receive a follow-up questionnaire.

Results

One hundred twenty-five participants (32 males and 93 females) mean age 46 (standard deviation: 11, range: 20-65) were randomized to ER (n = 62) or controls (n = 63). Overall 85.6% of participants returned postal questionnaires (87.1% ER group and 84.1% from controls). No significant differences were found between the two groups for response rate (difference between groups = 3.0%, 95% confidence interval [CI] = −10, 16; P = 0.64) or time to response after adjusting for age, gender, and treatment allocation (χ² _[3df] = 7.10; P = 0.07).

Conclusion

In the present RCT, we found little evidence for the effectiveness of ERs to increase response rates or time to respond for the return of questionnaires in this study population group.     

A randomized trial of electronic reminders showed a reduction in the time to respond to postal questionnaires

Objective

To assess the effect of electronic reminders (ERs) on response rate and time to response for the return of postal questionnaires.

Study Design and Setting

This open randomized controlled trial (RCT) was conducted at the University of York. Participants who were taking part in an established RCT and who provided an electronic mail address and/or mobile telephone number were eligible to take part in the study. The intervention group received ERs on the day they were expected to receive postal questionnaires.

Results

One hundred forty-eight participants (19 male and 129 female) aged 47 ± 11 (range, 19-65) years were studied. About 89.2% of participants returned postal questionnaires. There was no difference in questionnaire response rates in control (64 of 74 [86.5%]) vs. intervention (68 of 74 [91.9%]), groups (relative risk = 1.063, 95% confidence interval: 0.949-1.189). Median questionnaire time to response was 4 days less in the intervention group (10.0 ± 0.2; range, 10-14 days) compared with the control group (14.0 ± 1.4; range, 10-23 days) (χ²_1df = 5.27, P = 0.022).

Conclusion

ERs are useful tools for reducing participant time to response for postal questionnaires. We found little evidence for an effect of ERs on response rate for postal questionnaires.