A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers

ObjectiveTo propose a tool to assist trialists in making design decisions that are consistent with their trial's stated purpose.Study Design and SettingRandomized trials have been broadly categorized as either having a pragmatic or explanatory attitude. Pragmatic trials seek to answer the question, “Does this intervention work under usual conditions?,” whereas explanatory trials are focused on the question, “Can this intervention work under ideal conditions?” Design decisions make a trial more (or less) pragmatic or explanatory, but no tool currently exists to help researchers make the best decisions possible in accordance with their trial's primary goal. During the course of two international meetings, participants with experience in clinical care, research commissioning, health care financing, trial methodology, and reporting defined and refined aspects of trial design that distinguish pragmatic attitudes from explanatory.ResultsWe have developed a tool (called PRECIS) with 10 key domains and which identifies criteria to help researchers determine how pragmatic or explanatory their trial is. The assessment is summarized graphically.ConclusionWe believe that PRECIS is a useful first step toward a tool that can help trialists to ensure that their design decisions are consistent with the stated purpose of the trial.     

PRECIS研究进展

随机对照试验(RCT)可分为评估效力的解释性试验和评估临床效果的实用性临床试验(PCT)。PCT反映了临床实际条件下的干预效应,具有一定的外推性,但其内部真实性相对较差。与此相反,解释性试验是在理想条件下进行,其内部真实性较好,而外推性较差。但在RCT实际设计中,PCT和解释性试验并不是截然分离的两个极端,有许多RCT同时兼有两种设计的属性。PRECIS通过评价RCT设计的解释和实用两方面,指导如何实行干预和试验设计,使RCT在内部和外部真实性之间达到平衡。目前国内对PRECIS介绍甚少,更未见应用的报道,基于此以下介绍PRECIS基本原理、特点及应用,以期为临床试验设计提供参考。     

Content of Health Economics Analysis Plans (HEAPs) for Trial-Based Economic Evaluations: Expert Delphi Consensus Survey

ObjectivesHealth economics analysis plans (HEAPs) currently lack consistency, with uncertainty surrounding appropriate content. We aimed to develop a list of essential items that should be included in HEAPs for economic evaluations conducted alongside randomized trials.MethodsA list of potential items for inclusion was developed by examining existing HEAPs. An electronic Delphi survey was conducted among professional health economists. Respondents were asked to rate potential items from 1 (least important) to 9 (most important), suggest additional items, and comment on proposed items (round 1). A second survey (round 2) was emailed to participants, including the participant’s own scores from round 1 along with summary results from the whole panel; participants were asked to rerate each item. Consensus criteria for inclusion in the final list were predefined as >70% of participants rating an item 7-9 and <15% rating it 1-3 after round 2. A final item selection meeting was held to scrutinize the results and adjudicate on items lacking consensus.Results62 participants completed round 1 of the survey. The initial list included 72 potential items; all 72 were carried forward to round 2, and no new items were added. 48 round 1 respondents (77.4%) completed round 2 and reached consensus on 53 items. At the final meeting, the expert panel (n = 9) agreed that 58 items should be included in the essential list, moved 9 items to an optional list, and dropped 5 items.ConclusionsVia expert consensus opinion, this study identified 58 items that are considered essential in a HEAP.     

Core Items for a Standardized Resource Use Measure: Expert Delphi Consensus Survey

Background

Resource use measurement by patient recall is characterized by inconsistent methods and a lack of validation. A validated standardized resource use measure could increase data quality, improve comparability between studies, and reduce research burden.

Identification and expert panel rating of key structural approaches applied in health economic obesity models

ObjectivesThis study aims to assess the key structural modelling approaches applied in published obesity models, and to provide an expert consensus to improve the methodology and consistency of the application of decision-analytic modelling in obesity research.MethodsUsing a previously published systematic literature search as basis, ten individual interviews, and a face-to-face expert panel meeting were conducted. Within the expert panel meeting, the interview findings were presented and discussed, rated and where possible consensus statements were obtained. In particular, five topics of interest were assessed: time horizon, model type, obesity-related clinical events simulated, event simulation approaches and external event validation.ResultsIn addition to generic modelling standards, several obesity-specific recommendations were generated: Simulating a lifetime horizon was regarded as optimal (100% agreement); Ideally, both short and long-term results should be presented (100%); Using a risk equation approach for simulating the clinical events was the most preferred approach (60%) followed by applying a body mass index (BMI) related relative risk to a base risk estimate (30%); Continuous BMI approaches were preferred (100%); An individual patient/microsimulation state transition model was regarded as preferred modelling approach (90%); Discrete event simulation (DES) was regarded as the most flexible approach for building an obesity model but it was recognised as complex, and more difficult to build, populate and to disseminate; Performing an external validation was rated as important (100%).ConclusionsThe obtained insights, discussion and consensus can provide valuable information for developing decision-analytic models to generate high-quality and transparent economic evidence for obesity interventions.     

The CARE guidelines: consensus-based clinical case report guideline development

BackgroundA case report is a narrative that describes, for medical, scientific, or educational purposes, a medical problem experienced by one or more patients. Case reports written without guidance from reporting standards are insufficiently rigorous to guide clinical practice or to inform clinical study design.Primary ObjectiveDevelop, disseminate, and implement systematic reporting guidelines for case reports.MethodsWe used a three-phase consensus process consisting of (1) pre-meeting literature review and interviews to generate items for the reporting guidelines, (2) a face-to-face consensus meeting to draft the reporting guidelines, and (3) post-meeting feedback, review, and pilot testing, followed by finalization of the case report guidelines.ResultsThis consensus process involved 27 participants and resulted in a 13-item checklist—a reporting guideline for case reports. The primary items of the checklist are title, key words, abstract, introduction, patient information, clinical findings, timeline, diagnostic assessment, therapeutic interventions, follow-up and outcomes, discussion, patient perspective, and informed consent.ConclusionsWe believe the implementation of the CARE (CAse REport) guidelines by medical journals will improve the completeness and transparency of published case reports and that the systematic aggregation of information from case reports will inform clinical study design, provide early signals of effectiveness and harms, and improve healthcare delivery.     

PRECIS-2:基于研究目标的试验设计

采用随机对照试验（RCT）评价的干预措施效果，涉及理论疗效和实际的临床效果两方面。PRECIS（Pragmatic-Explanatory Continuum Indicator Summary）通过评价RCT设计的解释性和实用性两方面的程度，指导研究者如何实行干预和试验设计，使RCT在内部真实性和外部真实性之间达到一个平衡。在临床试验方案设计过程中，PRECIS的作用逐渐被研究者所认同。为了确保试验的设计与结果相关联，便于患者、临床医生和政策决策者应用，国际多团队专家对PRECIS进行了修正和升级，提出了新的PRECIS-2。PRECIS-2的9个维度都是为了帮助试验设计者思考研究设计的预期效果与试验结果的一致性和特定场景的适用性。本文旨在介绍PRECIS-2的研发、基本原理、特点及应用，以期为国内临床试验设计、决策提供帮助。     

Defining malnutrition: Mission or mission impossible?

ObjectiveAlthough screening for malnutrition in health care has expanded enormously, a gold standard for the optimal definition and operationalism of malnutrition is still lacking. This report reflects expert opinions on the elements of the definition and operationalism of malnutrition and is meant to trigger further debate within the nutritional societies.MethodsA Delphi study was performed consisting of three phases. After a literature review (phase 1), questions for a semistructured interviews (phase 2) were formulated. Subsequently, the results of these semistructured interviews were used to develop the final list of elements (for defining and operationalism of malnutrition). In phase 3 (final phase), experts were asked to provide written feedback regarding the ranking of elements concerning the importance of these elements.ResultsTwenty-two experts (response 73.3%) were included in the final phase of this Delphi study. No overall agreement could be reached. The elements deficiencies of energy or protein and decrease in fat-free mass were most often mentioned to be particularly important in defining malnutrition. Elements mentioned to be important in operationalism of malnutrition were involuntary weight loss, body mass index, and no nutritional intake. Opinions on cutoff points regarding these elements differed strongly among experts.ConclusionThis study shows that there is no full agreement among experts on the elements defining and operationalism of malnutrition. The results of this study may fuel the discussion within the nutritional societies, which will most ideally lead to an international consensus on a definition and operationalism of malnutrition.     

Pragmatic vs. explanatory: an adaptation of the PRECIS tool helps to judge the applicability of systematic reviews for daily practice

Objective

The Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) tool was designed to classify randomized clinical trials (RCT) as being more pragmatic or explanatory. We modified the PRECIS tool (called PRECIS-Review tool [PR-tool]) to grade individual trials and systematic reviews of trials. This should help policy makers, clinicians, researchers, and guideline developers to judge the applicability of individual trials and systematic reviews.

Study Design and Setting

To illustrate the usefulness and applicability of the PR-tool, we applied it to two systematic reviews. Each included RCT was scored on the 10 PRECIS domains on a scale of 1-5. After this scoring, a 10-domain average for each individual trial and for the systematic review a single domain average and an overall average was calculated.

Results

One review was more pragmatic with an average score of 3.7 (range, 2.9-4.6) on our PR-tool, whereas the other review was more explanatory with an average score of 1.9 (range, 1.1-3.3). The results also suggest that the included studies within each systematic review were rather uniform in their approach, although some domains seemed more prone to heterogeneity.

Conclusion

The PR-tool provides a useful estimate that gives insight by estimating quantitatively how pragmatic each RCT in the review is, which methodological domains are pragmatic or explanatory, and how pragmatic the review is.     

Enhancing credibility of chemical safety studies: emerging consensus on key assessment criteria

Objectives

We examined the extent to which consensus exists on the criteria that should be used for assessing the credibility of a scientific work, regardless of its funding source, and explored how these criteria might be implemented.

Data sources

Three publications, all presented at a session of the 2009 annual meeting of the Society for Risk Analysis, have proposed a range of criteria for evaluating the credibility of scientific studies. At least two other similar sets of criteria have recently been proposed elsewhere.

Data extraction/synthesis

In this article we review these criteria, highlight the commonalities among them, and integrate them into a list of 10 criteria. We also discuss issues inherent in any attempt to implement the criteria systematically.

Conclusions

Recommendations by many scientists and policy experts converge on a finite list of criteria for assessing the credibility of a scientific study without regard to funding source. These criteria should be formalized through a consensus process or a governmental initiative that includes discussion and pilot application of a system for reproducibly implementing them. Formal establishment of such a system should enable the debate regarding chemical studies to move beyond funding issues and focus on scientific merit.     

An International Consensus List of Potentially Clinically Significant Drug-Drug Interactions in Older People

ObjectivesWe aimed to establish an explicit list of potentially clinically significant drug-drug interactions (DDIs) in people aged ≥65 years.DesignA preliminary list of potentially clinically significant DDIs was compiled, based on 154 DDIs identified from literature review. Subsequently, a 2-round online Delphi survey was undertaken with a multidisciplinary expert panel. A consensus meeting and a final round were conducted to validate the final DDI list and the scope of information provided.Setting and ParticipantsTwenty nine experts, including geriatricians and clinical pharmacists from 8 European countries.MeasuresFor each DDI, in the first 2 rounds, experts were asked to score the severity of potential harm on a 5-point Likert-type scale. DDIs were directly included on the final list if the median score was 4 (major) or 5 (catastrophic). DDIs with a median score of 3 (moderate) were discussed at a consensus meeting and included if ≥75% of participants voted for inclusion in the final round.ResultsConsensus was achieved on 66 potentially clinically significant DDIs (28 had a median score of 4/5 and 48 of 3 in the Delphi survey). Most concerned cardiovascular, antithrombotic, and central nervous system drugs. The final list includes information on the mechanism of interaction, harm, and management. Treatment modification is recommended for three-quarters of DDIs.Conclusion and ImplicationsWe validated a list of potentially clinically significant DDIs in older people, which can be used in clinical practice and education to support identification and management of DDIs or to assess prevalence in epidemiologic and intervention studies.     

Systematic Review of Interdisciplinary Interventions in Nursing Homes

BackgroundThe role of interdisciplinary interventions in the nursing home (NH) setting remains unclear. We conducted a systematic evidence review to study the benefits of interdisciplinary interventions on outcomes of NH residents. We also examined the interdisciplinary features of successful trials, including those that used formal teams.Data SourcesMedline was searched from January 1990 to August 2011. Search terms included residential facilities, long term care, clinical trial, epidemiologic studies, epidemiologic research design, comparative study, evaluation studies, meta-analysis and guideline.Study SelectionWe included randomized controlled trials (RCTs) evaluating the efficacy of interdisciplinary interventions conducted in the NH setting.MeasurementsWe used the Cochrane Collaboration tools to appraise each RCT, and an RCT was considered positive if its selected intervention had a significant positive effect on the primary outcome regardless of its effect on any secondary outcome. We also extracted data from each trial regarding the participating disciplines; for trials that used teams, we studied the reporting of various team elements, including leadership, communication, coordination, and conflict resolution.ResultsWe identified 27 RCTs: 7 had no statistically significant effect on the targeted primary outcome, 2 had a statistically negative effect, and 18 demonstrated a statistically positive effect. Participation of residents’ own primary physicians (all 6 trials were positive) and/or a pharmacist (all 4 trials were positive) in the intervention were common elements of successful trials. For interventions that used formal team meetings, presence of communication and coordination among team members were the most commonly observed elements.ConclusionOverall interdisciplinary interventions had a positive impact on resident outcomes in the NH setting. Participation of the residents’ primary physician and/or a pharmacist in the intervention, as well as team communication and coordination, were consistent features of successful interventions.     

Empirical assessment suggests that existing evidence could be used more fully in designing randomized controlled trials

BackgroundMeta-analyses of randomized controlled trials (RCTs) provide the highest level of evidence regarding the effectiveness of interventions. Less is known about how they are used to inform the design and reporting of RCTs.MethodsA sample of RCTs published in leading medical journals in 2007 was assessed to establish whether authors considered previous trials in the design of their trial. An approach to calculate the sample size required for a significant pooled effect in an updated meta-analysis was applied to a subsample of the RCTs to illustrate the ways in which the results of an existing meta-analysis can be incorporated into the planning and reporting of new RCTs.ResultsSix of the 27 trials assessed (22%) reported the use of previous trial(s) for sample size calculations. Meta-analyses relating the results of the trial to previous research were cited in 37% (10 out of 27) of the report discussion sections. Previous evidence is formally incorporated into retrospective sample size calculations for three of the trials.Discussion/ConclusionConsulting previous research before embarking on a new trial and basing decisions about future research on the impact on an updated meta-analysis will make the reporting of research more coherent and the design of new RCTs more efficient.     

Developing a Fracture Risk Clinical Assessment Protocol for Long-Term Care: A Modified Delphi Consensus Process

ObjectivesTo develop a fracture risk Clinical Assessment Protocol (CAP) based on long-term care (LTC) fracture prevention recommendations and an embedded fracture risk assessment tool.DesignA modified Delphi consensus approach including 2 survey rounds and a face-to-face meeting was implemented to reach consensus on matching of LTC fracture prevention guideline statements to Fracture Risk Scale (FRS) risk levels.Setting and ParticipantsA national panel of recognized experts in osteoporosis, fractures, and long-term care, including an LTC resident and family members.MethodsRound 1 survey respondents (n = 24) were provided the LTC fracture prevention guidelines matched to FRS risk levels and were asked whether they agreed the guideline was appropriate for the risk level (yes, no, I don't know, I agree with some but not all of it) and to provide comments. In round 2, guideline statements that did not achieve consensus (≥80% agreement) were revised consistent with comments provided in round 1 and respondents were asked again if they agreed with the guideline statement. Statements that did not achieve consensus were to be discussed and resolved in an in-person meeting (n = 17).ResultsIn round 1 (75% response rate), consensus was achieved in 7/14 guideline statements. In round 2 (56% response rate), 5 statements were revised based on round 1 feedback and for 2 statements additional information was provided. Consensus was achieved in all but one statement related to the inappropriateness of pharmacologic therapy for residents with life expectancy less than 1 year. Following facilitated meeting discussions, consensus was obtained to revise the guideline statement to reflect that life expectancy was but one of several criteria that should be used to inform medication decisions.Conclusions and ImplicationsAn evidence-based fracture risk CAP was developed that will be embedded in international routine clinical assessment tools to guide fracture prevention in LTC.     

Proposals for person‐centred care in the COVID‐19 era. Delphi study

BackgroundIn this COVID‐19 era, we need to rethink the criteria used to measure the results of person‐centred care strategies.ObjectiveTo identify priorities, and criteria that health services can use to pursue actually the goal of achieving person‐centred care.DesignThree‐phase online qualitative study performed during May–July of 2020 using the Delphi technique.Setting and ParticipantsAn online platform was used for a consensus meeting of 114 participants, including health planning experts, health‐care institution managers, clinicians and patients.Main Outcome MeasuresCriteria and indicators for the achievement of person‐centred care.Main ResultsThe first round began with 125 proposals and 11 dimensions. After the second round, 28 ideas reached a high level of consensus among the participants. Ultimately, the workgroup agreed on 20 criteria for goals in the implementation of person‐centred care during the COVID‐19 era and 21 related indicators to measure goal achievement.DiscussionNine dimensions and 28 priorities were identified. These priorities are also in accordance with the quadruple aim approach, which emphasizes the need for care for health‐care professionals, without whom it is impossible to achieve a better quality of care.ConclusionsPerson‐centred care continues to be a key objective. However, new metrics are needed to ensure its continued development during the restoration of public health services beyond the control of COVID‐19.Patient or Public ContributionTwelve professionals and patient representatives participated voluntarily in the construction of the baseline questionnaire and in the selection of the criteria and indicators using an online platform for consensus meetings.     

Attitudes to evidence in acupuncture: an interview study

The use of complementary and alternative medicine is increasing in the Western world. However, there is no clear evidence of effect of alternative therapies. Moreover, there is no consensus between practitioners and researchers as to the right way of assessing the efficacy of alternative therapies. To investigate practitioners?? perspective on evidence and ways of assessing efficacy twelve in-depth interviews were conducted in Denmark with acupuncturists, including physicians practising acupuncture, acupuncturists with a health-related background, and acupuncturists without a health-related background. Two themes predominated the study, first, the interviewees?? general reflections on how it is possible to establish knowledge about an effect of acupuncture; and second the interviewees?? reflections on the use of randomized controlled trials in acupuncture, including obstacles and alternatives to conducting randomized controlled trials. Further, two conceptions of what constitutes evidence were identified: a biomedical conception and an experience-based conception. Most interviewees were sceptical about the use of randomized controlled trials in acupuncture. Two reasons, especially, were given for this scepticism. First, practical and instrumental reasons concerning the specific elements of the randomized controlled trial or relating to limited resources; and second, value-based reasons are concerning the nature of acupuncture. However, the interviewees were really opposed only to a certain kind of randomized controlled trial, the so-called explanatory trial. They would actually welcome a pragmatic trial. The study gives valuable insight into an under explored field and provide a platform for further investigation, and a better informed discussion of the subject.     

Nature and Predictors of Response Changes in Modified-Delphi Panels

ObjectivesTo describe the extent and nature of changes in participants’ responses after exposure to group feedback and discussion in modified-Delphi panels and to identify factors affecting those changes.MethodsWe analyzed data from 2 online modified-Delphi panels, each consisting of 2 rating rounds and an online discussion round. We included responses from 55 participants who answered 38 questions in both rating rounds. Because not all participants answered each question twice, our sample consisted of 1846 cases (response changes). We used mixed-effect logistic and multinomial logistic regression to identify factors predicting response changes and their direction relative to group median—our consensus measure.ResultsParticipants changed, on average, 49% of their responses. A response was changed in 47% of the 1846 cases: 28% of responses were changed toward consensus and 19% away from it. Although some measures of subjective participation experiences had a marginally significant impact on the propensity and direction of response changes, several objective measures of discussion engagement were statistically significant predictors of both the presence and direction of response changes.ConclusionOur results illustrate the nature of response changes and highlight the importance of exposing participants to alternative perspectives and encouraging them to explain their perspectives.