Impact Evaluation of CONSORT and STRICTA Guidelines on Reporting Quality for Randomized Controlled Trials of Acupuncture Conducted in China

Objective:To evaluate and compare the reports' qualities of acupuncture randomized controlled trials(RCTs) conducted in China before and after the implementation of two guidelines,i.e.,the Consolidated Standards of Reporting Trials(CONSORT) statement and the Standards for Reporting Interventions in Controlled Trials of Acupuncture(STRICTA).Methods:Proportions of studies that reported CONSORT and STRICTA items were compared for the years before and after implementation of these two guidelines.In addition,the total score of each item's reporting were calculated and reported differences during different date ranges were compared.Results:For CONSORT items(maximum score 8),there was evidence of a slight improvement in reporting between 1994-1995 and 1999-2000 combined(2.5 + 0.6) and 2004-2005 and2009-2010 combined(3.0 ±0.9;difference 0.4,95%confidence interval,0.3 to 0.6,P0.01).For STRICTA items(maximum score 17),there was evidence that a slight improvement in reporting between 1994-1995 and 1999-2000 combined(8.6±2.1) and 2004-2005 and 2009-2010 combined(10.1 ±1.8;difference1.5,95%confidence interval,1.1 to 1.9,P0.01).Conclusion:Quality of reporting for RCTs of acupuncture treatment conducted in China have generally improved since the introduction of the STRICTA and CONSORT guidelines.     

Guided medication dosing for elderly emergency patients using real-time, computerized decision support

Objective

To evaluate the impact of a real-time computerized decision support tool in the emergency department that guides medication dosing for the elderly on physician ordering behavior and on adverse drug events (ADEs).

Design

A prospective controlled trial was conducted over 26 weeks. The status of the decision support tool alternated OFF (7/17/06–8/29/06), ON (8/29/06–10/10/06), OFF (10/10/06–11/28/06), and ON (11/28/06–1/16/07) in consecutive blocks during the study period. In patients ≥65 who were ordered certain benzodiazepines, opiates, non-steroidals, or sedative-hypnotics, the computer application either adjusted the dosing or suggested a different medication. Physicians could accept or reject recommendations.

Measurements

The primary outcome compared medication ordering consistent with recommendations during ON versus OFF periods. Secondary outcomes included the admission rate, emergency department length of stay for discharged patients, 10-fold dosing orders, use of a second drug to reverse the original medication, and rate of ADEs using previously validated explicit chart review.

Results

2398 orders were placed for 1407 patients over 1548 visits. The majority (49/53; 92.5%) of recommendations for alternate medications were declined. More orders were consistent with dosing recommendations during ON (403/1283; 31.4%) than OFF (256/1115; 23%) periods (p≤0.0001). 673 (43%) visits were reviewed for ADEs. The rate of ADEs was lower during ON (8/237; 3.4%) compared with OFF (31/436; 7.1%) periods (p=0.02). The remaining secondary outcomes showed no difference.

Limitations

Single institution study, retrospective chart review for ADEs.

Conclusion

Though overall agreement with recommendations was low, real-time computerized decision support resulted in greater acceptance of medication recommendations. Fewer ADEs were observed when computerized decision support was active.     

Key principles for a national clinical decision support knowledge sharing framework: synthesis of insights from leading subject matter experts

Objective

To identify key principles for establishing a national clinical decision support (CDS) knowledge sharing framework.

Materials and methods

As part of an initiative by the US Office of the National Coordinator for Health IT (ONC) to establish a framework for national CDS knowledge sharing, key stakeholders were identified. Stakeholders'' viewpoints were obtained through surveys and in-depth interviews, and findings and relevant insights were summarized. Based on these insights, key principles were formulated for establishing a national CDS knowledge sharing framework.

Results

Nineteen key stakeholders were recruited, including six executives from electronic health record system vendors, seven executives from knowledge content producers, three executives from healthcare provider organizations, and three additional experts in clinical informatics. Based on these stakeholders'' insights, five key principles were identified for effectively sharing CDS knowledge nationally. These principles are (1) prioritize and support the creation and maintenance of a national CDS knowledge sharing framework; (2) facilitate the development of high-value content and tooling, preferably in an open-source manner; (3) accelerate the development or licensing of required, pragmatic standards; (4) acknowledge and address medicolegal liability concerns; and (5) establish a self-sustaining business model.

Discussion

Based on the principles identified, a roadmap for national CDS knowledge sharing was developed through the ONC''s Advancing CDS initiative.

Conclusion

The study findings may serve as a useful guide for ongoing activities by the ONC and others to establish a national framework for sharing CDS knowledge and improving clinical care.     

The impact of an electronic health record on nurse sensitive patient outcomes: an interrupted time series analysis

Objectives

To evaluate the impact of electronic health record (EHR) implementation on nursing care processes and outcomes.

Design

Interrupted time series analysis, 2003–2009.

Setting

A large US not-for-profit integrated health care organization.

Participants

29 hospitals in Northern and Southern California.

Intervention

An integrated EHR including computerized physician order entry, nursing documentation, risk assessment tools, and documentation tools.

Main outcome measures

Percentage of patients with completed risk assessments for hospital acquired pressure ulcers (HAPUs) and falls (process measures) and rates of HAPU and falls (outcome measures).

Results

EHR implementation was significantly associated with an increase in documentation rates for HAPU risk (coefficient 2.21, 95% CI 0.67 to 3.75); the increase for fall risk was not statistically significant (0.36; −3.58 to 4.30). EHR implementation was associated with a 13% decrease in HAPU rates (coefficient −0.76, 95% CI −1.37 to −0.16) but no decrease in fall rates (−0.091; −0.29 to 0.11). Irrespective of EHR implementation, HAPU rates decreased significantly over time (−0.16; −0.20 to −0.13), while fall rates did not (0.0052; −0.01 to 0.02). Hospital region was a significant predictor of variation for both HAPU (0.72; 0.30 to 1.14) and fall rates (0.57; 0.41 to 0.72).

Conclusions

The introduction of an integrated EHR was associated with a reduction in the number of HAPUs but not in patient fall rates. Other factors, such as changes over time and hospital region, were also associated with variation in outcomes. The findings suggest that EHR impact on nursing care processes and outcomes is dependent on a number of factors that should be further explored.     

An international expert survey on acupuncture in randomized controlled trials for low back pain and a validation of the low back pain acupuncture score

Background

Acupuncture is a promising treatment approach in patients with chronic low back pain (cLBP) but little is known about the quality of acupuncture in randomized controlled trials (RCT) of acupuncture cLBP.

Objective

To determine how international experts (IES) rate the quality of acupuncture in RCTs of cLBP; independent international validation of the Low Back Pain Acupuncture Score (LBPAS).

Methodology

Fifteen experts from 9 different countries outside China were surveyed (IES). They were asked to read anonymized excerpts of 24 RCTs of cLBP and answer a three-item questionnaire on how the method of acupuncture conformed to 1) Chinese textbook standards, 2) the expert''s personally preferred style, and 3) how acupuncture is performed in the expert''s country. Likert scale rating, calculation of the mode for each answer, and Spearman''s rank correlation coefficient between all three answers and the LBPAS were calculated.

Results

On comparison with Chinese textbook standards (question 1), 6 RCTs received a good rating, 8 trials a fair and 10 trials a poor or very poor rating. 5 of the 6 trials rated good, received at least a good rating also in question 2 or 3. We found a high correlation of 0.85 (p < 0.0001) between the IES and LBPAS ratings for question 1 and question 2, and a correlation of 0.66 (p < 0.0001) for question 3.

Conclusion

The international expert survey (IES) revealed that only 6 out of 24 (25%) RCTs of acupuncture for cLBP were rated "good" in respect to Chinese textbook acupuncture standards. There were only small differences in how the acupuncture quality was rated in comparison to Chinese textbook acupuncture, personally preferred and local styles of acupuncture. The rating showed a high correlation with the Low Back Pain Acupuncture Score LBPAS.     

Presence of key findings in the medical record prior to a documented high-risk diagnosis

Background

Failure or delay in diagnosis is a common preventable source of error. The authors sought to determine the frequency with which high-information clinical findings (HIFs) suggestive of a high-risk diagnosis (HRD) appear in the medical record before HRD documentation.

Methods

A knowledge base from a diagnostic decision support system was used to identify HIFs for selected HRDs: lumbar disc disease, myocardial infarction, appendicitis, and colon, breast, lung, ovarian and bladder carcinomas. Two physicians reviewed at least 20 patient records retrieved from a research patient data registry for each of these eight HRDs and for age- and gender-compatible controls. Records were searched for HIFs in visit notes that were created before the HRD was established in the electronic record and in general medical visit notes for controls.

Results

25% of records reviewed (61/243) contained HIFs in notes before the HRD was established. The mean duration between HIFs first occurring in the record and time of diagnosis ranged from 19 days for breast cancer to 2 years for bladder cancer. In three of the eight HRDs, HIFs were much less likely in control patients without the HRD.

Conclusions

In many records of patients with an HRD, HIFs were present before the HRD was established. Reasons for delay include non-compliance with recommended follow-up, unusual presentation of a disease, and system errors (eg, lack of laboratory follow-up). The presence of HIFs in clinical records suggests a potential role for the integration of diagnostic decision support into the clinical workflow to provide reminder alerts to improve the diagnostic focus.     

Clinical decision support implemented with academic detailing improves prescribing of key renally cleared drugs in the hospital setting

Objective

Lack of dose adjustment for renally cleared drugs in the presence of poor renal function is a common problem in the hospital setting. The absence of a clinical decision support system (CDSS) from direct clinician workflows such as computerized provider order entry (CPOE) hinders the uptake of CDSS. This study implemented CDSS in an environment independent of CPOE, introduced to prescribers via academic detailing, to address the dosing of renally cleared drugs.

Design

GFR+ was designed to automatically calculate and update renal function, doses of key drugs adjusted for renal function, and highlight clinically significant decreases in renal function. Prescribers were made aware of GFR+, its navigation, and surrounding clinical issues, using academic detailing.

Measurement

The rate of dosing conformity and management for key renally cleared drugs in hospitalized patients, before and after GFR+ implementation.

Results

Improvements were seen in dosing conformity for enoxaparin (from 68% to 86%, p=0.03), gentamicin (63–87%, p=0.01), and vancomycin (47–77%, p=0.07), as well as the appropriate use of gentamicin therapeutic drug monitoring (70–90%, p=0.02). During episodes of acute renal impairment, renally cleared drugs were held on 38% of instances in the pre-intervention period compared with 62% post-intervention (p=0.01).

Conclusion

Clinical decision support implemented with academic detailing improved dosing conformity and management of key renally cleared drugs in a hospitalized population. Academic detailing should be strongly considered to facilitate the introduction of CDSS systems that cannot be placed directly into the clinician workflow.     

Clinical decision support with automated text processing for cervical cancer screening

Objective

To develop a computerized clinical decision support system (CDSS) for cervical cancer screening that can interpret free-text Papanicolaou (Pap) reports.

Materials and Methods

The CDSS was constituted by two rulebases: the free-text rulebase for interpreting Pap reports and a guideline rulebase. The free-text rulebase was developed by analyzing a corpus of 49 293 Pap reports. The guideline rulebase was constructed using national cervical cancer screening guidelines. The CDSS accesses the electronic medical record (EMR) system to generate patient-specific recommendations. For evaluation, the screening recommendations made by the CDSS for 74 patients were reviewed by a physician.

Results and Discussion

Evaluation revealed that the CDSS outputs the optimal screening recommendations for 73 out of 74 test patients and it identified two cases for gynecology referral that were missed by the physician. The CDSS aided the physician to amend recommendations in six cases. The failure case was because human papillomavirus (HPV) testing was sometimes performed separately from the Pap test and these results were reported by a laboratory system that was not queried by the CDSS. Subsequently, the CDSS was upgraded to look up the HPV results missed earlier and it generated the optimal recommendations for all 74 test cases.

Limitations

Single institution and single expert study.

Conclusion

An accurate CDSS system could be constructed for cervical cancer screening given the standardized reporting of Pap tests and the availability of explicit guidelines. Overall, the study demonstrates that free text in the EMR can be effectively utilized through natural language processing to develop clinical decision support tools.     

A 30 year perspective of the quality of evidence published in 25 clinical journals: Signs of change?

Methods

The quality of clinical studies published in five different specialties, over three decades was evaluated. Computerised search of the Medline database was undertaken to evaluate the articles published in 25 clinical journals in 1983, 1993, and 2003 from five different specialties (medicine, surgery, paediatrics, anaesthesia, and psychiatry). The number of randomised controlled trials (RCTs), meta‐analyses, and other clinical trials (non‐RCT) were noted.

Results

From the 27 030 articles evaluated, there were 2283 (8.4%) RCTs, 166 (0.6%) meta‐analyses, and 4153 (15.4%) other clinical trials. For the proportion of RCTs, the rank order of the specialties was; anaesthesia (503; 18%), psychiatry (294; 9.6%), medicine (899; 8.1%), paediatrics (326; 6.4%), and surgery (261; 5.3%) (p<0.001). For the proportion of meta‐analysis, the rank order of the specialties was; psychiatry (36; 1.2%), medicine (105; 0.9%), paediatrics (15; 0.3%), anaesthesia (6; 0.2%), and surgery (4; 0.1%) (p<0.001). Overall, from 1983 to 2003, there were increases in the proportion of RCTs (449, 5.9% to 1027, 9.6%), meta‐analysis (0, 0% to 127, 1.2%), and other clinical trials (897, 12% to 1983, 19%) (p<0.001). This trend was apparent in each clinical specialty (p<0.001).

Conclusions

Over the three decades evaluated, clinical trials, notably RCTs and meta‐analysis form only a small proportion of articles published in prominent journals from five clinical specialties. This is notwithstanding the modest increases in the proportions of RCTs and meta‐analysis over the same period.     

Impact of a web-based personally controlled health management system on influenza vaccination and health services utilization rates: a randomized controlled trial

Objective

To assess the impact of a web-based personally controlled health management system (PCHMS) on the uptake of seasonal influenza vaccine and primary care service utilization among university students and staff.

Materials and methods

A PCHMS called Healthy.me was developed and evaluated in a 2010 CONSORT-compliant two-group (6-month waitlist vs PCHMS) parallel randomized controlled trial (RCT) (allocation ratio 1:1). The PCHMS integrated an untethered personal health record with consumer care pathways, social forums, and messaging links with a health service provider.

Results

742 university students and staff met inclusion criteria and were randomized to a 6-month waitlist (n=372) or the PCHMS (n=370). Amongst the 470 participants eligible for primary analysis, PCHMS users were 6.7% (95% CI: 1.46 to 12.30) more likely than the waitlist to receive an influenza vaccine (waitlist: 4.9% (12/246, 95% CI 2.8 to 8.3) vs PCHMS: 11.6% (26/224, 95% CI 8.0 to 16.5); χ²=7.1, p=0.008). PCHMS participants were also 11.6% (95% CI 3.6 to 19.5) more likely to visit the health service provider (waitlist: 17.9% (44/246, 95% CI 13.6 to 23.2) vs PCHMS: 29.5% (66/224, 95% CI: 23.9 to 35.7); χ²=8.8, p=0.003). A dose–response effect was detected, where greater use of the PCHMS was associated with higher rates of vaccination (p=0.001) and health service provider visits (p=0.003).

Discussion

PCHMS can significantly increase consumer participation in preventive health activities, such as influenza vaccination.

Conclusions

Integrating a PCHMS into routine health service delivery systems appears to be an effective mechanism for enhancing consumer engagement in preventive health measures.

Trial registration

Australian New Zealand Clinical Trials Registry ACTRN12610000386033. http://www.anzctr.org.au/trial_view.aspx?id=335463.     

Effects of an online personal health record on medication accuracy and safety: a cluster-randomized trial

Objective

To determine the effects of a personal health record (PHR)-linked medications module on medication accuracy and safety.

Design

From September 2005 to March 2007, we conducted an on-treatment sub-study within a cluster-randomized trial involving 11 primary care practices that used the same PHR. Intervention practices received access to a medications module prompting patients to review their documented medications and identify discrepancies, generating ‘eJournals’ that enabled rapid updating of medication lists during subsequent clinical visits.

Measurements

A sample of 267 patients who submitted medications eJournals was contacted by phone 3 weeks after an eligible visit and compared with a matched sample of 274 patients in control practices that received a different PHR-linked intervention. Two blinded physician adjudicators determined unexplained discrepancies between documented and patient-reported medication regimens. The primary outcome was proportion of medications per patient with unexplained discrepancies.

Results

Among 121 046 patients in eligible practices, 3979 participated in the main trial and 541 participated in the sub-study. The proportion of medications per patient with unexplained discrepancies was 42% in the intervention arm and 51% in the control arm (adjusted OR 0.71, 95% CI 0.54 to 0.94, p=0.01). The number of unexplained discrepancies per patient with potential for severe harm was 0.03 in the intervention arm and 0.08 in the control arm (adjusted RR 0.31, 95% CI 0.10 to 0.92, p=0.04).

Conclusions

When used, concordance between documented and patient-reported medication regimens and reduction in potentially harmful medication discrepancies can be improved with a PHR medication review tool linked to the provider''s medical record.

Trial registration number

This study was registered at ClinicalTrials.gov ({"type":"clinical-trial","attrs":{"text":"NCT00251875","term_id":"NCT00251875"}}NCT00251875).     

Improving completeness of electronic problem lists through clinical decision support: a randomized, controlled trial

Background

Accurate clinical problem lists are critical for patient care, clinical decision support, population reporting, quality improvement, and research. However, problem lists are often incomplete or out of date.

Objective

To determine whether a clinical alerting system, which uses inference rules to notify providers of undocumented problems, improves problem list documentation.

Study Design and Methods

Inference rules for 17 conditions were constructed and an electronic health record-based intervention was evaluated to improve problem documentation. A cluster randomized trial was conducted of 11 participating clinics affiliated with a large academic medical center, totaling 28 primary care clinical areas, with 14 receiving the intervention and 14 as controls. The intervention was a clinical alert directed to the provider that suggested adding a problem to the electronic problem list based on inference rules. The primary outcome measure was acceptance of the alert. The number of study problems added in each arm as a pre-specified secondary outcome was also assessed. Data were collected during 6-month pre-intervention (11/2009–5/2010) and intervention (5/2010–11/2010) periods.

Results

17 043 alerts were presented, of which 41.1% were accepted. In the intervention arm, providers documented significantly more study problems (adjusted OR=3.4, p<0.001), with an absolute difference of 6277 additional problems. In the intervention group, 70.4% of all study problems were added via the problem list alerts. Significant increases in problem notation were observed for 13 of 17 conditions.

Conclusion

Problem inference alerts significantly increase notation of important patient problems in primary care, which in turn has the potential to facilitate quality improvement.

Trial Registration

ClinicalTrials.gov: {"type":"clinical-trial","attrs":{"text":"NCT01105923","term_id":"NCT01105923"}}NCT01105923.     

Temporal artery biopsy… Who needs one?

Objective

To review a 10 year period of temporal artery biopsies, using the American College of Rheumatology (ACR) 1990 criteria: a five point scoring system for the diagnosis of giant cell arteritis (GCA).

Design

Population based, retrospective cohort analysis.

Setting

One district general hospital in the United Kingdom, over one decade.

Participants

All patients who underwent temporal artery biopsy from July 1994 to June 2004.

Main outcome measures

ACR score and temporal artery biopsy result.

Results

During the 10 year period 111 patients were identified. The median (range) age at presentation was 71 (29–85) years. Seventy five patients had an initial ACR score of three or four at presentation. There were 20 positive biopsy specimens. In 19 of these cases at least three of the other criteria were positive so there was already sufficient clinical information for a confident diagnosis. In only one case did the positive result influence the diagnosis by changing the ACR score from two to three. In our series, corticosteroid treatment before biopsy did not significantly reduce the yield of the biopsy.

Conclusions

The ACR score of three or more has a sensitivity of 93.5% and specificity of 91.2% for the diagnosis of GCA. Using these criteria, 68% of patients had sufficient clinical features when referred to make a confident diagnosis of GCA. Temporal artery biopsy was therefore unnecessary in this group. In the remaining group (ACR score ⩽2) there was one positive biopsy. The biopsy only changed the diagnosis in this one case—less than 3% of the uncertain cases and less than 1% of the total cases. Using the ACR criteria and restricting biopsy to those cases in which it might change the diagnosis will reduce the number of biopsies by two thirds without jeopardising diagnostic accuracy.     

Factors contributing to an increase in duplicate medication order errors after CPOE implementation

Objective

To evaluate the incidence of duplicate medication orders before and after computerized provider order entry (CPOE) with clinical decision support (CDS) implementation and identify contributing factors.

Design

CPOE with duplicate medication order alerts was implemented in a 400-bed Northeastern US community tertiary care teaching hospital. In a pre-implementation post-implementation design, trained nurses used chart review, computer-generated reports of medication orders, provider alerts, and staff reports to identify medication errors in two intensive care units (ICUs).

Measurement

Medication error data were adjudicated by a physician and a human factors engineer for error stage and type. A qualitative analysis of duplicate medication ordering errors was performed to identify contributing factors.

Results

Data were collected for 4147 patient-days pre-implementation and 4013 patient-days post-implementation. Duplicate medication ordering errors increased after CPOE implementation (pre: 48 errors, 2.6% total; post: 167 errors, 8.1% total; p<0.0001). Most post-implementation duplicate orders were either for the identical order or the same medication. Contributing factors included: (1) provider ordering practices and computer availability, for example, two orders placed within minutes by different providers on rounds; (2) communication and hand-offs, for example, duplicate orders around shift change; (3) CDS and medication database design, for example confusing alert content, high false-positive alert rate, and CDS algorithms missing true duplicates; (4) CPOE data display, for example, difficulty reviewing existing orders; and (5) local CDS design, for example, medications in order sets defaulted as ordered.

Conclusions

Duplicate medication order errors increased with CPOE and CDS implementation. Many work system factors, including the CPOE, CDS, and medication database design, contributed to their occurrence.     

Making electronic prescribing alerts more effective: scenario-based experimental study in junior doctors

Objective

Expert authorities recommend clinical decision support systems to reduce prescribing error rates, yet large numbers of insignificant on-screen alerts presented in modal dialog boxes persistently interrupt clinicians, limiting the effectiveness of these systems. This study compared the impact of modal and non-modal electronic (e-) prescribing alerts on prescribing error rates, to help inform the design of clinical decision support systems.

Design

A randomized study of 24 junior doctors each performing 30 simulated prescribing tasks in random order with a prototype e-prescribing system. Using a within-participant design, doctors were randomized to be shown one of three types of e-prescribing alert (modal, non-modal, no alert) during each prescribing task.

Measurements

The main outcome measure was prescribing error rate. Structured interviews were performed to elicit participants'' preferences for the prescribing alerts and their views on clinical decision support systems.

Results

Participants exposed to modal alerts were 11.6 times less likely to make a prescribing error than those not shown an alert (OR 11.56, 95% CI 6.00 to 22.26). Those shown a non-modal alert were 3.2 times less likely to make a prescribing error (OR 3.18, 95% CI 1.91 to 5.30) than those not shown an alert. The error rate with non-modal alerts was 3.6 times higher than with modal alerts (95% CI 1.88 to 7.04).

Conclusions

Both kinds of e-prescribing alerts significantly reduced prescribing error rates, but modal alerts were over three times more effective than non-modal alerts. This study provides new evidence about the relative effects of modal and non-modal alerts on prescribing outcomes.     

Governance for clinical decision support: case studies and recommended practices from leading institutions

Objective

Clinical decision support (CDS) is a powerful tool for improving healthcare quality and ensuring patient safety; however, effective implementation of CDS requires effective clinical and technical governance structures. The authors sought to determine the range and variety of these governance structures and identify a set of recommended practices through observational study.

Design

Three site visits were conducted at institutions across the USA to learn about CDS capabilities and processes from clinical, technical, and organizational perspectives. Based on the results of these visits, written questionnaires were sent to the three institutions visited and two additional sites. Together, these five organizations encompass a variety of academic and community hospitals as well as small and large ambulatory practices. These organizations use both commercially available and internally developed clinical information systems.

Measurements

Characteristics of clinical information systems and CDS systems used at each site as well as governance structures and content management approaches were identified through extensive field interviews and follow-up surveys.

Results

Six recommended practices were identified in the area of governance, and four were identified in the area of content management. Key similarities and differences between the organizations studied were also highlighted.

Conclusion

Each of the five sites studied contributed to the recommended practices presented in this paper for CDS governance. Since these strategies appear to be useful at a diverse range of institutions, they should be considered by any future implementers of decision support.