Not all systematic reviews are created equal: Considerations for appraisal

Systematic reviews can be a tremendous asset in the implementation of evidence-based practice, because they minimize some of the most-documented barriers to evidence-based practice. For example, by reading systematic reviews, clinicians may save time that would otherwise be dedicated to locating and appraising individual studies. Further, clinicians can rely on someone else's reviewing expertise, which reduces the knowledge and skill burden otherwise imposed on them. However, empirical studies have repeatedly demonstrated that there is great variability in the quality of systematic reviews. Thus, in order to harness their potential, it is imperative that clinicians distinguish high-quality systematic reviews from those of low quality. In this paper, we aim to discuss considerations for appraising the quality of systematic reviews.     

Critically appraising quantitative research

Critical appraisal of quantitative research is a skill that is necessary for adequate evidence-based practice. This paper describes critical appraisal and offers a process for its implementation. Four categories of quantitative studies are reviewed and associated links to relevant critical appraisal frameworks are provided. The implications for students, nursing faculty members, and practicing nurses are discussed.     

循证护理实践中正确评价和使用证据的研究

为提高对循证护理证据的认识和使用的正确性,减少偏差,本文从循证护理实践中关于证据的6个方哂进行了评述：（1）怎样看待和使用Meta分析证据;（2）怎样看待和使用定性系统评价证据;（3）怎样看待和使用随机对照试验证据;（4）怎样看待和使用对照但未随机分组的证据;（5）怎样看待和使用护理定性研究的证据;（6）怎样看待和使用专家意见和临床推理。     

Synthesis of single-case design research in communication sciences and disorders: Challenges,strategies, and future directions

The synthesis of scientific evidence has been undertaken for decades as a key mode of knowledge acquisition and development. With the growing evidence-based practice movement, the statistical aggregation tool of meta-analysis, with its techniques for measuring effect size, has become an integral method for assessing, analyzing, comparing, and quantifying findings from primary research studies. The role of meta-analysis in scientific and clinical decision making is expanding; however, evidence from single-case design research has been largely omitted from such efforts due to a lack of appropriate techniques, subpar methodology, and theoretical divisions. As single-case design research is a common methodology used in the study of low-incidence and heterogeneous populations, practice-based fields such as communication sciences and disorders are in need of methods to aggregate the available research and incorporate it into evidence-based decision making. This article outlines the challenges to synthesis of single-case design research, describes and critiques the available synthesis methods, and offers suggestions for current synthesis efforts and future research directions in communication sciences and disorders.

Source of funding: No source of funding reported.     

Reducing the use of diagnostic imaging in patients with suspected pulmonary embolism: validation of a risk assessment strategy

Objectives: The aims of this study were to measure the: (i) effects of implementation of a new risk assessment strategy for patients with suspected pulmonary embolism (PE) on the use of imaging and D‐dimer assay; (ii) negative predictive value for PE of a combination of low risk and negative D‐dimer assay; and (iii) compliance of ED clinicians with the strategy. Methods: A non‐randomized clinical trial was conducted in the ED of a 720‐bed teaching hospital between November 2002 and August 2003. Study subjects with suspected PE were compared with 191 randomly selected historical controls. The risk assessment strategy of Kline et al. was disseminated and implemented. Results: The negative predictive value for PE was 99% (95% confidence interval [CI] = 97–100%) in 114 patients with low risk and negative D‐dimer. There was a 21% absolute reduction in the rate of imaging following the implementation of the risk assessment strategy (56% vs 77%, P < 0.001). Conclusion: Low risk combined with a negative D‐dimer result may allow exclusion of PE without imaging.     

A longitudinal framework for fostering critical thinking and diagnostic reasoning

Nurses' reasoning skills are of interest to nurse researchers, managers, clinicians, educators and students. Recent emphasis on critical thinking has raised important questions regarding the modes of reasoning necessary for practice and appropriate measures to improve nurses' cognitive skills. This paper analyses critical thinking and diagnostic reasoning. A longitudinal framework for promoting both modes of reasoning, beginning with nursing students through to experienced clinicians, is proposed as a model worth testing in nurse samples.     

三级医院护士长绩效考评行为观察量表的建构

目的 运用关键事件技术对四川省三级医院的护士长岗位进行工作分析,开发护士长绩效考评行为观察量表.方法 运用关键事件技术访谈56名护理管理者收集到498件关键事件,归类获得15项考评指标、169个行为项目的 原始量表.48位考评者以五级利克特标度为基础对190名护士长进行预试考评,用项目分析筛选得到12项考评指标、54个行为项目的 正式行为观察量表.48位考评者用正式量表对192名护士长进行实测以检验其信度和效度.结果 该量表Cronbach's α系数在0.65～0.93,量表的重测信度在0.50～0.76;因子分析根据特征根>1的原则抽取了9个因子,累积贡献率68%.结论 基于关键事件技术的工作分析开发的护士长绩效考评行为观察量表具有良好的信度和效度.     

Validation of a diagnostic reminder system in emergency medicine: a multi-centre study

Background

Diagnostic error is a significant problem in emergency medicine, where initial clinical assessment and decision making is often based on incomplete clinical information. Traditional computerised diagnostic systems have been of limited use in the acute setting, mainly due to the need for lengthy system consultation. We evaluated a novel web‐based reminder system, which provides rapid diagnostic advice to users based on free text search terms.

Methods

Clinical data collected from patients presenting to three emergency departments with acute medical problems were entered into the diagnostic system. The displayed results were assessed against the final discharge diagnoses for patients who were admitted to hospital (diagnostic accuracy) and against a set of “appropriate” diagnoses for each case provided by an expert panel (potential utility).

Results

Data were collected from 594 patients (53.4% of screened attendances). Mean age was 49.4 years (95% CI 47.7 to 51.1) and the majority had significant past illnesses. Most were assessed first by junior doctors (70%) and 266/594 (44.6%) were admitted to hospital. Overall, the diagnostic system displayed the final discharge diagnosis in 95% of inpatients and 90% of “must‐not‐miss” diagnoses suggested by the expert panel. The discharge diagnosis appeared within the first 10 suggestions in 78% of cases.

Conclusions

The Isabel diagnostic aid has been shown to be of potential use in reminding junior doctors of key diagnoses in the emergency department. The effects of its widespread use on decision making and diagnostic error can be clarified by evaluating its impact on routine clinical decision making.     

Reviewing and selecting outcome measures for use in routine practice

For the successful achievement of evidence-based practice, clinicians, managers and purchasers need evidence on whether a particular intervention works and ways to judge the appropriateness of the outcome criteria and measures used. Guidance is needed on what outcome measure to use, especially within routine clinical care settings. Beginning with a re-clarification of the difference between a health status and an outcome measure, the paper presents an evaluative checklist for use by clinical audit and research staff to review outcome measures for use in routine care settings. Central features include the user-centredness of the measure, its psychometric properties, feasibility of use and utility. The applicability of the checklist is illustrated for outcome measurement in diabetes and stroke care. A modified form of the checklist is proposed for use by the busy clinician as an aid to the critical review of research papers within the context of evidence-based practice and to aid health care practitioners' choice of which outcome measure(s) to use within routine clinical care.     

The accuracy of primary care teams in diagnosing disorders of the shoulder

Objective The aim of this retrospective, observational study was to determine the accuracy of diagnoses given by a multitude of primary care services to patients referred to an orthopaedic upper limb surgeon with a focus towards shoulder pathology. Methods Records of all patients referred to a single upper limb surgeon over a 1‐year period were reviewed. The diagnosis in the primary care setting was compared with the initial working diagnosis in the specialist clinic and against the final diagnosis following specialist investigation/intervention. Results 114 eligible patients were identified. General practitioners referred 35% of patients, musculoskeletal triage services referred 63% of patients and independent physiotherapists referred 2% of patients. It was found that 37% of patients were not given a diagnosis by the primary care team. When a diagnosis was given, accuracy was 50% (κ = 0.28) when correlated against that given by an orthopaedic upper limb surgeon. Conclusions Our findings suggest that knowledge of shoulder conditions is limited amongst primary care practitioners. There does not appear to be any difference in accuracy between general practitioners and musculoskeletal triage services.     

Finding the evidence to change practice for assessing pain in children who are cognitively impaired

We report on a quality improvement project whose purpose was to systematically review behavioral pain assessment tools for children who are cognitively impaired, with the goal of identifying a valid and reliable tool for clinical practice. In addition, we sought to partner with parents and/or caregivers to expand their role in pain assessment. The project team conducted an extensive synthesis of the literature to examine the availability and quality of published pain assessment tools for use with children with cognitive impairment (CI) or developmental disability. Once completed, we identified 2 of the available 10 tools to test in the clinical setting. Data from this quality improvement project provided evidence to support the adoption of the revised Faces, Legs, Activity, Cry, and Consolability pain assessment tool for children with CI into clinical practice.     

The qual-CAT: Applying a rapid review approach to qualitative research to support clinical decision-making in speech-language pathology practice

Abstract

Systematic reviews and meta-syntheses of qualitative research produce “a sum greater than its parts”. Qualitative findings are examined alongside one another, and new insights are discovered about the topic under consideration. However, a barrier to clinicians undertaking these reviews is the time and knowledge required. Rapid review approaches, such as the Critically Appraised Topic (CAT), have recently been promoted in speech-language pathology, encouraging strategic, clinically driven examination of key research in focused topic areas, with the aim of supporting clinical decision-making. A strength of the CAT approach is that it supports interpretation and use of evidence by people who are not experts in the methodologies used. Additionally, CATs reduce the time input required by focusing on the best evidence available, and take a pragmatic approach to analysis across studies. While CAT methodologies have been detailed in relation to quantitative research, qualitative CAT methodologies have not specifically been discussed. Searching, ranking of research strength, appraisal and analysis methods are all likely to vary for qualitative CATs. This paper proposes and details a methodology for a “qual-CAT” approach, providing examples and steps that support the use of this methodology in practice.     

Appraising the quality of randomized controlled trials: inter-rater reliability for the OTseeker evidence database

RATIONALE AND AIMS: 'OTseeker' is an online database of randomized controlled trials (RCTs) and systematic reviews relevant to occupational therapy. RCTs are critically appraised and rated for quality using the 'PEDro' scale. We aimed to investigate the inter-rater reliability of the PEDro scale before and after revising rating guidelines. METHODS: In study 1, five raters scored 100 RCTs using the original PEDro scale guidelines. In study 2, two raters scored 40 different RCTs using revised guidelines. All RCTs were randomly selected from the OTseeker database. Reliability was calculated using Kappa and intraclass correlation coefficients [ICC (model 2,1)]. RESULTS: Inter-rater reliability was 'good to excellent' in the first study (Kappas >or= 0.53; ICCs >or= 0.71). After revising the rating guidelines, the reliability levels were equivalent or higher to those previously obtained (Kappas >or= 0.53; ICCs >or= 0.89), except for the item, 'groups similar at baseline', which still had moderate reliability (Kappa = 0.53). In study 2, two PEDro scale items, which had their definitions revised, 'less than 15% dropout' and 'point measures and variability', showed higher reliability. In both studies, the PEDro items with the lowest reliability were 'groups similar at baseline' (Kappas = 0.53), 'less than 15% dropout' (Kappas 相似文献   

Evidence-based assessment and autism spectrum disorders: A scoping review

Abstract

Assessment of language and communication in autism spectrum disorders (ASD) is a critical element in developing appropriate and effective interventions and supports. While there is a large literature on diagnosis of autism, the information available to guide clinical practice for communication and language specifically is sparser. In this scoping review, we sought to identify research articles and critical reviews dealing with communication assessment for individuals with autism. The first and second author each conducted an independent search of relevant scholarly databases to identify articles published between 1995 and 2013. Differences were resolved via discussion; 54 relevant articles were identified using the search terms: autism + language + assessment; autism + speech + assessment; and autism + communication + assessment. While guidelines for best practice are premature, especially given the diversity of developmental and cognitive profiles in ASD, emerging consensus was seen in the following areas: (a) formal norm- and criterion-referenced tests not designed specifically for ASD have been investigated and shown to be feasible and interpretable in use with individuals with autism; (b) language sample analysis and/or structured observational elicitations of communication in naturalistic contexts are widely advocated as a means to ensure the full profile of an individual’s strengths and challenges; (c) caregiver report measures in several studies compared favorably with formal tests, therefore many scholars support their use to supplement direct testing in clinical environments, because they can serve as probes of naturally occurring communication behaviors.     

The impact of workplace factors on evidence-based speech-language pathology practice for children with autism spectrum disorders

Abstract

Although researchers have examined barriers to implementing evidence-based practice (EBP) at the level of the individual, little is known about the effects workplaces have on speech-language pathologists’ implementation of EBP. The aim of this study was to examine the impact of workplace factors on the use of EBP amongst speech-language pathologists who work with children with Autism Spectrum Disorder (ASD). This study sought to (a) explore views about EBP amongst speech-language pathologists who work with children with ASD, (b) identify workplace factors which, in the participants’ opinions, acted as barriers or enablers to their provision of evidence-based speech-language pathology services, and (c) examine whether or not speech-language pathologists’ responses to workplace factors differed based on the type of workplace or their years of experience. A total of 105 speech-language pathologists from across Australia completed an anonymous online questionnaire. The results indicate that, although the majority of speech-language pathologists agreed that EBP is necessary, they experienced barriers to their implementation of EBP including workplace culture and support, lack of time, cost of EBP, and the availability and accessibility of EBP resources. The barriers reported by speech-language pathologists were similar, regardless of their workplace (private practice vs organization) and years of experience.     

Critical care cardiovascular nurse expert and novice diagnostic cue utilization

AIM: Critical care cardiovascular (CCCV) nursing diagnostic expertise was the focus of this research. The purpose of the study was to compare diagnostic cue utilization between expert and novice CCCV nurses. BACKGROUND: Knowledge related to objective measures of clinical nursing diagnostic expertise would enhance effective and efficient recognition, utilization, and reward of clinical expertise. METHODS: Five CCCV written simulations served as instruments in the study. Diagnostic content areas included left ventricular dysfunction, cardiac tamponade, sepsis, right ventricular failure, and hypovolemia related to internal abdominal haemorrhage. The sample was composed of 23 expert and 23 novice nurses. After reading each simulation, subjects were asked to verbally recall the simulation, give an impression of the predominant problem or diagnosis, and give a diagnostic explanation. Verbal recalls were audio-taped for protocol analysis. Diagnostic accuracy and cue utilization were determined through comparisons of subjects' recalled diagnoses and cues with results from an expert panel review consensus. The major variable was the mean recalled proportion of highly relevant cues to total cues (HRC/TC) on accurately diagnosed simulations. Chi-square analysis revealed that diagnostic accuracy was greater with experts than with novices. Differences between and among simulations, expertise, accuracy and the mean proportion of highly relevant cues to total cues were examined with a 4 x 2 x 2 factorial analysis of variance. RESULTS: When considering all accurately diagnosed simulations, experts had a higher HRC/TC than novices. The major limitations were the use of low fidelity written simulations and virgin verbal protocol methods. CONCLUSION: The findings generally support the idea that the development of diagnostic expertise is associated with the ability to focus on highly relevant cues.     

预防导尿管相关性尿路感染的循证实践策略的准备度评估

目的 了解在我院外科病房开展预防尿管相关性尿路感染(CAUTI)的循证实践策略的应用前准备度,识别项目应用中的障碍因素,根据障碍因素制定相对应的策略,以促进循证实践的开展。方法 采用基本资料调查表和循证护理实践准备度评估量表对我院开展预防CAUTI维护策略循证护理实践的科室护士进行问卷调查。结果CREBNA总量表得分为(137.71±16.48)分,达满分的88.85%。证据分量表得分为(52.75±6.54)分,组织环境分量表得分为(44.29±6.09)分,促进因素分量表得分为(40.66±4.81)分。得分最高的条目是“证据不违反国家的政策及法律法规”,得分最低的条目是“证据来源是可靠的”,t检验分析结果显示,量表总得分仅与开展循证实践的必要性有关(p=0.001),环境分量表得分,促进因素分量表得分与以上各因素没有相关性,而证据分量表得分与年龄分组有关(p=0.048)和开展循证护理实践的必要性有关(p=0.002)。结论 三甲医院护士循证实践准备情况较好,其中护士对循证护理实践的态度和年龄是影响循证实践的因素,科室管理者在临床开展循证实践时应重视提高护士对循证实践的态度,同时加强低年资护士对循证护理实践相关知识的培训,以促进循证实践在临床中的成功开展。     

Selecting analytical biomarkers for diagnostic applications: a first principles approach

Introduction: Biomarkers are objective indications of a medical state that can be measured accurately and reproducibly. Traditional biomarkers enable diagnosis of disease through detection of disease-specific molecules, disease-mediated molecular changes, or distinct physiological or anatomical signatures.

Areas covered: This work provides a framework for selecting biomarkers that are most likely to provide useful information about a patient’s disease state. Though the authors emphasize markers related to disease, this work is also applicable to biomarkers for monitoring physiological changes such as ovulation or pregnancy. Additionally, the scope was restricted to biomarkers that are amenable to analytical detection across a range of health care levels, including low resource settings. The authors describe trade-offs between biomarkers’ sensitivity/specificity for a disease-causing agent, the complexity of detection, and how this knowledge can be applied to the development of diagnostic tests. This report also details additional assessment criteria for successful tests.

Expert commentary: Biomarker selection should primarily be driven by an attempt to answer an explicit clinical question (preferably causative relationship of the biomarker to disease-state), and only then by test development expediency (ease of detection). This framework is useful for stakeholders from test developers to clinicians to identify the trade-offs for diagnostic biomarkers for any use case.     

Overview of Cochrane Systematic Reviews for Rehabilitation Interventions in Persons With Spinal Cord Injury: A Mapping Synthesis

ObjectiveThis article aims to describe the evidence on rehabilitation interventions for persons with spinal cord injury (SCI) identified in Cochrane Systematic Reviews (CSRs) selected for inclusion in the World Health Organization Rehabilitation Programme–Package of Interventions for Rehabilitation.Data SourcesThe CSRs search was led by the Cochrane Rehabilitation team, using the tagging process, using the terms “spinal cord injury” and “rehabilitation” in the Cochrane Library.Study SelectionWe performed an overview of all the CSRs according to the inclusion criteria defined with the World Health Organization: rehabilitation interventions in persons with SCI.Data ExtractionThe CSRs identified after the screening process were summarized using an evidence map, grouping outcomes, and comparisons of included CSRs indicating the effect and the quality of evidence to provide a comprehensive view of what is known.Data SynthesisOut of 248 CSRs from the past 10 years tagged in the Cochrane Rehabilitation database, 3 were related to SCI. They provide data on 13 outcomes analyzed within 11 comparisons for a total of 64 primary studies, including 2024 participants with SCI. Of these, 7 outcomes and 1 comparison focused on people with cervical SCI. Rehabilitation interventions might improve respiratory outcomes and pain relief in people with SCI. There is uncertainty whether bodyweight-supported treadmill training, robotic-assisted training, and functional electrostimulation affect walking speed and capacity.ConclusionsThe current evidence needs to be confirmed by better quality research. Therefore, future priorities are the improvement of methodological quality of the studies in people with SCI, particularly considering the complexity of this health condition. Further, there is a need for more CSRs in the field.