Prioritizing guideline topics: development and evaluation of a practical tool

RATIONALE, AIMS AND OBJECTIVES: A clear process for selecting and adopting clinical practice guidelines in the new topic areas is needed. The aim of this study is to design and develop a practical tool to assess guideline topics that have been suggested to the organization responsible for producing guidelines. METHODS: We carried out an iterative development, feasibility and validation study of a guideline topic prioritization tool. The setting included the guideline producer organization and the tax-funded health care system. In the first stage of the tool development, participants were researchers, members of the Current Care Board and experts from health care organizations. In the second stage, the evaluation was done internally within the project by three independent reviewers. The main outcome measures were responses to an evaluation questionnaire, qualitative process feedback and analysis of the performance of the instrument on a random set of guidelines. RESULTS: Evaluations by three independent reviewers revealed good agreement and face validity with respect to its feasibility as a planning tool at the guideline board level. Feedback from board members suggested that the instrument is useful in prioritizing guideline topics. CONCLUSION: This instrument was accepted for use by the Board. Further developments are needed to ensure feedback and acceptability of the instrument by those proposing topics.     

Are the outcomes of clinical pathways evidence‐based? A critical appraisal of clinical pathway evaluation research

Aim and objective To evaluate the validity of study outcomes of published papers that report the effects of clinical pathways (CP). Method Systematic review based on two search strategies, including searching Medline, CINAHL, Embase, Psychinfo and Picarta from 1995 till 2005 and ISI Web of Knowledge^SM. We included randomized controlled or quasi‐experimental studies evaluating the efficacy of clinical pathway application. Assessment of the methodological quality of the studies included randomization, power analysis, selection bias, validity of outcome indicators, appropriateness of statistical tests, direct (matching) and indirect (statistical) control for confounders. Outcomes included length of stay, costs, readmission rate and complications. Two reviewers independently assessed the methodological quality of the selected papers and recorded the findings with an evaluation tool developed from a set of items for quality assessment derived from the Cochrane Library and other publications. Results The study sample comprised of 115 publications. A total of 91.3% of the studies comprised of retrospective studies and 8.7% were randomized controlled studies. Using a quality‐scoring assessment tool, 33% of the papers were classified as of good quality, whereas 67% were classified as of low quality. Of the studies, 10.4% controlled for confounding by matching and 59.1% adopted parametric statistical tests without testing variables on normal distribution. Differences in outcomes were not always statistically tested. Conclusion Readers should be cautious when interpreting the results of clinical pathway evaluation studies because of the confounding factors and sources of contamination affecting the evidence‐based validity of the outcomes.     

Validity and reliability study on traditional Chinese FACT-C in Chinese patients with colorectal neoplasm

Objective To establish the validity and reliability of traditional Chinese version of the Functional Assessment of Cancer Therapy‐Colorectal (FACT‐C). Methods A total of 536 subjects self‐administered (n = 331) or interviewer‐administered (n = 205) FACT‐C (version 4), EORTC QLQ‐C30/CR38 and SF‐12v2 instruments for health‐related quality of life assessment. Construct validity was examined by item–scale correlation, scaling success and concurrent validity. Reliability was evaluated by test–retest reliability and internal consistency. Sensitivity was assessed by known‐groups comparisons. Results The completion rates for FACT‐C were almost perfect (>98%). The FACT‐C demonstrated item‐internal consistency and item discriminant validity through item–scale correlation. Scaling success and concurrent validity were satisfactory to support the construct validity. The five subscales of the FACT‐C showed good internal consistency with Cronbach alpha coefficient and substantial reproducibility, demonstrating good reliability. Sensitivity was supported when there were significant differences in scores related to physical condition between patients who were receiving treatment and those who were not. Conclusion Traditional Chinese version of the FACT‐C was demonstrated to have satisfactory psychometric properties in terms of applicability, reliability, validity and sensitivity in Chinese patients with colorectal neoplasm. The FACT‐C was valid colorectal‐specific health‐related quality of life tool for the Chinese population.     

Development of an interprofessional lean facilitator assessment scale

High reliability is important for optimising quality and safety in healthcare organisations. Reliability efforts include interprofessional collaborative practice (IPCP) and Lean quality/process improvement strategies, which require skilful facilitation. Currently, no validated Lean facilitator assessment tool for interprofessional collaboration exists. This article describes the development and pilot evaluation of such a tool; the Interprofessional Lean Facilitator Assessment Scale (ILFAS), which measures both technical and ‘soft’ skills, which have not been measured in other instruments. The ILFAS was developed using methodologies and principles from Lean/Shingo, IPCP, metacognition research and Bloom’s Taxonomy of Learning Domains. A panel of experts confirmed the initial face validity of the instrument. Researchers independently assessed five facilitators, during six Lean sessions. Analysis included quantitative evaluation of rater agreement. Overall inter-rater agreement of the assessment of facilitator performance was high (92%), and discrepancies in the agreement statistics were analysed. Face and content validity were further established, and usability was evaluated, through primary stakeholder post-pilot feedback, uncovering minor concerns, leading to tool revision. The ILFAS appears comprehensive in the assessment of facilitator knowledge, skills, abilities, and may be useful in the discrimination between facilitators of different skill levels. Further study is needed to explore instrument performance and validity.     

Qualitative research in health care: II. A structured review and evaluation of studies

A hand search of the original papers in seven medical journals over 5 years was conducted in order to identify those reporting qualitative research. A total of 210 papers were initially identified, of which 70 used qualitative methods of both data collection and analysis. These papers were evaluated by the researchers using a checklist which specified the criteria of good practice. Overall, 2% of the original papers published in the journals reported qualitative studies. Papers were more frequently positively assessed in terms of having clear aims, reporting research for which a qualitative approach was appropriate and describing their methods of data collection. Papers were less frequently positively assessed in relation to issues of data analysis such as validity, reliability and providing representative supporting evidence. It is concluded that the full potential of qualitative research has yet to be realized in the field of health care.     

Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research

Background The Cochrane Collaboration is strongly encouraging the use of a newly developed tool, the Cochrane Collaboration Risk of Bias Tool (CCRBT), for all review groups. However, the psychometric properties of this tool to date have yet to be described. Thus, the objective of this study was to add information about psychometric properties of the CCRBT including inter‐rater reliability and concurrent validity, in comparison with the Effective Public Health Practice Project Quality Assessment Tool (EPHPP). Methods Both tools were used to assess the methodological quality of 20 randomized controlled trials included in our systematic review of the effectiveness of knowledge translation interventions to improve the management of cancer pain. Each study assessment was completed independently by two reviewers using each tool. We analysed the inter‐rater reliability of each tool's individual domains, as well as final grade assigned to each study. Results The EPHPP had fair inter‐rater agreement for individual domains and excellent agreement for the final grade. In contrast, the CCRBT had slight inter‐rater agreement for individual domains and fair inter‐rater agreement for final grade. Of interest, no agreement between the two tools was evident in their final grade assigned to each study. Although both tools were developed to assess ‘quality of the evidence’, they appear to measure different constructs. Conclusions Both tools performed quite differently when evaluating the risk of bias or methodological quality of studies in knowledge translation interventions for cancer pain. The newly introduced CCRBT assigned these studies a higher risk of bias. Its psychometric properties need to be more thoroughly validated, in a range of research fields, to understand fully how to interpret results from its application.     

Psychometric testing of the Chinese Mandarin version of the MacNew Heart Disease Health‐related Quality of Life questionnaire for patients with myocardial infarction in mainland China

Tools to measure the quality of life of Chinese myocardial infarction (MI) patients in mainland China are small in numbers. For this study, 210 Chinese patients with MI were recruited to examine the psychometric properties of the Chinese Mandarin version of the MacNew Heart Disease questionnaire. Thirty‐five participants managed to complete the retest after 2 weeks. The MacNew Heart Disease questionnaire had acceptable internal consistency, test–retest reliability and concurrent validity. The principal components analysis identified three factors which accounted for 56.6% of the variance. The MacNew Heart Disease questionnaire demonstrated good reliability and validity to be used as a health status measurement tool, and is ready for clinical use in the assessment of the quality of life of Chinese MI patients in mainland China.     

Venous access in oncology and haematology patients: Part two

AIM: To assess the validity and inter-rater reliability of two tools, Venous Assessment Tool (VAT) and Deciding on IntraVenous Access (DIVA), using thematic analysis of feedback from nurses who used the tools, METHOD: Analysis methods used in grounded theory were employed to investigate the written feedback provided by nurses. Inter-rater reliability was tested statistically using percentage agreement and the kappa (K) statistic. FINDINGS: Thematic analysis of the nurses' feedback generated themes concerning the role of clinical judgement, how theory translates to practice and the role of patient preference when using VAT and DIVA. Exploring these themes confirmed the validity of the tools and highlighted the importance of these considerations when evaluating the usefulness of assessment tools in the clinical environment. When VAT was used by a group of nurses to assess patients, the strength of agreement statistically was 'moderate'. When using DIVA, the tool demonstrated 'very good' agreement statistically. CONCLUSION: When used with oncology and haematology patients, VAT and DIVA have been shown to generate agreement among the nurses who used them beyond that which would be expected by chance. As the only tools of this kind with evidence of inter-rater reliability, they provide clinicians with a useful resource which can be used in practice or research. However, the use of assessment tools cannot replace clinical judgement or override the preferences of patients.     

Clinical instruments: reliability and validity critical appraisal

Rationale, aim and objectives There is a lack of health care practitioners using objective clinical tools with sound psychometric properties. There is also a need for researchers to improve their reporting of the validity and reliability results of these clinical tools. Therefore, to promote the use of valid and reliable tools or tests for clinical evaluation, this paper reports on the development of a critical appraisal tool to assess the psychometric properties of objective clinical tools. Method A five-step process was followed to develop the new critical appraisal tool: (1) preliminary conceptual decisions; (2) defining key concepts; (3) item generation; (4) assessment of face validity; and (5) formulation of the final tool. Results The new critical appraisal tool consists of 13 items, of which five items relate to both validity and reliability studies, four items to validity studies only and four items to reliability studies. The 13 items could be scored as 'yes', 'no' or 'not applicable'. Conclusion This critical appraisal tool will aid both the health care practitioner to critically appraise the relevant literature and researchers to improve the quality of reporting of the validity and reliability of objective clinical tools.     

中文版高血压患者服药依从性量表测量学特性的系统评价

目的 系统评价中文版高血压患者服药依从性量表的测量学特性及研究的方法学质量,为医护人员选择高质量的量表提供循证依据。方法 系统检索PubMed、Embase、Web of Science、中国知网、维普数据库、万方数据库、中国生物医学文献数据库中与高血压患者服药依从性量表测量学特性评价相关的研究,检索时限为建库至2022年2月3日。按照健康测量工具遴选标准（consensus-based standards for the selection of health measurement instruments,COSMIN）要求,由2名研究者独立进行文献筛选和资料提取,采用COSMIN偏倚风险清单及质量标准评定量表测量学特性并形成最终推荐意见。结果 共纳入10项研究,涉及9个中文版高血压患者服药依从性量表,均未报告跨文化效度、测量误差、假设检验、反应度。因内容效度均为不充分或不确定,证据质量为中等或以下,最终9个量表均为B级推荐。结论 与其他8个量表相比,服药依从性自我效能量表修订版的测量学特性评价最为均衡,具有较好的信效度,但该量表的其他测量学特性仍有待验证。     

Pain assessment tools for unconscious or sedated intensive care patients: a systematic review

Title.  Pain assessment tools for unconscious or sedated intensive care patients: a systematic review.
Aim.  This paper is a report of a systematic review describing instruments developed for pain assessment in unconscious or sedated intensive care patients.
Background.  Intensive care patients who are unconscious or sedated are unable to communicate and therefore recognition and assessment of the pain is difficult. To assess these patients' pain, it is important to use a valid and reliable pain assessment tool.
Data sources.  A systematic bibliographical review was conducted, based on seven databases, covering the period from January 1987 to February 2007. A total of 1,586 abstracts was identified and reviewed, 58 papers were selected for full-text review and nine papers were included in the review.
Methods.  Two researchers independently reviewed the abstracts and three reviewers extracted the papers. The included papers were evaluated using a quality assessment instrument previously developed to evaluate pain assessment tools.
Results.  Five different pain assessment tools were identified that had been used with unconscious or sedated intensive care patients. All five instruments included behavioural indicators and three included physiological indicators. Their psychometric properties varied and it was not possible to deduce their clinical utility.
Conclusion.  All instruments were reasonably new. In most of them psychometric testing was in an early stage or even absent. Before any of the reported instruments can be chosen in preference to others, it is essential to test their validity, reliability and feasibility further.     

A new reliable tool (PVC assess) for assessment of peripheral venous catheters

Rationale and aims To evaluate the extensive use of peripheral venous catheters (PVCs), including catheter‐related complications, a reliable tool for PVC assessment is needed. The aim of this study was to develop such a tool to evaluate PVCs in relation to management, documentation and signs and symptoms of thrombophlebitis (TH), as well as to determine its inter‐rater and test‐retest reliability. Method The tool development included confirmation of content and face validity. Two groups of registered nurses used the new tool (PVC assess ) to assess PVC management and signs of TH independently. Group A (n = 3) assessed 26 items in 67 PVCs bedside (inter‐rater reliability). Group B (n = 3) assessed photographs (67 PVCs, 21 items) of the same PVCs as those in Group A with a 4‐week interval (test‐retest reliability). Proportion of agreement P(A) and Cohen's kappa were calculated to evaluate inter‐rater and test‐retest reliability. Results Among nurses assessing PVCs at bedside, the P(A) was good to excellent (0.80–1) in 96% of the items in PVC assess . In 80% of the items kappa was substantial to almost perfect (0.61–1). TH sign erythema fell into the fair range (kappa = 0.40). In test‐retest reliability analysis the P(A) was within the good and excellent range (0.80–1.0) and kappa varied from moderate to almost perfect (0.41–1.0) in 95% of the items. One item ‘outer dressing is clean’ was in fair range (0.21–0.40). Conclusions The PVC assess instrument shows satisfactory inter‐rater and test‐retest reliability. Reliability tests on reviewing documentation remain to be performed.     

Emotional care experienced by hospitalised patients: development and testing of a measurement instrument

Aims and objectives. This paper describes the development and preliminary psychometric testing of an instrument that evaluates the emotional care experienced by patients during hospitalisation. Background. Previous qualitative work using the Grounded Theory method identified the characteristics of interpersonal interactions that hospitalised patients perceived to be therapeutic. Three conditions were associated with the patient’s experience of emotional comfort: the patient’s perceived Level of Security, Level of Knowing and Level of Personal Value. Questions relating to each of these conditions were included in this new instrument. Design. Instrument development and psychometric testing. Method. Preliminary psychometric testing was carried out in four phases: construction of the instrument; assessment of face and content validity; testing for clarity and feasibility for use with hospitalised patients; assessment of reliability, construct validity of the tool and assessment of the internal structure. Conclusions. A hospital‐wide survey was carried out and the instrument was completed by a total sample of 132 patients. Two of the three sub‐scales achieved an internal consistency estimate of at least 0·70. The construct validity of the tool confirmed the previously identified characteristics of patients in need of additional emotional care. Exploratory factor analysis established two of the sub‐scales and identified a fourth sub‐scale which was named ‘Level of Connection’. Encouraging reliability and validity estimates were obtained and the instrument was improved. Further testing with larger samples is recommended. Relevance to clinical practice. This questionnaire, which is completed by hospitalised patients, differs from other instruments because it evaluates the interactions of all hospital staff rather than only nursing staff. This instrument can be used to identify patients who may be in need of additional emotional care and to evaluate the effectiveness of interventions directed at improving the emotional well‐being of patients.     

The Quality of Life in Alzheimer's Disease Scale: direct assessment of people with cognitive impairment

Background. A need exists, consistent with a person‐centred approach to dementia care, to facilitate collaborative approaches to the assessment of quality of life. Aims and objectives. To examine the concept of quality of life, to make the case for its centrality in dementia care provision and to consider the validity and reliability of one tool (Quality of Life in Alzheimer's Disease Scale, QOL‐AD). Conclusions. The use of the QOL‐AD aligns with person‐centred principles incorporating the subjective viewpoint of the person with dementia, thereby promoting a collaborative and inclusive approach to the measurement of the concept of quality of life. Relevance to clinical practice. Utilization of QOL‐AD in clinical practice facilitates a user‐centred and collaborative approach to the assessment of quality of life. This approach is in keeping with contemporaneous forces in health and social care delivery for people with dementia.     

Further assessment of a method to estimate reliability and validity of qualitative research findings

The reliability and validity of qualitative research findings are viewed with scepticism by some scientists. This scepticism is derived from the belief that qualitative researchers give insufficient attention to estimating reliability and validity of data, and the differences between quantitative and qualitative methods in assessing data. The danger of this scepticism is that relevant and applicable research findings will not be used. Our purpose is to describe an evaluative strategy for use with qualitative data, a strategy that is a synthesis of quantitative and qualitative assessment methods. Results of the strategy and factors that influence its use are also described.     

Focus on psychometrics use of health records in research: Reliability and validity issues

Data extracted from health records are commonly used in studies to address a variety of questions raised by health researchers. However, concern about the reliability and validity of such data generally is limited to an assessment of interrater reliability. Less attention has been paid to the reliability of the health record itself, and to the validity of both the health record and the data extracted from it. This article reviews the distinctions and overlaps among these types of reliability and validity and the factors that influence the validity and reliability of research data obtained from health records. Recommendations to investigators who use health record data in their research projects are offered.     

系统评价/Meta分析方法学质量的评价工具AMSTAR

AMSTAR是国外最新研发的用于评价系统评价/Meta分析方法学质量的测量工具,具有良好的效度、信度和反应度,得到了较为广泛的运用。本文就AMSTAR的研发过程、评价条目、运用现状等问题逐一介绍给国内研究者和使用者。     

简版脑卒中患者专门生存质量量表的汉化及信度、效度检验

目的 开发建立适用于我国脑卒中康复期患者生存质量评估的中文版量表.方法 在POST教授的简版脑卒中患者专门生存质量量表(the short version of the stroke specific quality of life scale,SV-SS-QoL)的基础上,采用双人翻译-回译的方法对英文版SV-SS-...     

Volume measurement using the Polhemus FastSCAN 3D laser scanning: a novel application for burns clinical research

The Polhemus FastSCAN system offers a noncontact method of quantifying limb edema volume by combining laser scanning with 3D spatial orientation. The aim of this study was to investigate the reliability, validity and clinical utility of this assessment technique in the burn environment. Pilot Studies: Completed in order to develop a standardised scanning procedure. Intertester Reliability: Two testers each scanned 30 uninjured upper limbs. Intratester Reliability: One tester conducted repeated scans for burn survivors (n = 6). Validity: The scan volumes were compared with water displacement measures for all volunteers (n = 36). Intertester reliability was moderate for whole arm scans (R = .59; 95% confidence intervals [CI] = 0.28-0.78) and excellent with hand truncated (R = .95; 95% CI = 0.90-0.98; minimum detectable difference = 242.1 cm3). Intratester reliability was inconclusive (R = .72; 95% CI = -0.07-0.96) but qualitative assessment confirmed poorer quality scans. The FastSCAN system overstated the arm volume by 49.3 cm3 on average, but 95% limits of agreement when compared with water displacement demonstrated the limitations as a clinical or research tool in the burn environment. The Polhemus FastSCAN system provides a noncontact method of limb volume measurement. The reliability was good only with the hand removed from calculations. The accuracy of the system compared poorly with water displacement measures in the burn clinical environment. Using the currently available software and method, the change detectable by this technique was too large for monitoring the efficacy of acute burn edema interventions.     

The impact of spinal cord stimulation on physical function and sleep quality in individuals with failed back surgery syndrome: A systematic review

The aim of this review was to determine the impact of spinal cord stimulation (SCS) on physical function and sleep quality in individuals with failed back surgery syndrome (FBSS). This review comprised three phases: an electronic database search (PubMed, Cinahl Plus, EMBASE, PsychInfo, Pedro, Cochrane Library) identified potential papers; these were screened for inclusion criteria, with extraction of data from accepted papers and rating of internal validity by two independent reviewers using the Effective Public Health Practice Project quality assessment tool, a tool designed to assess non‐RCTs (randomized controlled trials) as well as RCTs. Strength of the evidence was rated using the Agency for Health Care Policy and Research guidelines. The search generated 13 quantitative papers that fulfilled the inclusion criteria; all 13 studies investigated the impact of SCS on physical function, and nine studies investigated the impact of SCS on sleep quality. Consistent evidence (level C) found that SCS positively affected physical function, with improvements in participation in activities of daily living, leisure, social and work‐related activities. Similarly, consistent evidence (level C) found improvements in sleep quality following SCS. Improvements in sleep quantity, a reduction in awakenings and a decrease in sleep medication use were also noted (level D). The impact of SCS on cognitive function, i.e., memory and concentration, was also assessed using the same search strategy, no papers fulfilled the inclusion criteria for this study. Spinal cord stimulation effectively addressed many physical function and sleep problems associated with FBSS; however, there is a need for further high‐quality objective investigations to support this.