International Experts'' Perspectives on the State of the Nurse Staffing and Patient Outcomes Literature

PURPOSE: To assess the key variables used in research on nurse staffing and patient outcomes from the perspective of an international panel. DESIGN: A Delphi survey (November 2005-February 2006) of a purposively-selected expert panel from 10 countries consisting of 24 researchers specializing in nurse staffing and quality of health care and 8 nurse administrators. METHODS: Each participant was sent by e-mail an up-to-date review of all evidence related to 39 patient-outcome, 14 nurse-staffing and 31 background variables and asked to rate the importance/usefulness of each variable for research on nurse staffing and patient outcomes. In two subsequent rounds the group median, mode, frequencies, and earlier responses were sent to each respondent. FINDINGS: Twenty-nine participants responded to the first round (90.6%), of whom 28 (87.5%) responded to the second round. The Delphi panel generated 7 patient-outcome, 2 nurse-staffing and 12 background variables in the first round, not well-investigated in previous research, to be added to the list. At the end of the second round the predefined level of consensus (85%) was reached for 32 patient outcomes, 10 nurse staffing measures and 29 background variables. The highest consensus levels regarding measure sensitivity to nurse staffing were found for nurse perceived quality of care, patient satisfaction and pain, and the lowest for renal failure, cardiac failure, and central nervous system complications. Nursing Hours per Patient Day received the highest consensus score as a valid measure of the number of nursing staff. As a skill mix variable the proportion of RNs to total nursing staff achieved the highest consensus level. Both age and comorbidities were rated as important background variables by all the respondents. CONCLUSIONS: These results provide a snapshot of the state of the science on nurse-staffing and patient-outcomes research as of 2005. The results portray an area of nursing science in evolution and an understanding of the connections between human resource issues and healthcare quality based on both empirical findings and opinion.     

Preferred Self-Administered Questionnaires to Assess Depression,Anxiety and Somatization in People With Musculoskeletal Pain – A Modified Delphi Study

Depression, anxiety, and somatization influence the recovery of people with musculoskeletal pain. A Delphi study was conducted to reach consensus on the most appropriate self-administered questionnaires to assess these psychosocial factors in people at risk of developing persistent musculoskeletal pain. A multidisciplinary panel of international experts was identified via PubReMiner. The experts (N = 22) suggested 24 questionnaires in Round 1. In Round 2, experts rated the questionnaires on suitability, considering clinimetrics, content, feasibility, personal experiences, and expertise. The highest ranked questionnaires were retained for Round 3, in which the experts made a final assessment of the suitability of the questionnaires. Sensitivity analyses were performed to assess the impact of 1) not all experts having participated in each round, and 2) experts having been involved in relevant questionnaire development. Consensus (ie, ≥75% agreement) was reached for the following questionnaires. For depression: Patient Health Questionnaire-9, Beck Depression Inventory-II, Center for Epidemiological Studies-Depression Scale, and Depression Subscale of the Depression, Anxiety and Stress Scales. In the sensitivity analyses, consensus was also reached for the Depression Subscale of the Hospital Anxiety Depression Scale. For anxiety: Generalized Anxiety Disorder Scale-7, State and Trait Anxiety Inventory, and Pain Anxiety Symptoms Scale. For somatization: no recommendation could be made.PerspectiveThis study generated a short list of preferred questionnaires to assess depression, anxiety, and somatization in people with musculoskeletal pain. Broad implementation of these questionnaires by clinicians and researchers will facilitate easier comparison and pooling of baseline and outcome data. Some of the recommended questionnaires still require validation in this population.     

A Delphi study to identify performance indicators for emergency medicine

Objectives: The aim of this study was to identify performance indicators thought to reflect the quality of patient care in the emergency department.

Methods: A three round accelerated Expert Delphi study was conducted by email or fax. A panel of 33 experts drawn from the fields of emergency medicine, emergency nursing, professional service users, and patients were consulted. Participants were initially asked to propose performance indicators that reflected the quality of care given in the emergency department setting in the United Kingdom. In the second round these proposals were collated and scored using a 9 point Likert scale; those that had not reached consensus were returned for reconsideration in the light of group opinion. Those statements reaching a pre-defined consensus were identified.

Results: 224 performance indicators were proposed. Altogether 36 indicators reached consensus reflecting good departmental performance after round three; 24 of these were process measures.

Conclusions: 36 potential indicators of good quality of care in the emergency department in the UK have been identified.

     

Developing a Certificate Course for Hospice Nurses: A Delphi Survey of Subject Areas

A Delphi survey of 16 national experts in hospice nursing was conducted to address which subject areas were the most important to include in a certification course for hospice nurses. The three-round survey elicited 70 subject areas which were ranked in order of importance according to the group opinion of the panel of experts. The means, average deviations, standard deviations, and variances for each rank were calculated. The panel showed a high degree of consensus and there was little change in ranking after the second round. The survey produced results that should be of interest to organizations concerned with developing certification courses and examination procedures for hospice nurses.     

Development and Validation of a Mass Casualty Conceptual Model

Purpose: To develop and validate a conceptual model that provides a framework for the development and evaluation of information systems for mass casualty events.
Design: The model was designed based on extant literature and existing theoretical models. A purposeful sample of 18 experts validated the model. Open-ended questions, as well as a 7-point Likert scale, were used to measure expert consensus on the importance of each construct and its relationship in the model and the usefulness of the model to future research.
Methods: Computer-mediated applications were used to facilitate a modified Delphi technique through which a panel of experts provided validation for the conceptual model. Rounds of questions continued until consensus was reached, as measured by an interquartile range (no more than 1 scale point for each item); stability (change in the distribution of responses less than 15% between rounds); and percent agreement (70% or greater) for indicator questions.
Findings: Two rounds of the Delphi process were needed to satisfy the criteria for consensus or stability related to the constructs, relationships, and indicators in the model. The panel reached consensus or sufficient stability to retain all 10 constructs, 9 relationships, and 39 of 44 indicators. Experts viewed the model as useful (mean of 5.3 on a 7-point scale).
Conclusions: Validation of the model provides the first step in understanding the context in which mass casualty events take place and identifying variables that impact outcomes of care.
Clinical Relevance: This study provides a foundation for understanding the complexity of mass casualty care, the roles that nurses play in mass casualty events, and factors that must be considered in designing and evaluating information-communication systems to support effective triage under these conditions.     

Psychometric Analysis of the Brisbane Practice Environment Measure (B-PEM)

Purpose: To undertake rigorous psychometric testing of the newly developed contemporary work environment measure (the Brisbane Practice Environment Measure [B-PEM]) using exploratory factor analysis and confirmatory factor analysis.
Methods: Content validity of the 33-item measure was established by a panel of experts. Initial testing involved 195 nursing staff using principal component factor analysis with varimax rotation (orthogonal) and Cronbach's alpha coefficients. Confirmatory factor analysis was conducted using data from a further 983 nursing staff.
Results: Principal component factor analysis yielded a four-factor solution with eigenvalues greater than 1 that explained 52.53% of the variance. These factors were then verified using confirmatory factor analysis. Goodness-of-fit indices showed an acceptable fit overall with the full model, explaining 21% to 73% of the variance. Deletion of items took place throughout the evolution of the instrument, resulting in a 26-item, four-factor measure called the Brisbane Practice Environment Measure-Tested.
Conclusions: The B-PEM has undergone rigorous psychometric testing, providing evidence of internal consistency and goodness-of-fit indices within acceptable ranges. The measure can be utilised as a subscale or total score reflective of a contemporary nursing work environment.
Clinical Relevance: An up-to-date instrument to measure practice environment may be useful for nursing leaders to monitor the workplace and to assist in identifying areas for improvement, facilitating greater job satisfaction and retention.     

Desirable attributes of the ambulance technician, paramedic, and clinical supervisor: findings from a Delphi study

Methods: The Delphi technique was used to gain a consensus view from a panel of experts. The first round of the study asked the experts to list the attributes they believed were desirable for the ambulance technician, the ambulance paramedic, and the clinical supervisor. The first round of the study generated 3403 individual statements that were collapsed into 25 broad categories, which were returned to the experts, who were required to rate each of the attributes along a visual analogue scale in respect of each of the identified occupational groups.

Results: On completion of the second round the data were analysed to demonstrate rank ordering of desirable attributes by occupational group. The level of agreement within each group was determined by analysis using the Kendall coefficient of concordance. This showed high levels of agreement within the technician group but less agreement within the paramedic and clinical supervisor group. All were highly significant p<0.0001.

Conclusions: There was significant agreement among the experts as to the desirable attributes of ambulance staff, many of which do not feature in existing ambulance training curriculums. The findings of this study may therefore be of value in informing future curriculum development and in providing guidance for the selection of staff for each of the occupational groups.

     

Importance and emphasis of mycobacteriology competencies as perceived by educators.

OBJECTIVE: To compile a list of mycobacteriology competencies and to investigate the importance and emphasis of these competencies as perceived by clinical laboratory educators of three program types. DESIGN: Use of a survey instrument, which included 97 competency statements with Likert-type scales for the quantitative study of importance and emphasis of competencies. STUDY SELECTION: Randomly selected sample from all accredited college/university-based clinical laboratory science programs, hospital-based clinical laboratory science programs, and associate-degree clinical laboratory technician programs. DATA SYNTHESIS: For analysis, the competencies were divided into eight sub-categories. Means for each competency and each sub-category were calculated. Analysis of variance was used to determine any difference among the three program types for each sub-category of competencies. Chi-square analysis was used to determine significant differences for competencies requiring dichotomous responses. CONCLUSION: With the significant increase in tuberculosis and other mycobacterioses, the mycobacteriology experience for clinical laboratory students must be validated for relevance.     

Developing Program Performance Measures for Rural Emergency Medical Services

Background: The development of measures to monitor and evaluate the performance and quality of emergency medical services (EMS) systems has been a focus of attention for many years. The Medicare Rural Hospital Flexibility Program (Flex Program), established by Congress in 1997, provides grants to states to implement initiatives to strengthen rural healthcare delivery systems, including better integration of EMS into those systems of care. Objective: Building on national efforts to develop EMS performance measures, we sought to identify measures relevant to the rural communities and hospitals supported by the Flex Program. The measures are intended for use in monitoring rural EMS performance at the community level as well as for use by State Flex Programs and the Federal Office of Rural Health Policy (FORHP) to demonstrate the impact of the Flex Program. Methods: To evaluate the performance of EMS in rural communities, we conducted a literature search, reviewed research on performance measures conducted by key EMS organizations, and recruited a panel of EMS experts to identify and rate rurally-relevant EMS performance measures as well as emergent protocols for episodes of trauma, ST Elevation Myocardial Infarction (STEMI), and stroke. The rated measures were assessed for inclusion in the final measure set. Results: The Expert Panel identified 17 program performance measures to support EMS services in rural communities. These measures monitor the capacity of local agencies to collect and report quality and financial data, use the data to improve agency performance, and train rural EMS employees in emergent protocols for all age groups. Conclusion: The system of care approach on which this rural EMS measures set is based can support the FORHP's goal of better focusing State Flex Program activity to improve program impact on the performance of rural EMS services in the areas of financial viability, quality improvement, and local/regional health system performance.     

Identifying nurse sensitive indicators specific to haemodialysis nursing: A Delphi approach

BackgroundNurse sensitive indicators measure the quality of care that is specific to the practice of nurses. There are currently few indicators suitable to measure haemodialysis nursing practice.AimTo identify haemodialysis nurse sensitive indicators.MethodsThe Delphi technique, an iterative process, was used to attain consensus from a national expert panel of haemodialysis registered nurses. The panel rated their level of agreement, importance and frequency of nurse sensitive indicators using online surveys that included qualitative feedback. Results of each round shaped subsequent rounds. Consensus was set at greater than 70.0%.FindingsThe 38-member expert panel had a median haemodialysis experience of 17 years (IQR 8.5–22) and 89% had attained a specialist post-registration renal qualification. Round 1 assessed the suitability of 38 nurse sensitive indicators. Those indicators failing to achieve consensus were retested and removed after round 2 (n = 12). Round 3 assessed a modified list of 26 nurse sensitive indicators with all achieving consensus. The fourth and final round was used to both provide feedback to the expert panel and to elicit further qualitative feedback.DiscussionA comprehensive set of quality indicators to measure nursing-sensitive activities in the specialised context of haemodialysis have been developed.ConclusionThese 26 haemodialysis nurse sensitive indicators capture the structures, processes and outcomes which could measure the quality of nursing care provided to people receiving haemodialysis.     

Organizational Factors to Consider When Developing a Certification Program for Hospice Nurses: A Delphi Survey

A Delphi survey of 16 national experts in hospice nursing was conducted to address the question of which organizational factors should be considered when developing a certification program for hospice nurses. The three-round survey elicited many factors to consider which were arranged by category and ranked in order of importance according to the group opinion of the panel. The means, average deviations, standard deviations, and variances for each factor were calculated. The panel showed a high degree of consensus and there was little change in ranking after the second round. The survey produced results that should be of interest to organizations concerned with developing a certification program for hospice nurses.     

The Norwegian General Practice – Nursing Home criteria (NORGEP-NH) for potentially inappropriate medication use: A web-based Delphi study

Objective. To develop a set of explicit criteria for pharmacologically inappropriate medication use in nursing homes. Design. In an expert panel, a three-round Delphi consensus process was conducted via survey software. Setting. Norway. Subjects. Altogether 80 participants – specialists in geriatrics or clinical pharmacology, physicians in nursing homes and experienced pharmacists – agreed to participate in the survey. Of these, 62 completed the first round, and 49 panellists completed all three rounds (75.4% of those ultimately entering the survey). Main outcome measures. The authors developed a list of 27 criteria based on the Norwegian General Practice (NORGEP) criteria, literature, and clinical experience. The main outcome measure was the panellists’ evaluation of the clinical relevance of each suggested criterion on a digital Likert scale from 1 (no clinical relevance) to 10. In the first round panellists could also suggest new criteria to be included in the process. For each criterion, degree of consensus was based on the average Likert score and corresponding standard deviation (SD). Results. A list of 34 explicit criteria for potentially inappropriate medication use in nursing homes was developed through a three-round web-based Delphi consensus process. Degree of consensus increased with each round. No criterion was voted out. Suggestions from the panel led to the inclusion of seven additional criteria in round two. Implications. The NORGEP-NH list may serve as a tool in the prescribing process and in medication list reviews and may also be used in quality assessment and for research purposes.     

Improving quality in Medicaid: the use of care management processes for chronic illness and preventive care

BACKGROUND: Care management processes (CMPs), tools to improve the efficiency and quality of primary care delivery, are particularly important for low-income patients facing substantial barriers to care. OBJECTIVE: To measure the adoption of CMPs by medical groups, Independent Practice Associations, community clinics, and hospital-based clinics in California's Medicaid program and the factors associated with CMP adoption. METHODS: Telephone survey of every provider organization with at least 6 primary care physicians and at least 1 Medi-Cal HMO contract, Spring 2003. One hundred twenty-three organizations participated, accounting for 64% of provider organizations serving Medicaid managed care in California. We surveyed 30 measures of CMP use for asthma and diabetes, and for child and adolescent preventive services. RESULTS: The mean number of CMPs used by each organization was 4.5 for asthma and 4.9 for diabetes (of a possible 8). The mean number of CMPs for preventive services was 4.0 for children and 3.5 for adolescents (of a possible 7). Organizations with more extensive involvement in Medi-Cal managed care used more CMPs for chronic illness and preventive service. Community clinics and hospital-based clinics used more CMPs for asthma and diabetes than did Independent Practice Associations (IPAs), and profitable organizations used more CMPs for child and adolescent preventive services than did entities facing severe financial constraints. The use of CMPs by Medicaid HMOs and the presence of external (financial and nonfinancial) incentives for clinical performance were strongly associated with use of care management by provider organizations. CONCLUSIONS: Physician and provider organizations heavily involved in California's Medicaid program are extensively engaged in preventive and chronic care management programs.     

Improving patient safety in intensive care units in Michigan

PURPOSE: The aim of this study was to describe the design and lessons learned from implementing a large-scale patient safety collaborative and the impact of an intervention on teamwork climate in intensive care units (ICUs) across the state of Michigan. MATERIALS AND METHODS: This study used a collaborative model for improvement involving researchers from the Johns Hopkins University and Michigan Health and Hospital Association. A quality improvement team in each ICU collected and submitted baseline data and implemented quality improvement interventions. Primary outcome measures were improvements in safety culture scores using the Teamwork Climate Scale of the Safety Attitudes Questionnaire (SAQ); 99 ICUs provided baseline SAQ data. Baseline performance for adherence to evidence-based interventions for ventilated patients is also reported. The intervention to improve safety culture was the comprehensive unit-based safety program. The rwg statistic measures the extent to which there is a group consensus. RESULTS: Overall response rate for the baseline SAQ was 72%. Statistical tests confirmed that teamwork climate scores provided a valid measure of teamwork climate consensus among caregivers in an ICU, mean rwg was 0.840 (SD = 0.07). Teamwork climate varied significantly among ICUs at baseline (F98, 5325 = 5.90, P < .001), ranging from 16% to 92% of caregivers in an ICU reporting good teamwork climate. A subset of 72 ICUs repeated the culture assessment in 2005, and a 2-tailed paired samples t test showed that teamwork climate improved from 2004 to 2005, t(71) = -2.921, P < .005. Adherence to using evidence-based interventions ranged from a mean of 25% for maintaining glucose at 110 mg/dL or less to 89% for stress ulcer prophylaxis. CONCLUSION: This study describes the first statewide effort to improve patient safety in ICUs. The use of the comprehensive unit-based safety program was associated with significant improvements in safety culture. This collaborative may serve as a model to implement feasible and methodologically rigorous methods to improve and sustain patient safety on a larger scale.     

An Examination of Measures Related to Children's Exposure to Violence for Use by Both Practitioners and Researchers

Millions of children each year are exposed to violence in their homes, schools, and communities as both witnesses and victims. As a result, evidence-based programs for children and adolescents who have been exposed to traumatic events (CEV-EBPs) have been widely disseminated but rarely evaluated in their real-world applications. One crucial aspect of conducting such evaluations is finding appropriate measures that can be of use both to the practitioner and to the researcher. This review aims to provide guidance to the field by first identifying any gaps in the availability of psychometrically tested measures for certain outcome domains and age ranges and then recommending the measures that are most appropriate for use by both researchers and practitioners. Interviews with content experts in the measurement of trauma symptoms and parent-child relationships were conducted to identify the key outcome domains for measurement that are critical to the evaluation of CEV-EBPs and the criteria for dual-use measures, defined as measures that are useful to both researchers and practitioners. A database of 46 relevant measures was created by compiling measures from existing repositories and conducting a focused literature review. Our review of these measures found that existing repositories had few measures of depression, a major gap that should be addressed. Further, there were few measures for young children ages 0-3 years (n = 15) and only a handful of measures (n = 9) had both a child and parent version of the measure. Overall, although the different repositories that currently exist are helpful, researchers and practitioners would benefit from having a single reputable source (e.g., a centralized repository or item bank) to access when searching for measures to use in evaluating CEV-EBPs. Such a tool would hold promising to narrow the current gap between research and practice in the field of children's exposure to violence.     

Teaching evidence-based practice: the teachers consider the content

OBJECTIVE: To identify the views of evidence-based practice (EBP) experts about information that should be included in EBP courses. DESIGN: A two-round Delphi process. SETTING AND SUBJECT: The panel of experts in teaching EBP residing in developed and developing countries identified through purposive and snowball sampling. MAIN OUTCOME MEASURES: We developed a list of 49 topics representing key educational topics. The panel scored the importance of topics for inclusion in introductory and advanced courses. In the second round, we sent a summary of results to the panel and asked them to re-score in light of the group's responses. RESULTS: We used email to invite 105 EBP teachers to participate in the study. Fifty-one people from 15 countries agreed to participate, and 40 completed the second round. We achieved consensus that formulating clinical questions, searching pre-appraised resources, introduction to systematic reviews and critical appraisal of studies about therapy should be covered earlier in EBP courses while other critical appraisal topics and quantitative decision-making techniques should be left to more advanced levels. CONCLUSION: Experts concur that introductory EBP courses should be simple and certain topics should be avoided. Specifically, critical appraisal and statistical methods should be left to advanced courses.     

DEFINING RESEARCH CRITERIA TO CHARACTERIZE MEDICAL NECESSITY IN EMERGENCY MEDICAL SERVICES: A CONSENSUS AMONG EXPERTS AT THE NEELY CONFERENCE

Objective. Researchers interested in ensuring appropriate use of emergency medical services (EMS) resources have attempted to define safe and effective protocols for triage either at the time of dispatch or after on-scene evaluation. Published work in this area is difficult to evaluate because protocols and outcome criteria vary from study to study. The goal of the Neely Conference was to bring together EMS experts to define a set of criteria to be used in research studies evaluating dispatch triage and field triage systems. Methods. Thirty-one experts in EMS systems and research attended a day-long workshop to assess the current literature regarding dispatch triage and field triage, and make recommendations to standardize methods used to evaluate future triage protocols. Participants were surveyed during the workshop; consensus analysis techniques were used to determine if a formal consensus was reached. A Bayesian posterior probability of 0.99 was required to consider responses a “consensus.” Results. Participants considered current evidence regarding the usefulness of EMS triage criteria to be “weak.” However, respondents agreed on a set of research criteria that could define the need for an EMS response and/or EMS transport. Field triage criteria were considered more plausible than dispatch criteria. Valid outcome criteria for assessing the effectiveness of triage protocols included ED assessment and the need for immediate surgery. Hospital admission, final diagnosis, and expert opinion were not considered adequate outcome measures. Conclusion. EMS experts agreed on a standard set of triage criteria and outcome measures for evaluating triage protocols supporting alternative forms of transport and care.     

Implementing routine screening and brief alcohol intervention in primary health care: A Delphi survey of expert opinion

Aim To obtain a consensus of expert views on how best to implement screening and brief intervention (SBI) for excessive drinkers in a routine and enduring fashion in primary health care throughout England.

Method A Delphi survey of expert opinion in the UK.

Participants Seventy‐nine experts in SBI, of whom 53 (67%) remained in round 3 of the survey. The expert panel included primary health‐care professionals, alcohol‐service workers and researchers/ academics.

Measurements In round 3, 53 panel members (67% of an initial sample of 79) made ratings on a five‐point Likert scale of 157 items developed from responses to open ended questions in round 1 and fed back with group median ratings derived from round 2. Consensus was defined as an interquartile range of ≤1 and attention was mainly directed to items with consensus around median responses of strong agreement or disagreement.

Findings A number of clear conclusions emerged from the survey, including the recommendation of routine screening confined to new patient registrations, general health checks and special types of consultation. The employment of a specialist alcohol worker as a member of the primary health‐care team was strongly supported, but a model of interprofessional cooperation in the delivery of SBI could also be derived from findings. Other conclusions included the importance for the widespread implementation of SBI of a national alcohol strategy.     

Building a Performance Information System for Statewide Residential Treatment Services

This article presents a process for building a state-level performance information system for psychiatric residential treatment facilities for children and youth. Based on a consensus-based program model, researchers identified information needs of stakeholders for measuring clinical outcome measures and for monitoring outcomes after clients are discharged from programs. After reviewing the extant outcome measurement instruments and the array for information needs, researchers recommended a comprehensive package of measures for stakeholders. The article illustrates collaborative efforts made by researchers, state agency, and stakeholders to develop sets of indicators in three areas of performance information: access, process, and client status outcome.