Scaling the quality of clinical audit projects: a pilot study.

OBJECTIVE: To pilot the development of a scale measuring the quality of audit projects through audit project reports. DESIGN: Statements about clinical audit projects were selected from existing instruments, assessing the quality of clinical audit projects, to form a Likert scale. SETTING: The audit facilitators were based in Scottish health boards and trusts. STUDY PARTICIPANTS: The participants were audit facilitators known to have over 2 years experience of supporting clinical audit. The response at first test was 11 out of 14 and at the second test it was 27 out of 46. INTERVENTIONS: The draft scale was tested by 27 audit facilitators who expressed their strength of agreement or disagreement with each statement for three reports. MAIN OUTCOME MEASURES: Validity was assessed by test-re-test, item-total, and total-global indicator correlations. RESULTS: Of the 20 statements, 15 had satisfactory correlations with scale totals. Scale totals had good correlations with global indicators. Test-re-test correlation was modest. CONCLUSIONS: The wide range of responses means further research is needed to measure the consistency of audit facilitators' interpretations, perhaps comparing a trained group with an untrained group. There may be a need for a separate scale for reaudits. Educational impact is distinct from project impact generally. It may be more meaningful to treat the selection of projects and aims, methodology and impact separately as subscales and take a project profiling approach rather than attempting to produce a global quality index.     

A comparative analysis of six audit systems for mental health nursing.

PURPOSE: To devise an analytical framework to help identify strengths and weaknesses in the audit process as specified by existing psychiatric nursing audit systems, in order to analyse current audit practice and identify improvements for incorporation in the Newcastle Clinical Audit Toolkit for Mental Health. DATA SOURCES: Published material relating to the following six systems: the Central Nottinghamshire Psychiatric Nursing Audit; Psychiatric Nursing Monitor; Standards of Care and Practice; Achievable Standards of Care; Quartz; and Quest. DATA EXTRACTION: Comparison of the six systems according to an analytical framework derived from detailed empirical study (structures, processes and outcomes) of one of them in use and the educational evaluation literature. Examination of the extent to which guidance is provided for operating the systems and for wider process-related aspects of audit. RESULTS OF DATA SYNTHESIS: Five of the systems failed to specify some important elements of the audit process. Conceptually, the six systems can be divided into two main types: 'instrument-like' systems designed along psychometric lines and which emphasize the distance between the subjects of audit and the operators of the systems, and 'tool-like' systems which exploit opportunities for care setting staff to engage in the audit process. A third type of system is the locally-developed system which is offered to a wider audience but which does not make the same level of claim to universal applicability. CONCLUSION: The analytical framework allows different approaches to audit to be compared and contrasted not only according to the techniques used, but also according to process issues. The analysis of six systems revealed a variety of different techniques and procedures which can facilitate, in a methodologically rigorous manner, practitioner and other stakeholder involvement in audit processes.     

The challenge of balancing methodological research rigour and practical needs in low-income settings: What we are doing and what we need to do better

A roundtable discussion was held with 29 experienced Public Health researchers and policy makers from 9 countries at an international conference in Australia. Their experiences were discussed about establishing and maintaining research projects and health programs in developing countries. This paper aims to (1) Highlight methodological issues relevant to the conduct of research and development in low income settings as discussed at the roundtable; (2) List the main barriers in implementing research and project initiatives in low-income settings; and (3) present recommendations for meeting researchers' needs across cultures without jeopardising methodological rigour. Lessons learnt on barriers for the conduct of methodologically sound research were classified in four domains: investigator factors, community factors, strategic factors and methodological factors. Proposed solutions emerged in the areas of strategic guidelines, attitudinal changes and methodological approaches. Practical solutions to individual and systematic barriers were also suggested. It is hoped that these recommendations will influence current practice and assist other researchers in preparing and undertaking appropriate research methods and health program delivery strategies in low-income settings.     

Analysis of regional health policy for children: an example from Lorraine

Whereas today we have good, solid knowledge of the perception of health and its objectives from the perspective of youth, the recommendations of the High Commission on Public Health are geared towards the implementation of an overall health policy focusing on adolescent health. How is this policy implemented on a regional level? Is there coherence between actions undertaken within the same region? How do the various actors, elected officials, professionals, young people and families work together? A survey was conducted in the Lorraine region involving 217 people with a framework utilising both individual and group interviews. All categories of actors and stakeholders were represented. The results of this work demonstrate an often negative representation settings, and a partition of different approaches. Given that a variety of activities and actions exist, they are most often thematic. There undoubtedly remains a need to create links and build bridges between all of the projects carried out and to define in a more precise manner the common objectives focused on the health of young people.     

Medical audit cycle

The aim of medical audit is to improve the effectiveness and efficiency of medical care. Achieving this aim may involve a cycle of activities: (i) observing practice; (ii) setting a standard of practice; (iii) comparing the observed practice with the standard; (iv) implementing change; and (v) re-observing practice. This paper reviews those methods which have been used and tested in clinical practice at the various stages of the cycle. Some principles involved in the choice of what to audit are also described.
As definitive cost effective methods of auditing have not yet emerged, further initiatives and experimentation are required in clinical practice.     

浅谈医院建设工程竣工结算审核工作

探讨医院建设工程竣工审核问题。通过加强严格核对发包合同条款．认真检查隐蔽验收记录、落实设计变更签证、按图核定工程量、严格执行定额单价。着重讨论了医院建设工程审核节约工作的管理。     

Audit and summative assessment: system development and testing

Registrars in general practice have to submit an audit project as one of four parts of summative assessment. A criterion-referenced marking schedule has been developed in the West of Scotland, consisting of five independent criteria all of which have to be judged above minimum competence to pass. A system was developed to test the instrument using a marking exercise which calculated the sensitivity and specificity of the assessment process, for different combinations of assessors. One hundred and two registrar audit projects were then assessed by three independent assessors. Ninety-two (90%) passed and 10 projects (10%) were referred back to the registrar as being below minimum competence. After resubmission six projects (6%) passed, two projects (2%) were still below minimum competence, and two (2%) were not resubmitted. A referral process for assessing the audit projects of general practice registrars has been developed to maximize the opportunity of finding a project below minimum competence.     

Understanding variation in quality improvement: the treatment of sore throats in primary care

BACKGROUND: In 1988, two practices attempted to improve the prescribing of antibiotics for sore throat. The initiative produced only modest improvements in prescribing practice, a finding the authors found difficult to explain. This paper reanalyses the data from an audit of antibiotic prescribing for sore throat in general practice. OBJECTIVE: Our aim was to demonstrate the use of Shewhart control charts and to obtain fresh insight into the variations in clinical practice revealed in clinical audit data. METHODS: We use Shewhart control charts to explore variation in antibiotic prescribing between GPs and to suggest the action most likely to result in improvement. RESULTS: Using control charts, it is possible to distinguish two categories of GPs: low prescribers of antibiotics and high prescribers of antibiotics. Low prescribers of antibiotics show common cause variation, indicating that their prescribing is a stable process. Among low prescribers, improvement can best be achieved by changing the common underlying process. One high prescriber of antibiotics is affected by special cause variation. Among high prescribers, improvement can best be achieved by investigating the special causes affecting this GP and learning lessons from the findings. CONCLUSION: The original improvement effort took the same action on all GPs in both practices. Our analysis suggests that such an approach was unlikely to be successful and that different actions were needed for high and low prescribers. The control charts provide fresh insights on the original data and guide improvement efforts.     

Clinical audit. Check-up time

A survey of clinical audit in 21 trusts found wide variations in workload. The number of audits carried out in one year ranged from four to 218. Services most frequently audited were surgery, diagnostics, mental health and medicine. The average cost of an audit was 2,260 pounds. Most trusts reviewed progress on implemented agreed recommendations following an audit.     

Implementing alcohol and other drug interventions effectively: How does location matter?

Objective: To identify enabler and barrier mechanisms that impact project implementation, from a review of 127 completed projects intended to reduce harms associated with problematic alcohol and licit drug use in Australia. Design: Data comprised archival material (proposals, reports). A coding framework was developed from the literature and a sample of projects; two researchers developed and refined the framework. Open coding was used to identify factors impacting implementation, followed by pattern coding to identify underlying mechanisms. Sample: Project categories were developed from funding orientation and main activities projects were: enhancing organizational systems and processes (39), training and workforce development (18), community education and prevention (37) and client engagement and treatment (33). Thirty‐five projects (28%) were in non‐capital city locations. Results: Nine enabler and ten barrier mechanisms were identified, for example, ‘project planning and design’ and ‘wider service system challenges’. Three enabler mechanisms were more likely to be identified for non‐capital city projects; ‘external communication and relationships’ (83% vs 70%), ‘sensitivity to service users and settings’ (49% vs 40%) and ‘funding and resourcing’ (40% vs 35%). Most barrier mechanisms were identified for a higher proportion of non‐capital city projects, particularly ‘identifying and retaining personnel’ (54% vs 34%), ‘engaging communities and partners’ (46% vs 26%) and ‘organizational governance and capacity’ (29% vs 5%). Conclusions: Project implementation in non‐capital city locations requires particular attention to project planning and design, staffing and organizational well‐being. Policy initiatives and planning proposals might consider enabler mechanisms along with strategies to minimize barrier mechanisms to support successful implementation.     

Audit and summative assessment: a completed audit cycle

OBJECTIVE: To develop and test a system for delivering a completed audit cycle for summative assessment of general practitioner registrars in the United Kingdom. DESIGN: A trainer-based questionnaire on criteria for a completed audit cycle, followed by two marking exercises of audit projects submitted by general practice registrars. SETTING: Training practices in the West of Scotland between 1997 and 1998. SUBJECTS: Trainers and registrars in the above practices. RESULTS: 116 (89%) agreed that two collections of data were an essential or desirable part of an audit project. All 57 registrars who started in August 1997 successfully completed an audit cycle, seven (12%) after resubmission. Using two rather than three independent assessors to screen the projects, the marking instrument was shown to have a sensitivity of 95% (95% confidence interval (CI) +/-3.9%) and a specificity of 77% (95% CI +/-7.5%). All assessors found the new system easier to mark and 47 registrars (87%) found completing an audit cycle as or easier than expected. CONCLUSION: Evidence from the pilot project has shown that a general practice registrar's ability to review and critically analyse a piece of his/her work, with appropriate management of any necessary change, can be tested feasibly by means of a completed audit cycle within the registrar year. The process retains adequate levels of sensitivity and specificity and requires fewer assessors for marking the projects.     

Reflections on building community: a different perspective on academic-community partnerships using the integration matrix

Implementation of interdisciplinary rehabilitation services and clinical training in community settings is a complex process. Documented methods for analyzing, evaluating and monitoring the process within federally funded grant projects are limited. This paper reports on the use of one proposed method, the Integration Matrix of Konrad, as a framework for formative assessment and analysis of project operations. Based upon ongoing experiences implementing a Health Resources and Services Administration (HRSA) funded clinical training program in a community setting, a revision of the Integration Matrix is proposed to better reflect a community-centered, pervasive communication approach to program operations and analysis.     

Evaluating large and complex demonstrations: the CHAMPUS reform initiative experience. Civilian Health and Medical Program of the Uniformed Services

OBJECTIVE: To examine the evaluation process for the CHAMPUS Reform Initiative (CRI) both to highlight issues that evaluators must consider when undertaking such projects and to provide policymakers with tools to better assess demonstration project evaluations. DATA SOURCES: The CRI evaluation. STUDY DESIGN: Case study. DATA COLLECTION: Review of CRI evaluation reports. PRINCIPAL FINDINGS: Although policymakers increasingly rely on the evaluations of demonstration projects to determine whether to extend the scope and funding of many public programs, the results of these evaluations are often difficult to assess. Despite its analytical sophistication, the CRI evaluation was no exception. The somewhat artificial time constraints imposed by policymakers made projection of the CRI's performance beyond the demonstration period particularly difficult. CONCLUSIONS: Much uncertainty generally remains even after well-planned and well-executed evaluations of demonstration projects.     

Impact of a national audit project on gynaecologists in Scotland.

The objectives of the study were (a) to determine consultant gynaecologists' awareness of and views on a national audit project (the gynaecology audit project in Scotland) and (b) to measure changes in their reported practice in relation to 12 specific elements of care related to three audit topics (induced abortion, endometriosis, and vulvar carcinoma) for which recommendations for change had been made within the project. The study comprised a postal questionnaire survey of all 128 consultant gynaecologists in NHS practice in Scotland. The response rate was 90%. Of the respondents, 96% (109/113) recalled receiving feedback material from the audit project team and around 75% (range 66/89 to 84/105) had retained feedback reports for future reference. For the two more common clinical topics (induced abortion and endometriosis), over two thirds of the respondents indicated that they had been prompted to reconsider or change aspects of practice. Significant changes in reported practice, in line with project recommendations, were found for seven of the 12 specific elements of care examined. Thus, gynaecologists in Scotland showed a high level of awareness of and positive views towards a national audit project. Significant changes in reported practice, in accordance with circulated recommendations, were measurable in relation to several elements of clinical care.     

The European Council and health: the added value of a pan-European approach

The Council of Europe, from its creation in 1949, has been involved in establishing regulations in various field of skills. A group of conventions and actions shows the creation of status objectives. In the field of health, the Council's actions were to improve the economic and social progress in Europe. The European Committee of Health founded in 1954, is particularly in charge of health in order to establish recommendations for the members-states such as the implementation of experts groups and the preparation of precious reports. This article exposes the main decisions.     

Clinical effectiveness on an acute in-patient psychiatric unit

The paper outlines a model for addressing clinical effectiveness, and an illustration of how this is practically implemented through team working in a multidisciplinary in-patient environment. This is particularly relevant given the number of recent reports that highlight the need to develop and invest in the in-patient services. The difficulty in implementing evidence-based practice for mental health interventions is also addressed and initiatives being developed to enable a realistic approach in such an environment are described. The paper describes a structure and a process, using examples from audit, research and other initiatives particular to the unit, in providing accessible evidence based interventions for ward based staff, and improved clinical effectiveness generally.     

Audit in clinical practice: evaluating use of a nutrition screening tool developed for trauma nurses

Background: The Nutrition Checklist screening tool was developed by dietitians for the trauma nursing staff at the John Radcliffe Hospital, Oxford. Its purpose was to identify those at nutritional risk so that dietetic assessment and intervention could be implemented. This paper focuses on a single day audit that was devised 5 months post-initiation of the Nutrition Checklist. Method: Data from 48 nursing care plans was used to measure compliance of recording a nutrition score (and re-scoring where applicable) alongside the timing of dietetic referral and intervention. Results: Sixty-seven per cent (32/48) of trauma patients had a nutrition score recorded, however only 75% (24/32) of these patients were scored within 24 h of admission. Of those due for reassessment, only 38% (11/29) were rescored. Eighty-eight per cent (23/26) of the patients who scored ≥ 3 (automatic referrals) were referred to the dietitian; the dietitian documented 100% (23/23) of referrals. Conclusion: The audit suggests limited use of the screening tool by nurses. Collaboration between the nursing team, dietitian, consultant team, catering and clinical auditors could improve compliance in the use of the Nutrition Checklist and ultimately lead to improved clinical practice in nutritional care delivery. Despite its limitations, the audit exercise was a valuable learning experience in the maintenance of a nursing Nutrition Checklist. Further research is needed to assess whether the introduction of such a screening tool improves patient outcomes by minimizing nutrition-related complications.     

Audit in the therapy professions: some constraints on progress.

AIMS: To ascertain views about constraints on the progress of audit experienced by members of four of the therapy professions: physiotherapy, occupational therapy, speech and language therapy, and clinical psychology. METHODS: Interviews in six health service sites with a history of audit in these professions. 62 interviews were held with members of the four professions and 60 with other personnel with relevant involvement. Five main themes emerged as the constraints on progress: resources; expertise; relations between groups; organisational structures; and overall planning of audit activities. RESULTS: Concerns about resources focused on lack of time, insufficient finance, and lack of access to appropriate systems of information technology. Insufficient expertise was identified as a major constraint on progress. Guidance on designing instruments for collection of data was the main concern, but help with writing proposals, specifying and keeping to objectives, analysing data, and writing reports was also required. Although sources of guidance were sometimes available, more commonly this was not the case. Several aspects of relations between groups were reported as constraining the progress of audit. These included support and commitment, choice of audit topics, conflicts between staff, willingness to participate and change practice, and concerns about confidentiality. Organisational structures which constrained audit included weak links between heads of professional services and managers of provider units, the inhibiting effect of change, the weakening of professional coherence when therapists were split across directorates, and the ethos of regarding audit findings as business secrets. Lack of an overall plan for audit meant that while some resources were available, others equally necessary for successful completion of projects were not. CONCLUSION: Members of four of the therapy professions identified a wide range of constraints on the progress of audit. If their commitment to audit is to be maintained these constraints require resolution. It is suggested that such expert advice, but also that these are directed towards the particular needs of the four professions. Moreover, a forum is required within which all those with a stake in therapy audit can acknowledge and resolve the different agendas which they may have in the enterprise.     

Advance care planning for residents in aged care facilities: what is best practice and how can evidence-based guidelines be implemented?

Background Advance care planning in a residential care setting aims to assist residents to make decisions about future healthcare and to improve end-of-life care through medical and care staff knowing and respecting the wishes of the resident. The process enables individuals and others who are important to them, to reflect on what is important to the resident including their beliefs/values and preferences about care when they are dying. This paper describes a project conducted as part of the Joanna Briggs Institute Clinical Aged Care Fellowship Program implemented at the Manningham Centre in metropolitan Melbourne in a unit providing services for 46 low and high care residents. Objectives The objectives of the study were to document implementation of best practice in advance care planning in a residential aged care facility using a cycle of audit, feedback and re-audit cycle audit with a clinical audit software program, the Practical Application of Clinical Evidence System. The evidence-based guidelines found in 'Guidelines for a Palliative Approach in Residential Aged Care' were used to inform the process of clinical practice review and to develop a program to implement advance care planning. Results The pre-implementation audit results showed that advance care planning practice was not based on high level evidence as initial compliance with five audit criteria was 0%. The barriers to implementation that became apparent during the feedback stage included the challenge of creating a culture where advance care planning policy, protocols and guidelines could be implemented, and advance care planning discussions held, by adequately prepared health professionals and carers. Opportunities were made to equip the resident to discuss their wishes with family, friends and healthcare staff. Some residents made the decision to take steps to formally document those wishes and/or appoint a Medical Enduring Power of Attorney to act on behalf of the resident when they are unable to communicate wishes. The post-implementation audit showed a clear improvement as compliance ranged from 15-100% for the five audit criteria. Strong leadership by the project team was effective in engaging staff in this quality improvement program. Conclusion The outcomes of the project were extremely positive and demonstrate a genuine improvement in practice. All audit criteria indicate that the Manningham Centre is now positively working towards improved practice based on the best available evidence. It is hoped that as the expertise developed during this project is shared, other areas of gerontological practice will be similarly improved and more facilities caring for the older person will embrace evidence-based practice.