The General Medical Council's Performance Procedures: peer review of performance in the workplace

The General Medical Council procedures to assess the performance of doctors who may be seriously deficient include peer review of the doctor's practice at the workplace and tests of competence and skills. Peer reviews are conducted by three trained assessors, two from the same speciality as the doctor being assessed, with one lay assessor. The doctor completes a portfolio to describe his/her training, experience, the circumstances of practice and self rate his/her competence and familiarity in dealing with the common problems of his/her own discipline. The assessment includes a review of the doctor's medical records; discussion of cases selected from these records; observation of consultations for clinicians, or of relevant activities in non-clinicians; a tour of the doctor's workplace; interviews with at least 12 third parties (five nominated by the doctor); and structured interviews with the doctor. The content and structure of the peer review are designed to assess the doctor against the standards defined in Good Medical Practice , as applied to the doctor's speciality. The assessment methods are based on validated instruments and gather 700–1000 judgements on each doctor. Early experience of the peer review visits has confirmed their feasibility and effectiveness.     

Training the assessors for the General Medical Council's Performance Procedures

From July 1997, the General Medical Council (GMC) has had the power to investigate doctors whose performance is considered to be seriously deficient. Assessment procedures have been developed for all medical specialties to include peer review of performance in practice and tests of competence. Peer review is conducted by teams of at least two medical assessors and one lay assessor. A comprehensive training programme for assessors has been developed that simulates the context of a typical practice-based assessment and has been tailored for 12 medical specialties. The training includes the principles of assessment, familiarization with the assessment instruments and supervised practice in assessment methods used during the peer review visit. High fidelity is achieved through the use of actors who simulate third party interviewees and trained doctors who role play the assessee. A subgroup of assessors, selected to lead the assessment teams, undergo training in handling group dynamics, report writing and in defending the assessment report against legal challenge. Debriefing of assessors following real assessments has been strongly positive with regard to their preparedness and confidence in undertaking the assessment.     

开放评议与双盲评议在国际科技期刊质量控制中的价值

【目的】了解国际科技期刊在开放评议和双盲评议方面的实践,探索这两种评议方式的优缺点及对科技期刊质量控制的价值。【方法】对国际科技期刊实行开放评议和双盲评议的案例进行调研,如BMC系列期刊、F1000Research、Nature系列期刊等。【结果】有越来越多的期刊在尝试这两种方式,它们在减少传统单盲评议的偏见方面发挥了一定作用,但也带来了新的问题。【结论】开放评议和双盲评议适合于不同类型的期刊。期刊应根据自身的学科特点、学术定位来选择适合自己的同行评议方式,并在实践中不断评估,对之进行适当的调整。     

Procedures for establishing defensible programmes for assessing practice performance

The assessment of the performance of doctors in practice is becoming more widely accepted. While there are many potential purposes for such assessments, sometimes the consequences of the assessments will be 'high stakes'. In these circumstances, any of the many elements of the assessment programme may potentially be challenged. These assessment programmes therefore need to be robust, fair and defensible, taken from the perspectives of consumer, assessee and assessor. In order to inform the design of defensible programmes for assessing practice performance, a group of education researchers at the 10th Cambridge Conference adopted a project management approach to designing practice performance assessment programmes. This paper describes issues to consider in the articulation of the purposes and outcomes of the assessment, planning the programme, the administrative processes involved, including communication and preparation of assessees. Examples of key questions to be answered are provided, but further work is needed to test validity.     

Defending the boundaries of science: AIDS denialism, peer review and the Medical Hypotheses saga

This paper explores the boundary work undertaken by HIV scientists and activists against the journal Medical Hypotheses over its lack of peer review. Their action was sparked by the publication of an article by Peter Duesberg claiming that HIV does not cause AIDS and that antiretrovirals do more harm than good. Precisely because such 'AIDS denialism' can undermine HIV prevention and treatment interventions, as was demonstrably the case in South Africa under President Mbeki, the episode raised questions about when, in the interests of public health, the boundaries of legitimate scientific debate may be drawn to exclude unreasonable and unscholarly arguments. The paper argues that normative concerns motivated the complaints which resulted in the publisher withdrawing Duesberg's paper and imposing editorial policy changes on Medical Hypotheses. Concerns were raised about the implications for academic freedom of this boundary work in defence of peer review as a core practice in science. The paper concludes, however, that Duesberg's freedom to write what he likes remains intact, but that if he wants his work to carry the imprimatur of science, he now has to subject it to peer review.     

负责任同行评议:何谓、何以与何为

【目的】对“负责任同行评议”进行阐释,为促进同行评议质量提升提供参考建议。【方法】对“负责任同行评议”进行解构;针对什么是负责任同行评议、为什么需要负责任同行评议、负责任同行评议涵盖哪些方面,以及如何落实负责任同行评议分别展开讨论。【结果】阐释负责任同行评议的内涵,指出推行负责任同行评议的重要性与迫切性,以及提出推行实践负责任同行评议的保障措施。【结论】负责任同行评议为当前的科研评价问题提供了一个可能的解决方向,可以从理论层面引起研究者的关注,从实践层面引发评审人的思考。     

Current perspectives in assessment: the assessment of performance at work

BACKGROUND: Traditional assessment has improved significantly over the past 50 years. A number of new testing methods are now in place, the computer is improving both the fidelity and efficiency of examinations, and the psychometric principles on which assessment rests are more sophisticated than ever. AIM: There is growing interest in quality improvement and there are increasing demands for public accountability. This has shifted the focus of testing from education to work. The purpose of this paper is to describe the assessment of work. DISCUSSION: In contrast to traditional assessment, there are no 'methods' for the evaluation of work because the content and difficulty of the examination are not controlled in any fashion. Instead it is a matter of identifying the basis for the judgements (outcomes, process, or volume), deciding how the data will be gathered (practice records, administrative databases, diaries/logs, or observation), and avoiding threats to validity and reliability (patient mix, patient complexity, attribution, and numbers of patients). FUTURE DIRECTIONS: Overall, the assessment of doctors' performance at work is in its infancy and much research and development is needed. Nonetheless, it is being used increasingly in programmes of continuous quality improvement and accountability. It is critical that refinements occur quickly to ensure that patients receive the highest quality of care and that doctors are treated fairly and provided with the information they need to guide their professional development.     

The development of quality circles/peer review groups as a method of quality improvement in Europe. Results of a survey in 26 European countries

BACKGROUND: Peer review groups (PRGs) and quality circles (QCs) commenced in The Netherlands and have grown to become an important method of quality improvement in primary care in several other European countries. OBJECTIVE: Our aim was to provide an overview of QC/PRG activities and exemplary programmes in European countries. METHODS: A survey was performed in three consecutive steps by EQuiP (European Working Party on Quality in Family Practice), which is a representative association of experts from 26 European countries. The national representatives initially completed a structured questionnaire documenting the number and objectives of QCs/PRGs, sources of support and special programmes in their countries (step 1). In step 2, these sources were used to extend and validate the expert statements. Step 3 studied paradigmatic initiatives in depth. RESULTS: Step 1 took place in 2000; the response rate was 100% (26 countries). QCs/PRGs were very active in 10 countries; 16 countries showed little or no activity. Participation ranged from <2 to 86% of all GPs. Step 2 concentrated upon the countries with a high level of activity. Development appeared to be associated with establishment in private practice and the portion of GPs with vocational training. Eight programmes from six countries describing the establishment and the targeting of QC/PRG work are presented as case reports (step 3). CONCLUSION: In the last 10 years, substantial development of QCs/PRGs has taken place in The Netherlands, the UK, Denmark, Belgium, Ireland, Sweden, Norway, Germany, Switzerland and Austria. Further evaluation is needed to clarify the impact on quality of care.     

Assessing the performance of doctors in teams and systems

INTRODUCTION: Increasing attention is being directed towards finding ways of assessing how well doctors perform in clinical practice. Current approaches rely on strategies directed at individuals only, but, in real life, doctors' work is characterised by multiple complex professional interactions. These interactions involve different kinds of teams and are embedded within the overall context and systems of care. In addition to individual factors, therefore, we propose that the performance of doctors in health care teams and systems will also impact on the overall quality of patient care. Assessing these dimensions, however, poses a number of challenges. STRATEGIES: Taking a profile of a National Health Service, UK surgeon as an example, the team structures to which he or she may relate are illustrated. These include formal teams such as those found in the operating theatre, and those formed through various professional and collegial partnerships. The authors then propose a model for assessing doctors' performances in teams and systems, which incorporates the educational principles of continuous feedback to enhance future performance. DISCUSSION: To implement the proposed model, a wide range of professional, educational and regulatory bodies must collaborate. This raises a number of important implications for the future roles and relationships of these bodies, which are discussed. A strong and constructive partnership will be essential if the full potential of a more inclusive and representative assessment approach is to be realised.     

学术期刊公开同行评议的发展现状及发展趋势

目的 分析学术期刊公开同行评议的主要形式及其发展现状和趋势,为今后网络公评系统的建立提供参考。方法 参考现有的代表性公开同行评议期刊评审流程,提出建立网络公开同行评议的标准流程需考虑的质控指标。结果 建立网络公开同行评议的标准流程需要考虑：论文提交后的快速审查标准、审稿人的选择或自荐标准、审稿意见书的标准、审稿时限的标准及论文接收的标准等。结论 网络公开同行评议需建立能更大范围推广的标准流程,为学术期刊审稿模式的改进和发展提供建议。     

Discrepancies between explicit and implicit review: physician and nurse assessments of complications and quality

OBJECTIVE: To identify and characterize discrepancies between explicit and implicit medical record review of complications and quality of care. SETTING: Forty-two acute-care hospitals in California and Connecticut in 1994. STUDY DESIGN: In a retrospective chart review of 1,025 Medicare beneficiaries age >65, we compared explicit (nurse) and implicit (physician) reviews of complications and quality in individual cases. To understand discrepancies, we calculated the kappa statistic and examined physicians' comments. DATA COLLECTION: With Medicare discharge abstracts, we used the Complications Screening Program to identify and then select a stratified random sample of cases flagged for 1 of 15 surgical complications, 5 medical complications, and unflagged controls. Peer Review Organization nurses and physicians performed chart reviews. PRINCIPAL FINDINGS: Agreement about complications was fair (kappa = 0.36) among surgical and was moderate (kappa = 0.59) among medical cases. In discordant cases, physicians said that complications were insignificant, attributable to a related diagnosis, or present on admission. Agreement about quality was poor among surgical and medical cases (kappa = 0.00 and 0.13, respectively). In discordant cases, physicians said that quality problems were unavoidable, small lapses in otherwise satisfactory care, present on admission, or resulted in no adverse outcome. CONCLUSIONS: We identified many discrepancies between explicit and implicit review of complications and quality. Physician reviewers may not consider process problems that are ubiquitous in hospitals to represent substandard quality.     

The relationship between competence and performance: implications for assessing practice performance

OBJECTIVE: This paper aims to describe current views of the relationship between competence and performance and to delineate some of the implications of the distinctions between the two areas for the purpose of assessing doctors in practice. METHODS: During a 2-day closed session, the authors, using their wide experiences in this domain, defined the problem and the context, discussed the content and set up a new model. This was developed further by e-mail correspondence over a 6-month period. RESULTS: Competency-based assessments were defined as measures of what doctors do in testing situations, while performance-based assessments were defined as measures of what doctors do in practice. The distinction between competency-based and performance-based methods leads to a three-stage model for assessing doctors in practice. The first component of the model proposed is a screening test that would identify doctors at risk. Practitioners who 'pass' the screen would move on to a continuous quality improvement process aimed at raising the general level of performance. Practitioners deemed to be at risk would undergo a more detailed assessment process focused on rigorous testing, with poor performers targeted for remediation or removal from practice. CONCLUSION: We propose a new model, designated the Cambridge Model, which extends and refines Miller's pyramid. It inverts his pyramid, focuses exclusively on the top two tiers, and identifies performance as a product of competence, the influences of the individual (e.g. health, relationships), and the influences of the system (e.g. facilities, practice time). The model provides a basis for understanding and designing assessments of practice performance.     

Setting performance standards for medical practice: a theoretical framework

BACKGROUND: The assessment of performance in the real world of medical practice is now widely accepted as the goal of assessment at the postgraduate level. This is largely a validity issue, as it is recognised that tests of knowledge and in clinical simulations cannot on their own really measure how medical practitioners function in the broader health care system. However, the development of standards for performance-based assessment is not as well understood as in competency assessment, where simulations can more readily reflect narrower issues of knowledge and skills. This paper proposes a theoretical framework for the development of standards that reflect the more complex world in which experienced medical practitioners work. METHODS: The paper reflects the combined experiences of a group of education researchers and the results of literature searches that included identifying current health system data sources that might contribute information to the measurement of standards. CONCLUSION: Standards that reflect the complexity of medical practice may best be developed through an "expert systems" analysis of clinical conditions for which desired health care outcomes reflect the contribution of several health professionals within a complex, three-dimensional, contextual model. Examples of the model are provided, but further work is needed to test validity and measurability.