Factors influencing prescribing: an intern's perspective

OBJECTIVES: To examine the self-reported influences on intern prescribing practice. DESIGN: Qualitative interviews with a cross-sectional cohort. PARTICIPANTS AND SETTING: Ten interns practising in two urban teaching hospitals in New South Wales, Australia. RESULTS: The interns identified a number of factors that improve their confidence and perceived competence and allow them to extend their existing skills. These were approachable, available and up-to-date teachers (most often registrars and subspecialty nurses and pharmacists); timely, relevant and practical teaching (such as interactive bedside teaching); concise and widely accepted resources (such as prescribing pocket guides); and a constructive manner on the part of senior staff for dealing with prescribing errors. Interns also identified influences that are detrimental to confidence, conflict with their perceptions of appropriate prescribing and inhibit learning and skills acquisition. These were unapproachable, physically and mentally remote teachers (most often consultants); theoretical, inconsistent and irrelevant teaching (such as grand rounds or didactic education sessions); inconsistent and inaccessible resources; and a confrontational and accusatory way of dealing with prescribing errors. The added pressures of time, hospital hierarchies and the indirect influence of drug company promotion also impeded acquisition of good prescribing habits. CONCLUSIONS: At a critical time in skills development, interns encounter many forces that can potentially impact on prescribing practices in both positive and negative ways. Our data contribute to the understanding of the multifaceted learning environment of interns and may be useful in providing a foundation for prescriber education programmes tailored to the specific needs of junior doctors.     

Meeting the challenge of prescribing and administering medicines safely: structured teaching and assessment for final year medical students

OBJECTIVES: To promote safe prescribing and administration of medicines in the pre-registration house officer (PRHO) year through a programme of structured teaching and assessment for final year medical students. DESIGN: Forty final year medical students from two medical schools were randomly allocated either to participate in a pharmacist facilitated teaching session or to receive no additional teaching. Teaching comprised five practical exercises covering seven skills through which students rotated in small groups. One month later, a random sample of 16 taught and 16 non-taught students participated in a nine-station objective structured clinical examination (OSCE) to assess the impact of the teaching. SETTING: Manchester School of Medicine (MSM), and Kings College School of Medicine and Dentistry (KCSMD). PARTICIPANTS: Final year medical student volunteers. MAIN OUTCOME MEASURES: The need for teaching as indicated by student prior experience; questionnaire rating of student acceptability of teaching and assessment; self-rating of student confidence post-assessment, and student performance assessed by OSCE. RESULTS: The study demonstrated that the taught group achieved higher scores in eight OSCE stations. Four of these were statistically significant (P < or= 0.005). Taught students felt more confident performing the skills on five stations. From 0 to 47.5% students had prior experience of the skills taught. The post-teaching questionnaire evaluated exercises positively on several criteria, including provision of new information and relevance to future work. CONCLUSIONS: Structured teaching provided an effective and acceptable method of teaching the medicines management skills needed in the PRHO year. The structured approach complemented variable precourse clinical experience.     

Students' perceptions of relationships between some educational variables in the out-patient setting

BACKGROUND: Medical education uses the cognitive apprenticeship model of student learning extensively. Students rotate among different hospitals and out- patient clinics where they are exposed to a range of professionally relevant contexts. Here they learn to think and act in different domains under the supervision of experts. Previous research has shown that these learning situations involve little teaching. Students see a narrow range of patient problems and feedback is limited. The aim of this study is to investigate relationships among some educational variables in the out-patient clinic. METHOD: This paper provides a theoretical model that specifies the factors influencing the effectiveness of student rotations at out-patient clinics. The model makes distinctions between input variables, such as organizational quality, number of students contemporaneously involved and available space, and process variables, such as patient mix and supervision, and the output variable of the effectiveness of rotations in out-patient clinics. RESULTS: The model was tested against empirical data from evaluative surveys and showed a reasonable fit. The model offers suggestions for improving the learning environment of clinical rotations. DISCUSSION: The strength of this study lies in its process evaluation perspective which investigates interactions between intervening variables rather than the influence of particular variables in isolation from other variables.     

The use of portfolios for assessment of the competence and performance of doctors in practice

BACKGROUND: The use of portfolios can potentially provide flexibility in the summative assessment of doctors in practice. An assessment system should reflect and reinforce the active and planned professional development goals of individual doctors. This paper discusses some of the issues involved in developing such a system. RESULTS: To provide a complete picture of an individual doctor's practice, we suggest that a portfolio should encompass: (1) evidence covering all three domains of patient care, personal development and context management; (2) evidence that the person continuously undertakes critical assessment of their own performance, identifies and prioritises areas requiring enhanced performance and takes action to improve them as appropriate; (3) evidence that has been generated by assessments that are acceptably reliable, and (4) evidence which, taken in its entirety, is sufficient, valid, current and authentic. We include a suggested outline of the components of such a portfolio and suggest some criteria to determine the effectiveness of learning cycles. Portfolio reliability and validity requires sufficient evidence on which to base a judgement combined with reliable processes. CONCLUSION: Carefully specified portfolios can contribute to a system that ensures all doctors take an active part in identifying and meeting their own learning needs. Such a system, if properly implemented, would have a greatly beneficial impact on continuous quality improvement for the profession in general.     

Assessing the performance of doctors in teams and systems

INTRODUCTION: Increasing attention is being directed towards finding ways of assessing how well doctors perform in clinical practice. Current approaches rely on strategies directed at individuals only, but, in real life, doctors' work is characterised by multiple complex professional interactions. These interactions involve different kinds of teams and are embedded within the overall context and systems of care. In addition to individual factors, therefore, we propose that the performance of doctors in health care teams and systems will also impact on the overall quality of patient care. Assessing these dimensions, however, poses a number of challenges. STRATEGIES: Taking a profile of a National Health Service, UK surgeon as an example, the team structures to which he or she may relate are illustrated. These include formal teams such as those found in the operating theatre, and those formed through various professional and collegial partnerships. The authors then propose a model for assessing doctors' performances in teams and systems, which incorporates the educational principles of continuous feedback to enhance future performance. DISCUSSION: To implement the proposed model, a wide range of professional, educational and regulatory bodies must collaborate. This raises a number of important implications for the future roles and relationships of these bodies, which are discussed. A strong and constructive partnership will be essential if the full potential of a more inclusive and representative assessment approach is to be realised.     

The first batch of graduates of a new medical curriculum in Asia: how their teachers see them

OBJECTIVE: To examine teachers' views of the first batch of graduates of a revised medical curriculum in Asia. METHODS: A cross-sectional study using a structured questionnaire was carried out to obtain the views of all the clinical teachers involved in teaching final year students of the old curriculum in 2000-01 and the new curriculum in 2001-02 at the University of Hong Kong, which commenced curricular reform in 1997. RESULTS: Nearly 62% of respondents felt that better graduates were being produced with the new curriculum. The majority of them rated the new curriculum students better in nearly all the major goals of the new curriculum, such as self-directed learning initiative, problem solving skills, interpersonal skills and clinical performance in patient care. However, the core knowledge of the new curriculum students was of concern to some teachers. CONCLUSION: This study focused on the first complete cycle of a revised medical curriculum in Asia. Teachers' views of the new curriculum students were highly positive and they felt that better graduates were being produced.     

The effect of outcome information on doctors' evaluations of their own diagnostic decisions

BACKGROUND: Outcome bias is a psychological tendency to use outcome information in order to evaluate the quality of a previous decision, where decisions with bad outcomes are undervalued and decisions with good outcomes are overvalued. In some circumstances, being affected by outcome when evaluating a diagnostic decision might have important educational consequences: bad diagnostic decisions resulting, by chance, in good outcomes might be repeated in the future, whilst good diagnostic decisions with bad outcomes might be abandoned. However, no previous studies have tested whether outcome bias affects doctors' evaluations of their own diagnostic decisions. METHODS: A total of 91 practising Italian doctors (69 general practitioners, 22 neurologists) chose a diagnosis for a neurological case and then evaluated it after receiving outcome information (positive outcome, adverse outcome or no outcome). RESULTS: Outcome information affected the doctors in many ways: the quality of the decision was overrated following good outcomes, and underrated following bad outcomes; the difficulty of making the diagnosis was underrated after good outcomes and overrated after bad outcomes, and the memory of weightings assigned to signs and symptoms consistent with the diagnosis was overrated after good outcomes and underrated after bad outcomes (and vice versa for the weighting of evidences inconsistent with the diagnosis). CONCLUSIONS: Outcome bias in medical decisions is discussed in terms of its possible adverse effects on doctors' learning and knowledge revision processes: although sensitivity to outcomes may be desirable in some circumstances, in other circumstances it can cause either the fixing of incorrect knowledge or the improper revision of correct knowledge.     

Clinical clerkships: students can structure their own learning

In clinical clerkships, student learning is often unstructured and diverse. Even when curriculum objectives are explicit, they are seldom used by students to guide their learning. Student-determined learning objectives may help students to structure their learning. This study aimed to assess whether students can use this method to meet widely differing learning needs within the general practice clerkship and considered the relationship of student objectives to overall curriculum objectives. Students allocated a 5-week clerkship in general practice from April 2000 to April 2001 were required to set and meet personal learning objectives within a supportive learning environment. They recorded the strategies they used, how well objectives were met and which curriculum objectives they met. Objectives were categorised using a hierarchical system. A total of 143 (85%) students recorded 1549 objectives. Four categories accounted for 1092 (70%) of objectives: Consultation Skills; Study of Diseases; Practical Procedures, and Therapeutics. Otherwise the range of objectives set was wide, encompassing all curriculum objectives. A total of 1043 objectives were rated as satisfactorily or highly satisfactorily met and 185 (12%) were not met at all. Strategies that students used to meet objectives included clinical experience with doctors but also with practice-based nurses. Students were able to use student-determined learning objectives to meet a diverse range of learning needs within the general practice clerkship. The objectives set reflected the breadth of curriculum objectives. This method allows students to address gaps in their knowledge in a clerkship where teaching is largely based on opportunistic contact.     

Assessment of patient management skills and clinical skills of practising doctors using computer-based case simulations and standardised patients

CONTEXT: Standardised assessments of practising doctors are receiving growing support, but theoretical and logistical issues pose serious obstacles. OBJECTIVES: To obtain reference performance levels from experienced doctors on computer-based case simulation (CCS) and standardised patient-based (SP) methods, and to evaluate the utility of these methods in diagnostic assessment. SETTING AND PARTICIPANTS: The study was carried out at a military tertiary care facility and involved 54 residents and credentialed staff from the emergency medicine, general surgery and internal medicine departments. MAIN OUTCOME MEASURES: Doctors completed 8 CCS and 8 SP cases targeted at doctors entering the profession. Standardised patient performances were compared to archived Year 4 medical student data. RESULTS: While staff doctors and residents performed well on both CCS and SP cases, a wide range of scores was exhibited on all cases. There were no significant differences between the scores of participants from differing specialties or of varying experience. Among participants who completed both CCS and SP testing (n = 44), a moderate positive correlation between CCS and SP checklist scores was observed. There was a negative correlation between doctor experience and SP checklist scores. Whereas the time students spent with SPs varied little with clinical task, doctors appeared to spend more time on communication/counselling cases than on cases involving acute/chronic medical problems. CONCLUSION: Computer-based case simulations and standardised patient-based assessments may be useful as part of a multimodal programme to evaluate practising doctors. Additional study is needed on SP standard setting and scoring methods. Establishing empirical likelihoods for a range of performances on assessments of this character should receive priority.     

Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence

BACKGROUND: The intern year is a key time for the acquisition of clinical skills, both procedural and cognitive. We have previously described self-reported confidence and experience for a number of clinical skills, finding high levels of confidence among Australian junior doctors. This has never been correlated with an objective measure of competence. AIMS AND HYPOTHESIS: We aimed to determine the relationship between self-reported confidence and observed competence for a number of routine, procedural clinical skills. METHODS: A group of 30 junior medical officers in their first postgraduate year (PGY1) was studied. All subjects completed a questionnaire concerning their confidence and experience in the performance of clinical skills. A competency-based assessment instrument concerning 7 common, practical, clinical skills was developed, piloted and refined. All 30 PGY1s then completed an assessment using this instrument. Comparisons were then made between the PGY1s' self-reported levels of confidence and tutors' assessments of their competence. RESULTS: A broad range of competence levels was revealed by the clinical skills assessments. There was no correlation between the PGY1s' self-ratings of confidence and their measured competencies. CONCLUSIONS: Junior medical officers in PGY1 demonstrate a broad range of competence levels for several common, practical, clinical skills, with some performing at an inadequate level. There is no relationship between their self-reported level of confidence and their formally assessed performance. This observation raises important caveats about the use of self-assessment in this group.     

Selecting performance assessment methods for experienced physicians

BACKGROUND: While much is now known about how to assess the competence of medical practitioners in a controlled environment, less is known about how to measure the performance in practice of experienced doctors working in their own environments. The performance of doctors depends increasingly on how well they function in teams and how well the health care system around them functions. METHODS: This paper reflects the combined experiences of a group of experienced education researchers and the results of literature searches on performance assessment methods. CONCLUSION: Measurement of competence is different to measurement of performance. Components of performance could be re-conceptualised within a different domain structure. Assessment methods may be of a different utility to that in competence assessment and, indeed, of different utility according to the purpose of the assessment. An exploration of the utility of potential performance assessment methods suggests significant gaps that indicate priority areas for research and development.     

Quality of clinical supervision as perceived by specialist registrars in a university and district teaching hospital

BACKGROUND: Clinical supervision promotes the professional development of specialist registrars (SpRs). Our objective was to investigate and compare the perceived quality of supervision (PQS) in a university teaching hospital (UTH) and a district teaching hospital (DTH) and to identify aspects of supervision that could be improved. METHODS: The Cleveland Clinic's teaching effectiveness instrument (CCTEI) was used to measure the quality of supervision of attending doctors. Fifteen items reflecting good teaching ability were rated on a 5-point Likert-scale (1-5 = poor-excellent). RESULTS: SpRs rated 47 attending doctors using the CCTEI. A total of 416 ratings were obtained. Overall, the mean PQS was 3.85 (SD = 0.29) in the DTH and 3.56 (SD = 0.44) in the UTH (P = 0.02). A significant difference in PQS was found in 6 items. The supervisors in the DTH scored better on all these items. The best predictor of PQS was the item 'Organises time to allow for both teaching and care giving'. CONCLUSIONS: Overall, PQS was better in the DTH compared to the UTH. In both settings, generating a good learning environment and respecting the autonomy of the SpRs scored favourably. Supervisory roles focusing on improving cost-effective practice and communicative skills need more emphasis.     

Teaching doctors how to diagnose paroxysmal events: a comparison of two educational methods

Objectives  Two educational methods, facilitated case discussion and a computerised tutorial, were compared for teaching about childhood epilepsy. We used a comprehensive and clinically relevant assessment method to evaluate the hypothesis that a computerised tutorial more effectively increases knowledge acquisition than a facilitated case discussion.
Methods  Paediatric trainees ( n  = 66) were arbitrarily allocated to facilitated case discussion or computerised tutorial. The analysis of paroxysmal events was taught by the same teacher, using a standardised protocol and principles of active learning. Outcome measures included knowledge acquisition, clinical confidence and usefulness pre- and post-teaching, and at 3 months follow-up.
Results  Computerised tutorial participants scored significantly higher on knowledge acquisition post-teaching. There was gain in clinical confidence in both modalities post-teaching which did not differ between the groups. Confidence and knowledge were not related post-teaching. Both groups found the teaching relevant to clinical practice. However, facilitated case discussion participants rated the session as more enjoyable, and more useful in reinforcing and acquiring knowledge, and felt more motivated for further learning. At 3 months follow-up, participants in both modalities showed significant increases in knowledge acquisition, with no difference between modalities.
Conclusions  The computerised tutorial more effectively imparted knowledge immediately post-teaching. However, facilitated case discussion is the preferred modality in terms of participant enjoyment and perceived usefulness.     

Validation of a checklist to assess ward round performance in internal medicine

BACKGROUND: Ward rounds are an essential responsibility for doctors in hospital settings. Tools for guiding and assessing trainees' performance of ward rounds are needed. A checklist was developed for that purpose for use with trainees in internal medicine. OBJECTIVE: To assess the content and construct validity of the task-specific checklist. METHODS: To determine content validity, a questionnaire was mailed to 295 internists. They were requested to give their opinion on the relevance of each item included on the checklist and to indicate the comprehensiveness of the checklist. To determine construct validity, an observer assessed 4 groups of doctors during performance of a complete ward round (n = 32). The nurse who accompanied the doctor on rounds made a global assessment of the performance. RESULTS: The response rate to the questionnaire was 80.7%. The respondents found that all 10 items on the checklist were relevant to ward round performance and that the item collection was comprehensive. Checklist mean-item scores differed between levels of expertise: junior house officers 1.4 (1.0-1.9); senior house officers 2.0 (1.5-2.9); specialist trainees 2.5 (1.8-2.8), and specialists 2.7 (2.3-3.5); median (range) (P < 0.001). A significant correlation was found between global observer scores and nurse scores (r = 0.56, P < 0.001). CONCLUSION: The checklist, developed for assessing trainees' performance of ward rounds in internal medicine, showed high content validity. Construct validity was supported by the higher scores of experienced doctors compared to those with less experience and the significant correlation between the observer's and nurses' global scores. The developed checklist should be valuable in guiding and assessing trainees on ward round performance.     

When enough is enough: a conceptual basis for fair and defensible practice performance assessment

INTRODUCTION: An essential element of practice performance assessment involves combining the results of various procedures in order to see the whole picture. This must be derived from both objective and subjective assessment, as well as a combination of quantitative and qualitative assessment procedures. Because of the severe consequences an assessment of practice performance may have, it is essential that the procedure is both defensible to the stakeholders and fair in that it distinguishes well between good performers and underperformers. LESSONS FROM COMPETENCE ASSESSMENT: Large samples of behaviour are always necessary because of the domain specificity of competence and performance. The test content is considerably more important in determining which competency is being measured than the test format, and it is important to recognise that the process of problem-solving process is more idiosyncratic than its outcome. It is advisable to add some structure to the assessment but to refrain from over-structuring, as this tends to trivialise the measurement. IMPLICATIONS FOR PRACTICE PERFORMANCE ASSESSMENT: A practice performance assessment should use multiple instruments. The reproducibility of subjective parts should not be increased by over-structuring, but by sampling through sources of bias. As many sources of bias may exist, sampling through all of them may not prove feasible. Therefore, a more project-orientated approach is suggested using a range of instruments. At various timepoints during any assessment with a particular instrument, questions should be raised as to whether the sampling is sufficient with respect to the quantity and quality of the observations, and whether the totality of assessments across instruments is sufficient to see 'the whole picture'. This policy is embedded within a larger organisational and health care context.     

What can clinical teachers learn from Harry Potter and the Philosopher's Stone?

Many clinical teachers acquire a working knowledge of the principles of teaching and learning through observation, by adopting positive and rejecting negative examples of clinical instruction. Well selected vignettes of teaching behaviours taken from contemporary film and literature may provide rich substrate by which to engage clinical teachers in discourse about instructional technique. This paper draws on J K Rowling's novel and its companion film, Harry Potter and the Philosopher's Stone, and critically analyses the teaching styles of the staff at Hogwarts School of Wizardry and Witchcraft in the context of contemporary generic and medical education literature. Specifically, it argues that effective teachers demonstrate not only an in-depth knowledge of their discipline but possess a keen appreciation of the cognitive changes that occur in their students during the learning process. They are, furthermore, proficient in core instructional skills such as small group facilitation, feedback and questioning. Most importantly, effective teachers model appropriate attitudes in their professional setting and possess highly developed personal qualities such as creativity, flexibility and enthusiasm.     

Procedures for establishing defensible programmes for assessing practice performance

The assessment of the performance of doctors in practice is becoming more widely accepted. While there are many potential purposes for such assessments, sometimes the consequences of the assessments will be 'high stakes'. In these circumstances, any of the many elements of the assessment programme may potentially be challenged. These assessment programmes therefore need to be robust, fair and defensible, taken from the perspectives of consumer, assessee and assessor. In order to inform the design of defensible programmes for assessing practice performance, a group of education researchers at the 10th Cambridge Conference adopted a project management approach to designing practice performance assessment programmes. This paper describes issues to consider in the articulation of the purposes and outcomes of the assessment, planning the programme, the administrative processes involved, including communication and preparation of assessees. Examples of key questions to be answered are provided, but further work is needed to test validity.     

Monkey see, monkey do: a critique of the competency model in graduate medical education

BACKGROUND: Graduate medical education in the UK is in danger of being subsumed in a minimalist discourse of competency. ARGUMENT: While accepting that competence in a doctor is a sine qua non, the author criticises the construction of a graduate and specialist medical education based solely upon a competency model. Many competency models follow the concepts of either academic competence or operational competence, both of which have lately been subject to criticism. CONCLUSION: The author discusses the need for replacing such criterion-referenced models in favour of a model that engages the higher order competence, performance and understanding which represent professional practice at its best.     

Introducing resident doctors to complexity in ambulatory medicine

Objectives Medical education instructional methods typically imply one ‘best’ management approach. Our objectives were to develop and evaluate an intervention to enhance residents’ appreciation for the diversity of acceptable approaches when managing complex patients. Methods A total of 124 internal medicine residents enrolled in a randomised, crossover trial. Residents completed four web‐based modules in ambulatory medicine during continuity clinic. For each module we developed three ‘complex cases’. Cases were intended to be complex (numerous variables, including psychosocial and economic barriers) and to suggest multiple acceptable management strategies. Several experienced faculty members described how they would manage each case. Residents reviewed each case, answered the same questions, and compared their responses with expert responses. Participants were randomly assigned to complete two modules with, and two modules without complex cases. Results A total of 76 residents completed 279 complex cases. Residents agreed that complex cases enhanced their appreciation for the diversity of ‘correct’ options (mean ± standard error of the mean 4.6 ± 0.2 [1 = strongly disagree, 6 = strongly agree]; P < 0.001). Mean preference score was neutral (3.4 ± 0.2 [1 = strongly favour no cases, 6 = strongly favour cases]; P = 0.72). Knowledge post‐test scores were similar between modules with (76.0 ± 0.9) and without (77.8 ± 0.9) complex cases (95% confidence interval for difference ? 4.0 to 0.3; P = 0.09). Resident comments suggested that lack of time and cognitive overload impeded learning. Conclusions Residents felt complex cases made a valuable contribution to their learning, although preference was neutral and knowledge scores were not affected. Methods to facilitate trainee comfort in managing medically complex patients should be further explored.     

The relationship between competence and performance: implications for assessing practice performance

OBJECTIVE: This paper aims to describe current views of the relationship between competence and performance and to delineate some of the implications of the distinctions between the two areas for the purpose of assessing doctors in practice. METHODS: During a 2-day closed session, the authors, using their wide experiences in this domain, defined the problem and the context, discussed the content and set up a new model. This was developed further by e-mail correspondence over a 6-month period. RESULTS: Competency-based assessments were defined as measures of what doctors do in testing situations, while performance-based assessments were defined as measures of what doctors do in practice. The distinction between competency-based and performance-based methods leads to a three-stage model for assessing doctors in practice. The first component of the model proposed is a screening test that would identify doctors at risk. Practitioners who 'pass' the screen would move on to a continuous quality improvement process aimed at raising the general level of performance. Practitioners deemed to be at risk would undergo a more detailed assessment process focused on rigorous testing, with poor performers targeted for remediation or removal from practice. CONCLUSION: We propose a new model, designated the Cambridge Model, which extends and refines Miller's pyramid. It inverts his pyramid, focuses exclusively on the top two tiers, and identifies performance as a product of competence, the influences of the individual (e.g. health, relationships), and the influences of the system (e.g. facilities, practice time). The model provides a basis for understanding and designing assessments of practice performance.