Quality assessment in general practice trainers

INTRODUCTION: General practice trainers hold a key position in general practice training, especially through their provision of a role model. Their own competence in general practice care is important in this regard. The purpose of the study was to evaluate whether a quality assessment programme could identify the strengths and weaknesses of GP trainers in four main domains of general practice care. METHODS: The quality assessment programme comprised validated tests on four domains of general practice: general medical knowledge, knowledge of medical-technical skills, consultation skills and practice management. The criterion for the identification of relative strengths and weaknesses of GP trainers was a variation in the scores of trainers indicating higher and lower scores (strengths and weaknesses) within each domain. RESULTS: GP trainers (n=105) were invited to participate in the study and 90% (n=94) did so. The variation in scores allowed the indication of strengths and weaknesses. Main strengths were: general medical knowledge of the digestive system; knowledge of medical skills relating to the skin; consultation skills concerning empathy; practice management with regard to accessibility. Main weaknesses were: general medical knowledge of the neurological system; knowledge of the medical/technical skills relating to the endocrine metabolic and nutritional system; consultation skills regarding shared decision making; practice management involving cooperation with staff and other care providers. DISCUSSION: This first systematic evaluation of GP trainers identified their strengths and weaknesses. The weaknesses identified will be used in the improvement process as topics for collective improvement in the GP trainers' general curriculum and in individual learning plans.     

How can we prepare medical students for theatre‐based learning?

CONTEXT: The quality of medical undergraduate operating theatre-based teaching is variable. Preparation prior to attending theatre may support student learning. Identifying and agreeing key skills, competences and objectives for theatre-based teaching may contribute to this process of preparation. METHODS: We carried out a cross-sectional survey of consultant surgeons and students using a forced choice questionnaire containing 16 skills and competences classified as 'essential', 'desirable' or 'not appropriate', and a choice of 6 different teaching methods, scored for perceived effectiveness on a 5-point Likert scale. Questionnaire content was based on the findings from an earlier qualitative study. RESULTS: Comparative data analyses (Mann- Whitney and Kruskal-Wallis tests) were carried out using SPSS Version 14. A total of 42 consultant surgeons and 46 students completed the questionnaire (46% and 100% response rates, respectively). Knowledge of standard theatre etiquette and protocols, ability to scrub up adequately, ability to adhere to sterile procedures, awareness of risks to self, staff and patients, and appreciation of the need for careful peri-operative monitoring were considered 'essential' by the majority. Student and consultant responses differed significantly on 5 items, with students generally considering more practical skills and competences to be essential. Differences between students on medical and surgical attachments were also identified. CONCLUSIONS: Consultant surgeons and medical students agree on many aspects of the important learning points for theatre-based teaching. Compared with their teachers, students, particularly those on attachment to surgical specialties, are more ambitious - perhaps overly so - in the level of practical skills and risk awareness they expect to gain in theatre.     

Increasing the evidence base for selection for undergraduate medicine: four case studies investigating process and interim outcomes

INTRODUCTION: Prior to 1997, the University of Adelaide, Australia selected medical students according to matriculation marks alone. After extensive consultation with relevant stakeholders and examination of empirical evidence, the selection process is now based on a national written examination of reasoning and interaction skills, a structured oral assessment and a threshold matriculation score. This paper presents a series of 4 case studies examining the process related to the procedure and early interim outcomes, with the aim of adding to the evidence base for methods of medical student selection. METHODS: Data were collected from a range of sources between 1996 and 2001, including the University's central administration system, the Faculty of Health Sciences, and purpose-designed oral assessment forms and self-report student questionnaires. RESULTS: The oral assessment process has shown itself to be characterised by a high level of interassessor reliability. Equity of access has been addressed and the number of schools represented in the student intake has increased from between 10 and 15 in the final years of the old selection process to over 30 statewide under the new process. There has been a corresponding increase in the proportion of students from rural backgrounds (from an original 4-12% to 20-22%). DISCUSSION: These investigations add to the evidence base for medical school selection in that they demonstrate that an oral assessment process can be reliable and lead to early positive results in relation to student outcomes and access.     

What do students actually do on an internal medicine clerkship? A log diary study

BACKGROUND: There are limited data on the amount of time students spend on teaching and learning while on internal medicine clerkships, and existing data suggest a wide international variation. Community-based teaching of internal medicine is now widespread; but its strengths and weaknesses compared to traditional hospital based teaching are still unclear. AIM: To determine the proportion of time students spend on different activities on an internal medicine clerkship, and to determine whether this differs in general practice and in hospital. In addition we aimed to determine students' views on the educational value and enjoyment of various activities. METHODS: Prospective completion of log diaries recording student activities. Each student was asked to complete the diary for two separate weeks of their internal medicine clerkship: one week of general practice-based teaching and one week of hospital-based teaching. RESULTS: The response rate was 68% (88/130). Students spent approximately 5.5 h per day on teaching and learning activities in both environments, with more time (50 min vs. 30 min, P = 0.007) on unsupervised interaction with patients in hospital than in general practice, and more time (53 min vs. 21 min, P < 0.001) undergoingassessment in general practice than in hospital. Standard deviations were wide, demonstrating the heterogeneous nature of the data. Students perceived supervised interaction with patients and teaching by doctors as the most educational activities in both environments, but found it even more educationally valuable and enjoyable in general practice than in hospital (mean score for educational value: 4.27 in general practice, 3.88 in hospital, P = 0.048; mean score for enjoyment 4.13 in general practice, 3.66 in hospital, P = 0.03). CONCLUSIONS: Students greatly value interactions with patients, perceiving these as both educational and enjoyable. Curriculum planners must continue to place patient-based learning at the centre of undergraduate medical education. The heterogeneity of the data suggests that individual students have very different experiences, despite apparently similar timetables.     

Early identification of 'at-risk' students by the parents of paediatric patients

INTRODUCTION: Assessment of medical student clinical skills is best carried out using multiple assessment methods. A programme was developed to obtain parent evaluations of medical student paediatric interview skills for feedback and to identify students at risk of poor performance in summative assessments. METHOD: A total of 130 parent evaluations were obtained for 67 students (parent participation 72%, student participation 58%). Parents completed a 13-item questionnaire [Interpersonal Skills Rating Scale (IPS) maximum score 91, higher scores = higher student skill level]. Students received their individual parent scores and de-identified class mean scores as feedback, and participants were surveyed regarding the programme. Parent evaluation scores were compared with student performance in formative and summative faculty assessments of clinical interview skills. RESULTS: Parents supported the programme and participating students valued parent feedback. Students with a parent score that was less than 1 standard deviation (SD) below the class mean (low IPS score students) obtained lower faculty summative assessment scores than did other students (mean +/- SD, 59% +/- 5 versus 64% +/- 7; P < 0.05). Obtaining 1 low IPS score was associated with a subsequent faculty summative assessment score below the class mean (sensitivity 0.38, specificity 0.88). Parent evaluations combined with faculty formative assessments identified 50% of students who subsequently performed below the class mean in summative assessments. CONCLUSIONS: Parent evaluations provided useful feedback to students and identified 1 group of students at increased risk of weaker performance in summative assessments. They could be combined with other methods of formative assessment to enhance screening procedures for clinically weak students.     

Videotape recording in the assessment of the history-taking skills of medical students

The potential of videotape recording in the assessment of student history-taking skills has been explored in a third year general practice undergraduate course. By concentrating on priority areas in history-taking it has been possible to develop a profile of the strengths and weaknesses of individual students and of the group as a whole which should be of value to course organizers.     

Meeting the challenge of prescribing and administering medicines safely: structured teaching and assessment for final year medical students

OBJECTIVES: To promote safe prescribing and administration of medicines in the pre-registration house officer (PRHO) year through a programme of structured teaching and assessment for final year medical students. DESIGN: Forty final year medical students from two medical schools were randomly allocated either to participate in a pharmacist facilitated teaching session or to receive no additional teaching. Teaching comprised five practical exercises covering seven skills through which students rotated in small groups. One month later, a random sample of 16 taught and 16 non-taught students participated in a nine-station objective structured clinical examination (OSCE) to assess the impact of the teaching. SETTING: Manchester School of Medicine (MSM), and Kings College School of Medicine and Dentistry (KCSMD). PARTICIPANTS: Final year medical student volunteers. MAIN OUTCOME MEASURES: The need for teaching as indicated by student prior experience; questionnaire rating of student acceptability of teaching and assessment; self-rating of student confidence post-assessment, and student performance assessed by OSCE. RESULTS: The study demonstrated that the taught group achieved higher scores in eight OSCE stations. Four of these were statistically significant (P < or= 0.005). Taught students felt more confident performing the skills on five stations. From 0 to 47.5% students had prior experience of the skills taught. The post-teaching questionnaire evaluated exercises positively on several criteria, including provision of new information and relevance to future work. CONCLUSIONS: Structured teaching provided an effective and acceptable method of teaching the medicines management skills needed in the PRHO year. The structured approach complemented variable precourse clinical experience.     

Peer assessment of professional competence

BACKGROUND: Current assessment formats for medical students reliably test core knowledge and basic skills. Methods for assessing other important domains of competence, such as interpersonal skills, humanism and teamwork skills, are less well developed. This study describes the development, implementation and results of peer assessment as a measure of professional competence of medical students to be used for formative purposes. METHODS: Year 2 medical students assessed the professional competence of their peers using an online assessment instrument. Fifteen randomly selected classmates were assigned to assess each student. The responses were analysed to determine the reliability and validity of the scores and to explore relationships between peer assessments and other assessment measures. RESULTS: Factor analyses suggest a 2-dimensional conceptualisation of professional competence: 1 factor represents Work Habits, such as preparedness and initiative, and the other factor represents Interpersonal Habits, including respect and trustworthiness. The Work Habits factor had moderate, yet statistically significant correlations ranging from 0.21 to 0.53 with all other performance measures that were part of a comprehensive assessment of professional competence. Approximately 6 peer raters were needed to achieve a generalisability coefficient of 0.70. CONCLUSIONS: Our findings suggest that it is possible to introduce peer assessment for formative purposes in an undergraduate medical school programme that provides multiple opportunities to interact with and observe peers.     

Comparing health care delivery systems – initiating a student exchange project between Europe and the United States

BACKGROUND: Cross-cultural contact among different health care systems can provide a framework for identifying the strengths and weaknesses of one's own healthcare system. However, such contact has rarely had much impact upon medical education curricula. Despite intense debate on reforming the healthcare delivery systems (HCDS) in Europe and the United States, there is very little formal representation of this interdisciplinary field in our educational programs. DESCRIPTION: To address this problem, a medical student exchange program was conducted in which students developed case studies that produced comparative analyses of HCDS in Germany, Sweden, Denmark and the United States. Each case is intended to highlight critical differences among the systems. EVALUATION: Students and their faculty preceptors completed pre- and post-exchange questionnaires to assess perceived knowledge of the HCDS and the adequacy of time devoted to it in their curricula. Both perceived that too little attention was devoted to this content in their programs. Following the exchange, students described clear increases in perceived knowledge. DISCUSSION: Our common interest in curriculum reform was key to implementing the exchange. The written cases generated by the students are being developed as course material in some of the schools and a conference is planned to disseminate the cases and the implementation strategies for their inclusion in medical curricula.     

Formative assessment of the consultation performance of medical students in the setting of general practice using a modified version of the leicester assessment package

OBJECTIVE: To evaluate the use of a modified version of the Leicester Assessment Package (LAP) in the formative assessment of the consultation performance of medical students with particular reference to validity, inter-assessor reliability, acceptability, feasibility and educational impact. DESIGN: 180 third and fourth year Leicester medical students were directly observed consulting with six general practice patients and independently assessed by a pair of assessors. A total of 70 practice and 16 departmental assessors took part. Performance scores were subjected to generalizability analysis and students' views of the assessment were gathered by questionnaire. RESULTS: Four of the five categories of consultation performance (Interviewing and history taking, Patient management, Problem solving and Behaviour and relationship with patients) were assessed in over 99% of consultations and Physical examination was assessed in 94%. Seventy-six percent of assessors reported that the case mix was 'satisfactory' and 20% that it was 'borderline'; 85% of students believed it to have been satisfactory. Generalizability analysis indicates that two independent assessors assessing the performance of students across six consultations would achieve a reliability of 0.94 in making pass or fail decisions. Ninety-eight percent of students perceived that their particular strengths and weaknesses were correctly identified, 99% that they were given specific advice on how to improve their performance and 98% believed that the feedback they had received would have long-term benefit. CONCLUSIONS: The modified version of the LAP is valid, reliable and feasible in formative assessment of the consultation performance of medical students. Furthermore, almost all students found the process fair and believed it was likely to lead to improvements in their consultation performance. This approach may also be applicable to regulatory assessment as it accurately identifies students at the pass/fail margin.     

Approaches to learning and studying in medical students: validation of a revised inventory and its relation to student characteristics and performance

INTRODUCTION: Inventories to quantify approaches to studying try to determine how students approach academic tasks. Medical curricula usually aim to promote a deep approach to studying, which is associated with academic success and which may predict desirable traits postqualification. AIMS: This study aimed to validate a revised Approaches to Learning and Studying Inventory (ALSI) in medical students and to explore its relation to student characteristics and performance. METHODS: Confirmatory factor analysis was used to validate the reported constructs in a sample of 128 Year 1 medical students. Models were developed to investigate the effect of age, graduate status and gender, and the relationships between approaches to studying and assessment outcomes. RESULTS: The ALSI performed as anticipated in this population, thus validating its use in our sample, but a 4-factor solution had a better fit than the reported 5-factor one. Medical students scored highly on deep approach compared with other students in higher education. Graduate status and gender had significant effects on approach to studying and a deep approach was associated with higher academic scores. CONCLUSIONS: The ALSI is valid for use in medical students and can uncover interesting relationships between approaches to studying and student characteristics. In addition, the ALSI has potential as a tool to predict student success, both academically and beyond qualification.     

Perceptions of dissection by students in one medical school: beyond learning about anatomy. A qualitative study

INTRODUCTION: The practice of dissection, as part of undergraduate medical education, has recently resurfaced in the public eye. This paper focuses on a number of important learning outcomes that were reported by Year 1-5 medical students in a British medical school, during the dissection sessions in the first 2 years of their training, as part of a wider qualitative research project into undergraduate medical education. METHODS: A group of 29 students was selected by quota sampling, using the whole student population of the medical school as the sampling frame. Qualitative data were collected by 1 : 1 interviews with students and from formal non-participatory observations of dissection sessions. RESULTS: Apart from learning to cope with the overt 'emotional confrontation' with the cadavers which assists anatomical learning, 7 additional covert learning outcomes were identified by the students: teamwork, respect for the body, familiarisation of the body, application of practical skills, integration of theory and practice, preparation for clinical work, and appreciation of the status of dissection within the history of medicine. DISCUSSION: A number of medical schools have either removed the practical, hands-on aspect of dissection in the medical undergraduate curriculum or are seriously considering such a measure, on financial and/or human resource grounds. This study highlights the fact that dissection can impart anatomical knowledge as well as offer other relevant, positive learning opportunities to enhance the skills and attitudes of future doctors.     

Is it me or is it them? Factors that influence the passing of underperforming students

Context Research has found that clinical assessments do not always accurately reflect medical student performance. Barriers to failing underperformance in students have been identified in other vocational settings. Is ‘failure to fail’ an issue for medical educators in the UK, and, if so, what are its determinants? Methods We carried out a qualitative focus group study exploring the views of medical educators (general practitioners, hospital doctors and non‐clinical tutors) from two different UK medical schools. To make sense of a potential multitude of factors impacting on failure to fail, we selected the integrative model of behavioural prediction to underpin our data collection and analysis. Results Ten focus groups were carried out with 70 participants. Using both theory and data‐driven framework analysis, we identified six main themes relevant to the integrative model of behavioural prediction. These are: tutor attitudes towards an individual student; tutor attitudes towards failing a student; normative beliefs and motivation to comply; efficacy beliefs (self‐efficacy); skills and knowledge, and environmental constraints. Discussion Many different factors impact on medical educators’ failure to report underperformance in students. There are conflicts between these factors and the need to report competence accurately (i.e. duty to protect the public). Although some of the barriers identified are similar to those found in previous studies, using a theory‐based approach added value in that it facilitated a richer exploration of failure to fail. Insights offered in this study will be used to plan a questionnaire study and subsequent intervention to support medical educators in accurately reporting underperformance in students.     

Medical students' personality characteristics and academic performance: a five-factor model perspective

OBJECTIVES: This study investigates: (1) which personality traits are typical of medical students as compared to other students, and (2) which personality traits predict medical student performance in pre-clinical years. DESIGN: This paper reports a cross-sectional inventory study of students in nine academic majors and a prospective longitudinal study of one cohort of medical students assessed by inventory during their first preclinical year and by university examination at the end of each pre-clinical year. SUBJECTS AND METHODS: In 1997, a combined total of 785 students entered medical studies courses in five Flemish universities. Of these, 631 (80.4%) completed the NEO-PI-R (i.e. a measure of the Five-Factor Model of Personality). This was also completed by 914 Year 1 students of seven other academic majors at Ghent University. Year end scores for medical students were obtained for 607 students in Year 1, for 413 in Year 2, and for 341 in Year 3. RESULTS: Medical studies falls into the group of majors where students score highest on extraversion and agreeableness. Conscientiousness (i.e. self-achievement and self-discipline) significantly predicts final scores in each pre-clinical year. Medical students who score low on conscientiousness and high on gregariousness and excitement-seeking are significantly less likely to sit examinations successfully. CONCLUSIONS: The higher scores for extraversion and agreeableness, two dimensions defining the interpersonal dynamic, may be beneficial for doctors' collaboration and communication skills in future professional practice. Because conscientiousness affects examination results and can be reliably assessed at the start of a medical study career, personality assessment may be a useful tool in student counselling and guidance.     

Teaching sexual history taking to medical students and examining it: experience in one medical school and a national survey

OBJECTIVES: To assess the feasibility and acceptability of training and examining medical students in taking a sexual history and to compare practice with other medical schools in the UK. DESIGN: A training programme involving group work, role play and clinical attachments was developed and applied to 131 students at the University of Bristol Medical School. They then underwent an objective structured clinical assessment using simulated patients. The practice of other medical schools was surveyed by postal questionnaire. RESULT: The students felt that the examination was a good test of their knowledge and skills. One student failed. Sexual history taking is taught in 17 of 22 medical schools but examined in only six. CONCLUSION: Both teaching and examining of sexual history taking skills are possible and are likely to occur increasingly in UK medical schools.     

Medical student interviewing skills and mother-reported satisfaction and recall

BACKGROUND: Providing feedback to medical students about their interviewing skills is an important component of teaching programmes. There is very little information about mothers' views of medical student consultations in paediatrics, and in particular about what mothers consider to be the key elements of a successful consultation. Patient-centred interviewing is a model which emphasizes the active seeking of patient views. In association with appropriate clinical skills, it is reported to promote improved health outcomes. OBJECTIVES: To examine whether greater medical student clinical competence and more frequent use of patient-centred techniques is associated with higher maternal satisfaction, higher maternal rating of the medical student's interpersonal skills, and greater maternal recall of relevant diagnosis and treatment recommendations. METHOD: Two standardized 'medical student' videotaped interviews were created based on actual senior medical student consultations. Interview A demonstrated both higher student clinical competence and higher patient-centredness compared with interview B. Both videotaped interviews were viewed and then rated, using a questionnaire, by 11 mothers attending a teaching general practice. RESULTS: Significantly higher mean scores, indicating greater maternal satisfaction, were associated with interview A (P < 0.01 for all measures). Accurate recall for diagnosis and management was also significantly greater after interview A (mean diagnosis recall, interview A 35%, interview B 14%, P < 0.01; mean management recall, interview A 95%, interview B 57%, P < 0.01). CONCLUSIONS: Maternal satisfaction and recall were higher following a more clinically competent and patient-centred medical student interview. Maternal ratings of student interviews could be used as an additional method of assessment as well as providing feedback to medical students on their interview skills development.     

‘Sorry,it's my first time!' Will patients consent to medical students learning procedures?

Context The clinical teaching of medical students is essential to medicine; however, medical students often may not inform patients of their inexperience. Hence patients do not have the opportunity to consent specifically to the procedures being performed by medical students. Objectives The purpose of this study was to determine whether patients, when informed of the inexperience of a medical student, would still consent to the procedure. Methods Adult emergency department patients (114) were enrolled if they required one of the following procedures: sutures, intravenous (i.v.) access or splinting. Patients were first surveyed about his or her perceptions of medical student performing procedures. The first-year medical student then informed the patient of their inexperience (the number of procedures they had performed successfully). Finally, the patient was asked to consent to the student performing the procedure (i.e. ‘this is my first/third time suturing, may I suture you?’) The main outcome measure was patient consent to the student performing the procedure. Results Only 48% of participants knew they could be the first patient on whom a medical student might perform a procedure. A total of 66% thought they should be told if a student was performing his or her first procedure on them. The majority of patients (90%) consented (69/80 i.v., 20/20 splints, 13/14 sutures). For 7 of the 12 refusals, it was the student's first time performing the procedure. Conclusion Most patients will allow medical students to perform minor procedures, even when informed of the student's inexperience.     

Medical student experience of London general practice teaching attachments

Forty-eight students kept a log diary of activities during their central London general practice teaching attachments associated with the Department of Primary Health Care of University College and Middlesex School of Medicine. The students each saw on average 96 patients per week, of whom 69% were discussed by the general practitioner with the student after the consultation. Students spent an average of 21.5 hours a week sitting in with the general practitioner. While most of this time was as a passive observer, the students were also able to participate more actively, personally taking histories for a median of 1.25 hours a week and personally examining patients for a median of 1.7 hours a week. During these periods of active involvement each student personally took a mean of 10 short and 2.5 long histories per week and performed a mean of 25.5 short and 1.2 long examinations per week. General practitioners to whom the students were attached spent a mean of 4 hours a week on (patient-oriented) teaching. The tuition was highly rated by the students in terms of both usefulness and stimulation. Students also received a mean of 2.3 hours a week of teaching from other members of the primary health care team, which was somewhat less well received. Areas for improvement were: the relatively few home visits (median of 6 per week) per student; the limited time students spent on self-education (average of 65 minutes per week); and the few practical procedures performed by the students. Students could also be encouraged to play a more active role in examining and interviewing patients.     

Follow the patient: process and outcome evaluation of medical students' educational experiences accompanying outpatients

BACKGROUND: To instill patient-centred attitudes in medical students, several medical schools in Japan have recently started to offer educational experiences in which medical students accompany outpatients throughout entire visits to hospitals. OBJECTIVE: To evaluate the processes and outcomes of the educational experience of Year 5 medical students accompanying outpatients at Nagoya University Hospital. METHODS: An integrated, multimethod approach was adopted using a written survey with open-ended questions for students, focus groups with students, and a written evaluation survey for patients. In all, 99 students completed the survey, 19 students participated in 3 focus groups, and 46 patients participated in the evaluation. RESULTS: Many students were sceptical about the objectives of the exercise. We were able to gain insight into student perceptions about facets of the exercise such as the ratio of students to patients and whether or not students should wear white coats. In particular, there was consensus among students about the importance of the debriefing session after the experience. Students achieved different learning outcomes depending on their particular individual experiences. In the student survey, 49% were satisfied with this experience, 6% were dissatisfied, and 43% were neither. In contrast, patients were highly satisfied with the experience (mean score 4.2 out of 5.0 on a Likert scale). Some students expressed concern about being a burden to patients, while many patients reported feeling emotionally supported by being accompanied by students. CONCLUSION: An integrated approach to programme evaluation, using quantitative and qualitative methods, was useful in the process and outcome evaluation of this new educational experience. The results have been taken into consideration for quality improvement of this curricular element.