Paired basic science and clinical problem-based learning faculty teaching side by side: do students evaluate them differently?

INTRODUCTION: Many studies have evaluated the desirability of expert versus non-expert facilitators in problem-based learning (PBL), but performance differences between basic science and clinical facilitators has been less studied. In a PBL course at our university, pairs of faculty facilitators (1 clinician, 1 basic scientist) were assigned to student groups to maximise integration of basic science with clinical science. AIMS: This study set out to establish whether students evaluate basic science and clinical faculty members differently when they teach side by side. METHODS: Online questionnaires were used to survey 188 students about their faculty facilitators immediately after they completed each of 3 serial PBL cases. Overall satisfaction was measured using a scale of 1-7 and yes/no responses were gathered from closed questions describing faculty performance. results: Year 1 students rated basic science and clinical facilitators the same, but Year 2 students rated the clinicians higher overall. Year 1 students rated basic scientists higher in their ability to understand the limits of their own knowledge. Year 2 students rated the clinicians higher in several content expertise-linked areas: preparedness, promotion of in-depth understanding, and ability to focus the group, and down-rated the basic scientists for demonstrating overspecialised knowledge. Students' overall ratings of individual faculty best correlated with the qualities of stimulation, focus and preparedness, but not with overspecialisation, excessive interjection of the faculty member's own opinions, and encouragement of psychosocial issue discussion. CONCLUSION: When taught by paired basic science and clinical PBL facilitators, students in Year 1 rated basic science and clinical PBL faculty equally, while Year 2 students rated clinicians more highly overall. The Year 2 difference may be explained by perceived differences in content expertise.     

Early identification of 'at-risk' students by the parents of paediatric patients

INTRODUCTION: Assessment of medical student clinical skills is best carried out using multiple assessment methods. A programme was developed to obtain parent evaluations of medical student paediatric interview skills for feedback and to identify students at risk of poor performance in summative assessments. METHOD: A total of 130 parent evaluations were obtained for 67 students (parent participation 72%, student participation 58%). Parents completed a 13-item questionnaire [Interpersonal Skills Rating Scale (IPS) maximum score 91, higher scores = higher student skill level]. Students received their individual parent scores and de-identified class mean scores as feedback, and participants were surveyed regarding the programme. Parent evaluation scores were compared with student performance in formative and summative faculty assessments of clinical interview skills. RESULTS: Parents supported the programme and participating students valued parent feedback. Students with a parent score that was less than 1 standard deviation (SD) below the class mean (low IPS score students) obtained lower faculty summative assessment scores than did other students (mean +/- SD, 59% +/- 5 versus 64% +/- 7; P < 0.05). Obtaining 1 low IPS score was associated with a subsequent faculty summative assessment score below the class mean (sensitivity 0.38, specificity 0.88). Parent evaluations combined with faculty formative assessments identified 50% of students who subsequently performed below the class mean in summative assessments. CONCLUSIONS: Parent evaluations provided useful feedback to students and identified 1 group of students at increased risk of weaker performance in summative assessments. They could be combined with other methods of formative assessment to enhance screening procedures for clinically weak students.     

A comparison of learning outcomes and attitudes in student- versus faculty-led problem-based learning: an experimental study

OBJECTIVES: To compare learning outcomes and perceptions of facilitator behaviours and small-group process in problem-based learning (PBL) groups led by students and those led by faculty. DESIGN: A prospective, Latin-square cross-over design was employed. Second-year medical students participated in 11 PBL cases over the course of the academic year. For each case, half the student groups were led by faculty and the other half by a student group member selected randomly to serve in the facilitator role. Learning outcomes were assessed by performance on objective examinations covering factual materials pertinent to the case. Perceptions of facilitator behaviours and of group functioning were assessed with a questionnaire completed at the end of each individual case. Focus-group discussions were held to gain more in-depth information about student perceptions and experience. Student-led sessions were observed at random by the investigators. SETTING: A state-supported, US medical school with a hybrid lecture-based and problem-based curriculum. SUBJECTS: One hundred and twenty-seven second-year medical students and 30 basic science and clinical faculty. RESULTS: No differences were detected in student performance on the objective evaluation based on whether the facilitator was a faculty member or peer group member, nor were there any differences in the perceptions of group process. Students gave peer facilitators slightly higher ratings in the second semester of the experiment. In the focus-group discussions, students voiced a general preference for student-led groups because they felt they were more efficient. Observation and focus-group reports suggest that groups led by students sometimes took short cuts in the PBL process. CONCLUSION: In a hybrid lecture- and PBL-based curriculum, student performance on objective examinations covering PBL materials is unaffected by the status of the facilitator (student vs. faculty). However, in peer-facilitated groups, students sometimes took short cuts in the PBL process that may undermine some of the intended goals of PBL.     

Medical students' personality characteristics and academic performance: a five-factor model perspective

OBJECTIVES: This study investigates: (1) which personality traits are typical of medical students as compared to other students, and (2) which personality traits predict medical student performance in pre-clinical years. DESIGN: This paper reports a cross-sectional inventory study of students in nine academic majors and a prospective longitudinal study of one cohort of medical students assessed by inventory during their first preclinical year and by university examination at the end of each pre-clinical year. SUBJECTS AND METHODS: In 1997, a combined total of 785 students entered medical studies courses in five Flemish universities. Of these, 631 (80.4%) completed the NEO-PI-R (i.e. a measure of the Five-Factor Model of Personality). This was also completed by 914 Year 1 students of seven other academic majors at Ghent University. Year end scores for medical students were obtained for 607 students in Year 1, for 413 in Year 2, and for 341 in Year 3. RESULTS: Medical studies falls into the group of majors where students score highest on extraversion and agreeableness. Conscientiousness (i.e. self-achievement and self-discipline) significantly predicts final scores in each pre-clinical year. Medical students who score low on conscientiousness and high on gregariousness and excitement-seeking are significantly less likely to sit examinations successfully. CONCLUSIONS: The higher scores for extraversion and agreeableness, two dimensions defining the interpersonal dynamic, may be beneficial for doctors' collaboration and communication skills in future professional practice. Because conscientiousness affects examination results and can be reliably assessed at the start of a medical study career, personality assessment may be a useful tool in student counselling and guidance.     

Interpreting course evaluation results: insights from thinkaloud interviews with medical students

PURPOSE: To determine whether some of the fundamental assumptions that frequently underlie interpretation of course evaluation results are justified by investigating what medical students are thinking as they complete a typical basic science course evaluation. METHODS: A total of 24 students participated in thinkaloud cognitive interviews, voicing their thoughts while completing a typical evaluation instrument that included items on overall course design, educational materials and methods, and faculty teaching. Students' responses were organised to consider how they interpreted questions, formed judgements and selected response options. Major themes relevant to the meaningful interpretation of course evaluation data were identified. RESULTS: Medical students understood educational terms such as 'independent learning' in different ways from both one another and common usage. When formulating responses, students' judgements were sometimes based on unique or unexpected criteria, and they described editing their judgements by considering factors such as effort or caring on the part of teaching faculty. Students tended to avoid using the lower end of the rating scale, used the highest rating option selectively, but chose the second highest category indiscriminately. CONCLUSIONS: These results call into question fundamental assumptions that frequently underlie interpretation of course evaluation results, such as whether students understand the intended meanings of terms used in items; whether faculty members who receive the same rating are perceived similarly; whether ratings actually reflect teaching effectiveness, and whether 'positive' ratings reflect positive opinions. This study also demonstrates how thinkaloud interviews can be used in validity studies, providing information to supplement statistical and psychometric analyses.     

Can brief workshops improve clinical instruction?

CONTEXT: The impact of faculty development activities aimed at improving the teaching skills of clinical instructors requires elucidation. Since 2003, all instructors at our school of medicine have been required to undertake a brief workshop in basic clinical instructional skills as a prerequisite for promotion and tenure. The impact of this has, so far, remained unknown.OBJECTIVE: This study aimed to examine to what extent participation in a brief workshop can improve clinical instructors' performance in the long run, and which particular dimensions of performance are improved.METHODS: The study included a sample of 149 faculty members who undertook a required workshop in basic instructional skills. The teaching performance of these faculty members was measured by student feedback a year after the workshop. The study used pre- and post-test design, with a comparison group of 121 faculty members.RESULTS: Student ratings for 5 dimensions of clinical instruction increased significantly, but only for the study group who had participated in a workshop. The comparison group's ratings were unchanged. The highest improvement in the instructors' performance related to availability of teachers to students.CONCLUSIONS: The study supports previous findings about the added value gained by longterm improvement of instructional skills after participation in even a brief workshop. The meaningful improvement in instructor availability to students is associated with the workshops' emphasis on a learner-centred approach and the need to provide continuous feedback.     

Temporal and group-related trends in peer assessment amongst medical students

Peer assessment has been increasingly recommended as a way to evaluate the professional competencies of medical trainees. Prior studies have only assessed single groups measured at a single timepoint. Thus, neither the longitudinal stability of such ratings nor differences between groups using the same peer-assessment instrument have been reported previously. Participants were all members of 2 consecutive classes of medical students (n = 77 and n = 85) at the University of Rochester School of Medicine and Dentistry who completed Years 2 and 3 of medical school consecutively. All participants were evaluated by 6-12 classmates near the end of both Years 2 and 3. Main outcome measures were mean numerical ratings on peer-assessed scales of professional work habits (WH) and interpersonal attributes (IA). Both scales had high internal consistencies in both years (Cronbach's alpha 0.84-0.94). The IA and WH scales were moderately correlated with one another (r = 0.36 in Year 2, r = 0.28 in Year 3). Year 2 scores were predictive of Year 3 scores for both scales (WH: r = 0.64; IA; r = 0.62). Generalisability and decision analyses revealed that 1 class was consistently more discriminating with the WH scale, while the other was more discriminating with of the IA scale. Depending on the class, year and scale, the number of raters needed to achieve a reasonable reliability ranged between 7 and 28. Although Year 3 peer ratings were consistently higher than Year 2 peer ratings for both WH and IA, individual scores were highly correlated across the 2 years, despite the fact that different individuals were chosen as peer raters. Abilities appear to be stable between Years 2 and 3 of medical school. Groups may differ in their ability to discriminate different kinds of skills. Generalisability analysis can be used to discover these patterns within groups.     

Analysis and prospects for curricular reform of medical schools in Southeast Europe

AIM: To analyse the curricula of 16 medical schools in 6 countries in Southeast Europe in order to establish a prevailing standard curriculum against which a prospective curriculum reform could formulate its objectives. METHODS: Curricular information was gathered from a questionnaire sent via e-mail to the respective medical schools. The data collected ranged from the numbers of enrolled students to a breakdown of courses with distribution of instruction hours for certain teaching formats. For easier comparison the courses were clustered into 5 groups: pre-clinical, clinical, public health, liberal arts and electives. RESULTS: Belgrade has the highest number of undergraduate students, while Mostar has the lowest. Novi Sad, Foca/Srbinje, Zagreb, Split, Sofia, Ljubljana and Mostar have more than 5000 instruction hours, but Sarajevo lags behind with 4005 hours. Anatomy dominates the course load in Year 1, ranging from a share of 18.4% in Sofia to 11.3% in Novi Sad. Physiology dominates Year 2, ranging from 16.8% in Rijeka to 8.9% in Split, whereas in Year 3 the dominating course is pathology, reaching a peak of 13.7% in Sarajevo. Sofia has the highest number of class hours of clinical courses. The predominant public health courses are social medicine, family medicine and medical ecology. Medical English is taught at all medical schools (as electives in Ljubljana and Rijeka). CONCLUSION: There is considerable potential for curriculum improvement in the region. Teacher training, student participation, the definition of core competencies and the introduction of new methodologies should all be implemented in the process.     

Can student tutors act as examiners in an objective structured clinical examination?

Context  The dissemination of objective structured clinical examinations (OSCEs) is hampered by requirements for high levels of staffing and a significantly higher workload compared with multiple-choice examinations. Senior medical students may be able to support faculty staff to assess their peers. The aim of this study is to assess the reliability of student tutors as OSCE examiners and their acceptance by their peers.
Methods  Using a checklist and a global rating, teaching doctors (TDs) and student tutors (STs) simultaneously assessed students in basic clinical skills at 4 OSCE stations. The inter-rater agreement between TDs and STs was calculated by kappa values and paired t -tests. Students then completed a questionnaire to assess their acceptance of student peer examiners.
Results  All 214 Year 3 students at the University of Göttingen Medical School were evaluated in spring 2005. Student tutors gave slightly better average grades than TDs (differences of 0.02–0.20 on a 5-point Likert scale). Inter-rater agreement at the stations ranged from 0.41 to 0·64 for checklist assessment and global ratings; overall inter-rater agreement on the final grade was 0.66. Most students felt that assessment by STs would result in the same grades as assessment by TDs (64%) and that it would be similarly objective (69%). Nearly all students (95%) felt confident that they could evaluate their peers themselves in an OSCE.
Conclusions  On the basis of our results, STs can act as examiners in summative OSCEs to assess basic medical skills. The slightly better grades observed are of no practical concern. Students accepted assessment performed by STs.     

Follow the patient: process and outcome evaluation of medical students' educational experiences accompanying outpatients

BACKGROUND: To instill patient-centred attitudes in medical students, several medical schools in Japan have recently started to offer educational experiences in which medical students accompany outpatients throughout entire visits to hospitals. OBJECTIVE: To evaluate the processes and outcomes of the educational experience of Year 5 medical students accompanying outpatients at Nagoya University Hospital. METHODS: An integrated, multimethod approach was adopted using a written survey with open-ended questions for students, focus groups with students, and a written evaluation survey for patients. In all, 99 students completed the survey, 19 students participated in 3 focus groups, and 46 patients participated in the evaluation. RESULTS: Many students were sceptical about the objectives of the exercise. We were able to gain insight into student perceptions about facets of the exercise such as the ratio of students to patients and whether or not students should wear white coats. In particular, there was consensus among students about the importance of the debriefing session after the experience. Students achieved different learning outcomes depending on their particular individual experiences. In the student survey, 49% were satisfied with this experience, 6% were dissatisfied, and 43% were neither. In contrast, patients were highly satisfied with the experience (mean score 4.2 out of 5.0 on a Likert scale). Some students expressed concern about being a burden to patients, while many patients reported feeling emotionally supported by being accompanied by students. CONCLUSION: An integrated approach to programme evaluation, using quantitative and qualitative methods, was useful in the process and outcome evaluation of this new educational experience. The results have been taken into consideration for quality improvement of this curricular element.     

The reliability of summative judgements based on objective structured clinical examination cases distributed across the clinical year

CONTEXT: Objective structured clinical examinations (OSCEs) can be used for formative and summative evaluation. We sought to determine the generalisability of students' summary scores aggregated from formative OSCE cases distributed across 5 clerkships during Year 3 of medical school. METHODS: Five major clerkships held OSCEs with 2-4 cases each during their rotations. All cases used 15-minute student-standardised patient encounters and performance was assessed using clinical and communication skills checklists. As not all students completed every clerkship or OSCE case, the generalisability (G) study was an unbalanced student x (case : clerkship) design. After completion of the G study, a decision (D) study was undertaken and phi (phi) values for different cut-points were calculated. RESULTS: The data for this report were collected over 2 academic years involving 262 Year 3 students. The G study found that 9.7% of the score variance originated from the student, 3.1% from the student-clerkship interaction, and 87.2% from the student-case nested within clerkship effect. Using the variance components from the G study, the D study suggested that if students completed 3 OSCE cases in each of the 5 different clerkships, the reliability of the aggregated scores would be 0.63. The phi, calculated at a cut-point 1 standard deviation below the mean, would be approximately 0.85. CONCLUSIONS: Aggregating case scores from low stakes OSCEs within clerkships results in a score set that allows for very reliable decisions about which students are performing poorly. Medical schools can use OSCE case scores collected over a clinical year for summative evaluation.     

Factors in faculty evaluation of medical students' performance

CONTEXT: Faculty members often use global rating scales as a method of assessing various characteristics of medical students' clinical performance. The purpose of this study was to determine if some performance characteristics are more highly associated with the overall faculty grade than others. METHODS: The clinical performance of 211 surgery clerkship students was evaluated by 2 or 3 faculty preceptors. Faculty rated students on 10 specific performance characteristics, using a 5-point scale. Faculty then assigned a numerical grade summarising the faculty's view of the student's performance. Reliability of the ratings was estimated by the intraclass correlation, and 1-way (analysis of variance) anova was used to test for differences among the students' mean ratings. Logistic regression was employed to determine the accuracy of each performance measure in predicting students' grades (A or B). Stepwise logistic regression was used to determine if there was a combination of performance characteristics that best predicted students' grades. RESULTS: The inter-rater reliabilities were low (相似文献   

Growth of analytical thinking skills over time as measured with the MATCH test

Context Ber’s Comprehensive Integrative Puzzle aims to assess analytical clinical thinking in medical students. We developed a paediatric version, the MATCH test, in which we added two irrelevant options to each question in order to reduce guessing behaviour. We tested its construct validity and studied the development of integrative skills over time. Methods We administered a test (MATCH 1) to subjects from two universities, both with a 6‐year medical training course. Subjects included 30 students from university 1 who had completed a paediatric clerkship in Year 4, 23 students from university 2 who had completed a paediatric clerkship in Year 5, 13 students from both universities who had completed an advanced paediatric clerkship in Year 6, 28 paediatric residents and 17 paediatricians. We repeated this procedure using a second test with different domains in a new, comparable group of subjects (MATCH 2). Results Mean MATCH 1 scores for the respective groups were: Year 4 students: 61.2% (standard deviation [SD] 1.3); Year 5 students: 71.3% (SD 1.6); Year 6 students: 76.2% (SD 1.5); paediatric residents: 88.5% (SD 0.7), and paediatricians: 92.2% (SD 1.1) (one‐way anova F = 104.00, P < 0.0001). Students of both universities had comparable scores. MATCH 1 and 2 scores were comparable. Cronbach’s α‐values in MATCH 1 and 2 were 0.92 and 0.91, respectively, for all subjects, and 0.82 and 0.87, respectively, for all students. Conclusions Analytical clinical thinking develops over time, independently of the factual content of the course. This implies that shortened medical training programmes could produce less skilled graduates.     

Evaluation of procedural skills training in an undergraduate curriculum

CONTEXT: A substantial proportion of medical students enter their intern year without any basic skills experience. Lack of experience is a significant source of stress for many junior doctors. OBJECTIVES: To evaluate the effect of a basic procedural skills tutorial for Year 3 medical students on their competence in relevant skills at Year 5. SUBJECTS: The control group consisted of 93 medical students who completed Year 3 in 1996. The intervention group consisted of 92 medical students who completed Year 3 in 1997. The intervention group received a practical skills tutorial in Year 3; the control group did not. Both groups were assessed on their practical skills competence during Year 5. METHODS: A 3-hour practical tutorial on injection and suturing techniques was delivered to the intervention group. The effectiveness of the intervention was assessed by self-reported experience of giving injections, inserting sutures and sustaining needlestick injuries, and by teacher-rated competency in four basic procedural skills. RESULTS: Students who received the Year 3 tutorial were significantly more likely to record a satisfactory assessment for their performance in all four basic skills compared with students who did not receive the tutorial. They were less likely than controls to refuse invitations to give injections, but not invitations to insert a suture, during Years 4 and 5. CONCLUSIONS: A single session of formalised teaching in procedural skills in the early stages of a medical degree can have long-term effectiveness in basic skills competence and may increase students' confidence to practise their skills.     

Not knowing that they do not know: self-assessment accuracy of third-year medical students

PURPOSE: The development of self-regulated learning is a major focus of our problem-based learning (PBL) medical programme. Students who are unsuccessful in assessments often seem to lack insight into the standard of their own performance, yet the ability to self-assess accurately is essential for the effective self-management of learning. The aim of this project was to evaluate the accuracy of self- and peer-assessment according to academic performance. METHOD: In 2004, 175 3rd-year students undertook an integrated, case-based, short-essay, formative assessment. After the assessment they were provided with model answers and marking criteria. Students marked their own assessment paper and the paper of one of their peers. Assessment papers were subsequently marked by faculty members. The following data was available for each student: self-mark, faculty-mark, score awarded by a peer and the score that they awarded to their peer. Self-assessment and peer-assessment ability was compared to overall academic performance. RESULTS: Low-achieving students score themselves and their peers generously. High-achieving students score themselves more harshly than faculty. However, they score their peers accurately. CONCLUSION: In the 3rd year of the programme low-achieving students are unable to assess accurately the quality of their own work or the work of their peers in a formative written assessment. The PBL curriculum does not guarantee the appropriate development of self-assessment skills.     

Student perceptions of tutor skills in problem-based learning tutorials

OBJECTIVE: The problem-based learning (PBL) tutor plays a role that is different from the role of a teacher in a conventional teaching format. In the Faculty of Medicine and Health Sciences, United Arab Emirates, all students are Arab nationals and tutors are expatriates with different sociocultural backgrounds from the students. This study was designed to investigate how students evaluate tutors in PBL tutorials and whether student evaluations of tutors change with the progress of students in PBL tutorials. METHODS: Differences in tutor performance evaluation by male and female students were also analysed. The students evaluated 12 tutor skills in a scale of 1-3, 1 being 'below average' and 3, 'outstanding'. Student responses from a total of 314 (98.1%) completed forms collected over 2 academic years were analysed statistically. A total of 14 tutors participated in the PBL programme. RESULTS: The analysis revealed that tutors as a group were rated as having average to outstanding tutor skills in 10 items of the evaluation form. Students and faculty perceptions were different for the tutor skills of guiding students for information management. The students expected more support from tutors, whereas the tutors tried to emphasize self-learning in the PBL curriculum. Lower scores to the tutors in the 'problem' bringing sociocultural and religious issues for discussion showed that a gap in sociocultural/religious understanding between students and tutors might influence tutor skills. CONCLUSIONS: Differences in tutor evaluation by male and female students indicate necessity of adopting different strategies by tutors in a different sociocultural background. The results of the study have direct implications for faculty development.     

Peer assessment of professional competence

BACKGROUND: Current assessment formats for medical students reliably test core knowledge and basic skills. Methods for assessing other important domains of competence, such as interpersonal skills, humanism and teamwork skills, are less well developed. This study describes the development, implementation and results of peer assessment as a measure of professional competence of medical students to be used for formative purposes. METHODS: Year 2 medical students assessed the professional competence of their peers using an online assessment instrument. Fifteen randomly selected classmates were assigned to assess each student. The responses were analysed to determine the reliability and validity of the scores and to explore relationships between peer assessments and other assessment measures. RESULTS: Factor analyses suggest a 2-dimensional conceptualisation of professional competence: 1 factor represents Work Habits, such as preparedness and initiative, and the other factor represents Interpersonal Habits, including respect and trustworthiness. The Work Habits factor had moderate, yet statistically significant correlations ranging from 0.21 to 0.53 with all other performance measures that were part of a comprehensive assessment of professional competence. Approximately 6 peer raters were needed to achieve a generalisability coefficient of 0.70. CONCLUSIONS: Our findings suggest that it is possible to introduce peer assessment for formative purposes in an undergraduate medical school programme that provides multiple opportunities to interact with and observe peers.     

Does feedback matter? Practice‐based learning for medical students after a multi‐institutional clinical performance examination

CONTEXT: Achieving competence in 'practice-based learning' implies that doctors can accurately self- assess their clinical skills to identify behaviours that need improvement. This study examines the impact of receiving feedback via performance benchmarks on medical students' self-assessment after a clinical performance examination (CPX). METHODS: The authors developed a practice-based learning exercise at 3 institutions following a required 8-station CPX for medical students at the end of Year 3. Standardised patients (SPs) scored students after each station using checklists developed by experts. Students assessed their own performance immediately after the CPX (Phase 1). One month later, students watched their videotaped performance and reassessed (Phase 2). Some students received performance benchmarks (their scores, plus normative class data) before the video review. Pearson's correlations between self-ratings and SP ratings were calculated for overall performance and specific skill areas (history taking, physical examination, doctor-patient communication) for Phase 1 and Phase 2. The 2 correlations were then compared for each student group (i.e. those who received and those who did not receive feedback). RESULTS: A total of 280 students completed both study phases. Mean CPX scores ranged from 51% to 71% of items correct overall and for each skill area. Phase 1 self-assessment correlated weakly with SP ratings of student performance (r = 0.01-0.16). Without feedback, Phase 2 correlations remained weak (r = 0.13-0.18; n = 109). With feedback, Phase 2 correlations improved significantly (r = 0.26-0.47; n = 171). Low-performing students showed the greatest improvement after receiving feedback. CONCLUSIONS: The accuracy of student self-assessment was poor after a CPX, but improved significantly with performance feedback (scores and benchmarks). Videotape review alone (without feedback) did not improve self-assessment accuracy. Practice-based learning exercises that incorporate feedback to medical students hold promise to improve self-assessment skills.     

Investigating the use of sampling for maximising the efficiency of student-generated faculty teaching evaluations

PURPOSE: Surveys of medical students are widely used to evaluate course content and faculty teaching within the medical school. Gathering information that accurately reflects student perceptions requires that students buy into the evaluation process and be willing to provide thoughtful responses to the teaching evaluation. To maintain student commitment, it is important that medical students are not overburdened with poorly planned evaluations. Sampling might decrease the number of evaluations required of students and might also reduce the proportion of non-responses and other forms of inattentive response biases. METHODS: A sampling technique employed within a large medical lecture is described and evaluated. A generalisability study of the teacher evaluations is conducted. RESULTS: A high response rate and high levels of reliability were obtained by sampling a small proportion of the total class. The largest source of error was related to rater and utilising sufficient numbers of student-raters is critical to achieving reliable results. CONCLUSION: Sampling can reduce evaluation demands placed on students, and preserve reliability and increase the validity of mean evaluation scores. With computer presentation, efficient sampling techniques become practical and should be part of software packages used to present teacher evaluations.