Factor instability of clinical teaching assessment scores among general internists and cardiologists

CONTEXT: We are unaware of studies examining the stability of teaching assessment scores across different medical specialties. A recent study showed that clinical teaching assessments of general internists reduced to interpersonal, clinical teaching and efficiency domains. We sought to determine the factor stability of this 3-dimensional model among cardiologists and to compare domain-specific scores between general internists and cardiologists. METHODS: A total of 2000 general internal medicine and cardiology hospital teaching assessments carried out from January 2000 to March 2004 were analysed using principal factor analysis. Internal consistency and inter-rater reliability were calculated. Mean item scores were compared between general internists and cardiologists. RESULTS: The interpersonal and clinical teaching domains previously demonstrated among general internists collapsed into 1 domain among cardiologists, whereas the efficiency domain remained stable. Internal consistency of domains (Cronbach's alpha range 0.89-0.93) and inter-rater reliability of items (range 0.65-0.87) were good to excellent for both specialties. General internists scored significantly higher (P<0.05) than cardiologists on most items except for 4 items that more accurately assessed the cardiology teaching environment. CONCLUSIONS: We observed factor instability of clinical teaching assessment scores from the same instrument administered to general internists and cardiologists. This finding was attributed to salient differences between these specialties' educational environments and highlights the importance of validating assessments for the specific contexts in which they are to be used. Future research should determine whether interpersonal domain scores identify superior teachers and study the reasons why interpersonal and clinical teaching domains are unstable across different educational settings.     

Early identification of 'at-risk' students by the parents of paediatric patients

INTRODUCTION: Assessment of medical student clinical skills is best carried out using multiple assessment methods. A programme was developed to obtain parent evaluations of medical student paediatric interview skills for feedback and to identify students at risk of poor performance in summative assessments. METHOD: A total of 130 parent evaluations were obtained for 67 students (parent participation 72%, student participation 58%). Parents completed a 13-item questionnaire [Interpersonal Skills Rating Scale (IPS) maximum score 91, higher scores = higher student skill level]. Students received their individual parent scores and de-identified class mean scores as feedback, and participants were surveyed regarding the programme. Parent evaluation scores were compared with student performance in formative and summative faculty assessments of clinical interview skills. RESULTS: Parents supported the programme and participating students valued parent feedback. Students with a parent score that was less than 1 standard deviation (SD) below the class mean (low IPS score students) obtained lower faculty summative assessment scores than did other students (mean +/- SD, 59% +/- 5 versus 64% +/- 7; P < 0.05). Obtaining 1 low IPS score was associated with a subsequent faculty summative assessment score below the class mean (sensitivity 0.38, specificity 0.88). Parent evaluations combined with faculty formative assessments identified 50% of students who subsequently performed below the class mean in summative assessments. CONCLUSIONS: Parent evaluations provided useful feedback to students and identified 1 group of students at increased risk of weaker performance in summative assessments. They could be combined with other methods of formative assessment to enhance screening procedures for clinically weak students.     

Sources of validity evidence for an internal medicine student evaluation system: an evaluative study of assessment methods

Background  Medical students' final clinical grades in internal medicine are based on the results of multiple assessments that reflect not only the students' knowledge, but also their skills and attitudes.
Objective  To examine the sources of validity evidence for internal medicine final assessment results comprising scores from 3 evaluations and 2 examinations.
Methods  The final assessment scores of 8 cohorts of Year 4 medical students in a 6-year undergraduate programme were analysed. The final assessment scores consisted of scores in ward evaluations (WEs), preceptor evaluations (PREs), outpatient clinic evaluations (OPCs), general knowledge and problem-solving multiple-choice questions (MCQs), and objective structured clinical examinations (OSCEs). Sources of validity evidence examined were content, response process, internal structure, relationship to other variables, and consequences.
Results  The median generalisability coefficient of the OSCEs was 0.62. The internal consistency reliability of the MCQs was 0.84. Scores for OSCEs correlated well with WE, PRE and MCQ scores with observed (disattenuated) correlation of 0.36 (0.77), 0.33 (0.71) and 0.48 (0.69), respectively. Scores for WEs and PREs correlated better with OSCE than MCQ scores. Sources of validity evidence including content, response process, internal structure and relationship to other variables were shown for most components.
Conclusion  There is sufficient validity evidence to support the utilisation of various types of assessment scores for final clinical grades at the end of an internal medicine rotation. Validity evidence should be examined for any final student evaluation system in order to establish the meaningfulness of the student assessment scores.     

Validity of index of learning styles scores: multitrait-multimethod comparison with three cognitive/learning style instruments

Cognitive and learning styles research is limited by the lack of evidence supporting valid interpretations of style assessment scores. We sought evidence to support the validity of scores from 4 instruments: the Index of Learning Styles (ILS); the Learning Style Inventory (LSI); the Cognitive Styles Analysis (CSA), and the Learning Style Type Indicator (LSTI). The ILS assesses 4 domains: sensing-intuitive (SensInt), active-reflective (ActRefl), sequential-global (SeqGlob) and visual-verbal (VisVerb), each of which parallel a similar domain in at least 1 of the other instruments. We administered the ILS, LSI and CSA to family medicine and internal medicine residents and Year 1 and 3 medical students and applied the multitrait-multimethod matrix to evaluate convergence and discrimination. After 3 months participants repeated the ILS and completed the LSTI. A total of 89 residents and medical students participated. Multitrait-multimethod analysis showed evidence of both convergence and discrimination for ActRefl (ILS, LSI and LSTI) and SensInt (ILS and LSTI) scores. ILS SeqGlob and SensInt scores showed unanticipated correlation. No other domains met the criteria for convergence or discrimination. Test-retest reliabilities for ILS scores were 0.856 for SensInt, 0.809 for ActRefl, 0.703 for SeqGlob and 0.684 for VisVerb. Cronbach's alpha values were > or = 0.810 for LSI and 0.237-0.758 for LSTI. At least 9 participants misinterpreted the LSI instructions. These data support the validity of ILS active-reflective and sensing-intuitive scores, LSI active-reflective scores and LSTI sensing-intuitive scores for determining learning styles in this population. Cognitive style and learning style scores may not be interchangeable, even for constructs with similar definitions.     

Development of a teaching style inventory for tutor evaluation in problem-based learning

OBJECTIVE: To develop and validate a self-rating instrument to assess teaching styles among tutors in problem-based learning (PBL). METHODS: The development of the teaching style inventory (TSI) was based theoretically on four types of teaching behaviours: the assertive, suggestive, collaborative and facilitative styles, as proposed by Bibace et al. A 35-item questionnaire was generated and evaluated for content validity by a group of experienced tutors. The questionnaire was mailed to 196 tutors at the National Taiwan University College of Medicine. The results were submitted for item analysis, internal consistency testing and exploratory factor analysis. Longterm test-retest reliability was assessed by a sample of 50 tutors after a 6-month interval. RESULTS: Finally, 118 tutors returned the questionnaires. In the item reduction process, seven items were excluded due to low interscale correlation. Principle component factoring yielded a three-factor solution that accounted for 48.5% of the total variance. Internal consistency coefficients of the four hypothetical domains ranged from 0.73 to 0.83. All domains correlated to each other as expected. Assertive and facilitative styles, which are theoretically opposite teaching styles, showed a negative correlation with each other. Most of the items of each hypothetical domain correlated better with their own domain than with other domains. Longterm test-retest correlations of the four domains ranged from 0.54 to 0.81. CONCLUSION: The TSI demonstrated high internal consistency reliability, acceptable longterm test-retest reliability, and construct validity. Further psychometric testing should focus on applicability to other populations, predictive validity and short-term test-retest reliability. This instrument can be used by programme directors for the recruitment of tutors and can also be used to increase the self-awareness of tutors.     

Effectiveness of medical school admissions criteria in predicting residency ranking four years later

BACKGROUND: Medical schools across Canada expend great effort in selecting students from a large pool of qualified applicants. Non-cognitive assessments are conducted by most schools in an effort to ensure that medical students have the personal characteristics of importance in the practice of Medicine. We reviewed the ability of University of Toronto academic and non-academic admission assessments to predict ranking by Internal Medicine and Family Medicine residency programmes. METHODS: The study sample consisted of students who had entered the University of Toronto between 1994 and 1998 inclusive, and had then applied through the Canadian resident matching programme to positions in Family or Internal Medicine at the University of Toronto in their graduating year. The value of admissions variables in predicting medical school performance and residency ranking was assessed. RESULTS: Ranking in Internal Medicine correlated significantly with undergraduate grade point average (GPA) and the admissions non-cognitive assessment. It also correlated with 2-year objective structured clinical examination (OSCE) score, clerkship grade in Internal Medicine, and final grade in medical school. Ranking in Family Medicine correlated with the admissions interview score. It also correlated with 2nd-year OSCE score, clerkship grade in Family Medicine, clerkship ward evaluation in Internal Medicine and final grade in medical school. DISCUSSION: The results of this study suggest that cognitive as well as non-cognitive factors evaluated during medical school admission are important in predicting future success in Medicine. The non-cognitive assessment provides additional value to standard academic criteria in predicting ranking by 2 residency programmes, and justifies its use as part of the admissions process.     

Meeting the challenge of prescribing and administering medicines safely: structured teaching and assessment for final year medical students

OBJECTIVES: To promote safe prescribing and administration of medicines in the pre-registration house officer (PRHO) year through a programme of structured teaching and assessment for final year medical students. DESIGN: Forty final year medical students from two medical schools were randomly allocated either to participate in a pharmacist facilitated teaching session or to receive no additional teaching. Teaching comprised five practical exercises covering seven skills through which students rotated in small groups. One month later, a random sample of 16 taught and 16 non-taught students participated in a nine-station objective structured clinical examination (OSCE) to assess the impact of the teaching. SETTING: Manchester School of Medicine (MSM), and Kings College School of Medicine and Dentistry (KCSMD). PARTICIPANTS: Final year medical student volunteers. MAIN OUTCOME MEASURES: The need for teaching as indicated by student prior experience; questionnaire rating of student acceptability of teaching and assessment; self-rating of student confidence post-assessment, and student performance assessed by OSCE. RESULTS: The study demonstrated that the taught group achieved higher scores in eight OSCE stations. Four of these were statistically significant (P < or= 0.005). Taught students felt more confident performing the skills on five stations. From 0 to 47.5% students had prior experience of the skills taught. The post-teaching questionnaire evaluated exercises positively on several criteria, including provision of new information and relevance to future work. CONCLUSIONS: Structured teaching provided an effective and acceptable method of teaching the medicines management skills needed in the PRHO year. The structured approach complemented variable precourse clinical experience.     

A comprehensive assessment of medical schools in Bosnia and Herzegovina

OBJECTIVES: To perform internal and external evaluations of all 5 medical schools in Bosnia and Herzegovina against international standards. METHODS: We carried out a 2-stage survey study using the same 5-point Likert scale for internal and external evaluations of 5 medical schools in Bosnia and Herzegovina (Banja Luka, Foca/East Sarajevo, Mostar, Sarajevo and Tuzla). Participants consisted of managerial staff, teaching staff and students of medical schools, and external expert assessors. Main outcome measures included scores on internal and external evaluation forms for 10 items concerning aspects of school curriculum and functioning: 'School mission and objectives'; 'Curriculum'; 'Management'; 'Staff'; 'Students'; 'Facilities and technology'; 'Financial issues'; 'International relationships'; 'Internal quality assurance', and 'Development plans'. RESULTS: During internal assessment, schools consistently either overrated their overall functioning (Foca/East Sarajevo, Mostar and Tuzla) or markedly overrated or underrated their performance on individual items on the survey (Banja Luka and Sarajevo). Scores for internal assessment differed from those for external assessment. These differences were not consistent, except for the sections 'School mission and objectives', 'Curriculum' and 'Development plans', which were consistently overrated in the internal assessments. External assessments was more positive than internal assessments on 'Students' and 'Facilities and technology' in 3 of 5 schools. CONCLUSIONS: This assessment exercise in 5 medical schools showed that constructive and structured evaluation of medical education is possible, even in complex and unfavourable conditions. Medical schools in Bosnia and Herzegovina have successfully formed a national consortium for formal collaboration in curriculum development and reform.     

Investigating the use of sampling for maximising the efficiency of student-generated faculty teaching evaluations

PURPOSE: Surveys of medical students are widely used to evaluate course content and faculty teaching within the medical school. Gathering information that accurately reflects student perceptions requires that students buy into the evaluation process and be willing to provide thoughtful responses to the teaching evaluation. To maintain student commitment, it is important that medical students are not overburdened with poorly planned evaluations. Sampling might decrease the number of evaluations required of students and might also reduce the proportion of non-responses and other forms of inattentive response biases. METHODS: A sampling technique employed within a large medical lecture is described and evaluated. A generalisability study of the teacher evaluations is conducted. RESULTS: A high response rate and high levels of reliability were obtained by sampling a small proportion of the total class. The largest source of error was related to rater and utilising sufficient numbers of student-raters is critical to achieving reliable results. CONCLUSION: Sampling can reduce evaluation demands placed on students, and preserve reliability and increase the validity of mean evaluation scores. With computer presentation, efficient sampling techniques become practical and should be part of software packages used to present teacher evaluations.     

Developing teaching skills for medical educators in Russia: a cross-cultural faculty development project

CONTEXT: Faculty development programmes have proved successful for improving teaching skills. We investigated whether or not a successful US-based faculty development programme for improving the teaching skills of medical faculty could be transported to Russia. METHODS: Five seminars, based on the 7 categories of the Stanford Faculty Development Program model, were presented to 48 teachers at Kazan State Medical University in Kazan, Russia. The seminars were comprised of mini-lectures, reviews of actual videotaped teaching scenario re-enactments, interactive role plays of teaching situations, and personalised goal setting for future teaching performance. Evaluation was performed through participants' self-reported ratings of teaching ability based on a retrospective pretest/post-test questionnaire and fulfilment of commitment of change (CTC) statements written by workshop participants. Outcomes were measured at both 1 and 12 months post-intervention. RESULTS: Survey response rates were 98% (47/48) at 1 month and 81% (39/48) at 12 months. Global teaching performance improved (pretest = 38.4, 1 month post-test = 43.7, 12 months post-test = 42.5; P < 0.001) as did ratings of specific teaching behaviours (pretest = 100.2, 1 month post-test = 121.3, 12 months post-test = 116.8; P < 0.001). A total of 127 CTC statements were made and 90 (71%) were successfully instituted. CONCLUSIONS: Our faculty development intervention demonstrated a positive, lasting effect on the teaching skills of Russian faculty members. This suggests that our efforts in transporting this intervention across the 2 cultures were successful.     

Assessment of patient management skills and clinical skills of practising doctors using computer-based case simulations and standardised patients

CONTEXT: Standardised assessments of practising doctors are receiving growing support, but theoretical and logistical issues pose serious obstacles. OBJECTIVES: To obtain reference performance levels from experienced doctors on computer-based case simulation (CCS) and standardised patient-based (SP) methods, and to evaluate the utility of these methods in diagnostic assessment. SETTING AND PARTICIPANTS: The study was carried out at a military tertiary care facility and involved 54 residents and credentialed staff from the emergency medicine, general surgery and internal medicine departments. MAIN OUTCOME MEASURES: Doctors completed 8 CCS and 8 SP cases targeted at doctors entering the profession. Standardised patient performances were compared to archived Year 4 medical student data. RESULTS: While staff doctors and residents performed well on both CCS and SP cases, a wide range of scores was exhibited on all cases. There were no significant differences between the scores of participants from differing specialties or of varying experience. Among participants who completed both CCS and SP testing (n = 44), a moderate positive correlation between CCS and SP checklist scores was observed. There was a negative correlation between doctor experience and SP checklist scores. Whereas the time students spent with SPs varied little with clinical task, doctors appeared to spend more time on communication/counselling cases than on cases involving acute/chronic medical problems. CONCLUSION: Computer-based case simulations and standardised patient-based assessments may be useful as part of a multimodal programme to evaluate practising doctors. Additional study is needed on SP standard setting and scoring methods. Establishing empirical likelihoods for a range of performances on assessments of this character should receive priority.     

Observed versus reported precepting skills: teaching behaviours in a community ambulatory clerkship

PURPOSE: Community-based doctors are increasingly utilised for the clinical education of medical students. Faculty development programmes are frequently provided. However, data about the community faculty-student teaching interaction and the longterm impact of such programmes are limited. This study observes community faculty-student teaching interactions and assesses the use of clinical teaching methods taught during faculty development workshops. METHODS: Between March and August 2002, 13 rural, urban and suburban community-based faculty who completed at least 1 faculty development workshop were observed during faculty-student-patient encounters and interviewed. Observer-interviewers utilised a checklist to record teaching styles, methods and skills, and administered a questionnaire to obtain community faculty's self-perceptions. RESULTS: On observation, the assertive precepting style and didactic teaching predominated in 90% and 86% of teaching encounters, respectively. Interactive discussion and direct observation occurred less often, in 71% and 18% of encounters, respectively. Feedback was provided in 25% of encounters. On interview, community faculty reported that faculty development workshops resulted in improved precepting effectiveness and professional satisfaction with moderate to considerable improvement in feedback, interactive teaching and use of multiple precepting styles for 100%, 77% and 77% of community faculty, respectively. Community faculty reported that their skills in addressing problem students, direct observation and didactic teaching improved moderately. There was little statistical correlation between self-assessed and observed skills. CONCLUSIONS: Community-based faculty rate their teaching skills as improved following faculty development. Observation reveals that interactive discussion, direct observation, feedback and problem management skills are still lacking.     

Using standardised students in faculty development workshops to improve clinical teaching skills

PURPOSE: We describe the use of standardised students (SSs) in interdisciplinary faculty development programmes to improve clinical teaching skills. Standardised students are actual health professions students who are trained to portray a prototypical teaching challenge consistently across many encounters with different faculty participants. METHODS: The faculty development programmes described focused on the skills of providing feedback and brief clinical teaching. At the beginning of each session, each participant was videotaped in encounters with 2 different SSs. Using microteaching (an instructional method in which learners view short segments of their own videotaped performance and discuss the tapes with a facilitator, consultant or other workshop participants), each group of participants and instructors reviewed the tapes and reflected on the encounters, providing immediate feedback to participants and modelling different approaches to the same teaching problem. The same process was repeated with more complicated scenarios after 2 weeks and again after 6 months offering reinforcement, further practice and more sophisticated development of the strategies learned. Participants completed post-session evaluations and a follow-up telephone survey. RESULTS: A total of 36 faculty members from the colleges of medicine, dentistry, pharmacy and nursing participated in workshops in 2000-01. The workshops were rated as highly relevant to participants' teaching, and most participants reported that they had learned a great deal. Participants most appreciated reviewing the videotaped interactions, the feedback they received, the interactions with their colleagues, the interdisciplinary nature of the groups and the practical focus of the workshops. CONCLUSIONS: Standardised students provide a high fidelity, low risk, simulated environment in which faculty can reflect on and experiment with new teaching behaviours. Such encounters can enhance the effectiveness and impact of faculty development programmes to improve clinical teaching skills.     

Reliability: on the reproducibility of assessment data

CONTEXT: All assessment data, like other scientific experimental data, must be reproducible in order to be meaningfully interpreted. PURPOSE: The purpose of this paper is to discuss applications of reliability to the most common assessment methods in medical education. Typical methods of estimating reliability are discussed intuitively and non-mathematically. SUMMARY: Reliability refers to the consistency of assessment outcomes. The exact type of consistency of greatest interest depends on the type of assessment, its purpose and the consequential use of the data. Written tests of cognitive achievement look to internal test consistency, using estimation methods derived from the test-retest design. Rater-based assessment data, such as ratings of clinical performance on the wards, require interrater consistency or agreement. Objective structured clinical examinations, simulated patient examinations and other performance-type assessments generally require generalisability theory analysis to account for various sources of measurement error in complex designs and to estimate the consistency of the generalisations to a universe or domain of skills. CONCLUSIONS: Reliability is a major source of validity evidence for assessments. Low reliability indicates that large variations in scores can be expected upon retesting. Inconsistent assessment scores are difficult or impossible to interpret meaningfully and thus reduce validity evidence. Reliability coefficients allow the quantification and estimation of the random errors of measurement in assessments, such that overall assessment can be improved.     

The prediction of professional behaviour

Objective The purpose of this study was to establish outcome measures for professionalism in medical students and to identify predictors of these outcomes. Design Retrospective cohort study. Setting A US medical school. Participants All students entering in 1995 and graduating within 5 years. Measures Outcome measures included review board identification of professionalism problems and clerkship evaluations for items pertaining to professionalism. Pre-clinical predictor variables included material from the admissions application, completion of required course evaluations, students' self-reporting of immunisation compliance, students' performance on standardised patient (SP) exercises, and students' self-assessed performance on SP exercises. Results The outcome measures of clerkship professionalism scores were found to be highly reliable (alpha 0.88–0.96). No data from the admissions material was found to be predictive of professional behaviour in the clinical years. Using multivariate regression, failing to complete required course evaluations (B = 0.23) and failing to report immunisation compliance (B = 0.29) were significant predictors of unprofessional behaviour found by the review board in subsequent years. Immunisation non-compliance predicted low overall clerkship professional evaluation scores (B = − 0.34). Student self-assessment accuracy (SP score minus self-assessed score) (B = 0.03) and immunisation non-compliance (B = 0.54) predicted the internal medicine clerkship professionalism score. Conclusions This study identifies a set of reliable, context-bound outcome measures in professionalism. Although we searched for predictors of behaviour in the admissions application and other domains commonly felt to be predictive of professionalism, we found significant predictors only in domains where students had had opportunities to demonstrate conscientious behaviour or humility in self-assessment.     

Qualitative assessment of the long-term impact of a faculty development programme in teaching skills

CONTEXT: The long-term impact of faculty development programmes (FDPs) is poorly understood, and most assessments of them have been quantitative in nature. OBJECTIVE: This study aimed to use qualitative methods to better understand the long-term impact of an FDP in teaching skills (FDP/TS). METHODS: A survey was carried out in July 2002 of the 242 faculty members and fellows who had participated in a 9-month FDP/TS at any time from 1987 through 2000. The survey included 2 quantitative questions and an open-ended qualitative question about the impact of the programme on the participants' professional and personal lives. RESULTS: A total of 200 past participants (83%) responded to the survey. Participants from early and recent cohorts were similarly represented. In all, 82% of respondents said programme participation had had 'a moderate' or 'a lot' of impact on their professional life, and 49% said their personal life had been affected to this degree. Four major domains, each containing at least 3 subcategories, emerged from qualitative analysis. The domain intrapersonal development included changes participants reported in themselves and in their approach to self-management. Interpersonal development contained subcategories relating to how participants interact with others. Subcategories in the domain development as a teacher related to increased teaching ability and enjoyment. The domain career development included professional growth and career opportunities attributed to programme participation. CONCLUSIONS: Longitudinal FDPs can have broad and sustained positive effects on the professional and personal lives of participants. Qualitative evaluation methods may result in a richer and deeper understanding of the impact of these programmes.     

Setting school-level outcome standards

BACKGROUND: To establish international standards for medical schools, an appropriate panel of experts must decide on performance standards. A pilot test of such standards was set in the context of a multidimensional (multiple-choice question examination, objective structured clinical examination, faculty observation) examination at 8 leading schools in China. METHODS: A group of 16 medical education leaders from a broad array of countries met over a 3-day period. These individuals considered competency domains, examination items, and the percentage of students who could fall below a cut-off score if the school was still to be considered as meeting competencies. This 2-step process started with a discussion of the borderline school and the relative difficulty of a borderline school in achieving acceptable standards in a given competency domain. Committee members then estimated the percentage of students falling below the standard that is tolerable at a borderline school and were allowed to revise their ratings after viewing pilot data. RESULTS: Tolerable failure rates ranged from 10% to 26% across competency domains and examination types. As with other standard-setting exercises, standard deviations from initial to final estimates of the tolerable failure rates fell, but the cut-off scores did not change significantly. Final, but not initial cut-off scores were correlated with student failure rates (r = 0.59, P = 0.03). DISCUSSION: This paper describes a method to set school-level outcome standards at an international level based on prior established standard-setting methods. Further refinement of this process and validation using other examinations in other countries will be needed to achieve accurate international standards.     

The stability of maternal ratings of medical student interviews

INTRODUCTION: Little information is available on the intrarater reliability of parent ratings of medical student interview skills. The aim of this study was to compare maternal ratings of a videotaped medical student interview, with ratings by the same mother of the same interview seen a month later. METHOD: Thirty mothers rated 1 of 2 'medical student' interview videotapes (random allocation). An actor played the role of the student and demonstrated different student skill levels in each interview. Mothers rated the same interview again a month later (mean +/- SD 34 +/- 13 days). Maternal satisfaction was measured using the Medical Interview Satisfaction Scale (MISS, maximum score 203) and the Interpersonal Skills Rating Scale (IPS, maximum score 91) with higher scores indicating higher satisfaction. RESULTS: No significant differences in mean maternal ratings were observed following the first and second viewings of each interview. The average percentage difference in first and second viewing scores was MISS 4%, IPS 8% with correlation coefficients MISS 0.96, IPS 0.87 (P < 0.05). CONCLUSIONS: The high levels of stability in maternal evaluations of simulated medical student interviews supports the use of maternal evaluations for longitudinal monitoring of medical student performance.     

What do students actually do on an internal medicine clerkship? A log diary study

BACKGROUND: There are limited data on the amount of time students spend on teaching and learning while on internal medicine clerkships, and existing data suggest a wide international variation. Community-based teaching of internal medicine is now widespread; but its strengths and weaknesses compared to traditional hospital based teaching are still unclear. AIM: To determine the proportion of time students spend on different activities on an internal medicine clerkship, and to determine whether this differs in general practice and in hospital. In addition we aimed to determine students' views on the educational value and enjoyment of various activities. METHODS: Prospective completion of log diaries recording student activities. Each student was asked to complete the diary for two separate weeks of their internal medicine clerkship: one week of general practice-based teaching and one week of hospital-based teaching. RESULTS: The response rate was 68% (88/130). Students spent approximately 5.5 h per day on teaching and learning activities in both environments, with more time (50 min vs. 30 min, P = 0.007) on unsupervised interaction with patients in hospital than in general practice, and more time (53 min vs. 21 min, P < 0.001) undergoingassessment in general practice than in hospital. Standard deviations were wide, demonstrating the heterogeneous nature of the data. Students perceived supervised interaction with patients and teaching by doctors as the most educational activities in both environments, but found it even more educationally valuable and enjoyable in general practice than in hospital (mean score for educational value: 4.27 in general practice, 3.88 in hospital, P = 0.048; mean score for enjoyment 4.13 in general practice, 3.66 in hospital, P = 0.03). CONCLUSIONS: Students greatly value interactions with patients, perceiving these as both educational and enjoyable. Curriculum planners must continue to place patient-based learning at the centre of undergraduate medical education. The heterogeneity of the data suggests that individual students have very different experiences, despite apparently similar timetables.