Faculty development for teaching and evaluating professionalism: from programme design to curriculum change

INTRODUCTION: The recent emphasis on the teaching and evaluation of professionalism for medical students and residents has placed significant demands on medicine's educational institutions. The traditional method of transmitting professional values by role modelling is no longer adequate, and professionalism must be taught explicitly and evaluated effectively. However, many faculty members do not possess the requisite knowledge and skills to teach this content area and faculty development is therefore required. PROGRAMME DESCRIPTION: A systematic, integrated faculty development programme was designed to support the teaching and evaluation of professionalism at our institution. The programme consisted of think tanks to promote consensus and "buy-in", and workshops to convey core content, examine teaching strategies and evaluation methods, and promote reflection and self-awareness. PROGRAMME EVALUATION: The programme was evaluated using a CIPP (context, input, process, product) analysis. The institution supported this initiative and local expertise was available. A total of 152 faculty members, with key educational responsibilities, attended 1 or more faculty development activities. Faculty participation resulted in agreement on the cognitive base and attributes of professionalism, consensus on the importance of teaching and evaluating professionalism, and self-reported changes in teaching practices. This initiative also led to the development of new methods of evaluation, site-specific activities and curriculum change. DISCUSSION: A faculty development programme designed to support the teaching and evaluation of professionalism can lead to self-reported changes in teaching and practice as well as new educational initiatives. It can also help to develop more knowledgeable faculty members, who will, it is hoped, become more effective role models.     

The effect of professional skills training on patient-centredness and confidence in communicating with patients

OBJECTIVES: The effect of introducing professional skills training on students' patient-centred attitudes and perceptions of ability to communicate was examined. The professional skills training included weekly training in communication skills, ethics and law, and clinical skills. METHODS: Consecutive cohorts of medical students receiving a traditional pre-clinical curriculum (n = 199) and a new curriculum including professional skills training (n = 255) were compared. Students completed the Doctor-Patient Scale to assess patient-centred attitudes and an 11-item scale to assess confidence in their ability to communicate with patients. Students completed the measures at the start of Year 1 and the end of Year 2. RESULTS: Students receiving the professional skills training showed increased confidence in communicating with patients and increases in 2 dimensions of patient-centredness ('holistic care' and 'patient decision making'). Students receiving the traditional curriculum showed increased nervousness in talking to patients. Gender and ethnic differences were found in patient-centredness and confidence in communicating, which were maintained over time. CONCLUSIONS: The introduction of professional skills training was successful in improving students' confidence in their ability to perform specific communicative behaviours and increasing patient-centredness relative to a traditional curriculum.     

Early identification of 'at-risk' students by the parents of paediatric patients

INTRODUCTION: Assessment of medical student clinical skills is best carried out using multiple assessment methods. A programme was developed to obtain parent evaluations of medical student paediatric interview skills for feedback and to identify students at risk of poor performance in summative assessments. METHOD: A total of 130 parent evaluations were obtained for 67 students (parent participation 72%, student participation 58%). Parents completed a 13-item questionnaire [Interpersonal Skills Rating Scale (IPS) maximum score 91, higher scores = higher student skill level]. Students received their individual parent scores and de-identified class mean scores as feedback, and participants were surveyed regarding the programme. Parent evaluation scores were compared with student performance in formative and summative faculty assessments of clinical interview skills. RESULTS: Parents supported the programme and participating students valued parent feedback. Students with a parent score that was less than 1 standard deviation (SD) below the class mean (low IPS score students) obtained lower faculty summative assessment scores than did other students (mean +/- SD, 59% +/- 5 versus 64% +/- 7; P < 0.05). Obtaining 1 low IPS score was associated with a subsequent faculty summative assessment score below the class mean (sensitivity 0.38, specificity 0.88). Parent evaluations combined with faculty formative assessments identified 50% of students who subsequently performed below the class mean in summative assessments. CONCLUSIONS: Parent evaluations provided useful feedback to students and identified 1 group of students at increased risk of weaker performance in summative assessments. They could be combined with other methods of formative assessment to enhance screening procedures for clinically weak students.     

The 'Collaborative Care' curriculum: an educational model addressing key ACGME core competencies in primary care residency training

AIM: The 'Collaborative Care' curriculum is a 12-month senior resident class project in which one evidence-based clinical guideline is designed, implemented and evaluated in our residency practice. This curriculum specifically addresses three of the six Accreditation Council for Graduate Medical Education (ACGME) core competencies: Practice-Based Learning and Improvement, Interpersonal and Communication Skills and System-Based Practices. Additionally, the project enhances the quality of patient care within the model family practice centre in a family practice residency. METHODS: During the project, the third-year residency class selects the disease, develops the clinical guideline, leads its implementation and guides the evaluation process. Select faculty members serve as mentors and coach the resident class through each phase of the project. Specific educational objectives are developed for each content area: evidence-based medicine, clinical guideline development, continuous quality improvement and team leadership. A series of seminars are presented during the project year to provide 'just-in-time' learning for the key content and skills required for each step in the project. By working together to develop the practice guideline, then working with nurses and allied health staff to implement the guideline and review its effectiveness, the resident team gains competence in the areas of practice-based learning and improvement, interpersonal and communication skills and system-based practices. RESULTS: The self-reported level of resident confidence in skill acquisition for each content area was measured for each resident at the time of graduation from the residency programme. Results from the first 2 years of this curriculum are reported (resident n = 12), and demonstrate a high level of physician confidence in the skills addressed and their utility for future practice. CONCLUSIONS: The senior resident seminar and team project model reported here creates learning experiences that appear to address at least three of the ACGME general competency expectations: practice-based learning and improvement, interpersonal communication skills, and systems-based practice. From the initial resident feedback, this educational model seems to establish a high level of physician confidence in the skills addressed and their utility for future practice.     

Long-term follow-up of a 10-month programme in curriculum development for medical educators: a cohort study

Context  There is an ongoing need for curriculum development (CD) in medical education. However, only a minority of medical teaching institutions provide faculty development in CD. This study evaluates the long-term impact of a longitudinal programme in curriculum development.
Methods  We surveyed eight cohorts of participants ( n  =   64) and non-participants ( n  =   64) from 1988 to 1996 at baseline and at 6–13 years after completion of a 10-month, one half-day per week programme offered annually, which included a mentored CD project, workshops on CD steps, a final paper and a presentation.
Results  Fifty-eight participants (91%) and 50 non-participants (78%) returned completed follow-up surveys. In analyses, controlling for background characteristics and baseline self-rated proficiencies, participants were more likely than non-participants at follow-up to report having developed and implemented curricula in the past 5 years (65.5% versus 43.7%; odds ratio [OR] 2.41, 95% confidence interval [CI] 1.03–5.66), to report having performed needs assessment when planning a curriculum (86.1% versus 58.8%; OR 5.59, 95% CI 1.20–25.92), and to rate themselves highly in developing (OR 3.57, 95% CI 1.36–9.39), implementing (OR 3.04, 95% CI 1.16–7.93) and evaluating (OR 2.74, 95% CI 1.10–6.84) curricula. At follow-up, 86.2% of participants reported that the CD programme had made a moderate or great impact on their professional careers. Responses to an open-ended question on the impact confirmed continued involvement in CD work, confidence in CD skills, application of CD skills and knowledge beyond CD, improved time management, and lasting relationships formed because of the programme.
Conclusions  Our results suggest that a longitudinal faculty development programme that engages and supports faculty in real CD work can have long-lasting impact.     

Coming of age as communicators: differences in the implementation of common communications skills training in four residency programmes

OBJECTIVES: To determine thematic similarities and differences in the implementation of common-content communications skills training (CST) in medicine, surgery, paediatrics, and obstetrics and gynaecology residency programmes. METHODS: Communications skills training based upon the Kalamazoo consensus statement of communication skills in the clinical encounter was implemented in 4 residency programmes. Field notes of the CST sessions in each programme were analysed and coded for themes, considering the domains of Context, Input, Process and Product ('CIPP' methodology). Immediate learning outcomes were quantitatively assessed using retrospective pre/post methodology. RESULTS: Important differences were noted in the implementation of CST in the 4 disciplines. The 2 surgical disciplines showed relatively less reflective language and greater concentration on straight skill acquisition, whereas the 2 medical disciplines concentrated on the residents' role as teachers of communication skills for buy-in. Thematic similarities between disciplines included similar challenges to being good communicators in practice, as identified by residents (e.g. inadequate time and space), as well as lack of formal training. Quantitative learning outcome data from the educational intervention were significant in all groups (P < 0.05). CONCLUSIONS: Common material in CST can be adapted to different disciplines. By analysing for thematic similarities and differences in implementation in the 4 disciplines, a picture of different pedagogic 'subcultures' emerged, with different behavioural norms and values related to the doctor's role as communicator. In shared core competency training, it may be useful to consider these differences in planning, so that the training may be both sensitive to the behavioural norms of different disciplines, and effective.     

Assessing the relationship between peer and facilitator evaluations in case-based learning

OBJECTIVES: Attempts to validate peer evaluation and to incorporate it into the curriculum have met with mixed results. The purpose of this study was to assess the use of peer evaluations in a Year 1 case-based learning course. METHODS: As part of the formal grading process for the course, all faculty facilitators (n = 69 over 3 years) completed a 12-item evaluation form for each student at the conclusion of each case. As part of a course assignment, students (n = 415 over 3 years) completed brief evaluations of their peers based on 2 criteria: the overall quality of written reports, and participation in group discussion. In addition, students provided anonymous feedback in the written end-of-course evaluation about the peer evaluation process, and faculty were asked to comment during the wrap-up luncheon for small-group facilitators. RESULTS: Response rates for the 3 Year 1 medical student classes ranged from 95% to 99%. The average number of peer evaluations completed for each student was 4.6. The G coefficients for the rater-nested-within-person generalisability study were 0.52 for written reports and 0.60 for group participation; both were based on an average of 4-5 ratings. Correlation coefficients between peer and faculty evaluations in each of the 3 consecutive years of the course ranged from 0.46 to 0.63; all were statistically significant at P < 0.001. A correction for attenuation suggests that the true score correlation between faculty and peer measures is near 1.0. DISCUSSION: This study provides strong evidence that facilitator and peer ratings measure similar constructs and shows that, even among Year 1 medical students, peer evaluation can be conducted in a valid manner.     

Observed versus reported precepting skills: teaching behaviours in a community ambulatory clerkship

PURPOSE: Community-based doctors are increasingly utilised for the clinical education of medical students. Faculty development programmes are frequently provided. However, data about the community faculty-student teaching interaction and the longterm impact of such programmes are limited. This study observes community faculty-student teaching interactions and assesses the use of clinical teaching methods taught during faculty development workshops. METHODS: Between March and August 2002, 13 rural, urban and suburban community-based faculty who completed at least 1 faculty development workshop were observed during faculty-student-patient encounters and interviewed. Observer-interviewers utilised a checklist to record teaching styles, methods and skills, and administered a questionnaire to obtain community faculty's self-perceptions. RESULTS: On observation, the assertive precepting style and didactic teaching predominated in 90% and 86% of teaching encounters, respectively. Interactive discussion and direct observation occurred less often, in 71% and 18% of encounters, respectively. Feedback was provided in 25% of encounters. On interview, community faculty reported that faculty development workshops resulted in improved precepting effectiveness and professional satisfaction with moderate to considerable improvement in feedback, interactive teaching and use of multiple precepting styles for 100%, 77% and 77% of community faculty, respectively. Community faculty reported that their skills in addressing problem students, direct observation and didactic teaching improved moderately. There was little statistical correlation between self-assessed and observed skills. CONCLUSIONS: Community-based faculty rate their teaching skills as improved following faculty development. Observation reveals that interactive discussion, direct observation, feedback and problem management skills are still lacking.     

The first medical humanities programme in Turkey

THE MEDICAL HUMANITIES: The arts and humanities have been considered a recreational activity related to the interests and talents of the doctor, rather than to their practice of medicine. GOOD MEDICAL PRACTICE COURSE: At Hacettepe University, the 'good medical practice course' was added to the curriculum in September 2004. The goals of this course are to help the students achieve the skills and attitudes of a good doctor and to become competent in caring for and communicating with patients. IMPLEMENTATION: As a part of the programme, in the first 3 years students prepared 526 projects concerned with medical humanities in the broad areas of 'medicine and art', 'medicine and history' and 'man and medicine'. They presented 72 of the projects as short communications at the Medical Humanities Congress and the remaining projects as posters. Each project was also to be prepared as a portfolio. PROGRAMME EVALUATION: At the end of the programme, a questionnaire was given to both the students and the tutors to evaluate the course. The part of the programme rated most difficult was the medical humanities section, according to 67.1% of the students. They commented that it took a great deal of time to research and prepare the projects. PROPOSED CHANGES: The new concept and the extra workload, in addition to new medical knowledge, confused all the students. We have made some changes to the programme and have decided to develop a number of new activities. CONCLUSION: 'Medical humanities' is a new concept for almost all the students and the faculty; it needs to be well defined.     

The effect of an extended hospital induction on perceived confidence and assessed clinical skills of newly qualified pre-registration house officers

BACKGROUND: Recent studies raise concerns over the preparedness of newly qualified doctors for the role of the pre-registration house officer (PRHO). This study aimed to assess self-perception of preparedness, objective assessment of core clinical skills and the effect of an extended clinical induction programme prior to commencing full duties. METHODS: A group of 26 newly qualified doctors from 1 district general hospital underwent an extended 5-day, ward-based induction programme. The participants completed questionnaires on their own perceptions of their preparedness for PRHO duties and underwent an objective structured clinical examination (OSCE) of 4 core clinical skills prior to induction, on completion of induction and 1 month into working life. RESULTS: At the outset PRHOs had low perceptions of their own capabilities in all clinical scenarios and skills. Most perceptions improved after induction, although in 2 clinical areas they felt even less confident. One month into post there were significant improvements in all areas. Only 1 PRHO passed all 4 clinical skills assessments at the pre-induction assessment. Seven (26%) failed on 1 or more skills at the post-induction assessment. However, all participants were deemed competent in all skills at the 1-month assessment. CONCLUSION: Newly qualified doctors do not feel prepared for PRHO duties and objectively are not competent in basic clinical skills. An extended induction improves preparedness in some but not all clinical areas and improves performance of objectively assessed clinical skills.     

Effects of basic clinical skills training on objective structured clinical examination performance

OBJECTIVES: The aim of curriculum reform in medical education is to improve students' clinical and communication skills. However, there are contradicting results regarding the effectiveness of such reforms. METHODS: A study of internal medicine students was carried out using a static group design. The experimental group consisted of 77 students participating in 7 sessions of communication training, 7 sessions of skills-laboratory training and 7 sessions of bedside-teaching, each lasting 1.5 hours. The control group of 66 students from the traditional curriculum participated in equally as many sessions but was offered only bedside teaching. Students' cognitive and practical skills performance was assessed using Multiple Choice Question (MCQ) testing and an objective structured clinical examination (OSCE), delivered by examiners blind to group membership. RESULTS: The experimental group performed significantly better on the OSCE than did the control group (P < 0.01), whereas the groups did not differ on the MCQ test (P < 0.15). This indicates that specific training in communication and basic clinical skills enabled students to perform better in an OSCE, whereas its effects on knowledge did not differ from those of the traditional curriculum. CONCLUSION: Curriculum reform promoting communication and basic clinical skills are effective and lead to an improved performance in history taking and physical examination skills.     

Faculty opinions about a revised pre-clinical curriculum

INTRODUCTION: Medical schools having innovative curricula have been encouraged to ascertain the levels of satisfaction of faculty members with the curriculum. Faculty at schools that employ problem-based learning (PBL) have been shown to have positive perceptions, but not all schools are in a position to adopt PBL on a large scale. This study sought to determine faculty members' opinions about a new curriculum that is less ambitious than one utilizing true PBL. CONTEXT AND SETTING: Since 1997, the University of Otago Medical School (Dunedin, New Zealand) has had an integrated, modular pre-clinical curriculum that emphasizes clinical relevance. It has proved popular with students. This study focused on faculty members' impressions. METHODS: We surveyed faculty members' opinions with a questionnaire identical to one used in studies at PBL schools. Faculty compared the students and their own levels of satisfaction in the old and new curricula on 7 to 10 items. The overall response rate was 85.4% (152 of 178). RESULTS: Perceptions of the new curriculum were positive among teachers who taught during the pre-clinical years and those who taught the students only after they reached the clinical years. Results for individual questions were in the same direction and generally similar in magnitude to those reported on identical items for PBL. CONCLUSION: We conclude that a hybrid curriculum that is more acceptable to many traditional teachers and students than is PBL has almost as great a positive effect on faculty members' perceptions of students' abilities and of the curriculum as does PBL.     

Not knowing that they do not know: self-assessment accuracy of third-year medical students

PURPOSE: The development of self-regulated learning is a major focus of our problem-based learning (PBL) medical programme. Students who are unsuccessful in assessments often seem to lack insight into the standard of their own performance, yet the ability to self-assess accurately is essential for the effective self-management of learning. The aim of this project was to evaluate the accuracy of self- and peer-assessment according to academic performance. METHOD: In 2004, 175 3rd-year students undertook an integrated, case-based, short-essay, formative assessment. After the assessment they were provided with model answers and marking criteria. Students marked their own assessment paper and the paper of one of their peers. Assessment papers were subsequently marked by faculty members. The following data was available for each student: self-mark, faculty-mark, score awarded by a peer and the score that they awarded to their peer. Self-assessment and peer-assessment ability was compared to overall academic performance. RESULTS: Low-achieving students score themselves and their peers generously. High-achieving students score themselves more harshly than faculty. However, they score their peers accurately. CONCLUSION: In the 3rd year of the programme low-achieving students are unable to assess accurately the quality of their own work or the work of their peers in a formative written assessment. The PBL curriculum does not guarantee the appropriate development of self-assessment skills.     

Qualitative assessment of the long-term impact of a faculty development programme in teaching skills

CONTEXT: The long-term impact of faculty development programmes (FDPs) is poorly understood, and most assessments of them have been quantitative in nature. OBJECTIVE: This study aimed to use qualitative methods to better understand the long-term impact of an FDP in teaching skills (FDP/TS). METHODS: A survey was carried out in July 2002 of the 242 faculty members and fellows who had participated in a 9-month FDP/TS at any time from 1987 through 2000. The survey included 2 quantitative questions and an open-ended qualitative question about the impact of the programme on the participants' professional and personal lives. RESULTS: A total of 200 past participants (83%) responded to the survey. Participants from early and recent cohorts were similarly represented. In all, 82% of respondents said programme participation had had 'a moderate' or 'a lot' of impact on their professional life, and 49% said their personal life had been affected to this degree. Four major domains, each containing at least 3 subcategories, emerged from qualitative analysis. The domain intrapersonal development included changes participants reported in themselves and in their approach to self-management. Interpersonal development contained subcategories relating to how participants interact with others. Subcategories in the domain development as a teacher related to increased teaching ability and enjoyment. The domain career development included professional growth and career opportunities attributed to programme participation. CONCLUSIONS: Longitudinal FDPs can have broad and sustained positive effects on the professional and personal lives of participants. Qualitative evaluation methods may result in a richer and deeper understanding of the impact of these programmes.     

A comprehensive collaborative patient safety residency curriculum to address the ACGME core competencies

CONTEXT: Patient safety currently receives only scant attention in most residency curricula. Safety is a subject that transcends the US Accreditation Council for Graduate Medical Education's 6 core competencies. OBJECTIVE: To design and implement a new patient safety curriculum in collaboration with the Schools of Nursing and Pharmacy, in such a way as to address all 6 competencies. SETTING AND PARTICIPANTS: The curriculum applies to a university-based family medicine residency programme with 45 residents at 5 sites, including urban, suburban and rural sites. CURRICULUM DESIGN: The curriculum includes introductory workshops for faculty and residents, a series of didactic courses, individual portfolios and a series of small group exercises including chart reviews, case presentations and a longitudinal quality improvement project. The activities are run by a multidisciplinary team. OUTCOME MEASURES: Main outcome measures include assessment of resident performance in curriculum activities and in an annual objective structured clinical examination (OSCE) that includes standardised patient interviews, simulations and a written examination. Programme evaluation will include comparison of OSCE performance with that at a neighbouring residency. RESULTS: Residents identified safety problems and system-based solutions using a safety journal. Cases of polypharmacy were identified using journals and chart reviews, and medication changes proposed and discussed. At resident practice sites, residents identified safety priorities based on a staff survey and proposed system-based solutions. Results of the OSCE will be presented elsewhere. CONCLUSIONS: A new patient safety curriculum was successfully introduced into a family medicine residency. The curriculum integrates patient safety into residents' daily activities and incorporates input from the disciplines of nursing and pharmacy so as to help build more effective clinical teams and inculcate a culture of safety.     

Setting up a clinical skills learning facility

Objective  This paper outlines the considerations to be made when establishing a clinical skills learning facility.
Considerations  Establishing a clinical skills learning facility is a complex project with many possible options to be considered. A number of professional groups, undergraduate or postgraduate, may be users. Their collaboration can have benefits for funding, uses and promotion of interprofessional education. Best evidence and educational theory should underpin teaching and learning. The physical environment should be flexible to allow a range of clinical settings to be simulated and to facilitate a range of teaching and learning methods, supported by computing and audio-visual resources. Facilities should be available to encourage self-directed learning. The skills programme should be designed to support the intended learning outcomes and be integrated within the overall curriculum, including within the assessment strategy. Teaching staff may be configured in a number of ways and may be drawn from a variety of backgrounds. Appropriate staff development will be required to ensure consistency and quality of teaching with monitoring and evaluation to assure appropriate standards. Patients can also play a role, not only as passive teaching material, but also as teachers and assessors. Clinical, diagnostic and therapeutic equipment will be required, as will models and manikins. The latter will vary from simple part task trainers to highly sophisticated human patient simulators. Care must be taken when choosing equipment to ensure it matches specified requirements for teaching and learning.
Conclusion  Detailed planning is required across a number of domains when setting up a clinical skills learning facility.     

Achievement of non-cognitive goals of undergraduate medical education: perceptions of medical students, residents, faculty and other health professionals

BACKGROUND: Professionalism is increasingly emphasised in medical education. Non-cognitive goals, including values, attitudes and skills, remain challenging to define and measure. The purpose of this study was to better understand these goals and their achievement in the MD programme. METHODS: Graduating medical students, faculty preceptors, residents and other health professionals (OHPs) completed a systematically developed mailed survey, rating achievement of 25 attribute statements. Following analyses of means and standard deviations, factor analysis of responses was conducted. Responses were compared across respondent groups. RESULTS: The overall response rate was 50.1% (191/396), comprising 57.5% of the students, 54.1% of the faculty members, 30.9% of the residents and 50% of the OHPs. Five items received mean ratings over 4/5; none were below 3/5. Five factors explained 65% of variance. They were: 'Teamwork and interprofessional skills'; 'Duty and responsibility'; 'Communication and interpersonal skills'; 'Professionalism and values', and 'Trustworthiness and ethical behaviour'. The groups differed significantly on 2 factors: Teamwork and interprofessional skills (P < or = 0.0001) and Communication and interpersonal skills (P < or = 0.001). CONCLUSIONS: Important curriculum goals received high mean ratings. Ratings differed significantly across groups, suggesting differing perceptions of the extent to which goals were met. More study is needed to understand the basis of these perceptions.     

Humanising medical education through faculty development: linking self-awareness and teaching skills

BACKGROUND: We conducted a longitudinal faculty development programme for medical school faculty, focused on enhancing learner-centred teaching skills, by integrating traditional elements of education, focusing on knowledge, skills and attitudes, with the non-traditional process elements of community building, self-awareness and relationship formation. METHODS: This year-long programme enrolled faculty from a range of clinical departments at a single institution. The participants gathered for day-long sessions in each of 9 months and also met at lunchtime once a month for "booster" meetings. Sessions were organised according to content areas identified as relevant to enhancing teaching skills. In addition to each content area, a variety of experiential and process learning methods were incorporated and modelled in each session. The impact of the programme was assessed using self-report data after each session, and together with paired interviews at the completion of the programme. EVALUATION: Programme attendance was 97%. Qualitative thematic and content analysis showed that faculty reported renewed energy and enthusiasm for teaching, as well as skills mastery. Faculty also reported significant improvements in self-awareness and habits of lifelong learning. Many faculty members felt that a singular benefit of the programme was developing and nurturing personal as opposed to instrumental relationships. CONCLUSIONS: It is feasible to successfully convene clinical faculty from different departments in the same faculty development programme with little or no concern for competition and conflict. The key to success is integrating content and process dimensions into a framework of community building and collective engagement. From the participants' perspective, skills and confidence, which by self-report increased for all participants, was less consequential than the opportunity to learn about themselves and their relationships to others in a safe environment. We have conducted this programme for 5 successive years.