Conducting the symphony: a qualitative study of facilitation in problem-based learning tutorials

Context  Tutors in problem-based learning (PBL) tutorials have a complex role to play in facilitating students' learning. This includes providing support for students' acquisition of content knowledge and skills in critical thinking, coaching of group processes and modelling of reflective practice. Few studies which investigate the key role of tutors in the PBL tutorial process are qualitative in design.
Methods  This study explores the nature and technique of facilitation provided by PBL tutors from the students' viewpoint. Data were obtained from written responses to an open-ended question asking students about the effectiveness of their PBL tutor(s) and from in-depth interviews carried out with two randomly selected students.
Results  Three main themes arose from the inductive analysis of qualitative data: (i) role confusion by tutors; (ii) tutor management of sensitive issues, and (iii) facilitation 'style'. The theme of tutors' facilitation style was dominant and three sub-categories were apparent. These were: (i) managing the learning in PBL tutorials; (ii) facilitating group processes, and (iii) guiding group discussion.
Conclusions  Findings highlight the need for tutors to regularly review the PBL tutorial processes and group dynamics within the tutorial setting. These findings have implications for tutor training and programmes of ongoing professional development for PBL facilitators.     

An exploration of perceptions of tutor evaluation in problem‐based learning tutorials

Medical Education 2010: 44 : 892–899 Context Within problem‐based learning (PBL) tutorials, the relationship between student and tutor is predicated on the tutor adopting the role of mentor and metacognitive coach. This rapport differs considerably from the traditional teacher–student relationship and is likely to impact on the process and outcomes of tutor evaluations. Such evaluations are a ubiquitous means of providing feedback to tutors from students about the quality of their facilitation. Although critiqued in the literature as ‘popularity contests’, tutor evaluations are commonly used in tertiary institutions for purposes of recruitment, re‐employment and promotion. Methods This study seeks to provide insight into students’ and tutors’ perceptions of evaluations of teaching within PBL tutorials. As a unique teaching and learning environment, the PBL tutorial requires sophisticated facilitation skills of tutors and considerable autonomy from students. Qualitative data were gathered from three focus group discussions and one in‐depth interview with first‐ and second‐year medical students and their PBL tutors. Results Thematic analysis identified four major themes, including: defining the ‘ideal’ tutor; making unthinking or deliberately false evaluations; promoting a consumer mentality, and providing support for tutors. An underlying suspicion of the purpose of the evaluation process was apparent among tutors and students. Conclusions Findings suggest that, within the PBL tutorial environment at least, regularly evaluating tutors creates mistrust and confusion among the medical school, the tutor and the student on several levels. Suggestions for further research are proposed.     

Difficult incidents and tutor interventions in problem‐based learning tutorials

Context Tutors report difficult incidents and distressing conflicts that adversely affect learning in their problem‐based learning (PBL) groups. Faculty development (training) and peer support should help them to manage this. Yet our understanding of these problems and how to deal with them often seems inadequate to help tutors. Objectives The aim of this study was to categorise difficult incidents and the interventions that skilled tutors used in response, and to determine the effectiveness of those responses. Methods Thirty experienced and highly rated tutors in our Year 1 and 2 medical curriculum took part in semi‐structured interviews to: identify and describe difficult incidents; describe how they responded, and assess the success of each response. Recorded and transcribed data were analysed thematically to develop typologies of difficult incidents and interventions and compare reported success or failure. Results The 94 reported difficult incidents belonged to the broad categories ‘individual student’ or ‘group dynamics’. Tutors described 142 interventions in response to these difficult incidents, categorised as: (i) tutor intervenes during tutorial; (ii) tutor gives feedback outside tutorial, or (iii) student or group intervenes. Incidents in the ‘individual student’ category were addressed relatively unsuccessfully (effective < 50% of the time) by response (i), but with moderate success by response (ii) and successfully (> 75% of the time) by response (iii). None of the interventions worked well when used in response to problems related to ‘group dynamics’. Overall, 59% of the difficult incidents were dealt with successfully. Conclusions Dysfunctional PBL groups can be highly challenging, even for experienced and skilled tutors. Within‐tutorial feedback, the treatment that tutors are most frequently advised to apply, was often not effective. Our study suggests that the collective responsibility of the group, rather than of the tutor, to deal with these difficulties should be emphasised.     

Profiles of effective tutors in problem-based learning: scaffolding student learning

OBJECTIVES: Research on tutoring in problem-based learning has not focused so far on the variation in tutoring and how this variation can be interpreted by conceptions about effective tutoring. DESIGN: This study focuses on the profiles of tutors generated by means of an instrument, the so-called Tutor Intervention Profile (TIP), and tries to determine which profiles are more or less effective. The TIP contains four dimensions of tutor behaviour: (1) elaboration; (2) directing the learning process; (3) integration of knowledge; and (4) stimulating interaction and individual accountability. SETTING: The medical school of the University of Maastricht, The Netherlands. SUBJECTS: Sixty-seven tutors who run 67 tutorial groups across three units (courses) in the academic year 1996-97. RESULTS: It appeared that high, average and low performing tutors differ in their performance on each of the four dimensions of the TIP. Several different profiles of tutor performance could be distinguished, which were more or less effective. One group of tutors demonstrated a tutor intervention profile that was characterized as relying more on the use of expert knowledge, whereas another group of tutors was characterized as relying more on their abilities to stimulate the learning process in the tutorial group. The tutor intervention profile that was perceived by students as most effective showed high scores on each of the four dimensions, as expected. Notably, a tutor stressing the learning process in the tutorial group was perceived as more effective than a tutor stressing content (expert tutor). This is especially true for a relatively poor scoring tutor. CONCLUSIONS: The results of this study are consistent with research on human tutoring and research on tutoring in problem-based learning.     

Problem-Based Learning as an Effective Learning Tool in Community Medicine: Initiative in a Private Medical College of a Developing Country

Background:Knowledge of community medicine is essential for health care professionals to function as efficient primary health care physicians. Medical students learning Community Medicine as a subject are expected to be competent in critical thinking and generic skills so as to analyze community health problems better. However, current teaching by didactic lectures fails to develop these essential skills. Problem-based learning (PBL) could be an effective strategy in this respect. This study was hence done to compare the academic performance of students who were taught Community Medicine by the PBL method with that of students taught by traditional methods, to assess the generic skills of students taught in a PBL environment and to assess the perception of students toward PBL methodology.Results:A total of 77 students took part in the brainstorming session of PBL. The correlation between self-assigned scores of the participants and those assigned by the tutor in the brainstorming session of PBL was significant (r = 0.266, P = 0.05). Out of 54 students who took part in the presentation session, almost all 53 (98.1%) had good perception toward PBL. Demotivational scores were found to be significantly higher among males (P = 0.024). The academic performance of students (P < 0.001) and success rates (P = 0.05) in the examination were higher among students who took part in PBL compared to controls.Conclusion:PBL helped improve knowledge of students in comparison to those exposed only to didactic lectures. As PBL enabled students to identify the gaps in their knowledge and enhanced their group functioning and generic skills, we recommend PBL sessions: They would help optimize the training in Community Medicine at medical schools. Good correlation of tutor and self-assessment scores of participants in the brainstorming session suggests that the role of tutors could be restricted to assessment in presentation sessions alone. Demotivation, which hinders group performance in PBL, needs to be corrected by counselling and timely feedback by the tutors.     

Techniques used by "expert" and "non-expert" tutors to facilitate problem-based learning tutorials in an undergraduate medical curriculum

BACKGROUND: There is inconclusive debate within the literature as to whether the best problem-based learning (PBL) tutors are subject experts or not. The debate hinges on whether knowledgeable tutors are tempted to intervene too often in PBL discussions compared to non-expert tutors, and whether the latter may not be able to sufficiently challenge the students' level of understanding. PURPOSE: To describe approaches used by tutors in PBL tutorials and to identify differences between tutors from medical and non-medical backgrounds. METHODS: The research reported in this paper was undertaken during the academic session 1999-2000 at the University of Liverpool Faculty of Medicine. A qualitative exploratory case study method was used and two PBL groups were observed. One of these groups had a medically qualified tutor and the other had a tutor from a humanities background. The focus of the observation was the discourse between tutor and students, which was analysed using a framework drawn from linguistics. Results were fed back to both the tutors and the students to check their perceptions of the interactions. RESULTS: Analysis of the tutorial group interaction revealed that tutors from both backgrounds used similar techniques to raise students' awareness, facilitate the group process and direct students' learning. Differences were noted between the two tutors: the medical tutor set out to raise students' awareness by using questioning techniques herself, whereas the non-medical tutor expected students to question each other. The non-medical tutor was observed to facilitate the group process more often than the medical tutor. CONCLUSIONS: Qualitative analysis of spoken discourse in PBL tutorials provides valuable insights into the processes involved in PBL, thereby generating material which is useful for both training of and giving feedback to PBL tutors.     

Tutor intervention profile: reliability and validity

An instrument has been developed to provide tutors with feedback about their performance in problem-based tutorial groups. This questionnaire consists of 33 statements reflecting the behaviour of the tutor in the tutorial group when students analyse the case and generate learning issues and the behaviour of the tutor when students report back to the tutorial group about their self-study. The statements are grouped on theoretical dimensions found in the literature about co-operative learning and problem-based learning. This study focuses on the validity and the reliability of this instrument. Confirmatory factor analysis showed that a four-factor model showed a weak fit, whereas each separate factor fitted the data reasonably well. The four factors are: (1) elaboration; (2) directing the learning process; (3) integration of knowledge and (4) stimulating interaction and individual accountability. These factors were strongly correlated with the overall judgement of the tutor at the end of the unit, so the concurrent validity of this instrument was high. Generalizability studies indicated that the rating scales provide reliable information with a sample size of student responses falling within the range of real tutorial group sizes. The results of this study show that the instrument for providing tutors with feedback is valid and reliable. This questionnaire can be used in staff development activities and also provides needs assessment for faculty development.     

One Year Effects of a Workplace Integrated Care Intervention for Workers with Rheumatoid Arthritis: Results of a Randomized Controlled Trial

Purpose To evaluate the effectiveness of a workplace integrated care intervention on at-work productivity loss in workers with rheumatoid arthritis (RA) compared to usual care. Methods In this randomized controlled trial, 150 workers with RA were randomized into either the intervention or control group. The intervention group received an integrated care and participatory workplace intervention. Outcome measures were the Work Limitations Questionnaire, Work Instability Scale for RA, pain, fatigue and quality of life (RAND 36). Participants filled out a questionnaire at baseline, and after 6 and 12 months. We performed linear mixed models to analyse the outcomes. Results Participants were on average 50 years of age, and mostly female. After 12 months, no significant intervention effect was found on at-work productivity loss. We also found no significant intervention effects on any of the secondary outcomes. Conclusions We did not find evidence for the effectiveness of our workplace integrated care intervention after 12 months of follow up. Future studies should focus on investigating the intervention in groups of workers with severe limitations in work functioning, and an unstable work situation.     

Impact of orthodontic treatment on adolescents’ quality of life: a longitudinal evaluation of treated and untreated individuals

Purpose

To assess the impact of fixed appliance therapy on the quality of life of a cohort of Brazilian adolescents.

Methods

The treatment group was composed of individuals who started orthodontic treatment. The control group was composed of individuals not selected for immediate treatment. Adolescents answered the Brazilian short form of the Child Perceptions Questionnaire (CPQ_11–14). Higher scores indicate a greater negative impact on quality of life. Adolescents in the treatment group completed four sets of interviews: prior to appliances’ bonding (T1), 1 month (T2), 6 months (T3) and 12 months (T4) after appliances’ placement. Adolescents in the control group were assigned to a comparable schedule. Statistics included inter-group comparisons, Friedman and Wilcoxon tests, and mixed-effects models.

Results

In the treatment group, the functional limitation score was higher at T1 (p?=?0.004) and T2 (p?=?0.007) compared to T4. The emotional well-being score was higher at T1 compared to T2 (p?<?0.001), T3 (p?<?0.001) and T4 (p?<?0.001). The overall CPQ_11–14 score was higher at T1 compared to T2 (p?=?0.005), T3 (p?<?0.001) and T4 (p?<?0.001). The overall CPQ_11–14 score was also higher at T2 compared to T3 (p?=?0.001). No significant change was found in the control group. In the mixed-effects models, the interaction between group (treatment) and time was significant for functional limitations (p?<?0.001), emotional well-being (p?<?0.001), social well-being (p?=?0.004) and the overall CPQ_11–14 score (p?<?0.001).

Conclusion

Quality of life improved among adolescents undergoing orthodontic treatment. Orthodontists should inform patients what they might expect regarding the physical and the psychosocial repercussions of appliance therapy.

     

The Effect of Paid Leave on Maternal Mental Health

Objectives I examined the relationship between paid maternity leave and maternal mental health among women returning to work within 12 weeks of childbirth, after 12 weeks, and those returning specifically to full-time work within 12 weeks of giving birth. Methods I used data from 3850 women who worked full-time before childbirth from the Early Childhood Longitudinal Study—Birth Cohort. I utilized propensity score matching techniques to address selection bias. Mental health was measured using the Center for Epidemiologic Studies Depression (CESD) scale, with high scores indicating greater depressive symptoms. Results Returning to work after giving birth provided psychological benefits to women who used to work full-time before childbirth. The average CESD score of women who returned to work was 0.15 standard deviation (p?<?0.01) lower than the average CESD score of all women who worked full-time before giving birth. Shorter leave, on the other hand, was associated with adverse effects on mental health. The average CESD score of women who returned within 12 weeks of giving birth was 0.13 standard deviation higher (p?<?0.05) than the average CESD score of all women who rejoined labor market within 9 months of giving birth. However, receipt of paid leave was associated with an improved mental health outcome. Among all women who returned to work within 12 weeks of childbirth, those women who received some paid leave had a 0.17 standard deviation (p?<?0.05) lower CESD score than the average CESD score. The result was stronger for women who returned to full-time work within 12 weeks of giving birth, with a 0.32 standard deviation (p?<?0.01) lower CESD score than the average CESD score. Conclusions The study revealed that the negative psychological effect of early return to work after giving birth was alleviated when women received paid leave.     

Vision-related quality of life and psychological status in Chinese women with Sjogren’s syndrome dry eye: a case-control study

Background

Sjogren’s syndrome dry eye (SSDE) mainly affects middle-aged women and can negatively affect women’s psychological and social functioning. However, little is known about the correlation between vision-related quality of life (VR-QoL) and psychological status for women with SSDE. We therefore examined VR-QoL and psychological status in two groups of Chinese women: an SSDE group and a non-SSDE group. We also explored the associations between VR-QoL scores, sociodemographic measures, ophthalmologic parameters, and psychological status in women with SSDE.

Methods

The case-control study recruited 30 female outpatients with SSDE and 30 without SSDE from the Eye and Ear, Nose, and Throat (ENT) Hospital of Fudan University. Demographic and ophthalmologic data were collected from all participants. Ophthalmologic examinations included best-corrected visual acuity (BCVA), corneal fluorescein staining (CFS), tear break-up time (TBUT) and Schirmer test. Data collected using the National Eye Institute’s Visual Function Questionnaire (NEI-VFQ) and Ocular Surface Disease Index (OSDI) survey instruments were analyzed to identify potential differences in VR-QoL between the SSDE group and the non-SSDE group. We also used the Zung Self-Rating Anxiety and Self-Rating Depression Scales (SAS and SDS) to determine psychological status in both groups.

Results

The SSDE group scored significantly lower than the non-SSDE group on the NEI-VFQ subscales of general health, general vision, and long-distance vision activities (all p?<?0.05). The SSDE group achieved a significantly higher ocular symptoms score compared with the control group (p?=?0.0256). The SAS and SDS scores of the SSDE group were significantly higher than the non-SSDE group (p?=?0.0072 and 0.0162, respectively). The prevalence of anxiety and depression in the SSDE group was significantly higher than the non-SSDE group (p?=?0.0240 and 0.0200, respectively). Nine of twelve NEI-VFQ subscales were negatively correlated with SAS/SDS scores (all p values were <0.05). The exceptions were social function, color vision and peripheral vision. The composite OSDI score and its three subscale scores for the women in the SSDE group were all positively correlated with overall SAS/SDS scores (all p values were <0.05).

Conclusions

Both VR-QoL and psychological status were significantly worse in SSDE group than in the non-SSDE group. The VR-QoL of women with SSDE had a negative correlation with their anxiety and depression levels.

     

The effect of longterm vs shortterm tutors on the quality of the tutorial process and student performance

Small-group, problem- based learning can require a significant amount of individual faculty time when groups last as long as 10 weeks. One solution is to use two short-term (3–5 weeks) tutors instead of a single long-term (6–10 weeks) tutor. This study was performed to evaluate whether having short-term instead of a long-term tutors affected student performance and/or the quality of the tutorial process. The grade point averages for first and second year medical students in the classes of 2001 and 2002 were stratified by how many long-term tutors students had (2, 3, 4, 5, or 6) over a period of 17 months. In addition, students completed anonymously a 24-item tutor evaluation questionnaire utilizing a 5 point Likert scale (outstanding, good, satisfactory, marginal or unsatisfactory) at the mid-point and at the end of each tutorial. The evaluations were stratified by whether the tutor was short-term or long-term (to the mid-point or end of the tutorial, respectively). Statistical analysis was performed using analysis of variance. Grade point averages did not correlate with the number of long-term tutors that students had. Long-term tutors were rated significantly higher than short-term tutors in 17 of the 24 categories evaluated by the questionnaire, including development of students' critical thinking, information skills and self-learning skills; encouraging students to express themselves freely and to take responsibility for their own learning; helping students to balance basic science and clinical applications; and overall assessment. Short-term vs long-term tutors do not affect objective student performance as measured by grade point average. However, long-term tutors are ranked significantly higher by students innumerous areas related to the perceived quality of the tutorial process. Short-term tutoring may be less of a demand on an individual faculty's time, but may lessen the students' perceived quality of the tutorial process. This revised version was published online in September 2006 with corrections to the Cover Date.     

The Effectiveness of a Physical Activity Educational Campaign in a Rural Obstetrics and Gynecology Office

Objectives The objective of this study was to evaluate the effectiveness of an early intervention health education campaign to positively influence physical activity (PA) knowledge, intention, and performance among prenatal women and women of reproductive age. Methods This study employed a quantitative, quasi-experimental, control-group comparison design with nonprobability sampling methodology. Implemented in rural healthcare settings located in the Southeastern portion of the United States, participants included prenatal patients and patients of reproductive age (n = 325) from two separate obstetrics and gynecology (OB/GYN) offices. While the intervention group was solicited from an OB/GYN office where the information-based health education campaign was implemented, the comparison group was solicited from a comparable OB/GYN office that did not implement the health education campaign. Results The women exposed to the PA health education campaign were significantly more likely to report that PA information was provided at their physician’s office, scored higher on PA knowledge, and were more likely to meet the guidelines for vigorous PA and strength training (p < 0.05). Conclusions Physical activity educational campaigns are a cost effective intervention that can be implemented in healthcare settings to promote maternal and child health.     

Effectiveness and Cost-benefit Evaluation of a Comprehensive Workers’ Health Surveillance Program for Sustainable Employability of Meat Processing Workers

Objective To evaluate the effectiveness of a comprehensive workers’ health surveillance (WHS) program on aspects of sustainable employability and cost-benefit. Methods A cluster randomized stepped wedge trial was performed in a Dutch meat processing company from february 2012 until march 2015. In total 305 workers participated in the trial. Outcomes were retrieved during a WHS program, by multiple questionnaires, and from company registries. Primary outcomes were sickness absence, work ability, and productivity. Secondary outcomes were health, vitality, and psychosocial workload. Data were analyzed with linear and logistic multilevel models. Cost-benefit analyses from the employer’s perspective were performed as well. Results Primary outcomes sickness absence (OR?=?1.40), work ability (B?=??0.63) and productivity (OR?=?0.71) were better in the control condition. Secondary outcomes did not or minimally differ between conditions. Of the 12 secondary outcomes, the only outcome that scored better in the experimental condition was meaning of work (B?=?0.18). Controlling for confounders did not or minimally change the results. However, our stepped wedge design did not enable adjustment for confounding in the last two periods of the trial. The WHS program resulted in higher costs for the employer on the short and middle term. Conclusions Primary outcomes did not improve after program implementation and secondary outcomes remained equal after implementation. The program was not cost-beneficial after 1–3 year follow-up. Main limitation that may have contributed to absence of positive effects may be program failure, because interventions were not deployed as intended.     

The Effectiveness of an Intervention to Enhance Cooperation Between Sick-Listed Employees and Their Supervisors (COSS)

Introduction Early return-to-work (RTW) after sick leave is considered to support employees’ quality of life. Successful RTW requires adequate cooperation between absent employees and their supervisors. This study assesses the effectiveness of an intervention for COoperation regarding RTW between Sick-listed employees and their Supervisors (COSS; i.e. ‘conversation roadmap’, monitoring of cooperation and, if necessary, extra occupational physician support). Methods In this field study, employees on sick leave for 2–10 weeks, aged 18 up to and including 60, and performing paid labour for at least 12 h per week were included. Terminally ill were excluded. Multivariate regression (correcting for baseline quality of life) was used to compare 6-months follow up data regarding quality of life between the groups. Using Cox regression analyses, time until first-, full-, and sustainable RTW was compared between groups. Results In total 64 employees received COSS or common practice. No significant group differences were found regarding all study outcomes. The COSS group had a higher chance of work resumption than the common practice group. The hazard ratio was 1.39 for first RTW (95 % CI 0.81–2.37), 1.12 for full RTW (95 % CI 0.65–1.93) and 1.10 for sustainable RTW (95 % CI 0.63–1.95). Conclusions COSS has no significant effects. Yet, the results regarding work resumption show a tendency towards effectiveness. Therefore, COSS can be further developed and applied in practice. Researchers should try to prevent some limitations of the present study in future research, for instance by finding a more common research setting.     

Student perceptions of tutor skills in problem-based learning tutorials

OBJECTIVE: The problem-based learning (PBL) tutor plays a role that is different from the role of a teacher in a conventional teaching format. In the Faculty of Medicine and Health Sciences, United Arab Emirates, all students are Arab nationals and tutors are expatriates with different sociocultural backgrounds from the students. This study was designed to investigate how students evaluate tutors in PBL tutorials and whether student evaluations of tutors change with the progress of students in PBL tutorials. METHODS: Differences in tutor performance evaluation by male and female students were also analysed. The students evaluated 12 tutor skills in a scale of 1-3, 1 being 'below average' and 3, 'outstanding'. Student responses from a total of 314 (98.1%) completed forms collected over 2 academic years were analysed statistically. A total of 14 tutors participated in the PBL programme. RESULTS: The analysis revealed that tutors as a group were rated as having average to outstanding tutor skills in 10 items of the evaluation form. Students and faculty perceptions were different for the tutor skills of guiding students for information management. The students expected more support from tutors, whereas the tutors tried to emphasize self-learning in the PBL curriculum. Lower scores to the tutors in the 'problem' bringing sociocultural and religious issues for discussion showed that a gap in sociocultural/religious understanding between students and tutors might influence tutor skills. CONCLUSIONS: Differences in tutor evaluation by male and female students indicate necessity of adopting different strategies by tutors in a different sociocultural background. The results of the study have direct implications for faculty development.     

The Influence of Neighborhood Crime on Increases in Physical Activity during a Pilot Physical Activity Intervention in Children

The purpose of this study was to examine whether neighborhood crime moderated the response (increases in steps) to a pilot intervention to increase physical activity in children. Twenty-seven insufficiently active children aged 6–10 years (mean age?=?8.7 years; 56 % female; 59 % African American) were randomly assigned to an intensive intervention group (IIG) or minimal intervention group (MIG). Change in average daily number of steps from baseline was regressed against an index of neighborhood crime in a multilevel repeated-measures model that included a propensity score to reduce confounding. Safer neighborhoods were associated with higher increases in steps during the pilot intervention (interaction p?=?0.008). Children in the IIG living in low-crime neighborhoods significantly increased their physical activity (5275?±?1040 steps/day) while those living in high-crime neighborhoods did not (1118?±?1007) (p for difference?=?0.046). In the IIG, the increase in daily steps was highly correlated with neighborhood crime (r?=?0.58, p?=?0.04). These findings suggest the need for physical activity interventions to account for participants’ environments in their design and/or delivery. To promote healthy behaviors in less-supportive environments, future studies should seek to understand how environments modify intervention response and to identify mediators of the relationship between environment and intervention.     

Tutoring in problem-based learning: perceptions of teachers and students

This study examined teacher and student perceptions during the first 2 years of a complete transition from a conventional to a problem-based learning (PBL) curriculum at Dalhousie University. Teaching staff who had tutored in the two pre-clinical years (   n = 88  ) completed a questionnaire at the end of the 1993–94 academic year, and student assessments of their tutors were collated for all nine units (   n = 597  ).
Seven research questions were addressed in the study which examined the faculty, student and administrative aspects of tutoring. The results showed that faculty tutors rated PBL more highly than traditional medical school methods on eight of the nine items. Teaching staff were very satisfied with their tutoring experience, but expressed a need for further training in group facilitation, questioning, handling 'difficult' situations and evaluating students. They reported that their workload outside tutorials was cut almost in half in their second year of tutoring. Students expected a tutor to be a skilled group facilitator who would guide them in their learning, while helping to maintain a positive group climate. They did not want the tutor to teach the content as they perceived the task of learning to be their responsibility.
Several major administrative factors affected tutors' and students' perceptions of tutorials, including: changing tutorial group composition and tutor every 8–10 weeks; team tutoring; end-of-unit exam; conflicting demands of basic science 'vertical' units and ongoing 'horizontal' units; departmental budgetary requirements for basic medical education; recognition of tutoring in promotion and tenure decisions; and recruitment of tutors.     

Validation of a performance assessment instrument in problem-based learning tutorials using two cohorts of medical students

Although problem-based learning (PBL) has been widely used in medical schools, few studies have attended to the assessment of PBL processes using validated instruments. This study examined reliability and validity for an instrument assessing PBL performance in four domains: Problem Solving, Use of Information, Group Process, and Professionalism. Two cohorts of medical students (N = 310) participated in the study, with 2 years of PBL evaluation data extracted from archive rated by a total of 158 faculty raters. Analyses based on generalizability theory were conducted for reliability examination. Validity was examined through following the Standards for Educational and Psychological Testing to evaluate content validity, response processes, construct validity, predictive validity, and the relationship to the variable of training. For construct validity, correlations of PBL scores with six other outcome measures were examined, including Medical College Admission Test, United States Medical Licensing Examination (USMLE) Step 1, National Board of Medical Examiners (NBME) Comprehensive Basic Science Examination, NBME Comprehensive Clinical Science Examination, Clinical Performance Examination, and USMLE Step 2 Clinical Knowledge. Predictive validity was examined by using PBL scores to predict five medical school outcomes. The highest percentage of PBL total score variance was associated with students (60 %), indicating students in the study differed in their PBL performance. The generalizability and dependability coefficients were moderately high (Ep² = .68, ? = .60), showing the instrument is reliable for ranking students and identifying competent PBL performers. The patterns of correlations between PBL domain scores and the outcome measures partially support construct validity. PBL performance ratings as a whole significantly (p < .01) predicted all the major medical school achievements. The second year PBL scores were significantly higher than those of the first year, indicating a training effect. Psychometric findings provided support for reliability and many aspects of validity of PBL performance assessment using the instrument.     

The effect of midterm peer feedback on student functioning in problem-based tutorials

Within Problem-Based Learning successful learning depends on the quality of cognitive, social and motivational contributions students make to the tutorial group. But at the same time, not all students in PBL automatically contribute in a high quality manner, which might impede successful group functioning. This study investigated whether peer process feedback combined with goal setting can be used to improve the quality of students’ individual contributions. A mixed-methods explanatory design, in which 74 second-year Health Sciences students participated, combined a pre- and posttest with a focus group. The results indicated that the quality of the contributions only increased for students with a below average score on the pre-test. The qualitative data confirmed that the impact of the feedback could be increased by combining individual reflection by means of goal setting with face-to-face discussion. Another suggestion is to investigate whether midterm peer process feedback is more effective for first year students, because they are still developing their tutorial behavior, as opposed to second year students.