International recognition of basic medical education programmes

OBJECTIVE: This document aims to formulate a World Federation for Medical Education (WFME) policy and to open debate on the subject on international recognition of basic medical education institutions and programmes. METHODS: We carried out a systematic review of international quality assurance of medical education and recognition methodology, including accreditation procedures and alternative quality assurance methods, with a focus on the role of the WFME in international recognition of basic medical education programmes. RESULTS: In order to further the intentions of the WFME, the Federation will: continue its activity to establish new Global Directories of Health Professions Education Institutions (GDHPEI); set up a planning working group to prepare the work of the international advisory committee for GDHPEI; develop a database of relevant accrediting and recognising agencies; continue its project on the promotion of proper national accreditation; establish a working group to develop principles to be used in the evaluation of medical schools and other health professions education institutions and their programmes for the purpose of international recognition, especially when proper accreditation is not feasible, and work with partners on training programmes for advisors and assessors. CONCLUSIONS: The new directory for medical schools, which will include qualitative information about basic medical education programmes, will provide a basis for the meta-recognition of medical schools' programmes by stimulating the establishment of national accreditation systems and other quality assurance instruments.     

Selection of medical students: a controlled experiment

Objectives  We aimed to discover, through a controlled experiment, whether cognitive and non-cognitive assessment would select higher-achieving applicants to medical school than selection by lottery.
Methods  We carried out a prospective cohort study to compare 389 medical students who had been admitted by selection and 938 students who had been admitted by weighted lottery, between 2001 and 2004. Main outcome measures were dropout rates, study rate (credits per year) and mean grade per first examination attempt per year. Study rates in the 4 pre-clinical years of medical school were used to categorise students' performance as average or optimal.
Results  Pre-admission variables did not differ between the two groups. The main outcome of the selection experiment was that relative risk for dropping out of medical school was 2.6 times lower for selected students than for lottery-admitted controls (95% confidence interval 1.59–4.17). Significant differences between the groups in the percentage of optimally performing students and grade point average for first examination attempts were found only in the 2001 cohort, when results favoured the selected group. The results of the selection process took into account both the assessment procedure involved and the number of students who withdrew voluntarily.
Conclusions  This is the first controlled study to show that assessing applicants' non-cognitive and cognitive abilities makes it possible to select students whose dropout rate will be lower than that of students admitted by lottery. The dropout rate in our overall cohort was 2.6 times lower in the selected group.     

Effectiveness of medical school admissions criteria in predicting residency ranking four years later

BACKGROUND: Medical schools across Canada expend great effort in selecting students from a large pool of qualified applicants. Non-cognitive assessments are conducted by most schools in an effort to ensure that medical students have the personal characteristics of importance in the practice of Medicine. We reviewed the ability of University of Toronto academic and non-academic admission assessments to predict ranking by Internal Medicine and Family Medicine residency programmes. METHODS: The study sample consisted of students who had entered the University of Toronto between 1994 and 1998 inclusive, and had then applied through the Canadian resident matching programme to positions in Family or Internal Medicine at the University of Toronto in their graduating year. The value of admissions variables in predicting medical school performance and residency ranking was assessed. RESULTS: Ranking in Internal Medicine correlated significantly with undergraduate grade point average (GPA) and the admissions non-cognitive assessment. It also correlated with 2-year objective structured clinical examination (OSCE) score, clerkship grade in Internal Medicine, and final grade in medical school. Ranking in Family Medicine correlated with the admissions interview score. It also correlated with 2nd-year OSCE score, clerkship grade in Family Medicine, clerkship ward evaluation in Internal Medicine and final grade in medical school. DISCUSSION: The results of this study suggest that cognitive as well as non-cognitive factors evaluated during medical school admission are important in predicting future success in Medicine. The non-cognitive assessment provides additional value to standard academic criteria in predicting ranking by 2 residency programmes, and justifies its use as part of the admissions process.     

A learning resources centre: its utilization by medical students

Innovations in medical education have led to the increasing use of problem-based learning techniques, a committee system organization, and more time for independent study in many undergraduate programmes. There has been an increase in availability of alternative methods for presentation of information. To encourage self-directed learning, resources such as computers, videos and models, among others, should be readily available to students. The University of Calgary Faculty of Medicine has provided various resources contained in one area, called the Bacs Learning Resource Centre (BLRC). Since the maintenance and further development of such a centre requires significant resources, it is important to determine student utilization of the various components used in their learning. For those who are about to set up such a learning resource centre this information gives guidance on which materials are most useful.
The utilization of the centre by 69 first year medical students was studied using questionnaires. The utilization during a specific course was determined by analysing the entries in the individual log books given to the students at the beginning of the Integrative Course. With the exception of one student, all those who responded to the questionnaire used the Centre, with 20% or less of their total study time being spent there. The BLRC was most used for the Musculo-skeletal, Cardiovascular and Reticulo-Endothelial courses. All categories of resources were found to be useful, with the tape/slides least utilized. Utilization was most influenced by the quality of resources available and recommendations by peers. The development of a centre such as the BLRC, with a variety of resources concentrated in one area, suitable for individual or group study and accessible 24 hours a day, should be considered by all medical schools to enhance self-directed learning in medical students.     

Selection of medical students: the Beer-Sheva perspective

Summary: Summary. In many Western medical schools with a low attrition rate the selection of medical students represents the key hurdle for admission to the practice of medicine. The process therefore deserves careful attention. Described herein are impressions and conclusions from almost two decades' experience in selecting medical students at the Ben-Gurion University in Beer-Sheva, Israel.
Emphasis is placed on personal qualities as determined by an interview process of those students who pass a relatively lenient academic threshold. Interviewing is carried out by teams of two, one of whom is a doctor, and each candidate who is accepted is interviewed by four members of the admission committee. Emphasis is placed on clear goals for the interview process, carefully selected and trained interviewers and continual review of the process. In spite of the inherent limitations of a subjective interview process we feel that it offers considerable advantage over other approaches in student selection.     

Integrating cultural competency into the undergraduate medical curriculum

CONTEXT: A significant movement is underway to develop standardised curricula that provide medical students with a fundamental knowledge of cultural sensitivity. METHODS: This paper reviews the recent initiative to integrate cultural competency training into the curricula of American undergraduate medical institutions. RESULTS: Despite these initiatives, few medical institutions have achieved a comprehensive curriculum providing for continued cultural experiences and assessment of related skill sets. The article proposes a framework for the introduction of continued instruction in and assessment of cultural competency within the undergraduate medical education. CONCLUSIONS: The described components of cultural competency education in the undergraduate medical curricula provide a framework to facilitate early and continued exposure to experiences designed to promote cultural sensitivity.     

Demographic and socio-economic associations with academic attainment (UCAS tariff scores) in applicants to medical school

BACKGROUND: In the United Kingdom medical students are selected predominantly on their academic merit. Their academic achievement marks are equated via the tariff point score structure administered by the Universities and Colleges Admissions Service (UCAS). We studied the applicant databases for 1998-2003 for one English medical school to determine the factors that predict high tariff point scores. METHODS: Complete demographic data and relative socio-economic status, educational institution attended and tariff point score was available for 8997 UK applicants aged 21 years or younger to the 5-year Bachelor of Medicine/Bachelor of Surgery (BM BS) course at Nottingham University medical school (and partially complete data for a further 1891 applicants). The data were subjected to standard univariate and multivariate analyses and to path analysis. RESULTS: In these samples, the independent predictors of a high tariff point score were being younger and male. The effect sizes were small, although significant. Higher tariff point scores were achieved by those from households less materially disadvantaged. Ethnicity was also a predictor with white, Chinese and those of mixed ethnic origin achieving higher tariff point scores than those from other groups. Finally, the type of school attended predicted academic achievement with applicants from further education colleges, independent schools and grant-maintained schools achieving higher tariff point scores. CONCLUSION: Notwithstanding the relatively homogeneous (predominantly young, white, high academic achievers) applicant pool to a single UK medical school we identified consistent significant predictors of high tariff point scores. As high tariff point scores are still the major entry criterion to UK medical schools, our findings will be of value in informing policy decisions concerning 'widening access' schemes being established at government request.     

Academic standards and changing patterns of medical school admissions: a Malaysian study

Changing social demands made it necessary for the Medical Faculty of the University of Malaya to accommodate students with a wider range of academic experience than before. However, teachers sought to achieve comparable academic standards to those in the West by striving to maintain a close resemblance to the Western model of medical education in other respects. As a result teachers failed to adapt their teaching methods, assessment techniques and curriculum design to meet the educational needs of the students, thus compromising academic standards. Many students lack basic academic skills and do not know how to learn effectively. In order to help students overcome their learning difficulties innovative teaching was required during the first year at university, designed to foster the joint development of knowledge and basic skills. In the case of less well-prepared students who lack self-confidence, a caring and supportive learning environment is crucial to the achievement of meaningful learning. Lecturers needed to become facilitators of learning rather than transmitters of knowledge. However, teachers' objective to retain international recognition of the degree, which presumably reflected the importance of teaching, was not operationalized in terms of its incentive structure such that teachers were constrained not to try to fill the new roles demanded of them. It was assumed that academic distinction accrued through scientific research was essential for the achievement of academic excellence. However, under the prevailing circumstances the two aims were mutually exclusive and incompatible and teaching quality deteriorated.     

A comprehensive assessment of medical schools in Bosnia and Herzegovina

OBJECTIVES: To perform internal and external evaluations of all 5 medical schools in Bosnia and Herzegovina against international standards. METHODS: We carried out a 2-stage survey study using the same 5-point Likert scale for internal and external evaluations of 5 medical schools in Bosnia and Herzegovina (Banja Luka, Foca/East Sarajevo, Mostar, Sarajevo and Tuzla). Participants consisted of managerial staff, teaching staff and students of medical schools, and external expert assessors. Main outcome measures included scores on internal and external evaluation forms for 10 items concerning aspects of school curriculum and functioning: 'School mission and objectives'; 'Curriculum'; 'Management'; 'Staff'; 'Students'; 'Facilities and technology'; 'Financial issues'; 'International relationships'; 'Internal quality assurance', and 'Development plans'. RESULTS: During internal assessment, schools consistently either overrated their overall functioning (Foca/East Sarajevo, Mostar and Tuzla) or markedly overrated or underrated their performance on individual items on the survey (Banja Luka and Sarajevo). Scores for internal assessment differed from those for external assessment. These differences were not consistent, except for the sections 'School mission and objectives', 'Curriculum' and 'Development plans', which were consistently overrated in the internal assessments. External assessments was more positive than internal assessments on 'Students' and 'Facilities and technology' in 3 of 5 schools. CONCLUSIONS: This assessment exercise in 5 medical schools showed that constructive and structured evaluation of medical education is possible, even in complex and unfavourable conditions. Medical schools in Bosnia and Herzegovina have successfully formed a national consortium for formal collaboration in curriculum development and reform.     

A structured panel interview and group exercise in the selection of medical students

The selection of entrants for medical school, as now widely agreed, should include measurements of non-academic as well as academic attributes. The authors have developed a process to assess the personal attributes of applicants. This included a structured panel interview carried out twice on each applicant by independent interviewers and a group exercise in which several applicants were observed whilst discussing a problem. Training for interviewers took the form of a half-day interactive workshop.
One hundred and forty-one school-leaving applicants completed the new assessment. No relationship was found between academic achievement as reflected by marks in a national examination and scores in the panel interview, the group exercise, or the school principal's report. However, significant intercorrelations were found between the panel interview, group exercise and school report. The results of this experience have encouraged the Auckland School to continue to explore methods to measure these attributes in a carefully controlled study.     

Contribution of medical students to admission interviews

Objective  Medical students were included in 3-person interview panels selecting applicants for a 4-year medical programme for graduates and health professionals. We analysed student contributions to the interview panels.
Methods  A total of 188 semi-structured interviews were carried out over 2 years. The scores of 17 student interviewers were compared with those of the other 2 members of the panels.
Results  Students were positive about the interview process and their involvement. Overall interview scores did not differ between students and other interviewers. However, students gave lower scores overall and in a number of individual areas to those who had the highest consensus scores and were offered places on the programme. Students were more likely than other interviewers to be indecisive between 2 scores (25.2% versus 17.5%; P  <   0.01).
Conclusions  Students can be integrated effectively into the interview process. However, they tend to be more reluctant to give high scores and less decisive about committing to a single score than more experienced interviewers. Students value the experience of participating in the selection process, especially when combined with an exploration of the literature related to selection procedures.     

Lessons learned from complementary and integrative medicine curriculum change in a medical school

OBJECTIVES: This paper describes a pilot study that examined lessons learned from the introduction of complementary and alternative medicine (CAM) elements into a medical school curriculum. METHODS: A qualitative approach was selected as a first step in evaluating the phenomenological experience of introducing the CAM Educational Project in 2000-05. In 2005, semi-structured interviews were conducted with faculty staff and graduating students who had participated in all 4 years of the CAM Project. Qualitative content was analysed focusing on linguistic data and contextual meaning. RESULTS: The overall response to the integration of CAM curricular elements into the medical school curriculum was positive among all faculty and graduating medical students. Participant experiences were often dependent on the perceived rigour of alternative approaches to a presenting patient problem, along with the importance attributed to openness to patient perspectives as part of evidence-based practices. There was an appreciation of the importance of developing increased awareness and utilisation of CAM in medical practice, as well as a recognition of resistance by some medical school faculty to CAM approaches. CONCLUSIONS: This evaluation of a specific CAM educational project suggests potentially transferable findings to other medical schools. Integrating CAM into the medical school curriculum requires a dedicated team if it is to result in a significant change. This change requires that CAM practices are visible to both students and faculty, that there is a co-operative climate, accessible resources, and institutional support, and that CAM content is embedded into the existing curriculum. All these factors combined can lead to sustainable integration of CAM content issues into the medical school curriculum.     

Factors affecting the utility of the multiple mini-interview in selecting candidates for graduate-entry medical school

Context  We wished to determine which factors are important in ensuring interviewers are able to make reliable and valid decisions about the non-cognitive characteristics of candidates when selecting candidates for entry into a graduate-entry medical programme using the multiple mini-interview (MMI).
Methods  Data came from a high-stakes admissions procedure. Content validity was assured by using a framework based on international criteria for sampling the behaviours expected of entry-level students. A variance components analysis was used to estimate the reliability and sources of measurement error. Further modelling was used to estimate the optimal configurations for future MMI iterations.
Results  This study refers to 485 candidates, 155 interviewers and 21 questions taken from a pre- prepared bank. For a single MMI question and 1 assessor, 22% of the variance between scores reflected candidate-to-candidate variation. The reliability for an 8-question MMI was 0.7; to achieve 0.8 would require 14 questions. Typical inter-question correlations ranged from 0.08 to 0.38. A disattenuated correlation with the Graduate Australian Medical School Admissions Test (GAMSAT) subsection 'Reasoning in Humanities and Social Sciences' was 0.26.
Conclusions  The MMI is a moderately reliable method of assessment. The largest source of error relates to aspects of interviewer subjectivity, suggesting interviewer training would be beneficial. Candidate performance on 1 question does not correlate strongly with performance on another question, demonstrating the importance of context specificity. The MMI needs to be sufficiently long for precise comparison for ranking purposes. We supported the validity of the MMI by showing a small positive correlation with GAMSAT section scores.     

Comparison of performance of students from different high school systems in medical sciences at Kuwait University

Data, although limited, question the validity of the formula applied for admission to the medical school in which high school grades are the only preselection variable applied. Comparison between two groups of students from two different high school systems in Kuwait was carried out to determine if the admission criteria used currently for entry to the medical school are equally valid for both groups. The results are based on the students' performances in the first three-semester programme of medical sciences. Subjects covered were anatomy, physiology, biochemistry and behavioural sciences. The group derived from the High School National Diploma performed significantly better with a percentage pass rate of 82% while of those who were derived from the Course Credit System only 61% passed the final examination. In addition, only one of the latter group attained total marks of more than 80% compared to 12 students from the National Diploma group. The percentage failure according to subjects was consistently higher among the Course Credit graduates in all the subjects. All differences between the two groups are statistically significant (P less than 0.001).     

Overview of accreditation of undergraduate medical education programmes worldwide

Context  There is significant variation in the structure and quality of undergraduate medical education around the world. Accreditation processes can encourage institutional improvement and help promote high-quality education experiences.
Methods  To investigate the overseeing of medical education from an international perspective, the Foundation for Advancement of International Medical Education and Research (FAIMER^®) has developed, and continues to update, the Directory of Organizations that Recognize/Accredit Medical Schools (DORA). The directory includes information on the presence of national accrediting bodies and related data. Medical education accreditation information was pooled by World Health Organization (WHO) regions.
Results  Although over half of all countries with medical schools indicate that they have a national process for accrediting medical education programmes, the nature of the various authorities and levels of enforcement vary considerably.
Discussion  Despite global trends indicating an increasing focus on the quality of education programmes, data linking accreditation processes to the production of more highly skilled doctors and, ultimately, better patient care are lacking. Investigating current accreditation practices is a necessary step for further research. To this end, we will continue to gather data on medical education around the world and will explore opportunities for relating these processes to outcomes.     

Assessment of non-cognitive traits through the admissions multiple mini-interview

CONTEXT: Contemporary studies have shown that traditional medical school admissions interviews have strong face validity but provide evidence for only low reliability and validity. As a result, they do not provide a standardised, defensible and fair process for all applicants. METHODS: In 2006, applicants to the University of Calgary Medical School were interviewed using the multiple mini-interview (MMI). This interview process consisted of 9, 8-minute stations where applicants were presented with scenarios they were then asked to discuss. This was followed by a single 8-minute station that allowed the applicant to discuss why he or she should be admitted to our medical school. Sociodemographic and station assessment data provided for each applicant were analysed to determine whether the MMI was a valid and reliable assessment of the non-cognitive attributes, distinguished between the non-cognitive attributes, and discriminated between those accepted and those placed on the waitlist (waiting list). We also assessed whether applicant sociodemographic characteristics were associated with acceptance or waitlist status. RESULTS: Cronbach's alpha for each station ranged from 0.97-0.98. Low correlations between stations and the factor analysis suggest each station assessed different attributes. There were significant differences in scores between those accepted and those on the waitlist. Sociodemographic differences were not associated with status on acceptance or waiting lists. DISCUSSION: The MMI is able to assess different non-cognitive attributes and our study provides additional evidence for its reliability and validity. The MMI offers a fairer and more defensible assessment of applicants to medical school than the traditional interview.     

MOR: a simulation-based assessment centre for evaluating the personal and interpersonal qualities of medical school candidates

Context Medical school admissions traditionally rely heavily on cognitive variables, with non‐cognitive measures assessed through interviews only. In recognition of the unsatisfactory reliability and validity of traditional interviews, medical schools are increasingly exploring alternative approaches that can provide improved measures of candidates’ personal and interpersonal qualities. Methods An innovative assessment centre (MOR [Hebrew acronym for ‘selection for medicine’]) was designed to measure candidates’ personal and interpersonal attributes. Three assessment tools were developed: behavioural stations, including encounters with simulated patients and group tasks; an autobiographical questionnaire, and a judgement and decision‐making questionnaire. Candidates were evaluated by trained raters on four qualities: interpersonal communication; ability to handle stress; initiative and responsibility, and self‐awareness. Results In the years 2004–05, the 588 medical school candidates with the highest cognitive scores were tested; this resulted in a change of approximately 20% in the cohort of accepted students compared with previous admission criteria. Internal consistency ranged from 0.80 to 0.88; inter‐rater reliability ranged from 0.62 to 0.77 for the behavioural stations and from 0.72 to 0.95 for the questionnaires; test–retest score correlation was 0.7. The correlation between candidates’ MOR scores and cognitive scores approached zero, reflecting the value of MOR in the screening process. Feedback from participants indicated that MOR was perceived as fair and appropriate for medical school screening. Discussion MOR is a reliable tool for measuring non‐cognitive attributes in medical school candidates. It has high content and face validity. Furthermore, its implementation conveys the importance of maintaining humanist characteristics in the medical profession to students and faculty staff.