Quality of reporting of experimental studies in medical education: a systematic review

OBJECTIVE: Determine the prevalence of essential elements of reporting in experimental studies in medical education. DESIGN: Systematic review. DATA SOURCES: Articles published in 2003 and 2004 in Academic Medicine, Advances in Health Sciences Education, American Journal of Surgery, Journal of General Internal Medicine, Medical Education, and Teaching and Learning in Medicine. REVIEW METHODS: Articles describing education experiments, including evaluation studies with experimental designs, were identified (n = 185) by reviewing titles and abstracts. A random sample (n = 110) was selected for full review. The full text of each article was evaluated for the presence of guideline-based features of quality reporting: a critical literature review, conceptual framework, statement of study intent (e.g. aim, research question, or hypothesis), statement of study design, definition of main intervention and comparison intervention or control group, and consideration of human subject rights. RESULTS: Of the 105 articles suitable for review, 47 (45%) contained a critical literature review and 58 (55%) presented a conceptual framework. A statement of study intent was present in 80 articles (76%), among which the independent and dependent variables were operationally defined in 38 (47%) and 26 articles (32%), respectively. A total of 17 articles (16%) contained an explicit study design statement. Among the 48 studies with a comparison group, 35 (73%) clearly defined the comparison intervention or control group. Institutional review board approval or participant consent was reported in 44 articles (42%). CONCLUSIONS: The quality of reporting of experimental studies in medical education was generally poor. Criteria are proposed as a starting point for establishing reporting standards for medical education research.     

Finding POEMs in the medical literature.

Articles about primary care topics that measure patient-oriented outcomes (eg, morbidity, mortality, quality of life) should change practice, if the reported results are valid. We call these types of articles POEMs--Patient-Oriented Evidence that Matters. The extent and distribution of POEMs in the medical literature is unknown. We identified 85 medical journals of potential interest to primary care physicians, and counted 8085 original research articles over a 6-month period; 211 of those articles were POEMs (2.6%). Ten journals accounted for 50% of the POEMs identified during the study period: Journal of the American Medical Association, Annals of Internal Medicine, New England Journal of Medicine, Archives of Internal Medicine, Lancet, British Medical Journal, Obstetrics and Gynecology, Arthritis and Rheumatology, American Journal of Obstetrics and Gynecology, and Journal of the American College of Cardiology. Other journals with a high percentage of POEMs were: Journal of Family Practice, Journal of the American Board of Family Practice, and American Journal of Emergency Medicine. Half the journals we surveyed published 0 or 1 POEMs during the study period. Not all clinicians have the time, inclination, or access to review a large number of journals on a regular basis. By focusing on POEMs and the journals that publish them, busy physicians can avoid reading 98% of the original research published each month. This will not only drastically cut physicians' reading time, but also help them obtain the information that is most valuable for their patients.     

Medical students in ENT outpatient clinics: appointment times, patient satisfaction and student satisfaction

OBJECTIVES: Outpatient clinics are increasingly important in medical education. The effect of students on clinic times and patient satisfaction, as well as their own satisfaction, were studied. DESIGN: A prospective, non-randomized, controlled study using adult patient questionnaires, medical student questionnaires and clinic time sheets. SETTING: Two teaching hospital ENT clinics. SUBJECTS: Medical students and adult patients. RESULTS: Three hundred and twenty-five patient questionnaires were collected (77% response), including 135 student encounters. Students did not affect appointment durations (19 min +/- 0.48 (standard error)) except at centre B (35 min +/- 1.1, P < 0.0001) where patient numbers were cut for teaching. Patient satisfaction, generally high, was not affected by students, appointment duration or gender of doctor or patient. It was slightly higher in the lower social classes (rs = 0.20, P = 0. 003) and older patients (rs = 0.17, P = 0.002). Student acceptability scores were not affected by student numbers (up to four), social class or time spent alone with students. They were higher if time was spent alone with the doctor (75.3% +/- 4.9) than not (63.0% +/- 1.8, P = 0.024). Thirty-six per cent of patients preferred to have a student present; only 9% preferred not. Student satisfaction was higher at centre B (73.7% +/- 2.3) where appointments were longer and students spent more time alone with patients than centre A (64.3% +/- 2.3, P = 0.0052). CONCLUSIONS: Clinic appointments are not necessarily longer in the presence of students. When students have the chance to see patients alone during longer consultations, student satisfaction is higher. Patient satisfaction, generally high, is not altered by the presence of students, but patients given time alone with their doctor are more accepting of students. These findings have resource implications for the planning of NHS clinics in teaching hospitals.     

Contribution of medical student research to the MedlineTM-indexed publications of a German medical faculty

Medical students in Germany have to write a research thesis to acquire the title of medical doctor. This study evaluates the contribution of student research to the Medline?-indexed publications of a German medical faculty. A 1993–1995 Medline?-publication list, on which medical students among authors should be marked, was sent to medical faculty staff of the University of Würzburg, Germany (n = 238). Faculty members responded (106, 45%), 66 were working at a clinic, 26 at a clinic-associated institute and 14 at a basic science institute. Between 1993 and 1995, 1128 Medline?-indexed papers were published by these faculty members, who on average supervised 4·5 medical students (n = 477). Medical students were among the authors of 316 (28%) and were the first authors of 88 papers (7·8%). For 66% of medical students their research resulted in a Medline?-indexed publication. Medical student research activity can significantly influence the published output of a medical faculty.     

Patient-centred medicine through student-centred teaching: a student perspective on the key impacts of community- based learning in undergraduate medical education

CONTEXT: The UK General Medical Council has proposed that increased use of community settings is essential to enhancement of medical education. However, such curriculum developments have been directed by educationalists and clinical faculty; there is to date little to show whether student perspectives accord with such expectations. AIM: To examine student views on whether community-based learning during a UK undergraduate medical education course results in new learning in the areas expected, and to elucidate any process factors which enhance attainment of learning objectives. METHOD: Nominal group technique, to develop consensus on important learning outcomes and process factors, and questionnaire survey, developed from the views of the nominal groups. RESULTS: 89 students participated (response rate 70% for the nominal groups, and 88% for questionnaire). Students perceived increased learning in many of the areas expected. In particular, students reported significant learning from: witnessing the impact of a longer term and more personal relationship with patients; the visible impact of social environment on health; the importance of dealing with people rather than diseases, and the use of the whole team for care. In addition, they emphasized that tutor, staff and patient enthusiasm for student presence and learning greatly enhanced the student learning experience. CONCLUSIONS: Community settings appear to achieve the expected attitudinal adaptation of students. The role of the committed tutor and team is seen as pivotal to learning. The conclusions support an increased emphasis in contemporary medical education and related research activity on the key impact of relationships in the learning environment.     

Volunteer bias in medical education research: an empirical study of over three decades of longitudinal data

CONTEXT: The issue of whether medical education research outcomes can be biased by students' refusal to allow their data to be used in outcomes research should be empirically addressed to assure the validity of research findings. Given that institutions are expected to document the outcomes of their educational programmes, evaluations of clinical performance subsequent to medical school are crucial, but are often incomplete when graduates decline to permit data collection. OBJECTIVES: This study aimed to examine the demographic and performance differences between research volunteers and others. METHODS: A total of 7415 doctors graduated from Jefferson Medical College between 1970 and 2004; 75% (n = 5575) agreed to participate in medical education research by granting written permission for the collection of data from their postgraduate training directors on their behalf (research volunteers); 20% (n = 1489) refused to grant such permission (non-volunteers), and 5% (n = 351) did not return the permission form (non-respondents). This prospective longitudinal study compared research volunteers, non-volunteers and non-respondents on gender, ethnicity, performance measures prior to, during and after medical school, scores on medical licensing examinations, and board certification status. RESULTS: Doctors who granted permission (volunteers) generally performed better during and after medical school. In addition, they scored higher on medical licensing examinations and had a higher certification rate. Women and members of ethnic minority groups were less likely to grant permission. CONCLUSIONS: The study raises questions about the validity of research findings as a result of volunteerism in medical education research. The implications for guidelines regarding the protection of human subjects in medical education research, and for educational outcomes, are discussed.     

Patients as teachers: a qualitative study of patients'' views on their role in a community-based undergraduate project

OBJECTIVES: Patients have been used in clinical medical education for many years with, traditionally, a relatively passive role. Following the General Medical Council recommendations for curricular change and the development of more community-based teaching, 'ordinary patients' in the community are increasingly being partnered with undergraduate students for particular projects. Very little research has been undertaken on patients' perceptions of this role. DESIGN: Semi-structured interviews were carried out with 20 people to explore the views of patients taking part in a community-based undergraduate medical student project (the 'patient study') at Newcastle Medical School about their role as teachers of medical students, what they felt they had gained from participating, any problems or concerns and suggestions for change or improvement. SETTING: Newcastle Medical School, UK. SUBJECTS: Second-year medical students. RESULTS: Two major themes emerged. First, patients saw themselves in active roles as teachers: as experts in their medical condition; as exemplars of their condition; and as facilitators of the development of students' professional skills and attitudes. Secondly, patients felt they had benefited from participation, through talking about their problems; learning more about themselves; the satisfaction of helping; and from receiving gifts. In addition, a number of other issues were identified including interpersonal dynamics, gender and ethnic differences, inadequate briefing of participants and whether such community-based patient involvement might, in some situations, be felt to be exploitative. CONCLUSIONS: The study has shown that patients see themselves clearly as having specific contributions to make to medical students' education and training. This has implications for the further development of community-based teaching.     

A one-year audit of topics and domains in the Journal of the American Medical Association and the New England Journal of Medicine

BACKGROUND: The relative emphasis that major medical journals give to topic areas has a potential effect on priorities in patient care, policy decisions, and public awareness. We measured the distribution of topics in two journals, by disease categories and domains, over a calendar year. METHODS: All original investigations, reviews, editorials, and special articles published in 1998 by the Journal of the American Medical Association and the New England Journal of Medicine were classified by article type, disease category, and domain. The 12 domains ranged from basic science to health policy, and included primary and secondary prevention. RESULTS: The 1159 articles published in 1998 included 889 (77%) articles about specific diseases-590 falling within eight specialties-and 190 (16%) articles on generic topics. Eighty (7%) articles concerned the behaviors that cause disease. Primary prevention and screening were the subject of 71 (6%) and 29 (3%) articles, respectively. Most of these concerned uncommon issues in patient care. Although 27 (2%) articles dealt with essential health promotion (e.g., diet, exercise), and none included a study on how to help patients to exercise, stop smoking, or eat a healthy diet. In contrast, 451 (39%) articles concerned the diagnosis and treatment of patients with disease. CONCLUSIONS: The relative emphasis that journals gave to prevention during the sample period seems discordant with its importance to patients and public health. Potential explanations include poor volume and quality of submitted research and editorial concerns about importance and reader appeal.     

Advancing educators and education by defining the components and evidence associated with educational scholarship

OBJECTIVE: This study aimed to establish documentation standards for medical education activities, beyond educational research, for academic promotion consistent with principles of excellence and scholarship. METHODS: In 2006 a Consensus Conference on Educational Scholarship was convened by the Association of American Medical Colleges (AAMC) Group on Education Affairs (GEA) to outline a set of documentation standards for use by educators and academic promotion committees. Conference participants' work was informed by more than 15 years of literature on scholarship, educator portfolios and academic promotion standards. RESULTS: The 110 conference participants, including medical school deans, academic promotion committee members, department chairs, faculty and AAMC leaders, re-affirmed the 5 education activity categories (teaching, curriculum, advising and/or mentoring, education leadership and/or administration, and learner assessment), the contents of each category, and cross-category documentation standards. Educational excellence requires documentation of the quantity and quality of education activities. Documenting a scholarly approach requires demonstrating evidence of drawing from and building on the work of others, and documenting scholarship requires contributing work through public display, peer review and dissemination; both involve engagement with the community of educators. Implementation of these standards - quantity, quality and engagement with the education community - should occur in parallel with the development of an infrastructure to support educators, including sustained faculty development for educators, access to educational resources and journals, peer review mechanisms and consultation and support specific to each activity category. CONCLUSIONS: Educators' contributions to their institutions must be visible to be valued. The establishment of documentation standards for education activities provides the foundation for academic recognition of educators.     

Faculty attitudes towards medical communication and their perceptions of students' communication skills training at Dalhousie University

SETTING: Faculty of Medicine, Dalhousie University, Halifax, Nova Scotia, Canada. OBJECTIVES: (1) To assess the attitudes of full-time clinical faculty members towards medical communication using the newly developed Attitudes Towards Medical Communication Scale; (2) to determine faculty members' perceptions of communications training for students and residents. METHODS: An anonymous self-completion survey was sent to 233 full-time clinical faculty members. The questionnaire asked about faculty attitudes towards medical communication, and assessed faculty members' views of student and resident training in communication. RESULTS: Faculty scored highly in the Attitudes Towards Medical Communication Scale, with a mean score of 51.5 (SD 4.1) out of a possible 60. In univariate analysis, rating of personal enjoyment of teaching, rating of the importance of teaching, and having attended at least one faculty communications workshop in the previous 5 years were significantly associated with higher scale scores. When these factors were assessed using linear regression, only having attended a workshop and higher rating of the importance of teaching remained significant. Faculty assessed student training in communications skills poorly overall. When assessing seven specific communications areas, more than 20% rated this training as poor for six of the areas for third- and fourth-year students and for five of the areas for residents. CONCLUSIONS: Clinical faculty at Dalhousie have very positive attitudes towards medical communication, and more highly positive attitudes are found in those who have attended a communications workshop. Despite this evidence that faculty appreciate the importance of medical communication skills, many assessed students' training in this curriculum area as poor.     

Differences in curriculum emphasis in US undergraduate and generalist residency education programmes

OBJECTIVES: Little attention has been paid to the differential emphasis undergraduate and graduate medical education programmes place on the broad competencies that will be needed for practice in an increasingly managed health care environment. The purpose of this study was to determine differences in emphasis that undergraduate and primary care graduate medical education programmes are currently placing on 33 broad practice competencies, compared with the emphasis they ideally would like to give them, and the barriers they perceive to curriculum change. DESIGN: Subjects were surveyed by mailed questionnaire. A reminder postcard and follow-up mailing were sent to non-respondents. SETTING: US allopathic medical schools. SUBJECTS: Academic deans identified by the Association of American Medical Colleges (AAMC) and generalist (family medicine, internal medicine, paediatrics and obstetrics-gynaecology) residency programme directors identified by the American Council on Graduate Medical Education (ACGME). RESULTS: Findings revealed that residency programmes placed greater emphasis on the study's broad curriculum topics than did undergraduate medical education programmes. Statistically significant differences were found in current emphasis for 12 topics and ideal emphasis for six topics. Both groups identified an already crowded curriculum and inadequate funding as the top two barriers to curriculum change. CONCLUSIONS: The differences in curriculum emphases and perceived barriers to curriculum change most probably reflect the different realities of undergraduate and graduate medical education programmes, i.e. academics vs. a focus on immediate practice realities.     

'I don't have time': issues of fragmentation, prioritisation and motivation for education scholarship among medical faculty

Objectives  Although lack of time has been frequently cited as a barrier to scholarship, there has been little inquiry into what specific factors medical faculty staff perceive as contributing to this dilemma. The purpose of the present study was to explore, in greater detail, lack of time as a barrier for faculty interested in pursuing education scholarship.
Methods  In 2004, as part of a cross-sectional, mixed-methods needs assessment, 73 (67.6%) medical faculty completed a questionnaire probing areas related to education scholarship. Additionally, one year later, 16 respondents (60% of those invited) each participated in one of three focus groups.
Results  Despite their interest and regardless of their background training in education, faculty were able, on average, to devote only negligible amounts of time to education scholarship. The most commonly reported barrier to these pursuits was lack of protected time. Further analysis revealed that the time-related factor appeared to involve three themes: fragmentation (where opportunities to work on education projects are sporadic); prioritisation (where work responsibilities including after-hours work and administrative workload complete for time, and where there is difficulty in securing financially remunerated time), and motivation (where the degree of recognition and support for education work by both the department and colleagues is limited).
Conclusions  With respect to education scholarship, the dilemma caused by lack of time involves a complex, multi-faceted set of issues which extends beyond the number of hours available in a day. Personal interest and having background training in education do not appear to be sufficient to encourage involvement. Multiple institutional support mechanisms are necessary.     

A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach?

PURPOSE: To analyse the impact of computer-based patient record systems (CBPRS) on medical practice, quality of care, and user and patient satisfaction. DATA SOURCES: Manual and electronic search of the Medline, Cochrane, and Embase databases. STUDY SELECTION: Selected articles were published from 2000 to March 2003. CBPRS was defined as computer software designed to be used by clinicians as a direct aid in clinical decision making. To be included, the systems should have recorded patient characteristics and offered online advice, or information or reminders specific to clinicians during the consultation. DATA EXTRACTION: Keywords used for the search were: electronic record, informatic record, electronic medical record, electronic patient record, patient order entry, computer-based patient system, clinical decision support systems, and evaluation. RESULTS: Twenty-six articles were selected. Use of a CBPRS was perceived favourably by physicians, with studies of satisfaction being mainly positive. A positive impact of CBPRS on preventive care was observed in all three studies where this criterion was examined. The 12 studies evaluating the impact on medical practice and guidelines compliance showed that positive experiences were as frequent as experiences showing no benefit. None of the six studies analysing the impact of CBPRS on patient outcomes reported any benefit. CONCLUSIONS: CBPRS increased user and patient satisfaction, which might lead to significant improvements in medical care practices. However, the studies on the impact of CBPRS on patient outcomes and quality of care were not conclusive. Alternative approaches considering social, cultural, and organizational factors may be needed to evaluate the usefulness of CBPRS.     

The accountability of clinical education: its definition and assessment

BACKGROUND: Medical education is not exempt from increasing societal expectations of accountability. Competition for financial resources requires medical educators to demonstrate cost-effective educational practice; health care practitioners, the products of medical education programmes, must meet increasing standards of professionalism; the culture of evidence-based medicine demands an evaluation of the effect educational programmes have on health care and service delivery. Educators cannot demonstrate that graduates possess the required attributes, or that their programmes have the desired impact on health care without appropriate assessment tools and measures of outcome. OBJECTIVE: To determine to what extent currently available assessment approaches can measure potentially relevant medical education outcomes addressing practitioner performance, health care delivery and population health, in order to highlight areas in need of research and development. METHODS: Illustrative publications about desirable professional behaviour were synthesized to obtain examples of required competencies and health outcomes. A MEDLINE search for available assessment tools and measures of health outcome was performed. RESULTS: There are extensive tools for assessing clinical skills and knowledge. Some work has been done on the use of professional judgement for assessing professional behaviours; scholarship; and multiprofessional team working; but much more is needed. Very little literature exists on assessing group attributes of professionals, such as clinical governance, evidence-based practice and workforce allocation, and even less on examining individual patient or population health indices. CONCLUSIONS: The challenge facing medical educators is to develop new tools, many of which will rely on professional judgement, for assessing these broader competencies and outcomes.     

The Bachelor of Medical Science research degree as a start for clinician-scientists

A retrospective case-control study of Bachelor of Medical Science (BMedSc) graduates of the University of Queensland found that 20% were in full-time academic or research positions compared with only 0.9% in controls (P less than 0.001). A larger percentage graduated from the medical course with honours degrees compared with controls (47.8% vs 10.8%, P less than 0.001). At the time of the survey, a higher percentage held higher research degrees (17.3% vs 3.0%) or were working towards such degrees (12.0% vs 0%). As a group they had published more articles in refereed scientific journals and more books or chapters in books (P less than 0.001). The majority of BMedSc graduates (75%) stated that the programme was 'a worthwhile endeavour' and 73.8% said they would do the degree again if given the time over as a medical student. These results indicate that the BMedSc programme, if properly promoted, can be a valuable means of reversing the decline in clinician-scientist manpower.     

Successful teaching in evidence-based medicine

OBJECTIVES: Several published articles have described the importance of exposing medical trainees to the 'new paradigm' of evidence-based medicine (EBM). Recognizing this, we sought to develop and objectively evaluate a mini-course in EBM for third-year medical students. DESIGN: We developed a mini-course consisting of four sessions in which students learn to derive sequentially focused questions, search MEDLINE, review articles critically and apply information from the literature to specific clinical questions. To evaluate the teaching intervention, we performed a controlled educational study. Students at the intervention site (n=34) attended the EBM mini-course, while students at the control site (n=26) received more 'traditional' didactic teaching on various clinical topics. Intervention and control students were surveyed immediately before and after the mini-course to assess changes in reading and literature searching skills, as well as a tendency to use the literature to answer clinical questions. SETTING: Boston University School of Medicine. SUBJECTS: Third-year medical students. RESULTS: The intervention was associated with significant changes in students' self-assessed skills and attitudes. MEDLINE and critical appraisal skills increased significantly in the intervention group relative to the control group (significance of between group differences: P=0.002 for MEDLINE and P=0.0002 for critical appraisal), as did students' tendency to use MEDLINE and original research articles to solve clinical problems (significance of between group differences: P=0.002 and P=0.0008, respectively). CONCLUSIONS: We conclude that this brief teaching intervention in EBM has had a positive impact on student skills and attitudes at our medical school. We believe that the key elements of this intervention are (1) active student involvement, (2), clinical relevance of exercises and (3) integrated teaching targeting each of the component skills of EBM.