Predictive validity of the MCAT for students with two sets of scores

Addressed in the study reported here is the question of which set of scores for those students who retake the Medical College Admission Test (MCAT) yields a better predictive validity. The sample was comprised of 304 students who retook the MCAT prior to entering Jefferson Medical College between 1978 and 1981. Five sets of MCAT scores were considered as predictors in the study: earlier, later, higher, and lower sets of MCAT scores and the average of the earlier and later scores for each MCAT subtest. Twenty-five criteria were used, including grades earned in the freshman and sophomore years and scores on the subtests of Part I and Part II of the examinations of the National Board of Medical Examiners. Correlational techniques, such as bivariate and multiple correlation analyses and canonical correlation followed by redundancy analysis were utilized. The magnitude of redundancy indices indicated that the set of MCAT scores in which the earlier and later scores were averaged was the best predictor, followed by the earlier, lower, higher, and later sets of MCAT scores. The implications of these findings for the admissions process and for validity studies are discussed.     

History of not completing courses as predictor of academic difficulty among first-year students

The study reported here was undertaken to determine whether the prediction of academic difficulty in the first year of medical school is enhanced by a consideration of the number of courses withdrawn from, repeated courses, and incomplete courses on a student's undergraduate academic record. All students enrolled from 1981 through 1985 at Southern Illinois University School of Medicine who experienced academic difficulty in the first year were selected for the study. Successful students were matched with these students in terms of minority or majority status and served as a control group. Discriminant and classification analyses were performed in a hierarchical stepwise manner to predict success or difficulty in the first year. The variables that were significant in discriminating between minority students who had academic difficulty and those who did not were the science grade-point average (grades in biology, chemistry, physics, and mathematics), the score on the reading subtest of the Medical College Admission Test (MCAT), and the number of withdrawals from courses. For majority students, the significant discriminating variables were the score on the MCAT biology subtest and the number of incompletes taken for courses. The results of this study have implications for medical school admissions committees, premedical advisers, and premedical students.     

Predicting academic performance at a predominantly black medical school

A study was conducted by the authors that examined the validity of the Medical College Admission Test (MCAT), undergraduate grade-point average (GPA), and "competitiveness" of undergraduate college in predicting the performance of students at a predominantly black college of medicine. The performance measures used in the analysis consisted of course grades in all four years of medical school and scores on both Part I and Part II of the National Board of Medical Examiners (NBME) examinations. The predictive validities of the MCAT scores and undergraduate GPAs were found to be similar to those revealed in earlier studies conducted at predominantly white schools. Two exceptions to these similarities were found. First, the MCAT scores at the black school had a somewhat lower validity in predicting NBME examination scores than was the case at the other schools. Second, of the six MCAT subtest scores, Skills Analysis: Reading had the highest correlation with first-year grades, in contrast to results at the other schools. No differences between men and women were found in the validity of MCAT scores and the GPA. The competitiveness of the undergraduate college attended was found to contribute significantly to the prediction of all measures of medical school performance.     

Traditional criteria as predictors of minority student success in medical school.

There have been many reports stating that the traditional criteria of the Medical College Admission Test (MCAT) and undergraduate grade-point average (GPA) have little, if any, value in predicting success in the preclinical years of medical school among students from underrepresented (racial and ethnic) minority groups. In contrast to previous articles this report emphasizes that traditional criteria and the quality of the undergraduate college attended are of some statistical value in predicting success in the preclinical years of medical school among accepted students from under represented minority groups. Of these criteria, the one with the greatest predictive value is the selectivity of the undergraduate college attended.     

Relationship between past academic performance and results of specialty in-training examinations

In this study, the authors review the records of 63 graduates of Northwestern University Medical School who were residents in its graduate medical education programs of anesthesia and orthopedic surgery. They examine the relationship among college grades, medical school performance, and the results of assessment by annual, nationwide, medical specialty in-training examinations. For the anesthesia group, the best predictors of in-training examination performance were the Medical College Admission Test (MCAT) Verbal Ability score, the college grade-point average for nonscience subjects, and the MCAT Science, General Information, and Quantitative Ability scores. For the orthopedic group, the best predictors were the MCAT Verbal Ability score, the college grade-point average in nonscience subjects, the MCAT Science score, and the National Board of Medical Examiners Part I and Part II examination scores. The previous academic records for the 63 residents contained little to presage results in the in-training examination. The correlation obtained between nonscience college subjects and the in-training examination results was negative.     

Use of a medical reasoning aptitude test to help predict performance in medical school

A medical reasoning aptitude test (MRAT) was designed to assess aptitude for clinical problem-solving in medical school applicants. The purpose of the study reported here was to determine whether the information provided by this test, when used in conjunction with college grade-point averages (GPAs) and scores on the Medical College Admission Test (MCAT), would improve the prediction of medical school performance. Specifically, the authors investigated the incremental predictive value of the MRAT relative to students' overall performance in medical school and, more specifically, to their knowledge and clinical reasoning during preclinical years and clinical performance during the clerkship year. Overall, it was found that, with the exception of the first year, the addition of MRAT scores to the GPAs and MCAT scores increased precision in identifying students who performed poorly or exceptionally well in the second year and the clinical clerkship year. This last finding is especially useful, since few other tests have provided that information. On the basis of the preliminary findings, the authors propose further use and validation of the MRAT.     

Use of MCAT data in selecting students for admission to medical school

In the spring of 1986, medical school admissions personnel were surveyed on their institutions' admissions practices and the use of Medical College Admission Test (MCAT) data in student selection. The admissions officers listed sources of information considered in processing applications. The variables accorded high importance were: overall and science undergraduate grade-point averages, quality of degree-granting institutions, letters of evaluation, interview ratings, MCAT scores, extracurricular activities, work in areas related to health care, breadth and/or difficulty of course work, and state of legal residence. Variables judged of medium importance were: nonscience grade-point average, graduate study, narratives supplied through the American Medical College Application Service or supplemental narratives, demographic factors, and undergraduate research. The respondents divided the 15-point MCAT scale into exemplary, acceptable, and unacceptable ranges of performance. These responses varied widely among institutions. The mean response for the bottom of the acceptable range was a score of seven, and the mean for the beginning of the exemplary range was 11. In considering individual MCAT subtest scores, 34 percent of the admissions officers regarded the six scores individually and equally, 51 percent weighted the areas in ways related to their curricula, and 43 percent summed or averaged scores with equal weights at some point. Thirty-one percent used MCAT scores to adjust grade-point averages across undergraduate institutions.     

A model for predicting performance on the NBME Part I examination

Multiple regression analyses were employed to determine the relationships between achievement variables as predictors of the performance on Part I of the National Board of Medical Examiners examinations (NBME-I) of 366 medical students. The students' examination score averages in first- and second-year basic science courses were sequentially added to a composite Medical College Admission Test (MCAT) score to investigate the increases in prediction accuracy. Four prediction equations were cross validated on two subsequent medical school classes. While the multiple correlation between the predicted NBME-I score and the actual score significantly increased as first- and second-year course examination scores were added to the equations, the overall accuracy in predicting passing or failing did not substantially increase. The most useful equation employed a combination of the MCAT score and the first semester anatomy course examination score. The NBME-I predictions were given by letter to the 1984 and 1985 students approximately a year before they were to take the NBME-I examination so that students at risk of failure could undertake remedial study.     

Changes in the qualifications of medical school applicants, 1981 to 1985

The decline in the number of medical school applicants has prompted concerns among medical educators regarding the effects of this decrease on the qualifications of the applicant pool. Changes in the qualifications of medical school applicants were analyzed in terms of age, sex, and ethnicity or racial group using two measures of quality: Medical College Admission Test (MCAT) scores and grade-point averages. The distributions of these measures for 1981 and 1985 applicants were compared. The results showed that there were significant but modest gains over the years in the percentage of men and women applicants who scored from 10 to 15 on the MCAT biology, chemistry, physics, and science problems subtests. There were also significant percentage gains for applicants in all age categories, except applicants over age 32, and for all ethnic or racial groups except blacks. The authors conclude that a variety of sociodemographic, educational, and selection factors may account for these changes.     

The new MCAT and medical student performance

The "old" Medical College Admission Test (MCAT) has been replaced by the "new" MCAT. For many years the University of California, Irvine, has included the old MCAT in a prediction index as part of its admissions procedure. The use of the New MCAT has required this index to be recalculated. Since the predictive power of the new MCAT is unknown, it has been necessary to wait until the first class admitted completed its freshman year. This paper is an analysis of the new MCAT and its relationship to the criterion of class standing achieved in the school year ending in the spring of 1979.     

Evaluation of a preentrance enrichment program for minority students admitted to medical school

The authors report on an enrichment program for minority students and its effect on those students' performance in the first year of medical school. The six-week preentrance enrichment program (PEP) offers a curriculum of basic science courses, study skills training, and a seminar on special topics, typically on sociomedical issues. From 1973 through 1984, 115 students who had been admitted to Boston University School of Medicine participated in the PEP prior to matriculation. The PEP participants had lower scores on the Medical College Admission Test than minority nonparticipants admitted to the medical school. Analysis of the first-year performance of PEP participants, however, indicated that the participants had significantly higher proportions of pass and honors grades than minority nonparticipants in two courses and slightly higher (though not significant) proportions of pass and honors grades in six other courses of the nine courses in the first-year curriculum. In a related study, preliminary analysis showed that participation in the PEP was one of the consistent predictors of overall academic performance during the first year. Questionnaires completed by participants before and after the program and their responses to a survey at the end of their first year of medical school indicated the participants felt they had gained benefits from the program that enhanced their academic adjustment.     

Program for increasing enrollment of early acceptees in medical school

Early acceptees by medical schools generally have higher Medical College Admission Test (MCAT) scores and grade-point averages than those accepted later. Between 1977 and 1979, the number of these individuals actually enrolling at the University of Medicine and Dentistry of New Jersey Rutgers Medical School (UMDNJ-RMS) declined from 20.4 to 6.5 percent of all incoming medical students. This prompted the development in 1980 of a one-day, preenrollment program informing early acceptees about the UMDNJ-RMS educational program. Subsequently, more than 40 percent of the early acceptees have enrolled at the UMDNJ-RMS, and this noticeable increase appears to be related directly to the program.     

The effect of commercial coaching courses on performance on the MCAT

Two studies on the relationship between taking a commercial coaching course and performance on the Medical College Admission Test (MCAT) are reported. In both studies, one based on repeating examinees and another based on first-time examinees, differences in scores favoring coached examinees were found on the Science Knowledge and Science Problems subtests. The results on the Skills Analysis: Quantitative subtest were mixed, while neither study found differences between the two groups in scores on the Skills Analysis: Reading subtest. The analyses further revealed that the differences in scores on the Science Knowledge and Science Problems subtests were less pronounced for examinees with low scores on the Skills Analysis: Reading and Skills Analysis: Quantitative subtests, examinees with very low or very high undergraduate grade-point averages, and examinees from very selective undergraduate colleges. The substantive importance of the differences was also explored. The coaching effect was estimated to increase an average student's probability of acceptance by 5 percentage points. The results suggest that coaching for the MCAT has a limited effect that supports rather than undermines the validity of the test.     

A report on graduates of the Boston University six-year combined liberal-arts--medical program.

Graduates of a six-year combined Liberal-Arts-Medicine Program and their medical school classmates (traditional "eight-year" students) are compared as to their medical school performance and their professional postgraduate activities. On standardized examinations (Medical College Admission Test and examinations of the National Board of Medical Examiners) the six-year group was somewhat better than the eight-year group. In other aspects, such as class ranking, honors at graduation, and medicine clerkship grades, the six- and eight-year groups were similar. The two groups were remarkably similar in their postgraduate professional career choices and in achieving board certification. The data for the first three classes indicate that qualified high school students can succeed academically in an accelerated collegiate-degree program, do well in medical school, achieve postgraduate certification, and begin the practice of medicine at a younger age.     

医学生社会支持现状调查分析

目的 了解医学生的社会支持状况.方法采用社会支持评定量表对遵义医学院的886名学生的社会支持状况进行调查.结果 (1)不同性别医学生在社会支持各维度得分差异无统计学意义.(2)普通本科和二级学院的医学生在社会支持各维度得分差异无统计学意义.(3)农村生源的医学生在主观支持和总分上明显高于城镇医学生.(4)大一和大五医学生在社会支持利用度上得分较高,明显高于大二、大三、大四3个年级.结论 医学生获得的社会支持在性别和层次上无差异,但在生源地和年级上不同.     

Longitudinal study of medical students selected for admission to medical school by actuarial and committee methods.

This study evaluates a "mid period" follow-up evaluation of the outcomes of selection of medical students by customary committee review procedures versus actuarial selection. One-third of a freshman class was selected solely on the basis of a predictor index which was a previously validated, optimally weighted combination of scores on the Medical College Aptitude Test and the premedical grade-point average. The remaining two-thirds were selected by committee decision based on review of the total application file which, in addition to the aptitude test scores and academic record, included basic demographic data, information on extracurricular activities, avocational interests, work experience, letters of recommendation, personality test profiles, and interview impressions. In a previous study, it was reported that the two groups of students were undifferentiated with respect to their academic standing at the close of their sophomore year. In the present study, the actuarially selected and committee selected students were compared on class rank at the end of the junior year, total and subtest scores on part II of the National Board Examinations administered toward the close of their senior year, and type and location of internship, and practice or training status one year after graduation. The two groups were not reliably differentiated on any of these variables. Implications of the findings are discussed with respect to reliability, efficiency, and economy in the selection process and the function of the admissions committees with respect to borderline cases and issues of school policy and philosophy.     

Effect of two preclinical curricula on NBME Part I examination performance

The Michigan State University College of Human Medicine conducts two preclinical medical education programs. In Track I (lecture-based), students attend classes 24 hours per week, and lecture time totals 908 hours over a 50-week period. In Track II (problem-based), students attend classes only 12 hours per week, and lecture time totals 112 hours over the same 50-week period. Institution of the Part I examination of the National Board of Medical Examiners (NBME) as a graduation requirement provided an opportunity to compare the performances of students in both tracks. When students from each track with similar Science Problems subtest scores on the Medical College Admission Test were compared, no significant differences were observed in the students' total scores or pass rate on the NBME examination. However, there were significant differences in scores on the microbiology subtest of the NBME examination, with the Track I students achieving higher scores. The 1984 report of the Project Panel on the General Professional Education of the Physician and College Preparation for Medicine of the Association of American Medical Colleges stressed the need to examine critically and consider reducing the scheduled instructional and lecture hours in preclinical medical education programs. In the study reported here, the authors demonstrated that reduction of scheduled instructional time, when replaced by a guided problem-solving program, is not detrimental to students' performance on the NBME Part I examination.     

Recruitment and progress of minority medical school entrants 1970-1972.

This is a report on a national study of minority group applicants and entrants to the 1970, 1971, and 1972 entering classes of U.S. medical colleges. The aim of the investigation was to further understanding of the factors involved in attempting to increase minority representation in education for the practice of medicine. Data from the Association of American Medical Colleges are used to examine characteristics of successful and unsuccessful minority applicants to medical school. Socieconomic, personal, institutiona, and geographical factors that relate to the recruitment and progress of minority students in medicine are analyzed and evaluated. Differences between Caucasian and minority group students affecting admissions, retention, and promotion are documented. The investigators also compare the projections of a 1970 AAMC task force report with actual occurrences in the national effort to expand educational opportunities in medicine for blacks and other underrepresented minority students (that is, American Indians, Mexican Americans, and mainland Puerto Ricans). This comparison shows substantial progress toward the projected figures but a need for renewed commitment if they are to be reached. Suggestions are offered for improving the recruitment and progress of minority medical school entrants by such means as the AAMC Simulated Minority Admissions Exercises and by ongoing programs at individual medical schools. The study also yielded such pertinent findings as the following: 1. Confirmation that the racial characterizations self-reported by medical school applicants have a high degree of accuracy and an increasing degree of completeness. 2. An encouraging increase in the number of black premedical students who will potentially apply for the medical school classes entering in 1976 and 1977. 3. Growth in the enrollment of low-income medical students, most of it explained by the increase in the numbers of minority group members who have been admitted in recent years. 4. More mobility among blacks than Caucasians with regard to attending medical schools in other than their region of legal residence. 5. A higher proportion of women, of older, and of married students among minority medical school matriculants than among Caucasian matriculants. 6. A slightly higher medical school retention rate for Caucasians than for students from underrepresented minority groups, possibly explained in part by the greater diversity in the socioeconomic and educational backgrounds of the latter. 7. A positive relationship for blacks between the size of undergraduate college attended and successful completion of the first year of medical school.     

汉区少数民族医学生自我教育的调查分析

目的调查汉区少数民族医学生自我教育情况。方法应用医学生自我教育问卷对中国医科大学117名来自少数民族聚居地区的五年学制少数民族医学生进行自我教育情况的调查,并对结果进行分析。结果不同年级医学生自我教育总问卷、自我调控因子、自我认知因子、自我实现因子评分差异有统计学意义（P〈0.05）,三年级、五年级少数民族医学生的自我教育情况优于其他年级;女性少数民族医学生自我教育问卷评分显著高于男性少数民族医学生的评分（P〈0.05）;不同民族间医学生自我教育情况差异无统计学意义（P〉0.05）;少数民族医学生自我教育评分优于同校的医学生（P〈0.05）。结论医学教育有其特殊性,教育工作者应根据医学教育以及少数民族医学生的自我教育的特点,利用少数民族医学生自我教育自觉性的优势,帮助其实现学业发展目标。     

The stability of early specialty preferences among US medical school graduates in 1983

Medical graduates in 1983 were in preclinical training when the Graduate Medical Education National Advisory Committee forecast a surplus of 70,000 physicians by 1990. Among the problems identified was the nuclear role of medical schools in affecting specialty choices. To understand this role further, the current study determined the stability and evolution of specialty preferences between the time of the Medical College Admission Test and the senior year of medical school. The study included 10,321 US medical school graduates in 1983. Eighty percent changed their specialty preference during this interval, demonstrating the substantial effects that medical schools have on specialty selection. The stability of early preferences ranged from 41% to 1%. Interest in primary care specialties declined among both men and women; interest in specialty care and supporting services increased during this five-year longitudinal study. These findings parallel shifts away from primary care among US medical school graduates in 1978 and 1983.