Development of validated instrument to measure medical student learning in clinical urology: a step toward evidence based education

PURPOSE: To date published efforts to assess and improve medical student learning in urology have been limited due to the lack of an assessment tool with which to measure student learning. We report the development of a validated measure of medical student learning in urology. MATERIALS AND METHODS: Four core topics in clinical urology were selected as the focus of the test development, namely prostate cancer, screening with prostate specific antigen, benign prostatic hyperplasia and erectile dysfunction. Detailed curricula and multiple choice questions were created for each topic. Content validity of the curriculum and 28 item examination was established by a panel of 2 urologists and 2 medical physicians. Instrument reliability was determined by administering the test on line to third-year surgery students. Test construct validity was established through its administration to 19 urology residents and attending physicians. RESULTS: Reliability of the 28-item test instrument was measured by Cronbach's alpha at 0.76 and its 1-week test-retest reliability was 0.72. All urology experts performed well on the test. Mean urological expert scores were significantly higher than mean student post-test scores (24.9 +/- 2.1 vs 17.8 +/- 3.8, 2-tailed t test p <0.001). Urological experts with greater urological training had higher scores than those with less residency training. CONCLUSIONS: This study documents the development of a validated measure of medical student learning in urology. This validated instrument has the potential to improve educational quality control at medical schools and facilitate the development of effective, evidence based teaching methods.