Using cognitive models to develop quality multiple-choice questions |
| |
Authors: | Debra Pugh Andre De Champlain Mark Gierl Hollis Lai Claire Touchie |
| |
Affiliation: | 1. Department of Medicine, The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada;2. dpugh@toh.on.ca;4. Medical Council of Canada, Ottawa, Ontario, Canada;5. Faculty of Education, University of Alberta, Edmonton, Alberta, Canada;6. Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada;7. Medical Council of Canada, Ottawa, Ontario, Canada |
| |
Abstract: | With the recent interest in competency-based education, educators are being challenged to develop more assessment opportunities. As such, there is increased demand for exam content development, which can be a very labor-intense process. An innovative solution to this challenge has been the use of automatic item generation (AIG) to develop multiple-choice questions (MCQs). In AIG, computer technology is used to generate test items from cognitive models (i.e. representations of the knowledge and skills that are required to solve a problem). The main advantage yielded by AIG is the efficiency in generating items. Although technology for AIG relies on a linear programming approach, the same principles can also be used to improve traditional committee-based processes used in the development of MCQs. Using this approach, content experts deconstruct their clinical reasoning process to develop a cognitive model which, in turn, is used to create MCQs. This approach is appealing because it: (1) is efficient; (2) has been shown to produce items with psychometric properties comparable to those generated using a traditional approach; and (3) can be used to assess higher order skills (i.e. application of knowledge). The purpose of this article is to provide a novel framework for the development of high-quality MCQs using cognitive models. |
| |
Keywords: | |
|
|