Use of Professional Associate Ratings to Assess the Performance of Practicing Physicians: Past, Present, Future

The American Board of Family Practice (ABFP) is developing a computer-based testing system that will create realistic clinical encounters using an adaptation of an item generation process. Simulated patients' entire lives will be stochastically produced from a knowledge base, with constraints applied to prevent implausible simulations. The constraint mechanisms include knowledge acquisition decisions about grouping closely related medical concepts and widespread use of Bayesian networks to manage dependencies between concepts. Bayesian networks and fuzzy definitions provide stochastic variability between simulations produced from the same data. Examinees will interact with these patients using a large and stable set of queries and interventions. Multiple management plans associated with patient simulations provide a framework for scoring performance. All major components, including Health States, history generating "Lead To" objects, and Plans are reusable and often substitutable. Although initial knowledge acquisition demands are enormous, the system has good potential for low cost maintenance of content areas, and economies of scale as simulations and components are reused.