Reliability of consumer assessment of communication skills in a postgraduate family practice examination

Summary: Examinations of competence which may affect career prospects require measures which are of high reliability, and which can be demonstrated to be valid. In a New Zealand summative postgraduate examination of competence in family practice the doctor-patient communication skills of candidates were assessed by non-medically trained nominees of community organizations. The assessments were based on direct observation of the candidates' encounters with simulated patients. To estimate the reliability of the consumer examiner, after the examination the examiners re-scored a random selection of videotaped candidate encounters. The test-retest correlations of consumer scoring were demonstrated to be at a level consistent with adequate examination reliability (confidence interval 0.59-0-98).
Summary: Consumers may be valuable as a resource for the training and assessment of the communication skills of medical practitioners.