Difficulty and discrimination capacity of examinations estimated on the basis of repeated student failure

This paper examines the difficulty and discrimination capacity of several subjects for problematic students, i.e., those failing repeatedly (re-examined four or more times) in at least one subject. Four hundred and six problematic students were analysed. The difficulty in passing thirty-one mandatory subjects was assessed in two ways:

• 1 on the basis of the attrition rate of the students due to pass the subject,
• 2 on the basis of the percentage of number of failures over the number of examinations.

The discrimination capacity of the same subjects was assessed on the basis of ‘false positive’ and ‘false negative’ results. The result of the evaluation of one subject was considered a false positive if it gave an excellent mark to a problematic student; it was considered a false negative if a student had to be reexamined four or more times before passing, although he had at least one excellent passing mark in one or more subjects. There were 157 excellent marks given to 114 problematic students. The two ways to assess difficulty ranked the thirty-one subjects in a very significantly-similar way. The subjects of the first, second and sixth years of study were considerably more difficult than the other years. A high proportion of false-negative results in a subject did not imply necessarily increased difficulty and vice versa. Thus, only two of the five subjects with the highest proportion of false-negative results belonged to the group of the five most difficult subjects. Also, three of the five subjects with the highest proportion of false-positive results belonged to the group of the five easiest subjects. Finally, only one of the five subjects with the lowest proportion of false (positive plus negative) results belonged to the group of the five most difficult subjects. Examination systems involving oral examinations either alone or in conjunction with written ones were more difficult for the problematic students than those involving purely written examinations, while their discrimination capacity was not significantly different. The discrimination capacity of the subjects involving practical examinations as part of their evaluation system was significantly better than that of those not involving practical examinations. It is concluded that a very permissive examination system presents the problem of repeatedly-failing students. A quasi-open admission system, as in Athens University, associated with a very permissive examination system, leads to a pattern of difficulty, the peaks of which appear, mainly, in the first and last year of studies.