Inter-Rater Agreement between Trachoma Graders: Comparison of Grades Given in Field Conditions versus Grades from Photographic Review

ABSTRACT

Purpose: Trachoma surveillance is most commonly performed by direct observation, usually by non-ophthalmologists using the World Health Organization (WHO) simplified grading system. However, conjunctival photographs may offer several benefits over direct clinical observation, including the potential for greater inter-rater agreement. This study assesses whether inter-rater agreement of trachoma grading differs when trained graders review conjunctival photographs compared to when they perform conjunctival examinations in the field.

Methods: Three trained trachoma graders each performed an independent examination of the everted right tarsal conjunctiva of 269 children aged 0–9 years, and then reviewed photographs of these same conjunctivae in a random order. For each eye, the grader documented the presence or absence of follicular trachoma (TF) and intense trachomatous inflammation (TI) according to the WHO simplified grading system.

Results: Inter-rater agreement for the grade of TF was significantly higher in the field (kappa coefficient, κ, 0.73, 95% confidence interval, CI 0.67–0.80) than by photographic review (κ?=?0.55, 95% CI 0.49–0.63; difference in κ between field grading and photo grading 0.18, 95% CI 0.09–0.26). When field and photographic grades were each assessed as the consensus grade from the three graders, agreement between in-field and photographic graders was high for TF (κ?=?0.75, 95% CI 0.68–0.84).

Conclusions: In an area with hyperendemic trachoma, inter-rater agreement was lower for photographic assessment of trachoma than for in-field assessment. However, the trachoma grade reached by a consensus of photographic graders agreed well with the grade given by a consensus of in-field graders.