Interrater and intrarater reliability in the evaluation of velopharyngeal insufficiency within a single institution

OBJECTIVE: To explore the interrater and intrarater reliability in nasoendoscopic assessment of velopharyngeal (VP) function using the standardized reporting method described by Golding-Kushner within a single institution. DESIGN: Prospective blinded study. SETTING: Academic, tertiary care, pediatric hospital. PARTICIPANTS: Six health care providers (2 pediatric otolaryngology faculty members, 2 pediatric otolaryngology fellows, and 2 speech pathologists) independently rated 50 videotaped nasoendoscopy segments twice. The segments on the videotape were obtained in a clinical setting. MAIN OUTCOME MEASURES: The Golding-Kushner rating system was used to rate VP function. Raters described VP closure quantitatively by rating palatal and lateral pharyngeal wall movement for each segment. They also qualitatively described characteristics of the VP gap, rated gap size as none, small, medium, or large, and estimated the percentage gap size relative to the resting position. Reliability coefficients were calculated for the data sets. RESULTS: Fairly good interrater and intrarater reliability was seen in the quantitative measures. Faculty otolaryngologists rated segments more similarly to each other than did pediatric otolaryngology fellows, but intrarater reliability was similar for both the experienced and less experienced otolaryngologists. Less consistency was seen in the ratings of the speech pathologists. Raters tended to rate with less consistency when describing qualitative characteristics of the VP gap than when making quantitative measurements. CONCLUSIONS: The Golding-Kushner scale is a reasonably reliable tool for reporting nasoendoscopic findings at our institution. However, these data also indicate that there exists room for improvement and that rater training may increase reliability.