Reliability of the Welling scale (WS1) for rating temporal bone dissection performance

OBJECTIVE: To determine the inter- and intrarater reliability of using a new scale (Welling scale) for resident evaluation of temporal bone dissection performance. STUDY DESIGN: Prospective, double-blinded, randomized trial. METHODS: Twelve residents in otolaryngology (postgraduate year [PGY] 2-5) drilled 26 temporal bones (21 cadaveric, 5 plastic) with the objective to perform a complete mastoidectomy with facial recess approach. These bones were then rated using the Welling scale by six independent raters on two separate occasions (4-6 wk apart). Raters were blinded to PGY year. The Kappa statistic was calculated for inter- and intrarater reliability. RESULTS: Intrarater agreement was high for all raters, ranging from kappa = 0.65 to 0.72 (all P < .001), whereas the interrater agreement scores were more moderate (range, kappa = 0.49-0.64; all P < .01). CONCLUSION: The Welling scale can be used reliably to assess temporal bone dissection performance where performance is measured by assessment of end product (mastoidectomy with facial recess approach).     

Measuring edema in irradiated head and neck cancer patients

OBJECTIVES: Edema is a common side effect of radiotherapy for head and neck cancer. Systems have been developed to record and monitor changes that occur after radiotherapy. These lack the sensitivity to record edema in specific laryngo-pharyngeal structures. The aim of this study was to develop a rating scale to measure edema in the larynx and pharynx. METHODS: This was an exploratory study to develop a new measure, with the help of an expert panel, assessing interrater and intrarater reliability. A consensus group developed the rating scale. Eleven structures and 2 spaces were identified as areas sensitive to the development of edema. The terms no, mild, moderate, and severe were used to describe the degrees of edema. The scale was piloted and then tested for interrater and intrarater reliability on 5 speech and language therapists. They viewed 25 nasendolaryngoscopic images (23 patients who had had radiotherapy and 2 healthy volunteers). The images were rated with the scale. This process was repeated 1 week later. RESULTS: Images were taken from patients with oral, oropharyngeal, nasopharyngeal, or laryngeal cancer. All had had radiotherapy or chemoradiotherapy. All raters were experienced in viewing larynges via nasendolaryngoscopy. The interrater reliability for scoring the edema rating scale was moderate (weighted kappa, 0.54). Lower levels of agreement were found for the tongue base, valleculae, pharyngeal walls, and anterior commissure. The intrarater reliability was very good (weighted kappa, 0.84). CONCLUSIONS: The edema rating scale can be scored with very good test-retest reproducibility and moderate levels of agreement between clinicians. Modifications to the method are suggested to increase interrater reliability.     

Stuttering and speech naturalness: audio and audiovisual judgments.

Unsophisticated raters, using 9-point interval scales, judged speech naturalness and stuttering severity of recorded stutterer and nonstutterer speech samples. Raters judged separately the audio-only and audiovisual presentations of each sample. For speech naturalness judgments of stutterer samples, raters invariably judged the audiovisual presentation more unnatural than the audio presentation of the same sample; but for the nonstutterer samples, there was no difference between audio and audiovisual naturalness ratings. Stuttering severity ratings did not differ significantly between audio and audiovisual presentations of the same samples. Rater reliability, interrater agreement, and intrarater agreement for speech naturalness judgments were assessed.     

Reliability of the perceptual evaluation of adductor spasmodic dysphonia

OBJECTIVES: Although perceptual assessment by experienced voice clinicians remains the gold standard for the diagnosis and assessment of severity of adductor spasmodic dysphonia (ADSD), the interrater reliability of voice experts for this task has not been assessed. In addition, it is unknown whether telephone-recorded or -transmitted voice samples could be used for this task. The aims of this study were (1) to assess the reliability of perceptual analysis of ADSD severity by voice experts and (2) to compare the results between digitally recorded voice samples and those recorded over the telephone. METHODS: Five laryngologists randomly selected voice samples from 46 ADSD patients and rated the severity of ADSD on a 5-point rating scale. A set of digital voice recordings and a set of telephone voice recordings made from filtering the digital set via the telephone were rated, and each voice set was rated twice. Measures of intrarater and interrater reliability, as well as a measure of the probability of agreement among the raters, were calculated. RESULTS: There was a high level of agreement on ADSD severity, with excellent interrater and intrarater reliability (Cronbach's alpha, .93 to .96). The probabilities of rater agreement on the digitally recorded and telephone-filtered voice samples were similar (chi2, p = .07). The ratings of digital versus telephone voice samples were highly correlated (Pearson r, 0.99; p < .001). CONCLUSIONS: These results demonstrate that voice experts are reliably able to judge and agree on the severity of ADSD. Telephone-filtered voices appear to convey adequate ADSD perceptual cues for expert listeners to judge the severity of spasmodic dysphonia.     

Practice of pediatric otolaryngology: results of the future of pediatric education II project

OBJECTIVES: To define the practice of pediatric otolaryngology compared with general otolaryngology and to estimate pediatric otolaryngology workforce utilization and needs. METHODS: Survey of members of the American Academy of Pediatrics Section on Otolaryngology and Bronchoesophagology and the American Society of Pediatric Otolaryngology and of a random sample of the membership of the American Academy of Otolaryngology-Head and Neck Surgery. RESULTS: Pediatric otolaryngologists were more likely to practice in urban and/or academic settings than were general otolaryngologists. Children (age <18 years) comprised over 88% of the patients of pediatric otolaryngologists and 30% to 35% of the patients of general otolaryngologists. Pediatric otolaryngologists were more likely to see children with complicated diseases such as airway disorders or congenital anomalies than were general otolaryngologists. Pediatric otolaryngologists, unlike general otolaryngologists, reported an increasing volume of pediatric referrals, as well as increased complexity in the patients referred. The surveyed physicians estimated the present number of pediatric otolaryngologists in their communities as approximately 0.2 to 0.3 per 100 000 people. CONCLUSIONS: Most children receiving otolaryngologic care in the United States receive such care from general otolaryngologists. The patient profile and practice setting of the subspecialty of pediatric otolaryngology differ from those of general otolaryngology. The demand for pediatric otolaryngologists appears to be increasing, but many general otolaryngologists do not believe there is an increased need.     

Listener agreement for auditory-perceptual ratings of dysarthria.

PURPOSE: Darley, Aronson, and Brown (1969a, 1969b) detailed methods and results of auditory-perceptual assessment for speakers with dysarthrias of varying etiology. They reported adequate listener reliability for use of the rating system as a tool for differential diagnosis, but several more recent studies have raised concerns about listener reliability using this approach. METHOD: In the present study, the authors examined intrarater and interrater agreement for perceptual ratings of 47 speakers with various dysarthria types by 2 listener groups (inexperienced and experienced). The entire set of perceptual features proposed by Darley et al. was rated based on a 40-s conversational speech sample. RESULTS: No differences in levels of agreement were found between the listener groups. Agreement was within 1 scale value or better for 67% of the pairwise comparisons. Levels of agreement were lower when the average rating fell in the mid-range of the scale compared with samples that had an average rating near either of the scale endpoints; agreement was above chance level. No significant differences in agreement were found between the perceptual features. DISCUSSION: The levels of listener agreement that were found indicate that auditory-perceptual ratings show promise during clinical assessment for identifying salient features of dysarthria for speakers with various etiologies.     

Projected Societal Needs in Pediatric Otolaryngology

With the recent increase in pediatric otolaryngology fellowship training programs, fellows are being trained in record numbers and there is concern about the possibility that the supply of these physicians will soon exceed societal needs. This study was conducted to determine the number of fellowship-trained pediatric otolaryngologists that would meet the needs of society and when this need will be met. Information was collected from national organizations and from a survey of fellowship training programs. A hospital-based practitioner model was used to predict societal needs based on the anticipated changes in patient care, mainly managed-care capitated systems. The projected societal need is 382 pediatric otolaryngologists. If current fellowship training levels continue, this need will be met in just over 7 years.     

The effect of visible speech in the perceptual rating of pathological voices

OBJECTIVE: To test a simple method for improving consistency among raters for the perceptual evaluation of pathological voice quality by providing visible speech (spectrogram) as additional information because, to date, the interrater variability still limits the widespread clinical use of the best available rating system. DESIGN: Experimental comparison between 2 different ways (with and without the addition of visible speech) of perceptual rating by trained professionals of recorded pathological voices. Furthermore, the correlation between acoustical (jitter, shimmer, and noise-harmonic ratio) and perceptual parameters was investigated in both rating conditions. SUBJECTS: Six experts evaluated 70 recorded pathological voices using the GIRBAS (grade, instability, roughness, breathiness, asthenicity, and strain) scale in 2 separate sessions: first, conventionally, without visible speech as additional information, and several months later, with visible speech as additional information. MAIN OUTCOME MEASURES: The kappa interrater agreement and the correlation coefficient between GIRBAS scores and acoustic measures. RESULTS: We found a significant effect of visible speech on the agreement between the raters. The interrater agreement according to kappa statistics was significantly stronger with the addition of visible speech than without for rating grade, roughness, and breathiness. The correlation between acoustical and perceptual parameters showed no significant effect of visible speech. CONCLUSIONS: The addition of visible speech to the perceptual evaluation of pathological voices is an interesting clinical asset to enhance its reliability. The addition of visible speech to the clinical setting is feasible, since affordable computer programs are currently available that can provide the spectrogram in quasi-real time while conversing with the patient. The acoustical analysis might be applied in addition to perceptual rating in a multidimensional approach to assess voice quality.     

Zur auditiven Bewertung der Stimmqualität

BACKGROUND: For routine clinical purposes, dysphonic voices are assessed perceptually using the GRBAS scale or analogues. For clinical application, the crucial question is the interrater reliability (IRR) of the auditory perceptual assessment of voice quality. Therefore, the IRR of the four point RBH (roughness, breathiness, hoarseness vs overall grade) scale was studied. Other parameters, e.g. validity and intrarater reliability were not considered. METHODS: A total of 78 patients read a standard text "Der Nordwind und die Sonne". These samples were evaluated by 19 speech and voice therapy students according to the degree of roughness, breathiness and hoarseness. Data were subjected to reliability analysis. RESULTS: Our data indicate a high IRR with a Cronbach's alpha of 0.94. No single rating of the 19 raters could be omitted without decreasing the IRR. DISCUSSION: The data indicate that the perceptual assessment of hoarseness for running speech is highly reliable. The application of the RBH scale is suitable for clinical purposes. It should be considered as an outcome measure.     

The reliability of the assessment of endoscopic laryngeal findings associated with laryngopharyngeal reflux disease

OBJECTIVE: To determine the reliability of the assessment of laryngoscopic findings potentially associated with laryngopharyngeal reflux disease (LPRD). STUDY DESIGN: Prospective randomized blinded study. METHODS: One hundred twenty video segments of rigid fiberoptic laryngeal examinations were prospectively analyzed by five otolaryngologists blinded to patient information and were scored according to several variables potentially associated with LPRD. Separate assessments of the degree of erythema and degree of edema were scored on a five-point scale for the anterior commissure, membranous vocal fold, and interarytenoid region. Similarly, interarytenoid pachydermia, likelihood of LPRD involvement, and severity of LPRD findings were assessed. For each of these scored physical findings, inter-rater and intrarater reliabilities were determined. RESULTS: The inter-rater reliabilities of the laryngoscopic findings associated with LPRD were poor. Intraclass correlation coefficients were 0.161 and 0.461 for edema of the arytenoids and membranous vocal folds, respectively (P <.001). Intraclass correlation coefficients were 0.181 and 0.369 for erythema of the arytenoids and membranous vocal folds, respectively (P <.001). Raters demonstrated poor agreement as to the severity of LPRD findings (intraclass correlation coefficient, 0.265) and the likelihood of an LPRD component for dysphonia (intraclass correlation coefficient, 0.248). Similarly, intrarater reliability was extremely variable for the various physical findings, with Kendall correlation coefficients ranging from -0.121 to 0.837. CONCLUSIONS: Accurate clinical assessment of laryngeal involvement with LPRD is likely to be difficult because laryngeal physical findings cannot be reliably determined from clinician to clinician. Such variability makes the precise laryngoscopic diagnosis of LPRD highly subjective.     

Professional diversity and personal commitments of pediatric otolaryngologists

OBJECTIVE: To characterize the time demands and practice patterns of pediatric otolaryngologists. DESIGN: Prospective survey of members from the American Society of Pediatric Otolaryngology. RESULTS: The survey response rate was 54% (n = 136) of practicing members of the American Society of Pediatric Otolaryngology. Respondents described being actively engaged in clinical otolaryngology (99%), hospital or practice administration (71%), private enterprise (17%), research (71%), and teaching (89%) on a weekly basis. Sixty percent considered their time demands to be "too busy"; however, few anticipated changing their activities in 5 years. Among the responding physicians, 90% believed that nonotolaryngology peers within their institutions viewed pediatric otolaryngology favorably whereas only 50% thought that other otolaryngologists held the same opinion. CONCLUSIONS: Pediatric otolaryngologists participate in many activities beyond clinical medicine. While most considered their time demands to be too busy, few anticipated a change in their activities. This may be reflective of a high level of job satisfaction, financial constraints, or the relative youth of the subspecialty.     

The KTP laser: an emerging tool in pediatric otolaryngology

PURPOSE: To report the frequency and spectrum of use of the potassium titanyl phosphate (KTP) laser in a tertiary referral pediatric otolaryngology practice and to focus on a novel use for the KTP laser which has not previously been described in the literature. DESIGN: A retrospective chart review of the operative log database of a pediatric otolaryngologist in a tertiary referral setting over a seven year period. RESULTS: Out of 2886 cases, a total of 49 (1.7%) involved the use of the KTP laser. These included 7 otologic cases, 3 laryngeal cases, 31 subglottic/tracheal cases, 1 esophageal case and 7 nasal cases. One of these cases involved a previously unreported use of the KTP laser, closure of a tracheo-esophageal fistula (TEF). CONCLUSION: The KTP laser is an important operative tool in pediatric otolaryngology and new uses for this laser continue to emerge. One of these, KTP closure of a TEF offers pediatric otolaryngologists the potential for significant changes in management of this congenital problem with reduced surgical morbidity. Familiarity with the KTP laser and expertise in its use and applications is essential in providing state-of-the-art care to pediatric otolaryngology patients in a tertiary referral center.     

Adolescent otolaryngology

OBJECTIVE: To determine the pattern of disease amongst ambulatory adolescents referred to a pediatric otolaryngology outpatient department. METHODS: Retrospective chart review of adolescents newly referred to a tertiary pediatric otolaryngology outpatient department over a 12-month period. RESULTS: One hundred and fifteen patients were included (male 56, female 59) mean age 14.9 years. There were 36 (31%) patients who had previously required otolaryngology management for another condition, and 29 patients with complex medical conditions. Investigations, including audiology and medical imaging, were performed in 35 patients. The patients were managed surgically 34 (30%), medically 28 (24%), referred to other departments 10 (9%) had no intervention 21 (18%), while 22 (19%) failed to attend for follow up. CONCLUSION: Ambulatory adolescent patients present with a comparable spectrum of otolaryngological problems to other pediatric age groups. These patients appear to be well managed by pediatric otolaryngologists, and there does not seem to be a need to develop a free standing adolescent otolaryngology subspecialty within tertiary pediatric otolaryngology at this time.     

Teaching emergency airway management using medical simulation: a pilot program

OBJECTIVE: Simulation is a tool that has been used successfully in many high performance fields to permit training in rare and hazardous events. Our goal was to develop and evaluate a program to teach airway crisis management to otolaryngology trainees using medical simulation. METHODS: A full-day curriculum in the management of airway emergencies was developed. The program consists of three airway emergency scenarios, developed in collaboration between attending otolaryngologists and faculty from the Center for Medical Simulation. Following each scenario, the participants are led in a structured, video-assisted debriefing by a trained debriefer. Didactic material on team leadership and crisis management is built into the debriefings. Pediatric otolaryngology fellows, residents, and medical students have participated in the four courses that have been held to date. Participants evaluated the program on a five-point Likert scale. RESULTS: A total of 17 trainees participated in four pilot training courses. The survey data are as follows: overall program, 5.0 (SD, 0.00); course goals, 4.79 (SD, 0.43); realism, 4.36 (SD, 0.63); value of lecture, 4.71 (SD, 0.47); and quality of debriefings, 4.92 (SD, 0.28). Sample comments include: "This is a valuable tool for students and residents since true emergencies in ORL are often life-threatening and infrequent," and "This is a great course-really all physicians should experience it." Overall evaluation was extremely positive and both residents and fellows described the course as filling an important void in their education. CONCLUSION: Medical simulation can be an extremely effective method for teaching airway crisis management and teamwork skills to otolaryngology trainees at all levels.     

Deficiencies of cross-training between pediatrics and otolaryngology

OBJECTIVE: Conditions relating to the ear, nose and throat are very frequent problems encountered by general pediatricians. Similarly, a major percentage of patients seen and operated on by the general otolaryngologist are of the pediatric age group. It is my hypothesis that there is a deficiency of cross-training between these two specialties. METHODS: All pediatric and otolaryngology program directors in Canada were contacted by mail. They were asked to complete a questionnaire regarding the need for further cross-training, what training their residents are presently receiving, and clinical entities that needed more attention. RESULTS: Data were tabulated and analyzed. 62.5% of pediatric program directors and 83.3% of otolaryngology program directors responded. All pediatric program directors indicated a need for teaching by otolaryngologists for their residents and 90% identified an area of deficiency in training. Similarly, 90% of otolaryngology program directors indicated a need for teaching by pediatricians for their residents and 89% of those identified an area of deficiency in training. CONCLUSIONS: These results suggest that there is a deficiency in cross-training between pediatrics and otolaryngology.     

Predictors of future success in otolaryngology residency applicants

OBJECTIVE To evaluate the information available about otolaryngology residency applicants for factors that may predict future success as an otolaryngologist. DESIGN Retrospective review of residency applications; survey of resident graduates and otolaryngology clinical faculty. SETTING Otolaryngology residency program. PARTICIPANTS Otolaryngology program graduates from 2001 to 2010 and current clinical faculty from Barnes-Jewish Hospital/Washington University School of Medicine. MAIN OUTCOME MEASURE Overall ratings of the otolaryngology graduates by clinical faculty (on a 5-point scale) were compared with the resident application attributes that might predict success. The application factors studied are United States Medical Licensing Examination part 1 score, Alpha Omega Alpha Honor Medical Society election, medical school grades, letter of recommendation, rank of the medical school, extracurricular activities, residency interview, experience with acting intern, and extracurricular activities. RESULTS Forty-six graduates were included in the study. The overall faculty rating of the residents showed good interrater reliability. The objective factors, letters of recommendation, experience as an acting intern, and musical excellence showed no correlation with higher faculty rating. Rank of the medical school and faculty interview weakly correlated with faculty rating. Having excelled in a team sport correlated with higher faculty rating. CONCLUSIONS Many of the application factors typically used during otolaryngology residency candidate selection may not be predictive of future capabilities as a clinician. Prior excellence in a team sport may suggest continued success in the health care team.     

The residency experience in pediatric otolaryngology

OBJECTIVES: To assess the residency experience in pediatric otolaryngology, determine the impact of pediatric fellowship programs on residency training, and evaluate the need for fellowship training in pediatric otolaryngology. DESIGN: An anonymous, web-based survey of chief residents in otolaryngology. METHODS: Respondents described their experience in pediatric otolaryngology using a 5-point Likert scale and reported their comfort levels (yes/no) with various medical and surgical issues in pediatric otolaryngology. RESULTS: The survey was successfully completed by 70 respondents, representing a response rate of 26%. The majority of the respondents reported positive experiences with regard to the following aspects of pediatric otolaryngology training: didactics (81%), clinical research opportunities (78%), positive faculty role models (87%), career mentorship (74%), independent medical (84%) and surgical (81%) decision-making, and overall comprehensive residency experience (87%). Basic science research opportunities (50%) were reported as less available than clinical research opportunities (78%) (P = .002). Compared with other surveyed issues, a lower comfort level was reported for management of craniofacial anomalies (P < .001), excision of large lymphatic malformations (P < .001), cochlear implantation (P < .001), laryngotracheal reconstruction (P < .001), and surgical correction of velopharyngeal insufficiency (P < .001). No statistically significant difference was noted in responses based on the presence of a fellowship program at the institution. CONCLUSIONS: The residency experience in pediatric otolaryngology is perceived as comprehensive by graduating chief residents participating in this survey. The presence of a fellowship program does not appear to negatively impact the residency experience. Based on the reported comfort levels, the management of complex issues in pediatric otolaryngology may require additional training.     

Developing the next generation of otolaryngologist-researchers

The lifeblood of any specialty is innovation and discovery. It is important for the field of otolaryngology and its patients that we identify, recruit, train, and develop the next generation of researchers in otolaryngology. This article describes programs and resources currently available to otolaryngologists in training and early in their career for their development as clinician-scientists. We describe the background of the current generation of National Institutes of Health-funded otolaryngologists and discern where the next generation might come from. Special attention is given to the National Institute on Deafness and Other Communication Disorders, which focuses on supporting research and research training in hearing, balance, smell, taste, voice, speech, and language, and to programs aimed at the development of clinician-scientists.     

Establishing normative voice-related quality of life scores within the pediatric otolaryngology population

OBJECTIVE: To establish normative values for voice-related quality of life across a broad pediatric otolaryngology population using the Pediatric Voice Outcome Survey (PVOS). DESIGN: Longitudinal study. SETTING: Multiphysician outpatient pediatric otolaryngology practice. METHODS: The PVOS was completed by 385 parents of children and adolescents aged 2 to 18 years. Of the 385 parents, 75 were readministered the instrument 2 weeks after no intervention had been provided. Data were collected regarding the patients' age, main diagnosis, and operative intervention. RESULTS: The PVOS demonstrated robust internal consistency with an overall Cronbach alpha value of.70. Test-retest reliability demonstrated a weighted kappa value of 0.89 (95% confidence interval, 0.84-0.95) The mean +/- SD converted score (based on a 0-100 scale) for the overall population was 80.5 +/- 19.9. Converted PVOS scores are provided for each subpopulation according to main diagnosis. The PVOS scores varied significantly according to age (P<.05) and preoperative and postoperative status following adenoidectomy (P<.05). CONCLUSION: The PVOS represents a valid and reliable instrument to measure voice-related quality of life in a broad pediatric otolaryngology population.     

Workforce Issues for the Academic Otolaryngologist

Background The current medico‐economic environment has led to profound changes in our health care system and questions of physician surplus. These issues have particularly affected the academic health care system, as research funding and departmental support have decreased, and many young otolaryngologists are questioning academic careers because of these uncertainties. The current study was undertaken to assess the workforce environment for the academic otolaryngologist, particularly the young physician. Methods Surveys were sent to the academic chairmen of all accredited otolaryngology residency programs in the United States, requesting information on faculty appointments—actual and projected—as well as subspecialty appointments and expectations of young faculty. Results The response rate was 60% (59/98). Faculty additions have been relatively stable from 1994 to 1998, with approximately 37 assistant professor and 5 associate professor positions filled yearly. Faculty additions were the result of departmental expansion in 83% of cases and spanned many subspecialties. The subspecialty positions most frequently added from 1994 to 1998 were generalists (57), head and neck oncologists (53), pediatric otolaryngologists (48), and otologists (39), with generalists filling 15 positions in 1998. Ninety‐three percent of programs anticipate faculty additions in the next 5 years; most will be at the assistant professor level (77%), with 30% of positions for generalists, 20% for head and neck oncologists, and 18% for pediatric otolaryngologists. Faculty expectations are primarily clinical, with research being least important. Conclusions Academic positions are available for the young otolaryngologist, particularly in the fields of general otolaryngology, head and neck oncology, and pediatric otolaryngology.