The reliability and validity of the modified Unified Classification System for periprosthetic femoral fractures after hip arthroplasty

BackgroundThis study assessed the reliability and validity of the modified Unified Classification System for femur fractures after hip arthroplasty.MethodsFour hundred and two cases were evaluated by 6 observers, 3 experts and 3 trainee surgeons. Each observer read the radiographs on 2 separate occasions and classified each case as to its type. Reliability was assessed by looking at the intraobserver and interobserver agreement using the Kappa statistic. Validity was assessed within the B group by looking at the agreement between the radiographic classification and the intraoperative findings. Interobserver and intraobserver agreement and validity were analyzed, using weighted kappa statistics.ResultsThe mean k value for interobserver agreement was found to be 0.882 (0.833–0.929) for consultants (almost perfect agreement) and 0.776 (0.706–0.836) for the trainees (substantial agreement). Intraobserver k values ranged from 0.701 to 0.972, showing substantial to almost perfect agreement. Validity analysis of 299 type B cases revealed 89.854% agreement with a mean k value of 0.849 (0.770–0.946) (almost perfect agreement).ConclusionsThis study has shown that the modified Unified Classification System is reliable and valid. We believe it is useful to improve the judgment of the implant stability, and establish the therapeutic strategy for periprosthetic femoral fracture.     

Inter-observer reliability and intra-observer reproducibility of the Weber classification of ankle fractures

Our aim was to assess the reproducibility and the reliability of the Weber classification system for fractures of the ankle based on anteroposterior and lateral radiographs. Five observers with varying clinical experience reviewed 50 sets of blinded radiographs. The same observers reviewed the same radiographs again after an interval of four weeks. Inter- and intra-observer agreement was assessed based on the proportion of agreement and the values of the kappa coefficient. For inter-observer agreement, the mean kappa value was 0.61 (0.59 to 0.63) and the proportion of agreement was 78% (76% to 79%) and for intra-observer agreement the mean kappa value was 0.74 (0.39 to 0.86) with an 85% (60% to 93%) observed agreement. These results show that the Weber classification of fractures of the ankle based on two radiological views has substantial inter-observer reliability and intra-observer reproducibility.     

Inter- and intra-observer variability of the Crowe and Hartofilakidis classification systems for congenital hip disease in adults

Our study evaluated the reliability of the Crowe and Hartofilakidis classification systems for developmental dysplasia of the hip in adults. The anteroposterior radiographs of the pelvis of 145 patients with 209 osteoarthritic hips were examined twice by three experienced hip surgeons from three European countries and the abnormal hips were rated using both classifications. The inter- and intra-observer agreement was calculated. Interobserver reliability was evaluated using weighted and unweighted kappa coefficients and for the Crowe classification, among the three pairs there was a minimum kappa coefficient with linear weighting of 0.90 for observers A and C and a maximum kappa coefficient of 0.92 for observers B and C. For the Hartofilakidis classification, the minimum kappa value was 0.85 for observers A and B, and the maximum value was 0.93 for observers B and C. With regard to intra-observer reliability, the kappa coefficients with linear weighting between the two evaluations of the same observer ranged between 0.86 and 0.95 for the Crowe classification and between 0.80 and 0.93 for the Hartofilakidis classification. The reliability of both systems was substantial to almost perfect both for serial measurements by individual readers and between different readers, although the information offered was dissimilar.     

Inter- and intraobserver reliability of the Crowe and Hartofilakidis classifications in the assessment of developmental dysplasia of the hip in adult patients

Background

The purpose of this study is to investigate the inter- and intraobserver reliability of Crowe and Hartofilakidis classifications in the assessment of developmental dysplasia of the hip in adult patients.

Materials and methods

Two consultant orthopedic surgeons classified 141 dysplastic hips on 103 standard anterior–posterior pelvis radiographs according to the Crowe and Hartofilakidis classifications. Assessments were performed in random order by each observer on two separate occasions, at least 4?weeks apart. Kappa statistics were used to establish a relative level of agreement between observers for the two readings and between separate readings by the same observer.

Results

At the first readings, interobserver reliability analysis revealed kappa coefficient of 0.71 for the Crowe classification and 0.54 for the Hartofilakidis classification. At the second reading, the kappa coefficient was 0.72 for the Crowe classification and 0.75 for the Hartofilakidis classification. Intraobserver reliability analysis revealed kappa coefficients of 0.71 for the Crowe and 0.80 for the Hartofilakidis classification for observer A, and 0.76 and 0.70 for observer B.

Conclusions

In conclusion, we have found substantial inter- and intraobserver agreement for Crowe classification and substantial to moderate agreement for Hartofilakidis classification in this study. Both classification systems assess the different aspects of developmental dysplasia of hip in adults. Each system has advantages and disadvantages. We suggest using both of these classifications together to increase the accuracy.     

Reliability and validity of the Hartofilakidis classification system of congenital hip disease in adults

The reliability and validity of the Hartofilakidis et al. classification system in adults with congenital hip disease (CHD) were examined. The radiographs of 102 adult patients (158 hips) with CHD were independently assessed by three senior surgeons. Interobserver variability was assessed by examining the agreement between the three raters while validity of the classification system was assessed by examining the agreement between the assessment by either one of the three raters and the intraoperative finding (reference standard). The interobserver agreement between the three observers was high ranging from 0.720 to 0.854 (substantial to excellent) while the agreement of the preoperative prediction with the intraoperative findings was 87.4% (K = 0.823, excellent agreement). The Hartofilakidis et al. classification system reliably predicts from preoperative pelvis radiographs the bone deficiencies encountered during the operation.     

Reliability of the AO/ASIF classification for pertrochanteric femoral fractures

20 radiographs of pertrochanteric femoral fractures were classified as to fracture "group" and "sub-group" according to the AO/ASIF Fracture Classification (type 31A) by 15 observers. 3 months later, the same radiographs were reviewed by the same observers. Mean agreement of the observers with the final consensus ranged from 53% (with subgroup classification) to 81% (without subgroup). The mean kappa value for interobserver reliability was 0.33 and 0.34 for classification with subgroup in both observer sessions, respectively. Omission of the subgroup classification resulted in better mean kappa values (0.67 and 0.63, respectively). Mean intraobserver reliability was 0.48 in the fracture "subgroup" and 0.78 in the "group" classification. In conclusion, the results show that the AO/ASIF classification for pertrochanteric fractures is reliable for fracture subgroups 31A1, A2 or A3. The group classification should be used to compare scientific data and determine the best treatment. Further classification of fracture subgroups leads to poor reproducibility of results.     

Reliability of the AO/ASIF classification for pertrochanteric femoral fractures

20 radiographs of pertrochanteric femoral fractures were classified as to fracture "group" and "subgroup" according to the AO/ASIF Fracture Classification (type 31A) by 15 observers. 3 months later, the same radiographs were reviewed by the same observers. Mean agreement of the observers with the final consensus ranged from 53% (with subgroup classification) to 81% (without subgroup). The mean kappa value for interobserver reliability was 0.33 and 0.34 for classification with subgroup in both observer sessions, respectively. Omission of the subgroup classification resulted in better mean kappa values (0.67 and 0.63, respectively). Mean intraobserver reliability was 0.48 in the fracture "subgroup" and 0.78 in the "group" classification. In conclusion, the results show that the AO/ASIF classification for pertrochanteric fractures is reliable for fracture subgroups 31A1, A2 or A3. The group classification should be used to compare scientific data and determine the best treatment. Further classification of fracture subgroups leads to poor reproducibility of results.     

Reliability and intraoperative validity of preoperative assessment of standardized plain radiographs in predicting bone loss at revision hip surgery.

BACKGROUND: The most challenging aspect of revision hip surgery is the management of bone loss. A reliable and valid measure of bone loss is important since it will aid in future studies of hip revisions and in preoperative planning. We developed a measure of femoral and acetabular bone loss associated with failed total hip arthroplasty. The purpose of the present study was to measure the reliability and the intraoperative validity of this measure and to determine how it may be useful in preoperative planning. METHODS: From July 1997 to December 1998, forty-five consecutive patients with a failed hip prosthesis in need of revision surgery were prospectively followed. Three general orthopaedic surgeons were taught the radiographic classification system, and two of them classified standardized preoperative anteroposterior and lateral hip radiographs with use of the system. Interobserver testing was carried out in a blinded fashion. These results were then compared with the intraoperative findings of the third surgeon, who was blinded to the preoperative ratings. Kappa statistics (unweighted and weighted) were used to assess correlation. Interobserver reliability was assessed by examining the agreement between the two preoperative raters. Prognostic validity was assessed by examining the agreement between the assessment by either Rater 1 or Rater 2 and the intraoperative assessment (reference standard). RESULTS: With regard to the assessments of both the femur and the acetabulum, there was significant agreement (p < 0.0001) between the preoperative raters (reliability), with weighted kappa values of >0.75. There was also significant agreement (p < 0.0001) between each rater's assessment and the intraoperative assessment (validity) of both the femur and the acetabulum, with weighted kappa values of >0.75. CONCLUSIONS: With use of the newly developed classification system, preoperative radiographs are reliable and valid for assessment of the severity of bone loss that will be found intraoperatively.     

3D-radiographic analysis does not improve the Neer and AO classifications of proximal humeral fractures

The Neer and AO fracture classifications for fractures of the proximal humerus have shown poor reproducibility based on plain radiography. We wanted to investigate whether the addition of 3-dimensional (3D) reconstructions would increase the reproducibility of classification. 7 observers independently classified 24 fractures of the proximal humerus using both plain radiographs, CT and 3D and the classification was repeated 2 months later. There was a moderate interobserver agreement when using the Neer classification, but only a fair agreement with the AO classification. The Neer system had a mean kappa value of 0.44 and the AO had a value of 0.32 for the first assessment. In the second assessment, the mean kappa values were 0.49 and 0.34, respectively. Intraobserver reproducibility was fair to substantial agreement for Neer (kappa range 0.27-0.73) and for AO (kappa range 0.29-0.74). In conclusion, the addition of CT and 3D to plain radiographs did not improve the reproducibility of the classifications of Neer and AO of the proximal humerus.     

Reliability of the Lichtman classification of Kienböck's disease

The purpose of this study was to establish the interobserver reliability and intraobserver reproducibility of the staging of Kienb?ck's disease according to Lichtman's classification. Posteroanterior and lateral wrist radiographs of 64 patients with a diagnosis of Kienb?ck's disease and 10 control subjects were reviewed independently by 4 observers on 2 separate occasions. The reviewers included 3 hand fellowship-trained surgeons and 1 orthopedist who was not fellowship-trained in hand surgery. A stage was assigned to each set of radiographs according to the Lichtman classification. Paired comparisons for reliability among the 4 observers showed an average absolute percentage agreement of 74% and an average paired weighted kappa coefficient of 0.71. Furthermore, all the controls were correctly classified as stage I, which is in accordance with the Lichtman system. With regard to reproducibility, observers duplicated their initial readings 79% of the time with an average weighted kappa coefficient of 0.77. These results indicate substantial reliability and reproducibility of the Lichtman classification for Kienb?ck's disease.     

Two and three-dimensional computed tomography for the classification and management of distal humeral fractures. Evaluation of reliability and diagnostic accuracy

BACKGROUND: Complex fractures of the distal part of the humerus can be difficult to characterize on plain radiographs and two-dimensional computed tomography scans. We tested the hypothesis that three-dimensional reconstructions of computed tomography scans improve the reliability and accuracy of fracture characterization, classification, and treatment decisions. METHODS: Five independent observers evaluated thirty consecutive intra-articular fractures of the distal part of the humerus for the presence of five fracture characteristics: a fracture line in the coronal plane; articular comminution; metaphyseal comminution; the presence of separate, entirely articular fragments; and impaction of the articular surface. Fractures were also classified according to the AO/ASIF Comprehensive Classification of Fractures and the classification system of Mehne and Matta. Two rounds of evaluation were performed and then compared. Initially, a combination of plain radiographs and two-dimensional computed tomography scans (2D) were evaluated, and then, two weeks later, a combination of radiographs, two-dimensional computed tomography scans, and three-dimensional reconstructions of computed tomography scans (3D) were assessed. RESULTS: Three-dimensional computed tomography improved both the intraobserver and the interobserver reliability of the AO classification system and the Mehne and Matta classification system. Three-dimensional computed tomography reconstructions also improved the intraobserver agreement for all fracture characteristics, from moderate (average kappa [kappa2D] = 0.554) to substantial agreement (kappa3D = 0.793). The addition of three-dimensional images had limited influence on the interobserver reliability and diagnostic characteristics (sensitivity, specificity, and accuracy) for the recognition of specific fracture characteristics. Three-dimensional computed tomography images improved intraobserver agreement (kappa2D = 0.62 compared with kappa3D = 0.75) but not interobserver agreement (kappa2D = 0.24 compared with kappa3D = 0.28) for treatment decisions. CONCLUSIONS: Three-dimensional reconstructions improve the reliability, but not the accuracy, of fracture classification and characterization. The influence of three-dimensional computed tomography was much more notable for intraobserver comparisons than for interobserver comparisons, suggesting that different observers see different things in the scans-most likely a reflection of the training, knowledge, and experience of the observer with regard to these relatively uncommon and complex injuries.     

Interobserver and intraobserver reliability and validity of the Vancouver classification system of periprosthetic femoral fractures after hip arthroplasty

The Vancouver classification system of periprosthetic fractures has been revalidated in this study, using the radiographs of 45 patients. Three consultants and 3 trainees reviewed the radiographs independently, on 2 separate occasions, at least 2 weeks apart. Interobserver and intraobserver agreement and validity were analyzed, using weighted κ statistics. The mean κ value for interobserver agreement was found to be 0.69 (0.63-0.72) for consultants and 0.61 (0.56-0.65) for the trainees, both representing substantial agreement. Intraobserver κ values ranged from 0.74 to 0.90, showing substantial agreement. Validity analysis of 37 type B cases revealed 81% agreement within B1, B2, and B3 subgroups with a κ value of 0.68 (substantial agreement). This study has reconfirmed the reliability and validity of the Vancouver classification while it also emphasizes the intraoperative assessment of implant stability.     

Inter-observer variation of the Caterall and Herring classification in Perthes disease]

The reliability of the Caterall and the Herring classifications (inter-observer study) was compared using radiographs of 63 hips in 58 patients. The radiographs were reviewed by three independent observers with various clinical experience. The results were analyzed using kappa statistics and the percentage of inter-observer agreement. The less experienced observers seemed to be more prone to classify the hips as more affected than they actually were. They also seemed to produce a higher agreement rate. Disagreement was most common in the Caterall groups II and III. The Herring classification appears to be more reliable, also in children older than six, which perhaps makes it more useful in deciding between conservative or operative treatment.     

The classification of congenital talipes equinovarus

e have assessed the reliability of four classification systems for club foot. Four observers evaluated nine children (18 feet) at different stages in the first six months of life, a total of 180 examinations. Each observer independently assessed all feet according to the classification systems described by Catterall, Diméglio et al, Harrold and Walker, and Ponseti and Smoley. The variation between observers was assessed using the kappa test which for no more agreement than chance has a value of 0, and for complete agreement between observers a value of 1. The kappa values varied between 0.14 and 0.77 depending on which classification system was used. The system of Diméglio et al was found to have the greatest reliability. Our findings suggest that current classification systems for the analysis of congenital talipes equinovarus are not entirely satisfactory.     

Reliability and reproducibility of dens fracture classification with use of plain radiography and reformatted computer-aided tomography

BACKGROUND: The classification system of dens fractures by Anderson and D'Alonzo has been widely used in clinical studies. Of the three types of fractures, Type II and Type III are of particular importance because the distinction between them may affect treatment decisions. The purposes of this study were to assess whether this classification is reliable and reproducible and to determine whether computed tomography can improve its reliability and reproducibility. METHODS: Plain radiographs and spiral computed tomography images of dens fractures in eleven patients were assessed, and the fractures were assigned a classification of Type II or Type III at two readings, separated by six months, by two spine surgeons and three neuroradiologists. Kappa coefficients of agreement between the raters as well as the reproducibility of the classifications made by the individual raters were calculated independently for the fracture classifications based on the plain radiographs and those based on the reformatted computed tomography scans. RESULTS: The kappa coefficient for classifications based on plain radiographs was 0.30 and 0.25 (fair agreement) at the first and second readings, respectively. For classifications based on computed tomography scans, the corresponding kappa coefficients were 0.46 (moderate agreement) and 0.67 (substantial agreement). The kappa coefficients for intrarater reliability among the five raters averaged 0.56 (moderate agreement) when computed tomography scans were used and 0.28 (fair agreement) when plain radiographs were used. CONCLUSIONS: Substantial variation with regard to the classification of dens fractures was found within our group of raters. Greater agreement occurred when reformatted computed tomography scans rather than plain radiographs were used as the basis for classification. When classifying dens fractures according to the system of Anderson and D'Alonzo, one should consider using reformatted computed tomography scans and reaching a consensus with multiple raters.     

Inter-observer reliability of radiographic classifications and measurements in the assessment of Perthes' disease

We evaluated the inter-observer agreement of radiographic methods when evaluating patients with Perthes' disease. The radiographs were assessed at the time of diagnosis and at the 1-year follow-up by local orthopaedic surgeons (O) and 2 experienced pediatric orthopedic surgeons (TT and SS). The Catterall, Salter-Thompson, and Herring lateral pillar classifications were compared, and the femoral head coverage (FHC), center-edge angle (CE-angle), and articulo-trochanteric distance (ATD) were measured in the affected and normal hips. On the primary evaluation, the lateral pillar and Salter-Thompson classifications had a higher level of agreement among the observers than the Catterall classification, but none of the classifications showed good agreement (weighted kappa values between O and SS 0.56, 0.54, 0.49, respectively). Combining Catterall groups 1 and 2 into one group, and groups 3 and 4 into another resulted in better agreement (kappa 0.55) than with the original 4-group system. The agreement was also better (kappa 0.62-0.70) between experienced than between less experienced examiners for all classifications. The femoral head coverage was a more reliable and accurate measure than the CE-angle for quantifying the acetabular covering of the femoral head, as indicated by higher intraclass correlation coefficients (ICC) and smaller inter-observer differences. The ATD showed good agreement in all comparisons and had low interobserver differences. We conclude that all classifications of femoral head involvement are adequate in clinical work if the radiographic assessment is done by experienced examiners. When they are less experienced examiners, a 2-group classification or the lateral pillar classification is more reliable. For evaluation of containment of the femoral head, FHC is more appropriate than the CE-angle.     

Legg-Calve-Perthes disease. Part I: Classification of radiographs with use of the modified lateral pillar and Stulberg classifications

BACKGROUND: Accurate and reliable radiographic classifications of the relative severity and outcome of Legg-Calve-Perthes disease are essential in the study of that disease. As part of a prospective multicenter study, we sought to define more clearly the lateral pillar classification of severity and the Stulberg classification of outcome; we sought especially to define the borderlines between classification groups. METHODS: We performed interobserver and intraobserver trials of the lateral pillar and Stulberg classifications using sets of twenty radiographs chosen from a prospective study of 345 hips. To establish reliable definitions of the lateral pillar classification, we added a new, intermediate group termed the B/C border group, which includes femoral heads with a thin or poorly ossified lateral pillar and those with a loss of exactly 50% of the original height of the lateral pillar. The resulting classification consists of four groups: A, B, B/C border, and C. In our application of the classification system of Stulberg et al., we defined a class-II femoral head as round and fitting within 2 mm of a circle on both anteroposterior and frog-leg lateral radiographs. We defined a Stulberg class-III femoral head as out of round by more than 2 mm on either view and a Stulberg class-IV femoral head as one with at least 1 cm of flattening of the weight-bearing articular surface. To assess interobserver and intraobserver agreement, we performed two trials of each classification with six orthopaedic surgeons reviewing twenty radiographs or pairs of radiographs. RESULTS: In the first trial of the lateral pillar classification, there was 81% agreement per radiograph and the average weighted kappa was 0.71. In the second trial, there was 85% agreement per radiograph and the weighted kappa averaged 0.79. Intraobserver reliability testing showed a 77% match between Trials 1 and 2, an average weighted kappa of 0.81, and an average generalizability coefficient of 0.91. In Trial 1 of the Stulberg classification, there was 91% agreement per radiograph and an average weighted kappa of 0.82. In Trial 2, there was 92% agreement per radiograph and an average weighted kappa of 0.82. Intraobserver reliability testing showed an 89% match between Trials 1 and 2, an average weighted kappa value of 0.88, and an average generalizability coefficient of 0.92. CONCLUSIONS: The interobserver and intraobserver trials of these classifications produced kappa values and generalizability coefficients in the excellent range. The modified lateral pillar classification and the redefined Stulberg classification are sufficiently reliable and accurate for use in studies of Legg-Calve-Perthes disease.     

Reliability of the Severin classification in the assessment of developmental dysplasia of the hip.

This study was undertaken to investigate the reliability of Severin's classification at various ages, and to determine whether the reliability is improved by careful measurement rather than subjective assessment. The radiographs taken at ages 6, 10 and 16 years of 20 randomly selected patients treated for developmental dysplasia of the hip were graded by six observers on two separate occasions using parameters measured according to Severin's criteria. In addition, four of these observers regraded the same radiographs using subjective assessment without measurements being made on two other separate occasions. Agreements between and within observers were evaluated using the weighted Cohen's kappa statistic for each age group. Intraobserver reliability was good, there being a close association between the measured and the subjectively observed ratings. This accords with the subjective nature of this classification. The interobserver reliability was found to be poor although it improved when direct measurements were made. Overall agreement between observers improves as patient age increases. It is concluded that comparisons between different observers using the Severin classification system are not reliable. However, a single investigator comparing treatment modalities in the same study allowing for individual bias in assessing deformity and subluxation would produce reliable results.     

Mammography: interobserver variability in breast density assessment

Our objective was to determine the interobserver variability of breast density assessment according to the Breast Imaging Reporting and Data System (BI-RADS) and to examine potential associations between breast density and risk factors for breast cancer. Four experienced breast radiologists received instructions regarding the use of BI-RADS and they assessed 57 mammograms into BI-RADS density categories of 1-4. The weighted kappa values for breast density between pairs of observers were 0.84 (A, B) (almost perfect agreement); 0.75 (A, C), 0.74 (A, D), 0.71 (B, C), 0.77 (B, D), 0.65 (C, D) (substantial agreement). The weighted overall kappa, measured by the intraclass correlation coefficient (ICC), was 0.77 (95% CI: 0.69-0.85). Body mass index was inversely associated with high breast density. In conclusion, overall interobserver agreement in mammographic interpretation of breast density is substantial and therefore, the BI-RADS classification for breast density is useful for standardization in a multicentre study.