Observer Variability of an Angiographic Grading Scale Used for the Assessment of Intracranial Aneurysms Treated with Flow-Diverting Stents

BACKGROUND AND PURPOSE:Novel angiographic grading scales for the assessment of intracranial aneurysms treated with flow-diverting stents have been recently developed because previous angiographic grading scales cannot be applied to these aneurysms. The purpose of this study was to evaluate the inter- and intraobserver variability of the novel O''Kelly Marotta grading scale, which was developed specifically for the angiographic assessment of aneurysms treated with flow-diverting stents.MATERIALS AND METHODS:Multiple raters (n = 31) from the disciplines of neuroradiology and neurosurgery were presented with pre- and posttreatment angiographic images of 14 aneurysms treated with intraluminal flow diverters. Raters were asked to classify pre- and posttreatment angiograms by using the OKM grading scale. Statistical analyses were subsequently performed with calculation of a generalized multirater κ statistic for assessment of inter- and intraobserver variability and by performing a Wilcoxon signed rank sum test for assessment of group differences.RESULTS:Variability analysis of the OKM grading scale yielded substantial (κ = 0.74) and almost perfect (κ = 0.99) inter- and intraobserver agreement, respectively, with no statistically significant differences between raters with a background of neuroradiology versus neurosurgery or attending physician versus trainee.CONCLUSIONS:The OKM grading scale for the assessment of intracranial aneurysms treated with flow-diverting stents is a reliable grading scale that can be used equally well by users of varying backgrounds and levels of training. Comparison with interobserver variability of pre-existing angiographic grading scales shows equal or better performance.

The development of flow-diverting stents has ushered in an era of new paradigms and possibilities for the endovascular treatment of intracranial aneurysms. These devices are garnering momentum for difficult intracranial aneurysms that are wide-neck, have fusiform configuration, are located in perforator territories, or have complex geometry. Their mechanism of action relies on a reduction of filling of the aneurysmal sac with flow diversion toward the parent vessel, leading to stasis and subsequent thrombosis within the aneurysm. This process usually occurs with time, and angiographic evidence of aneurysm protection is usually not seen in immediate posttreatment angiograms. Rather, the process of stasis and thrombosis may not be complete for months after initial flow-diverting stent placement. While residual filling is a suboptimal or unacceptable posttreatment angiographic result for an aneurysm treated with endosacular coiling, this can be optimal and acceptable for an aneurysm treated by using flow-diverting devices. Traditional grading scales, such as the 3-point grading scale of Roy and Raymond used for aneurysms treated with coiling or clipping,¹ do not apply to aneurysms treated with flow-diverting stents. For example, after treatment, filling within the aneurysm body would be classified as residual aneurysm and would not be considered a desirable result according to the Roy and Raymond classification, but in the case of flow-diverting stent placement, this would be an expected immediate posttreatment result.A new grading scale specifically tailored to the angiographic assessment of aneurysms treated with flow-diverting stents was published by O''Kelly et al,² termed the O''Kelly Marotta grading scale. This novel grading scale incorporates 2 dimensions as parameters, which reflect the mechanism by which flow-diverting devices accomplish aneurysm protection: reduction in aneurysm sac filling (filling grade), which reflects an anatomic aspect, and promotion of stasis within the aneurysm sac (stasis grade), which reflects a more dynamic or physiologic parameter (Fig 1).Open in a separate windowFig 1.OKM angiographic grading scale for assessment of aneurysms treated with flow-diverting stents. Aneurysms are assigned grades on the basis of the amount of contrast filling of the aneurysm lumen (filling grades, A, B, C, D) and how long contrast persists in the aneurysm lumen with respect to angiographic phase (stasis grades 1, 2, 3). A grade consisting of a letter and a number is assigned to an aneurysm (eg, an aneurysm that fills its lumen completely with contrast persisting within the lumen into the venous phase of the angiogram is assigned grade A3).² Modified from O''Kelly CJ, Krings T, Fiorella D, et al. A novel grading scale for the angiographic assessment of intracranial aneurysms treated using flow diverting stents. Interv Neuroradiol 2010;16:133–37 and reproduced with permission from Centauro srl Publishing.With any novel grading scale, an evaluation of performance in the hands of the evaluating end user is necessary for an assessment of reliability. We present here an analysis of the inter- and intraobserver variability of the OKM grading scale based on evaluation of conventional angiographic images of aneurysms pre- and posttreatment with flow-diverting stents by 31 evaluators with a background in neurointervention from the disciplines of neuroradiology and neurosurgery.