Virtual-reality simulation to assess performance in hip fracture surgery

Background and purpose

Internal fixation of hip fractures is a common and important procedure that orthopedic surgeons must master early in their career. Virtual-reality training could improve initial skills, and a simulation-based test would make it possible to ensure basic competency of junior surgeons before they proceed to supervised practice on patients. The aim of this study was to develop a reliable and valid test with credible pass/fail standards.

Methods

20 physicians (10 untrained novices and 10 experienced orthopedic surgeons) each performed 3 internal fixation procedures of an undisplaced femoral neck fracture: 2 hook-pins, 2 screws, and a sliding hip screw. All procedures were preformed on a trauma simulator. Performance scores for each procedure were obtained from the predefined metrics of the simulator. The inter-case reliability of the simulator metrics was explored by calculation of intra-class correlation coefficient. Validity was explored by comparison between novices’ and experts’ scores using independent-samples t-test. A pass/fail standard was set by the contrasting-groups method and the consequences were explored.

Results

The percentage of maximum combined score (PM score) showed an inter-case reliability of 0.83 (95% CI: 0.65–0.93) between the 3 procedures. The mean PM score was 30% (CI: 7–53) for the novices and 76% (CI: 68–83) for the experienced surgeons. The pass/fail standard was set at 58%, resulting in none of the novices passing the test and a single experienced surgeon failing the test.

Interpretation

The simulation-based test was reliable and valid in our setting, and the pass/fail standard could discriminate between novices and experienced surgeons. Potentially, training and testing of future junior surgeons on a virtual-reality simulator could ensure basic competency before proceeding to supervised practice on patients.Worldwide, hip fractures account for substantial healthcare costs and high mortality, morbidity, and reoperation rates. Fractures are often treated with different types of internal fixation, which is a great contributor to training of surgical skills through the principle of the master-apprentice model. This model is not without risk, though. Inexperienced trainees contribute to a higher rate of re-admissions and reoperations (Palm et al. 2007, Leblanc et al. 2013).A review of 609 studies found that virtual-reality simulation training improved operative skills (Cook et al. 2011), and simulation-based training in orthopedic surgery is starting to emerge. Most of the development is in arthroscopy, but simulation-based training in fracture fixation is also being developed (Blyth et al. 2007, 2008, Mabrey et al. 2010, Atesok et al. 2012, Rambani et al. 2013). Current papers describe the simulators and explore construct validity of the simulator metrics (Tillander et al. 2004, Froelich et al. 2011). Reliable and valid tests with credible pass/fail standards are necessary to ensure basic competency of trainees before allowing them to proceed to supervised practice on patients (Stefanidis et al. 2012a, Konge et al. 2013).The objective of this study was to develop a test, to explore the reliability of this test, and to gather validity evidence. Furthermore, we wanted to establish a credible pass/fail standard and explore the consequences of this standard. The research questions were: (1) Which simulator metrics were able to discriminate between novices and experienced orthopedic surgeons?; (2) How many procedures on the simulator must be performed to ensure sufficient reliability?; and (3) What was a credible pass/fail standard in the test?