Success in orthopaedic training: resident selection and predictors of quality performance

Multiple studies have attempted to determine which attributes are predictive of success during residency as well as the optimal method of selecting residents who possess these attributes. Factors that are consistently ranked as being important in the selection of candidates into orthopaedic residency programs include performance during orthopaedic rotation, United States Medical Licensing Examination (USMLE) Step 1 score, Alpha Omega Alpha Honor Medical Society membership, medical school class rank, interview performance, and letters of recommendation. No consensus exists regarding the best predictors of resident success, but trends do exist. High USMLE Step 1 scores have been shown to correlate with high Orthopaedic In-Training Examination scores and improved surgical skill ratings during residency, whereas higher numbers of medical school clinical honors grades have been correlated to higher overall resident performance, higher residency interpersonal skills grading, higher resident knowledge grading, and higher surgical skills evaluations. Successful resident performance can be measured by evaluating psychomotor abilities, cognitive skills, and affective domain.     

Psychomotor testing and the ability to perform an anastomosis in junior surgical trainees.

Ten junior surgical trainees underwent objective testing of manual dexterity and visuospatial ability and were required to carry out five consecutive anastomoses on fresh porcine jejunum. Anastomoses were scored by a single observer and a cumulative error score (CES) derived for each procedure. In the first anastomosis there was little correlation between the psychomotor test results and the anastomosis scores. In subsequent trials there were significant negative correlations between aspects of manual dexterity and the CES. Over the five anastomoses there were significant negative correlations between improvement and manual dexterity, but there was a positive correlation between improvement and visuospatial ability (rs = 0.76, P < 0.005). Visuospatial skills are more important than pure motor ability in predicting the capacity to perform an anastomosis and tests of manual dexterity may be misleading in this context.     

Adaptive ageing surgeons

The surgical workforce is ageing. This will impact on future workforce supply and planning, as well as the professional performance and welfare of surgeons themselves. This paper is a ‘call to arms’ to surgeons to consider the complex problem of advancing years and surgical performance. We aim to promote discussion about the issue of ageing as it relates to surgeons, while exploring ways in which successful ageing in surgeons may be promoted. The task‐specific aspects of surgical practice suggest that it is a physically and cognitively demanding task, reliant on a range of fine motor, sensory, visuospatial, reasoning, memory and processing skills. Many of these skills potentially decline with age, although there is great inter‐individual variation, particularly in cognitive performance. Nevertheless, there is some consensus in the literature that age‐related cognitive changes exist in a proportion of surgeons, and there is an increase in operative mortality rates for certain surgical procedures performed by older and more experienced surgeons. In the absence of mandatory retirement, guidance is needed in regard to individualizing the timing of retirement and encouraging reflective and adaptive practice based on insight into how one's skills and performance may change with age. This may be best facilitated by some form of informed and guided self‐monitoring or ‘self‐screening’. It should be emphasized that self‐screening is not a form of self‐treatment but aims to enhance insight, using a tool kit of resources to promote adaptive ageing. Moreover, self‐screening should not be restricted to cognition, which is only part of the picture of ageing, but extended to emphasize the maintenance of mental and physical wellness, and the acceptance of independent professional treatment and support when required.     

Age, gender, lateral dominance, and prediction of operative skill among general surgery residents

Ability patterns and surgical proficiency were examined in matched groups of general surgery residents selected on the basis of age, gender, or hand preference from a population of 141 residents who had completed neuropsychologic tests of visuospatial, psychomotor, and stress tolerance abilities and had been rated on 12 aspects of technical skill exhibited during 1480 operative procedures. Older residents (ages 28 to 42 years) exhibited less motor speed (p less than 0.05) and coordination (p less than 0.005) and more caution in avoiding psychomotor errors (p less than 0.05) than did their younger counterparts. No differences were found for visuospatial abilities, stress tolerance, or rated surgical skill. These findings indicate that although age does appear to adversely affect pure motor skills, these are not important components of operative proficiency. Female residents exhibited superior (p less than 0.05) academic achievement (MCAT, Verbal and National Boards Part II) as compared with their male counterparts. They also excelled on a signal detection task requiring identification of visual patterns. However, the women scored less well (p less than 0.05) than men on a visuomotor task demonstrated to be a significant predictor of operative skill. Greater cautiousness in avoiding errors may be a contributing factor to their reduced efficiency on this task. In comparison to male controls, female residents received consistently lower surgical skills ratings, particularly on items measuring confidence and task organization. Left-handed residents were more reactive to stress (p less than 0.03), more cautious (p less than 0.04), and more proficient on a neuropsychologic test of tactile-spatial abilities (p less than 0.03) than right-handed counterparts. Although these traits correlated positively (p less than 0.05) with rated operative skill within the left-handed group, the group received consistently lower ratings than did right-handed residents. The inconvenience of assisting left-handed residents may overshadow attending surgeons' perceptions of their innate abilities. These findings demonstrate significant, neuropsychologically based differences among surgery residents that pose unique challenges to persons responsible for their selection and training.     

Objective classification of residents based on their psychomotor laparoscopic skills

Background  

From the clinical point of view, it is important to recognize residents’ level of expertise with regard to basic psychomotor skills. For that reason, surgeons and surgical organizations (e.g., Acreditation Council for Graduate Medical Education, ACGME) are calling for assessment tools that credential residents as technically competent. Currently, no method is universally accepted or recommended for classifying residents as “experienced,” “intermediates,” or “novices” according to their technical abilities. This study introduces a classification method for recognizing residents’ level of experience in laparoscopic surgery based on psychomotor laparoscopic skills alone.     

Hand-Eye Dominance and Depth Perception Effects in Performance on a Basic Laparoscopic Skills Set

Objectives:

Our study determined whether depth perception defects and hand-eye dominance affect an individual''s ability to perform laparoscopic skills.

Methods:

The study cohort comprised 104 third-year medical students from LSU School of Medicine who completed a questionnaire including information on handedness and were tested for eye dominance and depth perception by using standardized methods. Training sessions involved an initial recorded performance, a 20-minute practice session, followed by a final recorded performance. Recorded sessions were randomized and rated by using a visual analog scale (maximal possible score = 16) based on overall performance (OPS) and depth perception (DPS). A general linear model was used to correlate depth perception defects and hand-eye dominance with assessment scores for OPS and DPS.

Results:

Students with depth perception defects scored significantly lower on their initial performance than did those with normal depth perception (OPS, 4.80 vs. 7.16, P=0.0008; DPS, 5.25 vs. 6.93, P=0.0195). After training, the depth perception defect group continued to have lower scores compared with the normal depth perception group. However, the 2 groups showed similar increases in pre- to posttraining performance scores (OPS, 3.84 vs. 3.18, P=0.0732). Hand-eye dominance did not significantly affect scores.

Conclusions:

Depth perception defects appear to compromise an individual''s ability to perform basic laparoscopic skills. Individuals with defects can improve their skills by a proportion comparable to that of people with uncompromised depth perception. Differences in hand-eye dominance do not correlate with performance differences in basic laparoscopic skills. Although further research is necessary, the findings indicate that training can be tailored for individuals with depth perception defects to improve laparoscopic performance.     

SE16VIRTUAL REALITY LAPAROSCOPIC TRAINING IN AUSTRALIA

Opinions on the role of virtual reality simulation in surgical training are diverse and usually not evidence‐based. Several adequately powered, randomised and blinded studies have demonstrated an improvement in the operative performance of those who have undergone virtual reality training. Nonetheless there remains reluctance by the medical profession to incorporate simulation into training. This seems to be based on three misconceptions.

• 1 Simulators are expensive
• 2 It is difficult to incorporate them into an already overcrowded curriculum
• 3 The transfer of skill to the real world has not been demonstrated to reduce risk or improve patient outcomes.

We report on our attempts to assess the feasibility of establishing virtual training for local trainees and to assess its outcome on operative performance. 16 Basic surgical trainees were randomised into control and training groups. All underwent baseline testing of their psychomotor ability before the training group were allowed 24 hour access to a LapSim virtual reality simulator. It was not possible to complete this study and we will discuss the reasons for this which we believe has important implications for the future use of simulation in surgical training. 16 final year medical students were also recruited, and similarly randomised and assessed before being allowed access to the simulator. After four weeks of training, we were able to demonstrate a significant improvement in their simulated laparoscopic surgical skills when compared with the skills of local specialists. Using an animal model, we were able to demonstrate the effect of this on their operative performance.     

Videotape mentoring and surgical simulation in laparoscopic courses

As laparoscopic approaches to core urologic procedures continue to supplant their open counterparts, the demand to train urologists who received inadequate exposure to these techniques during residency has intensified. The acquisition of laparoscopic skills has been aided greatly by the introduction of hand-assisted laparoscopic surgery (HALS). In another training method, participants completed the standard animate and inanimate course training, then entered a mentoring relationship with their instructor, including an observational period and performance of several complex laparoscopic operations with the assistance of the mentor surgeon. However, the time commitment, compensation issues, and need for temporary operating privileges are obstacles to this approach. A number of studies have demonstrated that laparoscopic skills can be measured on a videotrainer and that ability improves with repetitive performance. Senior urologists with minimal initial knowledge may benefit from laparoscopic skills training videotape analysis and critique. Laparoscopic simulators can improve, not only the psychomotor skills required to perform laparoscopy, but operative performance as well. Ultimately, preoperative images and data may be interfaced with robotic simulation software to allow practice of virtual operations with realistic tissue photo-representation prior to performing them on patients. Improvements in laparoscopic surgical simulation and application of these newly acquired skills to a simulated patient will ultimately eliminate the learning curve on actual patients and provide a useful means of establishing competence.     

Gaze training enhances laparoscopic technical skill acquisition and multi-tasking performance: a randomized, controlled study

Background  

The operating room environment is replete with stressors and distractions that increase the attention demands of what are already complex psychomotor procedures. Contemporary research in other fields (e.g., sport) has revealed that gaze training interventions may support the development of robust movement skills. This current study was designed to examine the utility of gaze training for technical laparoscopic skills and to test performance under multitasking conditions.     

Correlation between perceptual, visuo-spatial, and psychomotor aptitude to duration of training required to reach performance goals on the MIST-VR surgical simulator

Given the dynamic nature of modern surgical education, determining factors that may improve the efficiency of laparoscopic training is warranted. The objective of this study was to analyze whether perceptual, visuo-spatial, or psychomotor aptitude are related to the amount of training required to reach specific performance-based goals on a virtual reality surgical simulator. Sixteen MS4 medical students participated in an elective skills course intended to train laparoscopic skills. All were tested for perceptual, visuo-spatial, and psychomotor aptitude using previously validated psychological tests. Training involved as many instructor-guided 1-hour sessions as needed to reach performance goals on a custom designed MIST-VR manipulation-diathermy task (Mentice AB, Gothenberg, Sweden). Thirteen subjects reached performance goals by the end of the course. Two were excluded from analysis due to previous experience with the MIST-VR (total n = 11). Perceptual ability (r = -0.76, P = 0.007) and psychomotor skills (r = 0.62, P = 0.04) significantly correlated with the number of trials required. Visuo-spatial ability did not significantly correlate with training duration. The number of trials required to train subjects to performance goals on the MIST-VR manipulation diathermy task is significantly related to perceptual and psychomotor aptitude.     

Spatial ability, experience, and skill in laparoscopic surgery

BACKGROUND: Previous research showing correlations between spatial ability and surgical skills has used participants in relatively early stages of training. Research in skill acquisition has shown that the role of cognitive abilities can diminish as skills become increasingly automatic. In this study, we explored the role of spatial ability in laparoscopic surgical skills in two groups, one experienced and the other relatively inexperienced. METHODS: Subjects were recruited from two videoscopic courses: an advanced course for experienced surgeons and a laparoscopic urological surgery course attended by participants with relatively little laparoscopic experience. Three measures were obtained: spatial abilities, videoscopic experience, and operative skills. RESULTS: A significant correlation (r = 0.393) was found between spatial ability and skills in the lower experience group but not among the experienced surgeons (r = 0.020). CONCLUSIONS: The results are consistent with the prediction that the importance of spatial ability in performance of laparoscopic skills should diminish with experience.     

ASSESSMENT OF LAPAROSCOPIC PSYCHOMOTOR SKILLS IN INTERNS USING THE MIST VIRTUAL REALITY SIMULATOR: A PREREQUISITE FOR THOSE CONSIDERING SURGICAL TRAINING?

Selection for surgical training in Australia is currently based on assessment of a structured curriculum vitae, referral reports from selected clinicians and an interview. The formal assessment of laparoscopic psychomotor skill and ability to attain skills is not currently a prerequisite for selection. The aim of this study was to assess the innate psychomotor skills of interns and also to compare interns with an interest in pursuing a surgical career to interns with those with no interest in pursuing a surgical career. Twenty-two interns were given the opportunity to carry out tasks on the Minimal Invasive Surgical Trainer, Virtual Reality (Mentice, Gothenburg, Sweden) Simulator. The candidates were required to complete six tasks, repeated six times each. Scores for each task were calculated objectively by the simulator software. Demographic data were similar between the two groups. Although some candidates who were interested in pursuing a surgical career performed poorly on the simulator, there was no significant difference when comparing the two groups. The Minimal Invasive Surgical Trainer, Virtual Reality (Mentice) Simulator provides an objective and comparable assessment of laparoscopic psychomotor skills. We can conclude that interns have varying inherent ability as judged by the simulator and this does not seem to have an influence on their career selection. There was no significant difference in the scores between the two groups. Interns with and without inherent abilities have aspirations to pursue surgical careers and their aptitude does not seem to influence this decision. Surgical colleges could use psychomotor ability assessments to recruit candidates to pursue a career in surgery. Trainees needing closer monitoring and additional training could be identified early and guided to achieve competency.     

Establishing a curriculum for the acquisition of laparoscopic psychomotor skills in the virtual reality environment

BackgroundThe unique psychomotor skills required in laparoscopy result in reduced patient safety during the early part of the learning curve. Evidence suggests that these may be safely acquired in the virtual reality (VR) environment. Several VR simulators are available, each preloaded with several psychomotor skills tasks that provide users with computer-generated performance metrics. This review aimed to evaluate the usefulness of specific psychomotor skills tasks and metrics, and how trainers might build an effective training curriculum.MethodsWe performed a comprehensive literature search.ResultsThe vast majority of VR psychomotor skills tasks show construct validity for one or more metrics. These are commonly for time and motion parameters. Regarding training schedules, distributed practice is preferred over massed practice. However, a degree of supervision may be needed to counter the limitations of VR training.ConclusionsIn the future, standardized proficiency scores should facilitate local institutions in establishing VR laparoscopic psychomotor skills curricula.     

Teaching and assessing non-technical skills

The terms human factors and non-technical skills have recently been introduced to the language of surgical education. Both tend to be used interchangeably and yet each has a specific definition. More importantly, however, is the fact that the attributes and qualities contained within these headings relate to behaviours, attitudes and cognitive skills. They are recognised as crucially important in the practice of surgery, but are often poorly articulated during surgical performance, during training, during any assessment process and, indeed, seldom measured with reference to any metric in any of these activities. Most research in this area addresses non-technical skills in the operating theatre and it remains to be seen whether the same attributes and skills are used outwith theatre, particularly in the ward setting. However, the contribution that these aspects of performance make to a safe and successful outcome following surgery is being increasingly appreciated and there is increasing recognition of the need to train and assess.     

Perceptual, visuospatial, and psychomotor abilities correlate with duration of training required on a virtual-reality flexible endoscopy simulator

BACKGROUND: Trainees acquire endoscopic skills at different rates. Fundamental abilities testing could predict the amount of training required to reach a performance goal on a virtual-reality simulator. METHODS: Eleven medical students were tested for fundamental abilities. Baseline endoscopic proficiency was evaluated with the GI Mentor II VR simulator (Simbionix, USA, Cleveland, OH). Subjects trained on the simulator with a defined performance goal. Subjects who achieved the goal were then reassessed. RESULTS: All subjects completed at least 10 trials or reached the performance goal. The <10 trial group (n=6) tested better for all fundamental abilities and baseline endoscopic performance than the >10 trial group (n=5). The number of trials required to reach the performance goal correlated significantly with both perceptual (r=.92, P=0.001) and visuospatial ability (r=.76, P=.03). Multiple regression showed strong correlation of all three abilities with duration of training (r=.95, P=.015). CONCLUSIONS: Most of the variability in acquisition of endoscopic skills can be accounted for by differences in fundamental abilities of trainees. Testing of fundamental abilities could help identify trainees who will require additional training to achieve desired performance objectives.     

Teaching surgeons to operate—Principles of psychomotor skills training

Summary Although the ultimate success of surgery depends on the use of adequate psychomotor skills, the evaluation of the abilities of a trainee and the teaching of these skills has not been systematized. The choice of a trainee in surgery should be based at least partially on his innate abilities, and his training should be begun at an appropriate level. The procedures he may do should be analyzed to determine the skills required for their performance. Then these skills should be taught specifically, initially in non-threatening situations such as laboratory settings, and their acquisition assessed so that he can be progressed to more advanced work at the appropriate time. Ultimately a decision must be made whether to train a candidate to a skill level or whether to train him for a set period and then counsel him regarding which procedures he has the skills to perform. There are well developed concepts in educational psychology that may be used in developing improved methods to assess and train prospective surgeons.     

Determining the content of a surgical curriculum

The purpose of undergraduate surgical education is to prepare the student for both the residency and eventual practice of medicine. To help determine the surgical knowledge and skills that would eventually the useful to the student, we conducted a survey of residents in training, physicians in practice, and surgical educators (department chairpersons and clerkship directors). Members of the Curriculum Committee of the Association for Surgical Education developed a questionnaire in which the respondents were asked to grade the functional importance of 84 areas of knowledge and 46 skills (0 = unnecessary, 3 = proficiency necessary). Using a modified Delphi technique to collect information, we sent the questionnaire to eight medical school graduation classes of 1975 (730 persons) and 1980 (776 persons) and all department chairpersons and/or clerkship directors (179). The results of the survey (46% response) revealed considerable agreement about the importance of certain skills and areas of knowledge, enabling us to rank order skills and knowledge based on mean responses (0.0 to 3.0). Physicians in practice, residents, and educators believed that certain areas of knowledge (e.g., acute abdominal problems, appendicitis, shock, cancer of the breasts) and skills (e.g., history taking and physical examination, gowning, suture removal) were very important (greater than 2.250, while other areas of knowledge (e.g., transplantation, liver abscess, soft tissue sarcomas) and skills (e.g., insertion of Swan-Ganz catheter, abdominal paracentesis, cricothyroidotomy) were less important (less than 1.3). This approach allows us to assign priorities to areas of knowledge and skills when determining curriculum content and to include functional criteria when developing educational objectives.     

Visuospatial and technical ability in the selection and assessment of higher surgical trainees in the London deanery

INTRODUCTION

Despite awareness of the limitations of current selection and competency assessments, there is little consensus and alternatives have not been readily accepted. Essential surgical skills include visuospatial and technical ability. The aim of this study was to survey current methods of higher surgical trainee selection and assessment. We suggest ways to improve the process.

MATERIALS AND METHODS

Nine surgical training programmes in the London deanery were surveyed through questionnaires to programme directors, existing trainees and examination of deanery publications.

RESULTS

Testing of visuospatial and technical ability was piloted at selection only in a single general surgical department. Practical skills were assessed in 3/9 (33%) specialties (ENT, plastic and general surgery). Once selected, no specialty tested visuospatial and technical ability. Practical skills were tested in only 1/9 (11%) specialties (plastic surgery). The remaining 8/9 (89%) were ‘assessed’ by interview.

CONCLUSIONS

Lack of visuospatial and technical ability assessment was identified at selection and during higher surgical training. Airlines have long recognised early identification of these qualities as critical for efficient training. There is a need for more objective methods in this area prior to selection as time to assess surgical trainees during long apprenticeships is no longer available. We advocate a suitably validated competency-based model during and at completion of training.     

Correlation between psychometric test scores and learning tying of surgical reef knots

We have investigated the correlation between the scores attained on a computerised psychometric test, measuring psychomotor aptitude and learning tying of a surgical reef knot. Fifteen surgical trainees performed a test of psychomotor aptitude (ADTRACK 2) from the MICROPAT testing system. They then performed a simple test of their ability to tie a surgical reef knot and were assessed by a panel of experts prior to embarking on a standardised course of instruction and practice session. The knot-tying test was repeated at the end of the day and the differences in average scores recorded. There was a significant correlation between the means of the differences in knot tying scores and ADTRACK 2 scores (r = -0.533, P < 0.05). Psychomotor abilities appear to be determinants of trainees' initial proficiency in learning to tie a surgical reef knot.     

Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery

Training and assessment paradigms for laparoscopic surgical skills are evolving from traditional mentor-trainee tutorship towards structured, more objective and safer programs. Accreditation of surgeons requires reaching a consensus on metrics and tasks used to assess surgeons' psychomotor skills. Ongoing development of tracking systems and software solutions has allowed for the expansion of novel training and assessment means in laparoscopy. The current challenge is to adapt and include these systems within training programs, and to exploit their possibilities for evaluation purposes. This paper describes the state of the art in research on measuring and assessing psychomotor laparoscopic skills. It gives an overview on tracking systems as well as on metrics and advanced statistical and machine learning techniques employed for evaluation purposes. The later ones have a potential to be used as an aid in deciding on the surgical competence level, which is an important aspect when accreditation of the surgeons in particular, and patient safety in general, are considered. The prospective of these methods and tools make them complementary means for surgical assessment of motor skills, especially in the early stages of training. Successful examples such as the Fundamentals of Laparoscopic Surgery should help drive a paradigm change to structured curricula based on objective parameters. These may improve the accreditation of new surgeons, as well as optimize their already overloaded training schedules.