Musculoskeletal education in physical medicine and rehabilitation residency programs: graduating resident perspective

OBJECTIVES: To characterize graduating physical medicine and rehabilitation (PM&R) residents physicians' perceptions of their current musculoskeletal (MSK) training, to identify barriers perceived by resident physicians to improving MSK education experiences, and to compare the views of resident physicians with those of PM&R residency program directors. DESIGN: Fourth-year PM&R residents graduating in 2004 whose program directors attended the 2004 Association of Academic Physiatrists annual meeting were asked to complete an MSK education survey developed by the authors. Data were compared with a previous MSK education survey that had been completed by PM&R residency program directors. RESULTS: Ninety-three of 156 (61%) fourth-year PM&R residents responded after multiple contacts. According to residents, the most frequently used MSK education formats during residency were MSK lecture series, MSK journal clubs, and MSK workshops. Potential barriers to improved MSK education during residency included staff, money, and time. If given unlimited resources, most residents would greatly increase the use of visiting lecturers, MSK workshops, and MSK lecture series. CONCLUSION: Graduating PM&R residents as well as residency program directors indicated a strong interest in expanding resident MSK education through the use of visiting lecturers. Differences were noted with respect to the use of hands-on learning (i.e., MSK workshops [residents]) vs. passive learning (i.e., CD ROMS/DVDs and videos [program directors]). Both groups described how limited resources including staff, money, and time are barriers to resident MSK education.     

Pilot study of a 360-degree assessment instrument for physical medicine & rehabilitation residency programs

OBJECTIVE: To perform a pilot test on a new format for multidisciplinary assessment of resident physicians' professionalism and clinical performance in acute inpatient rehabilitation settings. DESIGN: In this pilot study, a 26-item ratings instrument was developed for use by therapists, nurses, social workers, case managers, and psychologists to rate inpatient residents. RESULTS: A total of 421 ratings forms were returned over four academic years. Alpha reliability coefficient for instrumentation sample was 0.99. chi2 and analysis of variance procedures examined item mean differences. Significant differences (P 相似文献   

Electrodiagnostic medicine skills competency in physical medicine and rehabilitation residents: a method for development and assessment

This project sought to create an educational module including evaluation methodology to instruct physical medicine and rehabilitation (PM&R) residents in electrodiagnostic evaluation of patients with neuromuscular problems, and to verify acquired competencies in those electrodiagnostic skills through objective evaluation methodology. Sixteen residents were trained by board-certified neuromuscular and electrodiagnostic medicine physicians through technical training, lectures, and review of self-assessment examination (SAE) concepts from the American Academy of Physical Medicine & Rehabilitation syllabus provided in the Archives of Physical Medicine and Rehabilitation. After delivery of the educational module, knowledge acquisition and skill attainment were measured in (1) clinical skill in diagnostic procedures via a procedure checklist, (2) diagnosis and ability to design a patient-care management plan via chart simulated recall (CSR) exams, (3) physician/patient interaction via patient surveys, (4) physician/staff interaction via 360-degree global ratings, and (5) ability to write a comprehensive patient-care report and to document a patient-care management plan in accordance with Medicare guidelines via written patient reports. Assessment tools developed for this program address the basic competencies outlined by the Accreditation Council for Graduate Medical Education (ACGME). To test the success of the standardized educational module, data were collected on an ongoing basis. Objective measures compared resident SAE scores in electrodiagnostics (EDX) before and after institution of the comprehensive EDX competency module in a PM&R residency program. Fifteen of 16 residents (94%) successfully demonstrated proficiency in every segment of the evaluation element of the educational module by the end of their PGY-4 electrodiagnostic rotation. The resident who did not initially pass underwent remedial coursework and passed on the second attempt. Furthermore, the residents' proficiency as demonstrated by the evaluation after implementation of the standardized educational module positively correlated to an increase in resident SAE scores in EDX compared with resident scores before implementation of the educational module. Resident proficiency in EDX medicine skills and knowledge was objectively verified after completion of the standardized educational module. Validation of the assessment tools is evidenced by collected data correlating with significantly improved SAE scores and American Association of Neuromuscular and Electrodiagnostic Medicine (AANEM) exam scores, as outlined in the result section. In addition, the clinical development tool (procedure checklist) was validated by residents being individually observed performing skills and deemed competent by an AANEM-certified physician. The standardized educational module and evaluation methodology provide a potential framework for the definition of baseline competency in the clinical skill area of EDX.     

The Mayo-Portland Participation Index: A brief and psychometrically sound measure of brain injury outcome

OBJECTIVE: To evaluate the internal consistency, interrater agreement, concurrent validity, and floor and ceiling effects of the 8-item Participation Index (M2PI) of the Mayo-Portland Adaptability Inventory (MPAI). DESIGN: M2PI data derived from MPAIs completed independently by the people with acquired brain injury undergoing evaluation, their significant others, and rehabilitation staff were submitted to Rasch Facets analysis to determine the internal consistency of each independent rater group and of composite measures that combined rater groups. Correlations with the full-scale MPAI were examined to assess concurrent validity, as was interrater agreement. SETTING: Outpatient rehabilitation in academic physical medicine and rehabilitation department. PARTICIPANTS: People with acquired brain injury (N=134) consecutively seen for evaluation, significant others, and evaluating staff. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: The MPAI and M2PI. RESULTS: The M2PI showed satisfactory internal consistency, concurrent validity, interrater agreement, and minimal floor and ceiling effects, although evidence of rater bias was also apparent. Composite indices showed more desirable psychometric properties than ratings by individual rater groups. CONCLUSIONS: The M2PI, particularly in composite indices and with attention to rater biases, provides an outcome measure with satisfactory psychometric qualities and the potential to represent the varying perspectives of people with acquired brain injury, significant others, and rehabilitation staff.     

Comparison of caregivers', residents', and community-dwelling spouses' opinions about expressing sexuality in an institutional setting.

Opinions regarding the appropriateness of elderly residents' sexual behaviors in a chronic care hospital and how to respond to inappropriate behaviors were surveyed. Study participation was open to all staff (N = 1,205), eligible residents (N = 182) and community-dwelling spouses (N = 103). Participation rates were 40% (residents), 42% (spouses), 34% (nursing staff), 50% (allied health staff), and 22% (support staff). Staff completed the questionnaire independently, while residents and spouses were offered self-completion or a structured interview. Almost all selected the interview. Residents and spouses were less tolerant than staff of residents' masturbating, engaging in sexual relationships, viewing sexual materials, and making sexual approaches to staff. Privacy was the primary determinant of appropriateness for behaviors for all groups. Staff and spouses were more likely to endorse counseling when behaviors were perceived as inappropriate than residents. Nurses endorsed counseling less frequently than allied health professionals and support staff. Nurses were more likely to have been approached sexually by a resident. Differences of opinion are interpreted in terms of cohort influences on values and contextual influences on behavior.     

Exploring Individual Opinions of Potential Evaluators in a 360-Degree Assessment: Four Distinct Viewpoints of a Competent Resident

Background: Despite the highly acclaimed psychometric features of a 360-degree assessment in the fields of economics, military, and education, there has been increased interest in developing 360-degree instruments to assess competencies in graduate medical education only in the past recent years. Most of the effort to date, however, has focused on developing instruments and testing their reliability and feasibility. Insufficient attention has gone into issues of construct validity and particularly understanding the underlying constructs on which the instruments are based as well as the phenomena that affect ratings. Purpose: In preparation for developing a 360-degree assessment instrument, we explored variations in evaluators' opinion type of a competent resident and offer observation about evaluator's professional background and opinions. Method: Evaluators from two residency programs ranked 36 opinion statements, using a relative-ranking model, based on their opinion of a competent resident. By-person factor analysis was used to structure opinion types. Results: Factor analysis of 156 responses identified four factors interpreted as four different opinion types of a competent resident: (a) altruistic, compassionate healer (n = 42 evaluators), (b) scientifically grounded clinician (n = 30), (c) holistic, humanistic clinician (n = 62), and (d) patient-focused, health manager (n = 31). Although 72% of nurses/respiratory therapist evaluators expressed type C, 28% expressed other types just as often. Only 14% of evaluator physicians expressed type D, and the remainders were evenly split among the other types. Conclusions: Our evaluators in 360-degree system expressed four opinion types of a competent resident. The individual opinion and not professional background influences the characteristics an evaluator values in a competent resident. We propose that these values will have an impact on competency assessment and should be taken into account in a 360-degree assessment.     

Developing a Valid Evaluation for Interpersonal and Communication Skills

Objectives The Accreditation Council for Graduate Medical Education requires that residency programs evaluate the acquisition of six general competencies, including Interpersonal and Communication Skills (ICS). To develop a 360-degree method for accomplishing this, a semantic-differentiation matrix for various communication traits for nurses to evaluate physician ICS was developed. The authors sought to determine whether this evaluation method could discriminate between more experienced medical communicators (faculty) and less experienced medical communicators (residents).
Methods A 98-item questionnaire measured several communication dimensions by using an eight-element semantic-differentiation scale. In addition, global assessment ratings assessed nursing perceptions of physician ICS skills. This process was repeated for various clinical scenarios.
Results There were 26 nurse evaluators, 19 emergency medicine (EM) residents (EM2 and EM3), and 30 EM faculty. Each physician received five independent evaluations (total, 245 evaluations). All EM residents (EM2 and EM3) were compared with the EM faculty. All eight items on the semantic-differentiation scale were compared. Likewise, the global assessment scores were compared. In every category, the faculty scored higher (Mann-Whitney U: p < 0.001).
Conclusions An evaluation process integrating a semantic-differentiation matrix was applied to various clinical scenarios (as well as global assessment items) and demonstrated discriminatory results. Faculty physicians scored significantly higher than resident physicians. The ability to provide discriminatory results is a requisite in the development of a valid evaluation process. The described semantic-differentiation matrix and global assessment questions may be valid measurements of ICS.     

Article 2: Comparability of mayo-portland adaptability inventory ratings by staff, significant others, and people with acquired brain injury

Objective: To determine the internal consistency, reliability, and comparability of the Mayo-Portland Adaptability Inventory (MPAI-4) and subscales completed independently by people with acquired brain injury, family and significant others, and rehabilitation staff. Design: Rater agreement for MPAI-4 protocols was evaluated and data were submitted to Rasch facets analysis to determine the internal consistency for each rater group and for a composite measure based on all rater groups. Setting: Outpatient rehabilitation. Participants: 134 people with acquired brain injury consecutively seen for outpatient rehabilitation evaluation, significant others, and staff. Interventions: Not applicable. Main Outcome Measure: MPAI-4 and subscales (ability, adjustment, and participation indices). Results: Rasch indicators of internal consistency were entirely within acceptable limits for 3-rater composite full scale and subscale measures, and generally within acceptable limits for measures for single rater groups. Item agreement was generally acceptable; disagreements suggested varying perspectives and sources of bias for specific rater groups. Conclusions: The MPAI-4 possesses satisfactory internal consistency regardless of rating source. For research and program evaluation, the composite measure provides a method to synthesize multiple perspectives on current status. In clinical settings, assessment of varying these perspectives during initial evaluation is helpful in developing effective rehabilitation plans.     

360-degree Feedback: Possibilities for Assessment of the ACGME Core Competencies for Emergency Medicine Residents

The Accreditation Council for Graduate Medical Education (ACGME) has challenged residency programs to provide documentation via outcomes assessment that all residents have successfully mastered the six core competencies. A variety of assessment "tools" has been identified by the ACGME for outcomes assessment determination. Although rarely cited in the medical literature, 360-degree feedback is currently in widespread use in the business sector. This tool provides timely, consolidated feedback from sources in the resident's sphere of influence (emergency medicine faculty, emergency medicine residents, off-service residents and faculty, nurses, ancillary personnel, patients, out-of-hospital care providers, and a self-assessment). This is a significant deviation from both the peer review process and the resident review process that almost exclusively use physicians as raters. Because of its relative lack of development, utilization, and validation as a method of resident assessment in graduate medical education, a great opportunity exists to develop the 360-degree feedback tool for resident assessment.     

Assessment of musculoskeletal examination skills: physiatry residents as evaluators and models

OBJECTIVE: To evaluate the musculoskeletal examination (MSKE) skills of junior (postgraduate year [PGY] 2) physical medicine and rehabilitation (PM&R) residents and self-confidence with these skills, and to demonstrate changes in self-confidence in the MSKE skills of senior (PGY3 and PGY4) residents, who served as evaluators and models. DESIGN: Forty-one PGY2-4 residents participated in this retrospective cohort study, which was conducted within a residency program affiliated with two medical schools. Senior residents attended an instructional session in performing and evaluating MSKE skills, taught by a musculoskeletal physiatrist. The following week, junior residents were tested on their MSKE skills; nine seniors served as models, and another nine served as evaluators. Six seniors attended the instructional session only and did not participate in the evaluation. Juniors received a posttest teaching session on MSKE skills, before an unannounced repeat evaluation 5 mos later. All residents completed a survey regarding self-confidence in MSKE skills pre- and posttest teaching sessions. Performance of MSKE skills (based on PASSOR guidelines) and application of ACGME core competencies (medical knowledge, professionalism, interpersonal skills) were measured, and a survey was administered regarding self-confidence in MSKE skills. RESULTS: Posttest results showed a significant improvement of MSKE skills among juniors in the shoulder, lumbar spine, and knee examinations (P < 0.008), with the most robust improvement in the shoulder exam (P < 0.0001). Self-confidence of juniors in their MSKE skills increased significantly (P < 0.005). There was significant improvement (P < 0.008) in self-confidence in the MSKE skills of seniors who served as models and evaluators, but not in those who only attended the instructional session (P = 0.06). CONCLUSIONS: This evaluation and instructional method resulted in a significant improvement of MSKE skills of junior residents on formal testing. Using senior residents as evaluators and models improved their confidence in their own MSKE skills.     

Intra- and inter-rater reliability of the assessment of capacity for myoelectric control.

OBJECTIVE: To examine the reliability of the Assessment of Capacity for Myoelectric Control (ACMC) in children and adults with a myoelectric prosthetic hand. DESIGN: Intra-rater and inter-rater reliability estimated from reported assessments by 3 different raters. PATIENTS: A sample of convenience of 26 subjects (11 males, 15 females) with upper limb reduction deficiency or amputation and myoelectric prosthetic hands were video-taped during a regular clinical visit for ACMC. Participants' ages ranged from 2 to 40 years. METHODS: After instruction, 3 occupational therapists with no, 10 weeks' and 15 years' clinical experience of myoelectric prosthesis training and follow-up independently rated the 30 ACMC items for each patient. The ratings were repeated after 2-4 weeks. Inter- and intra-rater reliability in items was examined by using weighted kappa statistics and Rasch-measurement analyses. RESULTS: The mean intra-rater agreement in items was excellent (kappa 0.81) in the more experienced raters. Fit statistics showed too much variation in the least experienced rater, who also had only good (kappa 0.65) agreement in items. The stability of rater calibrations between first and second assessment showed that no rater varied beyond chance (>0.50 logit) in severity. The mean inter-rater agreement in items was fair; kappa 0.60, between the experienced raters and kappa 0.47 between raters with no and 10 weeks' experience. CONCLUSION: Overall, the agreement was higher in the more experienced raters, indicating that reliable measures of the ACMC require clinical experience from myoelectric prosthesis training.     

Use of video review to evaluate house staff performance of well‐baby examinations: A preliminary investigation

Pediatric training programs need to evaluate many house staff skills, including those involved in performing well‐baby examinations. To assess pediatric resident performance, we videotaped 23 pediatric residents performing well‐baby examinations at the beginning and end of a 6‐month ambulatory pediatric block rotation.

Each pediatric resident and two faculty members (a clinician and a behavioral scientist) participated in an interactive review of each videotape shortly after the well‐baby examination. These faculty‐resident triads simultaneously rated each well‐baby examination on 31 items listed in the American Academy of Pediatrics “Guidelines for Well Child Care.”;¹

After the 6‐month ambulatory experience, the residents as a group demonstrated improved coverage of 13 items, no change on 11 items (6 of which all residents completed on both the preevaluation and postevaluation), and a decreased coverage of 7 items on the questionnaire. The results suggest that video‐assisted interactive review may be a feasible and effective means of teaching and evaluating performance of the well‐baby examination.     

Inter-rater reliability of the Barthel Index, the Activity Index, and the Nottingham Extended Activities of Daily Living: The use of ADL instruments in stroke rehabilitation by medical and non medical personnel

This investigation addressed the question whether non-medical personnel could produce similar ratings to physicians when applying ADL scales. A sports scientist was trained in the assessment of stroke patients with the Barthel Index, the Activity Index and the Nottingham Extended Activities of Daily Living Scale. He and a rehabilitation physician assessed 20 stroke patients in first in-patient rehabilitation with these instruments. Measurements of inter-rater reliability were calculated for scores, subscales and single items, and for the latter also rater correspondence. Inter-rater reliability was good to excellent for all scores and subscales (ICC: 0.82-0.99). Reliability and correspondence was good to excellent for the items of the Barthel Scale, satisfactory to excellent for those of the Activity Index. Some 25 % of the items of the Nottingham Extended Activities of Daily Living revealed unsatisfactory reliability but still high inter-rater correspondence. Results indicate that physicians and non-medical personnel are able to apply these scales reliably to stroke patients.     

Reliability of the PEDro scale for rating quality of randomized controlled trials

BACKGROUND AND PURPOSE: Assessment of the quality of randomized controlled trials (RCTs) is common practice in systematic reviews. However, the reliability of data obtained with most quality assessment scales has not been established. This report describes 2 studies designed to investigate the reliability of data obtained with the Physiotherapy Evidence Database (PEDro) scale developed to rate the quality of RCTs evaluating physical therapist interventions. METHOD: In the first study, 11 raters independently rated 25 RCTs randomly selected from the PEDro database. In the second study, 2 raters rated 120 RCTs randomly selected from the PEDro database, and disagreements were resolved by a third rater; this generated a set of individual rater and consensus ratings. The process was repeated by independent raters to create a second set of individual and consensus ratings. Reliability of ratings of PEDro scale items was calculated using multirater kappas, and reliability of the total (summed) score was calculated using intraclass correlation coefficients (ICC [1,1]). RESULTS: The kappa value for each of the 11 items ranged from.36 to.80 for individual assessors and from.50 to.79 for consensus ratings generated by groups of 2 or 3 raters. The ICC for the total score was.56 (95% confidence interval=.47-.65) for ratings by individuals, and the ICC for consensus ratings was.68 (95% confidence interval=.57-.76). DISCUSSION AND CONCLUSION: The reliability of ratings of PEDro scale items varied from "fair" to "substantial," and the reliability of the total PEDro score was "fair" to "good."     

End-of-Life Care in an Australian Rehabilitation Facility for Older People: Staff Focus Groups

A qualitative study of staff experiences of end-of-life care for older people in a subacute rehabilitation facility was undertaken using three focus groups with senior multidisciplinary staff (5), junior nurses (8), and junior allied health staff (7). Content analysis revealed four major themes: being a key contact person; the quality of end–of-life care; referring to off-site service providers; and differing perspectives. These data have implications for multidisciplinary practice including staff education and capacity to change focus of care in facilities for older people dedicated to rehabilitation rather than palliation or end-of-life care.     

An internship for college students in physical medicine and rehabilitation: effects on awareness, career choice, and disability perceptions

OBJECTIVE: To show the impact that an internship program in Physical Medicine and Rehabilitation (PM&R) for college students has on their knowledge about the field, career choice, and perceptions about people with disabilities. DESIGN: Twelve students were selected to participate in the study. Students observed patient therapies and followed faculty and physiatry resident physicians. Students also participated in research studies in rehabilitation research. Group discussions regarding specific projects, research methods, career choice, and perceptions about disability were part of a didactic curriculum. Surveys about PM&R knowledge, attitudes toward people with disabilities, demographics, and course evaluations were administered. RESULTS: Results showed that the program increased knowledge about PM&R (P < 0.008). Premedical students missed significantly fewer questions (8.2 vs. 11.7; P = 0.04) on this survey than did other participants. Results also showed that this program affected their attitudes toward people with disabilities and student choice to pursue a career in health care. CONCLUSIONS: This type of internship experience provides an educational environment for college students to become acquainted with PM&R, interact positively with people with disabilities, and influence career choice in the allied health professions.     

The intra- and interrater reliability of the action research arm test: a practical test of upper extremity function in patients with stroke

OBJECTIVES: To determine the intra- and interrater reliability of the Action Research Arm (ARA) test, to assess its ability to detect a minimal clinically important difference (MCID) of 5.7 points, and to identify less reliable test items. DESIGN: Intrarater reliability of the sum scores and of individual items was assessed by comparing (1) the ratings of the laboratory measurements of 20 patients with the ratings of the same measurements recorded on videotape by the original rater, and (2) the repeated ratings of videotaped measurements by the same rater. Interrater reliability was assessed by comparing the ratings of the videotaped measurements of 2 raters. The resulting limits of agreement were compared with the MCID. PATIENTS: Stratified sample, based on the intake ARA score, of 20 chronic stroke patients (median age, 62yr; median time since stroke onset, 3.6yr; mean intake ARA score, 29.2). MAIN OUTCOME MEASURES: Spearman's rank-order correlation coefficient (Spearman's rho); intraclass correlation coefficient (ICC); mean difference and limits of agreement, based on ARA sum scores; and weighted kappa, based on individual items. RESULTS: All intra- and interrater Spearman's rho and ICC values were higher than .98. The mean difference between ratings was highest for the interrater pair (.75; 95% confidence interval, .02-1.48), suggesting a small systematic difference between raters. Intrarater limits of agreement were -1.66 to 2.26; interrater limits of agreement were -2.35 to 3.85. Median weighted kappas exceeded .92. CONCLUSION: The high intra- and interrater reliability of the ARA test was confirmed, as was its ability to detect a clinically relevant difference of 5.7 points.     

Criteria for selection of physical medicine and rehabilitation residents. A survey of current practices and suggested changes

The growing number of medical graduates seeking residency training in Physical Medicine and Rehabilitation (PM&R) has made the resident selection process increasingly difficult. This circumstance has prompted a search for more efficient selection criteria. Forty-one PM&R residency program directors in the United States and Puerto Rico completed a survey on the way they selected their residents. The results indicate that personal applicant characteristics, as determined by interview, were more important than academic criteria in ranking residency candidates. Of the academic variables presented, participation in a PM&R elective at the program director's hospital and the Dean's letter were rated highest. The interview experience has a major influence on the ranking of an applicant for acceptance into a PM&R training program. Academic criteria should have more influence on applicant selection than they do at present. In addition, early identification and vigorous recruitment of resident candidates with an interest in research is essential for increasing the number of academic physiatrists within our specialty.     

Residency training in physical medicine and rehabilitation. I: Clinical and didactic experience

A survey was conducted to determine the type of clinical and didactic training experience that was provided to resident trainees in physical medicine and rehabilitation (PM&R) in the 1987-1988 academic year. Chief residents from 43 (61%) of the 70 PM&R programs accredited by the Accreditation Council for Graduate Medical Education responded. According to respondents, the programs averaged 12.6 residents. The residents spent an average of 18.5 months on an inpatient bedservice, 12.6 months on outpatient exposures, and the remainder of the time attending other clinical experiences and didactic training. Forty percent of those responding reported that their programs required in-house call in postgraduate years (PGYs) 2 through 4, and 53% of the programs required no in-house call during the same clinical years. Seven percent of the programs required in-house call in PGYs 2 and 3, but none in PGY 4. The average time spent in electrodiagnostic studies was 7.6 months (range = 2 to 19 months). Electromyography exposure by completion of PGY 4 also varied widely, from 40 to 500 studies. Resident trainee exposure to inpatient and outpatient spinal cord injury, closed head injury, pediatric rehabilitation, sports medicine, and geriatric medicine, and rehabilitation fellowship positions being offered through the responding PM&R residency training programs were also surveyed. Some instances of apparent program imbalances or inadequate training which could reduce the scope of a resident's educational experience were noted.