Clinimetric evaluation of shoulder disability questionnaires: a systematic review of the literature

OBJECTIVE: To identify all available shoulder disability questionnaires designed to measure physical functioning and to evaluate evidence for the clinimetric quality of these instruments. METHODS: Systematic literature searches were performed to identify self administered shoulder disability questionnaires. A checklist was developed to evaluate and compare the clinimetric quality of the instruments. RESULTS: Two reviewers identified and evaluated 16 questionnaires by our checklist. Most studies were found for the Disability of the Arm, Shoulder, and Hand scale (DASH), the Shoulder Pain and Disability Index (SPADI), and the American Shoulder and Elbow Surgeons Standardised Shoulder Assessment Form (ASES). None of the questionnaires demonstrated satisfactory results for all properties. Most questionnaires claim to measure several domains (for example, pain, physical, emotional, and social functioning), yet dimensionality was studied in only three instruments. The internal consistency was calculated for seven questionnaires and only one received an adequate rating. Twelve questionnaires received positive ratings for construct validity, although depending on the population studied, four of these questionnaires received poor ratings too. Seven questionnaires were shown to have adequate test-retest reliability (ICC >0.70), but five questionnaires were tested inadequately. In most clinimetric studies only small sample sizes (n<43) were used. Nearly all publications lacked information on the interpretation of scores. CONCLUSION: The DASH, SPADI, and ASES have been studied most extensively, and yet even published validation studies of these instruments have limitations in study design, sample sizes, or evidence for dimensionality. Overall, the DASH received the best ratings for its clinimetric properties.     

COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures

Purpose

The original COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed to assess the methodological quality of single studies on measurement properties of Patient-Reported Outcome Measures (PROMs). Now it is our aim to adapt the COSMIN checklist and its four-point rating system into a version exclusively for use in systematic reviews of PROMs, aiming to assess risk of bias of studies on measurement properties.

Methods

For each standard (i.e., a design requirement or preferred statistical method), it was discussed within the COSMIN steering committee if and how it should be adapted. The adapted checklist was pilot-tested to strengthen content validity in a systematic review on the quality of PROMs for patients with hand osteoarthritis.

Results

Most important changes were the reordering of the measurement properties to be assessed in a systematic review of PROMs; the deletion of standards that concerned reporting issues and standards that not necessarily lead to biased results; the integration of standards on general requirements for studies on item response theory with standards for specific measurement properties; the recommendation to the review team to specify hypotheses for construct validity and responsiveness in advance, and subsequently the removal of the standards about formulating hypotheses; and the change in the labels of the four-point rating system.

Conclusions

The COSMIN Risk of Bias checklist was developed exclusively for use in systematic reviews of PROMs to distinguish this application from other purposes of assessing the methodological quality of studies on measurement properties, such as guidance for designing or reporting a study on the measurement properties.

     

Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist

Background  

The COSMIN checklist is a tool for evaluating the methodological quality of studies on measurement properties of health-related patient-reported outcomes. The aim of this study is to determine the inter-rater agreement and reliability of each item score of the COSMIN checklist (n = 114).     

Are factor analytical techniques used appropriately in the validation of health status questionnaires? A systematic review on the quality of factor analysis of the SF-36

Factor analysis is widely used to evaluate whether questionnaire items can be grouped into clusters representing different dimensions of the construct under study. This review focuses on the appropriate use of factor analysis. The Medical Outcomes Study Short Form-36 (SF-36) is used as an example. Articles were systematically searched and assessed according to a number of criteria for appropriate use and reporting. Twenty-eight studies were identified: exploratory factor analysis was performed in 22 studies, confirmatory factor analysis was performed in five studies and in one study both were performed. Substantial shortcomings were found in the reporting and justification of the methods applied. In 15 of the 23 studies in which exploratory factor analysis was performed, confirmatory factor analysis would have been more appropriate. Cross-validation was rarely performed. Presentation of the results and conclusions was often incomplete. Some of our results are specific for the SF-36, but the finding that both the application and the reporting of factor analysis leaves much room for improvement probably applies to other health status questionnaires as well. Optimal reporting and justification of methods is crucial for correct interpretation of the results and verification of the conclusions. Our list of criteria may be useful for journal editors, reviewers and researchers who have to assess publications in which factor analysis is applied.     

Facial disfigurement: is it in the eye of the beholder? A study in patients with Graves’ ophthalmopathy

OBJECTIVES: The importance of facial disfigurement in many diseases necessitates a reliable and valid measure of disfigurement severity for clinical studies. The hypothesis is that a universal concept of disfigurement exists and can be measured in a reliable way. The objectives of this study were to investigate if persons, in particular patients and physicians, can agree on facial disfigurement severity; and to determine the relative contribution of predefined clinical characteristics of patients with Graves' ophthalmopathy (GO) to the overall rating of facial disfigurement severity. DESIGN: A panel study was carried out in four different panels, each consisting of four members. PATIENTS: We randomly selected 100 slide pairs of GO patients from four available study populations, involving mild, moderate and severe GO patients (mean age 49 years, 76% female) who were treated with either radiotherapy, sham-irradiation, prednisone or orbital decompression. MEASUREMENTS: All panel members individually scored the disfigurement severity of 100 GO patients shown on standardized slides on a Visual Analog Scale. In total, 1600 ratings were collected. We calculated within- and between-panel agreement of disfigurement severity and identified determinants of disfigurement. RESULTS: Agreement within a panel varied from 0.65 to 0.79 and was highest within the panel of ophthalmologists. Between-panel agreement was 0.67 and was highest between ophthalmologists and laypersons. Compared with the global average, patients overrated and endocrinologists underrated disfigurement severity. Female panellists rated the patients, on average, more disfigured than male panellists. Important determinants of disfigurement were eyelid retraction, severe eyelid swelling and proptosis. Their relative importance was consistent across panels and in contrast to current measures of GO severity. CONCLUSION: Facial disfigurement severity can be measured in a reliable way using panels of panellists. Except for some systematic differences between panellists, facial disfigurement does not seem to be in the eye of the beholder.     

Minimally important change determined by a visual method integrating an anchor-based and a distribution-based approach

Background: Minimally important changes (MIC) in scores help interpret results from health status instruments. Various distribution-based and anchor-based approaches have been proposed to assess MIC. Objectives: To describe and apply a visual method, called the anchor-based MIC distribution method, which integrates both approaches. Method: Using an anchor, patients are categorized as persons with an important improvement, an important deterioration, or without important change. For these three groups the distribution of the change scores on the health status instrument are depicted in a graph. We present two cut-off points for an MIC: the ROC cut-off point and the 95% limit cut-off point. Results: We illustrate our anchor-based MIC distribution method determining the MIC for the Pain Intensity Numerical Rating Scale in patients with low back pain, using two conceivable definitions of minimal important change on the anchor. The graph shows the distribution of the scores of the health status instrument for the relevant categories on the anchor, and also the consequences of choosing the ROC cut-off point or the 95% limit cut-off point. Discussion: The anchor-based MIC distribution method provides a general framework, applicable to all kind of anchors. This method forces researchers to choose and justify their choice of an appropriate anchor and to define minimal importance on that anchor. The MIC is not an invariable characteristic of a measurement instrument, but may depend, among other things, on the perspective from which minimal importance is considered and the baseline values on the measurement instrument under study. A balance needs to be struck between the practicality of a single MIC value and the validity of a range of MIC values.     

Analyse der Dynamik des Ziliarmuskels w?hrend der Akkommodation

Hintergrund. Das Ausma? der anterioaxialen Verschiebung des humanen Ziliarmuskels w?hrend der Akkommodation sowie dessen etwaige altersabh?ngige ?nderung entzieht sich bis heute unserer Kenntnis.     

Interobserver reproducibility of the visual estimation of range of motion of the shoulder

OBJECTIVES: To assess interobserver reproducibility (agreement and reliability) of visually estimated shoulder range of motion (ROM) and to study the influence of clinical characteristics on the reproducibility. DESIGN: Test-retest analyses. SETTING: Various health care settings in the Netherlands. PARTICIPANTS: Consecutive patients with shoulder complaints (N = 201) referred by 20 general practitioners, 2 orthopedic physicians, and 20 rheumatologists. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Independent visual estimation by 2 physiotherapists of the ROM. Agreement was calculated as the mean difference in visual estimation between examiners +/-1.96 x standard deviations of this mean difference. The intraclass correlation coefficient (ICC) was calculated as a measure of reliability, based on a 2-way random effects analysis of variance. RESULTS: The lowest level of agreement was for visual estimation of active and passive elevation (limits of agreement, -43.4 to 39.8 and -46.7 to 41.5, respectively, for the difference between the affected and contralateral sides), for which the level of agreement was most clearly associated with pain severity and disability. The ability to differentiate between subjects was acceptable for all movements for the difference between the affected and contralateral sides (ICCs, > .70) except for horizontal adduction (ICC = .49). CONCLUSIONS: Interobserver agreement was low for the assessment of active and passive elevation, especially for patients with a high pain severity and disability. Except for horizontal adduction, visual estimation seems suitable for distinguishing differences between affected and contralateral ROM between subjects.