Distribution-based criteria for change in health-related quality of life in Parkinson's disease

BACKGROUND AND OBJECTIVE: To be useful, results from health-related quality of life (HRQoL) measures must be interpretable. The objective of this article is to examine statistical (distributional) approaches to interpretability. The standard error of measurement (SEM) and the standard error of the difference (S(diff)) are used in data on individuals with Parkinson's disease to calculate the minimum change scores required to be statistically meaningful for each dimension of an instrument to assess HRQoL in Parkinson's disease, the PDQ-39. METHODS: Data was collected from both a community and a clinic study; in both studies the PDQ-39 was administered at baseline and follow-up. RESULTS: The patterns of SEMs and S(diff)s were similar both across time periods and between samples, for all dimensions except Social Support. CONCLUSIONS: The results suggest that, for example, six points change on a 0-100 transformed scoring of the Mobility dimension may be considered on distributional grounds a minimum meaningful change. The demonstrated consistency across occasions and types of sample of SEMs and S(diff) for the majority of the dimensions of the PDQ-39, is evidence of the theoretically claimed advantage of this measure of sample independence, and supports use of this distributional approach to minimum meaningful change.     

Using the standard error of measurement to identify important changes on the Asthma Quality of Life Questionnaire

Objectives: To establish a link between the minimal important difference (MID) and the standard error of measurement (SEM) for all responsive dimensions of the Asthma Quality of Life Questionnaire (AQLQ). Methods: Secondary data analysis of baseline and follow-up interview data from 198 outpatients with asthma enrolled in a randomized controlled trial and receiving care at a major urban academic medical center's general medicine clinics. Domain statistics for baseline and follow-up interviews were examined for the AQLQ. The baseline SEM values were compared with established AQLQ MID standards using weighted values. Results: One SEM identified the MID in responsive AQLQ dimensions. Weighted values (0.88–0.93) validated excellent agreement between these two criteria. Conclusion: This is the third study to support using one SEM to identify important individual change in health-related quality of life (HRQoL) measures. However, refinement of the process for determining a measure's clinically meaningful differences is still needed to secure a link between the SEM and the identification of relevant HRQoL change over time.     

Measuring Clinically Meaningful Change Following Mental Health Treatment

Assessment of clinically meaningful change is useful for treatment planning, monitoring progress, and evaluating treatment response. Outcome studies often assess statistically significant change, which may not be clinically meaningful. Study objectives are to: (1) evaluate responsiveness of the BASIS-24 using three methods for determining clinically meaningful change: reliable change index (RCI), effect size (ES), and standard error of measurement (SEM); and (2) determine which method provides an estimate of clinically meaningful change most concordant with other change measures. BASIS-24 assessments were obtained at two time points for 1,397 inpatients and 850 outpatients. The proportion showing clinically meaningful change using each method was compared to the proportion showing change in global mental health, retrospectively reported change, and clinician-assessed change. BASIS-24 demonstrated responsiveness at both aggregate and individual levels. Regarding clinically meaningful improvement and decline, SEM was most concordant with all three outcome measures; regarding no change, RCI was most concordant with all three measures.     

Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life.

This study used the standard error of measurement (SEM) to evaluate intra-individual change on both the Chronic Respiratory Disease Questionnaire (CRQ) and the SF-36. After analyzing the reliability and validity of both instruments at baseline among 471 COPD outpatients, the SEM was compared to established minimal clinically important difference (MCID) standards for three CRQ dimensions. A value of one SEM closely approximated the MCID standards for all CRQ dimensions. This SEM-based criterion was then validated by cross-classifying the change status (improved, stable, or declined) of 393 follow-up outpatients using the one-SEM criterion and the MCID standard. Excellent agreement was achieved for all three CRQ dimensions. Although MCID standards have not been established for the SF-36, the one-SEM criterion was explored in these change scores. Among SF-36 scales demonstrating acceptable reliability and reasonable variance, the percent of individuals within each change category was consistent with those seen in the CRQ dimensions. These results replicate previous findings where a value of one SEM also closely approximated MCIDs for all dimensions of the Chronic Heart Disease Questionnaire among cardiovascular outpatients. The one-SEM criterion should be explored in other health-related quality of life instruments with established MCIDs.     

Minimally important change was estimated for the Manchester–Oxford Foot Questionnaire after foot/ankle surgery

ObjectivesTo ascertain the smallest amounts of change for the three Manchester–Oxford Foot Questionnaire (MOXFQ) domains that are likely to be clinically meaningful and beyond measurement error for conditions affecting the foot/ankle. Estimates were compared with those from the Short-Form 36 (SF-36).Study Design and SettingA prospective observational study of 671 consecutive patients undergoing foot or ankle surgery at an orthopedic hospital. Before and 9 months after surgery, patients completed the MOXFQ and SF-36; transition items (anchor) asked about perceived changes in foot/ankle pain or problems since the surgery.ResultsFour hundred ninety-one patients completed pre- and postoperative questionnaires. Anchor-based minimal clinically important change (MCIC) values were ∼13 points for each of the MOXFQ Walking/standing (W/S), Pain, and Social Interaction (S-I) domains [and greater than the standard error of measurement (SEM)]. MCIC values for all SF-36 domains fell within the SEM. Between-group MCIDs for the MOXFQ were W/S, 16.2; Pain, 9.9; S-I, 9.3. Distribution-based minimal detectable change (MDC₉₀) values for the MOXFQ were ∼11, ∼12, and ∼16 score points for the W/S, Pain, and S-I scales, respectively.ConclusionThis article provides information for aiding the interpretability of MOXFQ outcomes data and for planning future studies. The SF-36 is not recommended as a primary outcome for foot/ankle surgery.     

Estimating Clinically Meaningful Changes for the Functional Assessment of Cancer Therapy—Prostate: Results from a Clinical Trial of Patients with Metastatic Hormone-Refractory Prostate Cancer

Objective:  To determine clinically meaningful changes (CMCs) for the Functional Assessment of Cancer Therapy–Prostate (FACT–P).
Methods:  We obtained data from a Phase III trial of atrasentan in metastatic hormone-refractory prostate cancer patients (n = 809). We determined anchor-based differences using Karnofsky Performance Status (KPS), bone alkaline phosphatase (BAP), hemoglobin, time to disease progression (TTP), adverse events (AE), and survival. One-third and one-half standard deviation and standard error of measurement (SEM) were used as distribution-based criteria for CMCs. Comparison across baseline FACT–P domains and derived scales [FACT–P total score, Trial Outcome Index (TOI) score, prostate cancer subscale (PCS) score, pain-related score, and FACT Advanced Prostate Symptom Index (FAPSI)] were conducted for KPS, BAP, and hemoglobin using Student's t tests. Twelve-week change scores were compared for TTP, AE, and survival using ANCOVA.
Results:  CMCs were estimated as 6 to 10 for FACT–P total score, 5 to 9 for FACT–P TOI score, 2 to 3 for FACT–P PCS, 1 to 2 for the 4 PCS pain-related questions, and 2 to 3 for FAPSI. CMCs were also estimated using distribution-based criteria. Kappa statistics were computed to determine the degree of correspondence between the recommended guideline of 1.0 SEM and empirically derived standards. Most of the kappas for health-related quality of life domains and SEM standards had "substantial" to "almost perfect" concordance.
Conclusions:  The significant relationship between clinical and quality of life data provides support for the use of CMCs to increase interpretability of FACT–P scores.     

Standard error of measurement as a valid alternative to minimally important difference for evaluating the magnitude of changes in patient-reported outcomes measures

OBJECTIVE: To assess the degree of agreement between standard error of measurement (SEM) and minimally important difference (MID) criteria to evaluate the magnitude of the change caused by a medical intervention. STUDY DESIGN AND SETTING: Data were obtained from a cohort of 603 patients with neuropathic pain undergoing analgesic treatment with gabapentin who completed four health scales: Medical Outcomes Study Sleep Scale, Sheehan Disability Scale, Covi Anxiety Scale, and Raskin Depression scale. After calculating MID and SEM values for all scales, patients were classified into three categories: improvement, no change, and worsening. Agreement between the two criteria was assessed using Cohen's kappa index of agreement and Kendall's tau-b linear correlation coefficient. RESULTS: The 1 SEM criterion showed the highest agreement (kappa=0.68-1.00) and correlation (tau-b=0.75-1.00) with the MID criterion. Sensitivity analysis performed in gabapentin responders and nonresponders confirmed the results of the main analysis. CONCLUSION: The 1 SEM criterion is a valid alternative to the MID criterion to evaluate the magnitude of the change produced in patient-reported health outcomes measures.     

A practical approach to Bland-Altman plots and variation coefficients for log transformed variables

OBJECTIVE: Indicators of reproducibility for log-transformed variables can often not be calculated straightforwardly and are subsequently incorrectly interpreted. METHODS AND RESULTS: We discuss meaningful Coefficients of Variation (CV) for log-transformed variables, which can be derived directly from the standard error of the log-transformed measurements. To provide easy interpretable Bland and Altman plots, we calculated limits of inter and intraobserver agreement (LA) for log-transformed variables and transform them back to the original scale. These LAs for agreement are subsequently plotted on the original scale in a conventional Bland and Altman plot. Both approaches were illustrated in a clinical example on the reproducibility of skinfold thickness measurements. CONCLUSION: In reproducibility, it is important to calculate meaningful CVs, LAs, and Bland-Altman plots for log-transformed variables. We provide a practical approach in which existing statistical methods are applied in the field of reproducibility, thus leading to parameters of reproducibility which can be interpreted on the original scale.     

Standard equations are not accurate in assessing resting energy expenditure in patients with amyotrophic lateral sclerosis

OBJECTIVE: To assess the utility of standard equations for calculating caloric requirements in patients with amyotrophic lateral sclerosis (ALS). BACKGROUND: Malnutrition substantially increases the risk of death in ALS. Weight loss can be stabilized and survival prolonged with early gastrostomy feeding. However the use of standard nutrition equations has not been validated in this population. We therefore compared measured caloric expenditure to 2 predictive equations in patients with varying stages of ALS. METHODS: Thirty-four patients were studied. Caloric expenditure and respiratory quotient (R) were measured using indirect calorimetry. Results were compared with the Harris-Benedict equation. RESULTS: The prediction error for the Harris-Benedict equation was 18.6 + 14.9%. Limits of agreement showed this equation could overestimate caloric expenditure by 591 kcal/d and underestimate requirements by 677 kcal/d. R was >0.86 in 11 patients, suggesting overfeeding, and <0.8 in 15 patients, suggesting underfeeding. The difference between predicted and measured caloric expenditure did not correlate with disease severity, disease duration, or body mass index. Mechanically ventilated patients had higher than predicted energy expenditure. CONCLUSIONS: We found that standard equations used to calculate energy expenditure were not valid for patients with ALS. Moreover, the majority of our patients were either overfed or underfed. As underfeeding can cause diaphragm impairment, and overfeeding can increase ventilatory load, indirect calorimetry should be considered in ALS patients to determine optimal caloric requirement.     

Improving progress test score estimation using bayesian statistics

Medical Education 2011: 45 : 570–577 Objectives Progress tests give a continuous measure of a student’s growth in knowledge. However, the result at each test instance is subject to measurement error from a variety of sources. Previous tests contain useful information that might be used to reduce this error. A Bayesian statistical approach to using this prior information was investigated. Methods We first developed a Bayesian model that used the result from only one preceding test to update both the current estimated test score and its standard error of measurement (SEM). This was then extended to include results from all previous tests. Results The Bayesian model leads to an exponentially weighted combination of test scores. The results show smoothing of test scores when all previous tests are included in the model. The effective sample size is doubled, leading to a 30% reduction in measurement error. Conclusions A Bayesian approach can give improved score estimates and smaller SEMs. The method is simple to use with large cohorts of students and frequent tests. The smoothing of raw scores should give greater consistency in rank ordering of students and hence should better identify both high‐performing students and those in need of remediation.     

Patients' health-related quality-of-life and health state values for motor neurone disease/amyotrophic lateral sclerosis

Motor neurone disease (MND) has a severe impact on patient quality of life, especially in later stages of the disease. This study assesses the health-related quality of life (HRQL) of MND patients, and for the first time elicits health state values from patients for their present health state. A structured interview was conducted with 77 patients. Patients completed a disease specific health status measure (ALSAQ-40), a generic health status measure (EuroQol EQ-5D), a visual analogue scale (VAS) and a standard gamble (SG) exercise. The ALSAQ-40 was sensitive to disease severity. Patients' mean VAS rating of their own health ranged from 0.74 for stage 1 (early) disease severity (n = 15), to 0.37 for stage 4 (late stage) disease severity (n = 19). Utilities elicited via SG varied from a mean of 0.79 for stage 1 disease severity to a mean of 0.45 for stage 4 disease severity. The EQ-5D derived single index ranged from a mean of 0.63 for stage 1 disease severity to a mean of –0.01 for stage 4 disease severity. This study demonstrates that it is feasible and practical to obtain health state values from MND patients and it provides evidence that patients place a high value on their HRQL, even in cases where health status is very poor.     

中学生预防艾滋病教育题库信效度分析

目的检验学校预防艾滋病教育初、高中题库的应用效果,为发挥其在评估中的教育引导功能提供参考依据。方法在北京、内蒙古、新疆、陕西、上海5个省(市),采用方便整群抽样方法抽取20所初中与20所高中共3 881名学生进行问卷调查并进行分析。结果学校预防艾滋病教育初、高中题库中产生的问卷具有较好的区分度和内容效度;项目省知识和总分得分均高于非项目省,差异均有统计学意义(P值均<0.01);项目省知识和总分的合格率均高于非项目省,差异均有统计学意义(P值均<0.01);初、高中问卷的知识、态度得分和总分与"中学生生活技能评价量表"的8个维度和量表总分之间均呈弱的正相关,差异均有统计学意义(P值均<0.01)。初、高中问卷技能维度的题目有待增加和修订。结论初、高中题库产生的问卷在整体上具有较好的区分度和内容效度,可以作为用于学校艾滋病教育评估的测试工具。     

Comparison of anchor-based and distributional approaches in estimating important difference in common cold

Context Evaluative health-related quality-of-life instruments used in clinical trials should be able to detect small but important changes in health status. Several approaches to minimal important difference (MID) and responsiveness have been developed. Objectives To compare anchor-based and distributional approaches to important difference and responsiveness for the Wisconsin Upper Respiratory Symptom Survey (WURSS), an illness-specific quality of life outcomes instrument. Design Participants with community-acquired colds self-reported daily using the WURSS-44. Distribution-based methods calculated standardized effect size (ES) and standard error of measurement (SEM). Anchor-based methods compared daily interval changes to global ratings of change, using: (1) standard MID methods based on correspondence to ratings of “a little better” or “somewhat better,” and (2) two-level multivariate regression models. Participants About 150 adults were monitored throughout their colds (1,681 sick days.): 88% were white, 69% were women, and 50% had completed college. The mean age was 35.5 years (SD = 14.7). Results WURSS scores increased 2.2 points from the first to second day, and then dropped by an average of 8.2 points per day from days 2 to 7. The SEM averaged 9.1 during these 7 days. Standard methods yielded a between day MID of 22 points. Regression models of MID projected 11.3-point daily changes. Dividing these estimates of small-but-important-difference by pooled SDs yielded coefficients of .425 for standard MID, .218 for regression model, .177 for SEM, and .157 for ES. These imply per-group sample sizes of 870 using ES, 616 for SEM, 302 for regression model, and 89 for standard MID, assuming α = .05, β = .20 (80% power), and two-tailed testing. Conclusions Distribution and anchor-based approaches provide somewhat different estimates of small but important difference, which in turn can have substantial impact on trial design.     

Measurement precision for repeat examinees on a standardized patient examination

Examinees who initially fail and later repeat an SP-based clinical skills exam typically exhibit large score gains on their second attempt, suggesting the possibility that examinees were not well measured on one of those attempts. This study evaluates score precision for examinees who repeated an SP-based clinical skills test administered as part of the US Medical Licensing Examination sequence. Generalizability theory was used as the basis for computing conditional standard errors of measurement (SEM) for individual examinees. Conditional SEMs were computed for approximately 60,000 single-take examinees and 5,000 repeat examinees who completed the Step 2 Clinical Skills Examination(?) between 2007 and 2009. The study focused exclusively on ratings of communication and interpersonal skills. Conditional SEMs for single-take and repeat examinees were nearly indistinguishable across most of the score scale. US graduates and IMGs were measured with equal levels of precision at all score levels, as were examinees with differing levels of skill speaking English. There was no evidence that examinees with the largest score changes were measured poorly on either their first or second attempt. The large score increases for repeat examinees on this SP-based exam probably cannot be attributed to unexpectedly large errors of measurement.     

The minimal detectable change cannot reliably replace the minimal important difference

ObjectiveWe compared the minimal important difference (MID) with the minimal detectable change (MDC) generated by distribution-based methods.Study DesignStudies of two quality-of-life instruments (Chronic Respiratory Questionnaire [CRQ] and Rhinoconjunctivitis Quality of Life Questionnaire [RQLQ]) and two physician-rated disease-activity indices (Pediatric Ulcerative Colitis Activity Index [PUCAI] and Pediatric Crohn's Disease Activity Index [PCDAI]) provided longitudinal data. The MID values were calculated from global ratings of change (small change for CRQ and RQLQ; moderate for PUCAI and PCDAI) using receiver-operating characteristic (ROC) curve and mean change. Results were compared with five distribution-based strategies.ResultsOf the methods used to calculate the MDC, the 95% limits of agreement and the reliable change index yielded the largest estimates. In the patient-rated psychometric instruments, 0.5 SD was always greater than 1 standard error of measurements (SEM), and both fell between the mean change and the ROC estimates, on two of four occasions. The reliable change index came closest to MID of moderate change.ConclusionFor patient-rated psychometric instruments, 0.5 SD and 1 SEM provide values closest to the anchor-based estimates of MID derived from small change, and the reliable change index for physician-rated clinimetric indices based on moderate change. Lack of consistency across measures suggests that distribution-based approaches should act only as temporary substitutes, pending availability of empirically established anchor-based MID values.     

The Quality of Life of Parents of Children with Atopic Dermatitis: Interpretation of PIQoL-AD Scores

The PIQoL-AD has been used in several trials assessing quality of life (QoL) in parents of children with atopic dermatitis (AD), treated with pimecrolimus cream 1%. Secondary analysis of data from four trials (n = 621) was undertaken to help interpret the meaningfulness (significance) of the QoL results. A combination of anchor-based and distribution-based methods of interpreting instrument scores were employed. For each level on four AD severity indicators, mean PIQoL-AD scores and 95% confidence intervals were calculated. Low levels of association (r_s range = 0.12–0.45) were observed between QoL and clinical indicators; confirming previous findings. When data from the different trials and time-points were combined clear patterns emerged indicating the clinical meaning of PIQoL-AD scores. Distribution-based statistics such as effect sizes, the standard error of measurement (SEM) and the standard error of difference (S_diff) were also calculated. Despite a lack of consensus on how best to determine the meaningfulness of QoL scores, a large database of trial data was successfully used to link QoL scores to disease severity, aiding clinical interpretation of the former. Effect size, SEM and S_diff values were consistent across the trials indicating that a change of 2 to 3 PIQoL-AD points over time could be considered meaningful. It is argued that it would be useful for clinicians to gain an understanding of the meaning of QoL scores through routine use of such measures in clinical practice.     

Meta-analytic evaluation of the association between head injury and risk of amyotrophic lateral sclerosis

Head injury is considered as a potential risk factor for amyotrophic lateral sclerosis (ALS). However, several recent studies have suggested that head injury is not a cause, but a consequence of latent ALS. We aimed to evaluate such a possibility of reverse causation with meta-analyses considering time lags between the incidence of head injuries and the occurrence of ALS. We searched Medline and Web of Science for case–control, cross-sectional, or cohort studies that quantitatively investigated the head-injury-related risk of ALS and were published until 1 December 2016. After selecting appropriate publications based on PRISMA statement, we performed random-effects meta-analyses to calculate odds ratios (ORs) and 95% confidence intervals (CI). Sixteen of 825 studies fulfilled the eligibility criteria. The association between head injuries and ALS was statistically significant when the meta-analysis included all the 16 studies (OR 1.45, 95% CI 1.21–1.74). However, in the meta-analyses considering the time lags between the experience of head injuries and diagnosis of ALS, the association was weaker (OR 1.21, 95% CI 1.01–1.46, time lag ≥ 1 year) or not significant (e.g. OR 1.16, 95% CI 0.84–1.59, time lag ≥ 3 years). Although it did not deny associations between head injuries and ALS, the current study suggests a possibility that such a head-injury-oriented risk of ALS has been somewhat overestimated. For more accurate evaluation, it would be necessary to conduct more epidemiological studies that consider the time lags between the occurrence of head injuries and the diagnosis of ALS.     

New Zealand and United Kingdom experiences with the RAND modified Delphi approach to producing angina and heart failure criteria for quality assessment in general practice

OBJECTIVES: (1) To describe the development of minimum review criteria for the general practice management in New Zealand (NZ) of two chronic diseases: stable angina and systolic heart failure, and (2) to compare the NZ angina criteria with a set produced in Manchester to assess the extent to which use of the same approach to criteria development yields similar criteria. METHODS: A modified Delphi approach, based on the RAND consensus panel method, was used to produce minimum criteria for reviewing the recorded management of heart failure and angina in NZ general practice. The criteria for angina were compared with those produced in the UK, including assessment of the extent to which each set describes actions that the other panel agrees are necessary to record. RESULTS: For each condition we report minimum criteria describing actions rated as (a) necessary to record and (b) inappropriate to take but, if taken, necessary to record. Although strong scientific evidence underpins approximately one quarter and one third, respectively, of the final sets of NZ and UK angina criteria for actions necessary to record, the NZ criteria agree strongly with the UK criteria (33 of 39 criteria, 85%) but there is less UK agreement with the NZ angina criteria (28 of 40 criteria, 70%). CONCLUSION: Despite the lack of scientific evidence for up to three quarters of angina care in general practice, the RAND based approach to criteria development was used in NZ to reproduce most of the UK angina criteria for actions rated as necessary to record in general practice. It is important to make explicit whether ratings of necessity and appropriateness apply to the recording of actions or to the actions themselves.     

Minimal important change and minimal detectable change in activities of daily living in community-living older people

Objective

To estimate the minimal important change (MIC) and the minimal detectable change (MDC) of the Katz-activities of daily living (ADL) index score and the Lawton instrumental activities of daily living (IADL) scale.

Design

Data from a cluster-randomized clinical trial and a cohort study.

Setting

General practices in the Netherlands.

Participants

3184 trial participants and 51 participants of the cohort study with a mean age of 80.1 (SD 6.4) years.

Measurements

At baseline and after 6 months, the Katz-ADL index score (0-6 points), the Lawton IADL scale (0-7 points), and self-perceived decline in (I)ADL were assessed using a self-reporting questionnaire. MIC was assessed using anchor-based methods: the (relative) mean change score; and using distributional methods: the effect size (ES), the standard error of measurement (SEM), and 0.5 SD. The MDC was estimated using SEM, based on a test-retest study (2-week interval) and on the anchor-based method.

Results

Anchor-based MICs of the Katz-ADL index score were 0.47 points, while distributional MICs ranged from 0.18 to 0.47 points. Similarly, anchor-based MICs of the Lawton IADL scale were between 0.31 and 0.54 points and distributional MICs ranged from 0.31 to 0.77 points. The MDC varies by sample size. For the MIC to exceed the MDC at least 482 patients are needed.

Conclusion

The MIC of both the Katz-ADL index and the Lawton IADL scale lie around half a point. The certainty of this conclusion is reduced by the variation across calculational methods.

     

Push and pull: Exposure of young Taiwanese women to sexually explicit materials

This study was designed to understand young Taiwanese women’s perception of sexually explicit materials (SEMs). Researchers conducted six focus group discussions with 38 young women between the ages of 18 and 22 in Taiwan in 2009–2010 and used content analysis to analyze the data based on the push-pull theory. The results showed that the exposure of young women to SEMs was a sexual exploration process from no sexual activity to future sexual activity. This process was affected by the interactions of three powers: push power, pull power, and personal factors. The push power included factors, such as parents and social values, which failed to satisfy their sexual curiosity and provide them with autonomy. The pull power included SEMs and peer influence, which increased sexual arousal stimuli and curiosity to try sexual activity. The most important personal factors were young women’s growth, including sexual curiosity, cognition of SEMs, and gender equity in freedom to make sexual decisions. Understanding this push-pull process regarding SEM can help health-care providers with their own discourses in addressing sex and influence young women’s participation in desired, protected, and enjoyable sex when sufficiently ready.