Scale Linking to Enable Patient-Reported Outcome Performance Measures Assessed with Different Patient-Reported Outcome Measures

Background

Patient-reported outcome performance measures (PRO-PMs) incorporate outcomes from the patient’s perspective into performance measures and may have great potential to impact health care. The various patient-reported outcome measures (PROMs) used to assess the same outcome challenge widespread use of PRO-PMs. A potential solution is to statistically link PROMs to provide equivalent PRO-PM conclusions to be drawn regardless of which PROM was used.

Frequently Used Patient-Reported Outcome Measures of General Physical Function Were Highly Correlated With a Multitask Performance Outcome Test Battery

ObjectivesThis study aimed to determine the relationship between frequently used patient-reported outcome (PRO) measures and a multitask performance outcome (PerfO) measure of general physical function (PF) and to examine the association of these measures with depressive mood, pain, and age.MethodsFrequently used PRO measures of general PF (Patient-Reported Outcomes Measurement Information System [PROMIS] PF item bank, PROMIS PF Short Form 20a, Short Form 36 Physical Function Scale) and a PerfO test battery, namely, the Physical Performance Test (PPT), were administered to 78 adult patients from 3 inpatient clinics (cardiology and angiology, rheumatology and clinical immunology, and psychosomatic medicine) at Charité – Universitätsmedizin Berlin. Pearson correlations were used to investigate the associations between PRO measures and the PPT. To explore the predictive value of age, depressive symptoms, and pain intensity, we conducted multiple linear regression analysis for each PF measure.ResultsWe found strong linear relationships between PRO measures and PPT sum scores. Correlations between PPT sum scores and PROMIS PF T-scores were r > 0.75. For all PRO and PerfO measures, age was a predictor of general PF whereas depressive mood was not found to be a relevant predictor. Moreover, pain intensity was found to be a significant predictor of PRO measures but not for PPT sum scores.ConclusionsOur findings suggest that frequently used PRO measures and a multitask PerfO measure of general PF can be used to measure a common PF construct. Nevertheless, PF scores based on PRO measures should ideally be controlled for self-rated pain intensity.     

Common Patient-Reported Outcomes Within the Food and Drug Administration Voice of the Patient Reports

ObjectivesProponents of disease-specific patient-reported outcome measurements (PROMs) often argue disease-agnostic measures do not adequately capture their patient population’s experience. Patient-Reported Outcomes Measurement Information System (PROMIS) provides a disease-agnostic domain set that may adequately cover many diseases. This study seeks to investigate whether PROMIS’s quality of life domain coverage can span patient-reported outcomes (PROs) elicited from patients across unrelated diseases.MethodsThe Food and Drug Administration Voice of the Patient reports were an initiative to elevate patient voices regarding their condition and associated treatments. Two reviewers extracted patient-reported health-related (quality of life) domains from the reports and categorized them into PROMIS domains or non-PROMIS domains. Domain coverage was summarized for each report. Any extracted PROs not covered by PROMIS domains were placed in an “other” category and analyzed for common themes.ResultsAcross 26 reports, PROMIS covered 216 of 374 (70%) of the reports’ PRO domains. The heritable bleeding disorders report had the highest coverage (82%). Human immunodeficiency virus had the lowest coverage (50%). The most common PROMIS domain, “ability to participate in social roles,” appeared in 25 reports (96%). The most common domains not included in PROMIS were stigma, sensitivities, and sensory deficits as evident in 19 (73%), 18 (69%), and 18 reports (69%), respectively. If the top 3 unincluded domains were amended into PROMIS, the total domain coverage would increase to 84%.ConclusionsPRO domains elicited in the Food and Drug Administration Voice of the Patient reports were widely captured by PROMIS, suggesting domains patients experience contain enough overlap to be recorded by appropriate PROMIS domains. PROMIS could increase its coverage by adding domains.     

Using Patient-Reported Outcome Measures to Evaluate Care for Patients With Inflammatory Chronic Rheumatic Disease

ObjectivesFew countries integrate patient-reported outcome measures (PROMs) in routine performance assessment and those that do focus on elective surgery. This study addresses the challenges of using PROMs to evaluate care in chronic conditions. We set out a modeling strategy to assess the extent to which changes over time in self-reported health status by patients with inflammatory chronic rheumatic disease are related to their biological drug therapy and rheumatology center primarily responsible for their care.MethodsUsing data from the Portuguese Register of Rheumatic Diseases, we assess health status using the Health Assessment Questionnaire-Disability Index for rheumatic patients receiving biological drugs between 2000 and 2017. We specify a fixed-effects model using the least squares dummy variables estimator.ResultsPatients receiving infliximab or rituximab report lower health status than those on etanercept (the most common therapy) and patients in 4 of the 26 rheumatology centers report higher health status than those at other centers.ConclusionsPROMs can be used for those with chronic conditions to provide the patient’s perspective about the impact on their health status of the choice of drug therapy and care provider. Care for chronic patients might be improved if healthcare organizations monitor PROMs and engage in performance assessment initiatives on a routine basis.     

Toward System-Wide Implementation of Patient-Reported Outcome Measures: A Framework for Countries,States, and Regions

ObjectivesThis study aimed to develop a framework facilitating (1) the maturity assessment of healthcare systems regarding patient-reported outcome measure (PROM) implementation and (2) the comparison of different healthcare systems’ PROM implementation levels to guide discussions and derive lessons for regional, state-level, and national PROM initiatives.MethodsGuided by the grounded theory methodology, a PROM healthcare system implementation framework was developed following multiple steps. Based on interviews with 28 experts from 12 countries and a literature review, a framework was drafted and refined through 29 additional validation interviews.ResultsThe resulting framework comprises 5 implementation stages along 7 dimensions. Implementation stages range from “first experimentation” to “system-wide adoption and a vibrant ecosystem.” The dimensions are grouped into patient-reported outcome (PRO) measurement and PRO utilization, the former with the dimensions “scope and condition coverage,” “metric and process standardization,” and “tools and information technology–based solutions” and the latter with “patient empowerment and clinical decision support,” “reporting and quality improvement,” and “rewarding and contracting.” The “culture and stakeholder involvement” dimension connects both groups. Although a concerted implementation approach across dimensions can be observed in advanced countries, others show a more uneven adoption.ConclusionsThe framework and its preliminary application to different healthcare systems demonstrate (1) the importance of coherent progress across complementing dimensions and (2) the relevance of PROM integration across clinical specialties and care sectors to strengthen patient-centered care. Overall, the framework can facilitate dialogues between stakeholders to analyze the current PROM implementation status and strategies to advance it.     

Updated Recommendations on Evidence Needed to Support Measurement Comparability Among Modes of Data Collection for Patient-Reported Outcome Measures: A Good Practices Report of an ISPOR Task Force

The ISPOR Task Force on measurement comparability between modes of data collection for patient-reported outcome measures (PROMs) has updated the good practice recommendations from the 2009 ISPOR electronic patient-reported outcome and 2014 patient-reported outcome mixed modes Good Research Practices Task Force reports in light of accumulated evidence of measurement comparability among different modes of PROM data collection. Furthermore, with the increasing use of electronic formats of clinical outcome assessments in clinical trials and the US Food and Drug Administration’s encouragement of electronic data collection, this new task force report provides stakeholders with best practice recommendations reflecting the current body of evidence and enables them to respond to future developments in research and technology.This task force recommends an evidence-based approach to determine whether new research is needed to evaluate measurement comparability for a given questionnaire or technology. The suitability of existing evidence depends upon whether it satisfactorily demonstrates that the change in data collection mode has not affected the PROM’s measurement properties. In cases where sufficient evidence of measurement comparability exists and best practices for faithful migration are followed, this task force concludes that further testing of measurement comparability among the data collection modes is unnecessary, including cases of “mixing modes” within clinical trials such as bring your own device designs.     

Item-Level Psychometric Properties for a New Patient-Reported Psoriasis Symptom Diary

ObjectivesThis research evaluated the psychometric properties of a new Psoriasis Symptom Diary, identified diary responder definitions for use in determining whether a patient has experienced clinically meaningful change, and refined diary item content for use in future clinical trials.MethodsThe Psoriasis Symptom Diary was administered in a phase 2 clinical trial of AIN457 to US adult outpatients (N = 172) with physician-diagnosed moderate to severe chronic plaque-type psoriasis. Participant compliance with daily diary administration and item score variability, reliability, construct and discriminant validity, sensitivity to change, and interpretation were all evaluated.ResultsParticipants completed 94% of scheduled diary assessments across 12 study weeks. Diary items were generally normally distributed, and no floor or ceiling effects were observed. Item reliability (reproducibility) was acceptable (intraclass correlation coefficients > 0.80), with an exception for one item (skin color). At week 12, items significantly related to criterion measures as predicted (Psoriasis Area and Severity Index r = 0.27–0.57; Investigator’s Global Assessment r = 0.25–0.59), with the exception of items that measured skin color and difficulty using hands. Most items generated change scores that were synchronous to changes as measured by the Psoriasis Area and Severity Index, Investigator’s Global Assessment, Dermatology Life Quality Index (r > 0.37), as well as the Patient Global Impression of Change. Responders experienced a 2- to 3-point and 3- to 5-point change in item scores for minimal and large improvements, respectively. Four items that did not perform well were dropped from the diary.ConclusionsThe 16-item Psoriasis Symptom Diary demonstrated favorable psychometric properties and is a brief, useful tool for measuring patient-based symptoms and the impact of chronic plaque psoriasis.     

Development of EMPRO: A Tool for the Standardized Assessment of Patient-Reported Outcome Measures

Objective:  This study was aimed to develop a tool for the standardized assessment of patient-reported outcomes (PROs) to assist the choice of instruments.
Methods:  An expert panel adapted the eight attributes proposed by the Medical Outcomes Trust as evaluation review criteria, created items to evaluate them, and included a response scale for each item. A pilot test was designed to test the new tool's feasibility and to obtain preliminary information concerning its psychometric properties. The Spanish versions of five measures were selected for assessment: the SF-36 Health Survey, the Nottingham Health Profile, the COOP-WONCA charts, the EuroQol-5D, and the Quality of Life Questionnaire EORTC-QLQ-C30. We assessed the new tool's reliability (Cronbach's alpha and intraclass correlation coefficient [ICC]) and construct validity.
Results:  The new EMPRO (Evaluating the Measurement of Patient-Reported Outcomes) tool has 39 items covering eight key attributes: conceptual and measurement model, reliability, validity, responsiveness, interpretability, burden, alternative modes of administration, and cross-cultural and linguistic adaptations. Internal consistency was high (α = 0.95) as was interrater concordance (ICC: 0.87–0.94). Positive associations consistent with a priori hypotheses were observed between EMPRO attribute scores and the number of articles identified for the measures, the years elapsed since the publication of the first article, and the number of citations.
Conclusion:  A new tool for the standardized assessment of PRO measures is available. It has shown good preliminary reliability and validity and should be a useful aid to investigators who need to choose between alternative measures. Further assessment of the tool is necessary.     

A Systematic Review of Generic Multidimensional Patient-Reported Outcome Measures for Children,Part II: Evaluation of Psychometric Performance of English-Language Versions in a General Population

ObjectivesThe objectives of this systematic review were 1) to identify studies that assess the psychometric performance of the English-language version of 35 generic multidimensional patient-reported outcome measures (PROMs) for children and young people in general populations and evaluate their quality and 2) to summarize the psychometric properties of each PROM.MethodsMEDLINE, EMBASE, and PsycINFO were searched. The methodological quality of the articles was assessed using the COnsensus-based Standards for selection of health Measurement INstruments checklist. For each PROM, extracted evidence of content validity, construct validity, internal consistency, test-retest reliability, proxy reliability, responsiveness, and precision was judged against standardized reference criteria.ResultsWe found no evidence for 14 PROMs. For the remaining 21 PROMs, 90 studies were identified. The methodological quality of most studies was fair. Quality was generally rated higher in more recent studies. Not reporting how missing data were handled was the most common reason for downgrading the quality. None of the 21 PROMs has had all psychometric properties evaluated; data on construct validity and internal consistency were most frequently reported.ConclusionsOverall, consistent positive findings for at least five psychometric properties were found for Child Health and Illness Profile, Healthy Pathways, KIDSCREEN, and Multi-dimensional Student Life Satisfaction Scale. None of the PROMs had been evaluated for responsiveness to detect change in general populations. Further well-designed studies with transparent reporting of methods and results are required.     

The Role of Patient-Reported Outcome Measures in Value-Based Payment Reform

The U.S. health care system is currently experiencing profound change. Pressure to improve the quality of patient care and control costs have caused a rapid shift from traditional volume-driven fee-for-service reimbursement to value-based payment models. Under the 2015 Medicare Access and Children’s Health Insurance Program Reauthorization Act, providers will be evaluated on the basis of quality and cost efficiency and ultimately receive adjusted reimbursement as per their performance. Although current performance metrics do not incorporate patient-reported outcome measures (PROMs), many wonder whether and how PROMs will eventually fit into value-based payment reform. On November 17, 2016, the second annual Patient-Reported Outcomes in Healthcare Conference brought together international stakeholders across all health care disciplines to discuss the potential role of PROs in value-based health care reform. The purpose of this article was to summarize the findings from this conference in the context of recent literature and guidelines to inform implementation of PROs in value-based payment models. Recommendations for evaluating key perspectives and measurement goals are made to facilitate appropriate use of PROMs to best benefit and amplify the voice of our patients.     

A Novel Patient-Derived Conceptual Model of the Impact of Celiac Disease in Adults: Implications for Patient-Reported Outcome and Health-Related Quality-of-Life Instrument Development

Background

Celiac disease is a chronic inflammatory condition with wide ranging effects on individual’s lives caused by a combination of symptoms and the burden of adhering to a gluten-free diet (GFD).

Recommendations on Evidence Needed to Support Measurement Equivalence between Electronic and Paper-Based Patient-Reported Outcome (PRO) Measures: ISPOR ePRO Good Research Practices Task Force Report

BackgroundPatient-reported outcomes (PROs) are the consequences of disease and/or its treatment as reported by the patient. The importance of PRO measures in clinical trials for new drugs, biological agents, and devices was underscored by the release of the US Food and Drug Administration's draft guidance for industry titled “Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims.” The intent of the guidance was to describe how the FDA will evaluate the appropriateness and adequacy of PRO measures used as effectiveness end points in clinical trials. In response to the expressed need of ISPOR members for further clarification of several aspects of the draft guidance, ISPOR's Health Science Policy Council created three task forces, one of which was charged with addressing the implications of the draft guidance for the collection of PRO data using electronic data capture modes of administration (ePRO). The objective of this report is to present recommendations from ISPOR's ePRO Good Research Practices Task Force regarding the evidence necessary to support the comparability, or measurement equivalence, of ePROs to the paper-based PRO measures from which they were adapted.MethodsThe task force was composed of the leadership team of ISPOR's ePRO Working Group and members of another group (i.e., ePRO Consensus Development Working Group) that had already begun to develop recommendations regarding ePRO good research practices. The resulting task force membership reflected a broad array of backgrounds, perspectives, and expertise that enriched the development of this report. The prior work became the starting point for the Task Force report. A subset of the task force members became the writing team that prepared subsequent iterations of the report that were distributed to the full task force for review and feedback. In addition, review beyond the task force was sought and obtained. Along with a presentation and discussion period at an ISPOR meeting, a draft version of the full report was distributed to roughly 220 members of a reviewer group. The reviewer group comprised individuals who had responded to an emailed invitation to the full membership of ISPOR. This Task Force report reflects the extensive internal and external input received during the 16-month good research practices development process.Results/RecommendationsAn ePRO questionnaire that has been adapted from a paper-based questionnaire ought to produce data that are equivalent or superior (e.g., higher reliability) to the data produced from the original paper version. Measurement equivalence is a function of the comparability of the psychometric properties of the data obtained via the original and adapted administration mode. This comparability is driven by the amount of modification to the content and format of the original paper PRO questionnaire required during the migration process. The magnitude of a particular modification is defined with reference to its potential effect on the content, meaning, or interpretation of the measure's items and/or scales. Based on the magnitude of the modification, evidence for measurement equivalence can be generated through combinations of the following: cognitive debriefing/testing, usability testing, equivalence testing, or, if substantial modifications have been made, full psychometric testing. As long as only minor modifications were made to the measure during the migration process, a substantial body of existing evidence suggests that the psychometric properties of the original measure will still hold for the ePRO version. Hence, an evaluation limited to cognitive debriefing and usability testing only may be sufficient. However, where more substantive changes in the migration process has occurred, confirming that the adaptation to the ePRO format did not introduce significant response bias and that the two modes of administration produce essentially equivalent results is necessary. Recommendations regarding the study designs and statistical approaches for assessing measurement equivalence are provided.ConclusionsThe electronic administration of PRO measures offers many advantages over paper administration. We provide a general framework for decisions regarding the level of evidence needed to support modifications that are made to PRO measures when they are migrated from paper to ePRO devices. The key issues include: 1) the determination of the extent of modification required to administer the PRO on the ePRO device and 2) the selection and implementation of an effective strategy for testing the measurement equivalence of the two modes of administration. We hope that these good research practice recommendations provide a path forward for researchers interested in migrating PRO measures to electronic data collection platforms.     

A Review of Patient-Reported Outcome Labeling of FDA-Approved New Drugs (2016-2020): Counts,Categories, and Comprehensibility

ObjectivesA review of new drug approvals (NDAs) by the US Food and Drug Administration (FDA) for 2006 to 2015 showed that approximately 20% of new drugs had labeling based on patient-reported outcomes (PROs). The purpose of this study was to review labeling text based on PRO endpoints for NDAs from 2016 to 2020, with a special focus on the comprehensibility of such statements when included.MethodsWe reviewed drug approval reports on the Drugs@FDA web page of the FDA website to determine the number of NDAs from 2016 to 2020. For all identified NDAs, drug approval package and product labels were reviewed. NDAs from 2016 to 2020 were grouped by disease category as per International Classification of Diseases 10th Revision. Data were summarized for diseases that traditionally rely on PROs for evaluating treatment benefit (PRO dependent) and for diseases that traditionally do not rely on PROs (non-PRO dependent). Results were compared with NDAs from 2006 to 2010.ResultsNDAs amounting to 228 were identified from 2016 to 2020, 26.3% of which had labeling statements based on PRO endpoints. From 2006 to 2015 and from 2016 to 2020, PRO labeling statements were included in 46.5% (46 of 99) and 50.0% (47 of 94), respectively, of NDAs for PRO-dependent new molecular entities and in 6.0% (12 of 199) and 9.7% (13 of 199), respectively, of NDAs for non–PRO-dependent new molecular entities. Comprehensibility of labeling statements based on PRO endpoints was judged to be complex in 56.7% of product labels.ConclusionsThe increase in labeling text based on PRO endpoints in product labels is encouraging. However, there is room for improvement on the comprehensibility of labeling statements based on PRO endpoints.     

US Food and Drug Administration Analysis of Patient-Reported Diarrhea and Its Impact on Function and Quality of Life in Patients Receiving Treatment for Breast Cancer

ObjectivesMany trials conclude “no clinically meaningful detriment” to health-related quality of life (HRQL) or function between arms, even when notable differential toxicity is observed. Mean change from baseline analyses of function or HRQL can possibly obscure important change in subgroups experiencing symptomatic toxicity. We evaluate the impact of diarrhea, a key treatment arm toxicity, on patient-reported HRQL and functioning in clinical trials submitted to US Food and Drug Administration.MethodsThis study used 4 randomized, breast cancer trials (adjuvant to late-line metastatic) as case examples. Diarrhea, physical functioning (PF), and global health status and quality of life (GHS/QoL) from the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire-C30 were analyzed at baseline and approximately 3 and 6 months.ResultsGenerally, patients reporting very much diarrhea at months 3 and 6 had worse PF (9-19 points lower) and GHS/QoL (16-19 points lower) than patients reporting no diarrhea regardless of treatment arm. In the change from baseline analysis, patients reporting very much diarrhea also experienced a greater decrease in PF (6-13 points) and GHS/QoL (6-16 points) versus patients reporting no diarrhea in both arms.ConclusionsIn trials with moderate to large differences in symptomatic toxicity by arm, reporting “no meaningful difference in functioning and HRQL between arms” based on mean change from baseline analysis is insufficient and may obscure important impacts on subgroups experiencing symptomatic adverse events. Additional exploratory analyses with simple data visualizations evaluating functioning or HRQL in patient subgroups experiencing expected symptomatic toxicities can further inform the safety and tolerability of an investigational agent.