Comparison of Benefit-Risk Assessment Methods for Prospective Monitoring of Newly Marketed Drugs: A Simulation Study

ObjectivesTo compare benefit-risk assessment (BRA) methods for determining whether and when sufficient evidence exists to indicate that one drug is favorable over another in prospective monitoring.MethodsWe simulated prospective monitoring of a new drug (A) versus an alternative drug (B) with respect to two beneficial and three harmful outcomes. We generated data for 1000 iterations of six scenarios and applied four BRA metrics: number needed to treat and number needed to harm (NNT|NNH), incremental net benefit (INB) with maximum acceptable risk, INB with relative-value–adjusted life-years, and INB with quality-adjusted life-years. We determined the proportion of iterations in which the 99% confidence interval for each metric included and excluded the null and we calculated mean time to alerting.ResultsWith no true difference in any outcome between drugs A and B, the proportion of iterations including the null was lowest for INB with relative-value–adjusted life-years (64%) and highest for INB with quality-adjusted life-years (76%). When drug A was more effective and the drugs were equally safe, all metrics indicated net favorability of A in more than 70% of the iterations. When drug A was safer than drug B, NNT|NNH had the highest proportion of iterations indicating net favorability of drug A (65%). Mean time to alerting was similar among methods across the six scenarios.ConclusionsBRA metrics can be useful for identifying net favorability when applied to prospective monitoring of a new drug versus an alternative drug. INB-based approaches similarly outperform unweighted NNT|NNH approaches. Time to alerting was similar across approaches.     

Illustrating Emerging Good Practices for Quantitative Benefit-Risk Assessment: A Hypothetical Case Study of Systemic Biologic Treatments for Plaque Psoriasis

ObjectivesQuantitative benefit-risk assessment (qBRA) is a structured process to evaluate the benefit-risk balance of treatment options to support decision making. The ISPOR qBRA Task Force was recently established to provide recommendations for the design, conduct, and reporting of qBRA. This report presents a hypothetical case study illustrating how to apply the Task Force’s recommendations toward a qBRA to inform the benefit-risk assessment of brodalumab at the time of initial marketing approval. The qBRA evaluated 2 dosing regimens of brodalumab (210 mg or 140 mg twice weekly) compared with weight-based dosing of ustekinumab and placebo.MethodsWe followed the 5 steps recommended by the Task Force. Attributes included treatment response (≥75% improvement in Psoriasis Area and Severity Index), suicidal ideation and behavior, and infections. Performance data were drawn from pivotal clinical trials of brodalumab. The qBRA used multicriteria decision analysis and preference weights from a hypothetical discrete choice experiment. Sensitivity analyses examined the robustness of benefit-risk ranking to uncertainty in clinical effect and preference estimates, consideration of a subgroup (nail psoriasis), and the maintenance phase of treatment (52 weeks instead of 12).ResultsResults from this hypothetical qBRA suggest that brodalumab 210 mg had a more favorable benefit-risk profile compared with ustekinumab and placebo. Ranking of brodalumab compared with ustekinumab was dependent on brodalumab’s dose. Sensitivity analyses demonstrated robustness of benefit-risk ranking to uncertainty in clinical effect and preference estimates, as well as choice of attributes and length of follow-up.ConclusionThis case study demonstrates how to implement the ISPOR Task Force’s good practice recommendations on qBRA.     

Two Approaches to Incorporate Clinical Data Uncertainty into Multiple Criteria Decision Analysis for Benefit-Risk Assessment of Medicinal Products

BackgroundThe Problem formulation, Objectives, Alternatives, Consequences, Trade-offs, Uncertainties, Risk attitude, and Linked decisions (PrOACT-URL) framework and multiple criteria decision analysis (MCDA) have been recommended by the European Medicines Agency for structured benefit-risk assessment of medicinal products undergoing regulatory review.ObjectiveThe objective of this article was to provide solutions to incorporate the uncertainty from clinical data into the MCDA model when evaluating the overall benefit-risk profiles among different treatment options.MethodsTwo statistical approaches, the δ-method approach and the Monte-Carlo approach, were proposed to construct the confidence interval of the overall benefit-risk score from the MCDA model as well as other probabilistic measures for comparing the benefit-risk profiles between treatment options. Both approaches can incorporate the correlation structure between clinical parameters (criteria) in the MCDA model and are straightforward to implement.ResultsThe two proposed approaches were applied to a case study to evaluate the benefit-risk profile of an add-on therapy for rheumatoid arthritis (drug X) relative to placebo. It demonstrated a straightforward way to quantify the impact of the uncertainty from clinical data to the benefit-risk assessment and enabled statistical inference on evaluating the overall benefit-risk profiles among different treatment options.ConclusionsThe δ-method approach provides a closed form to quantify the variability of the overall benefit-risk score in the MCDA model, whereas the Monte-Carlo approach is more computationally intensive but can yield its true sampling distribution for statistical inference. The obtained confidence intervals and other probabilistic measures from the two approaches enhance the benefit-risk decision making of medicinal products.     

Quantitative Benefit-Risk Assessment in Medical Product Decision Making: A Good Practices Report of an ISPOR Task Force

Benefit-risk assessment is commonly conducted by drug and medical device developers and regulators, to evaluate and communicate issues around benefit-risk balance of medical products. Quantitative benefit-risk assessment (qBRA) is a set of techniques that incorporate explicit outcome weighting within a formal analysis to evaluate the benefit-risk balance. This report describes emerging good practices for the 5 main steps of developing qBRAs based on the multicriteria decision analysis process. First, research question formulation needs to identify the needs of decision makers and requirements for preference data and specify the role of external experts. Second, the formal analysis model should be developed by selecting benefit and safety endpoints while eliminating double counting and considering attribute value dependence. Third, preference elicitation method needs to be chosen, attributes framed appropriately within the elicitation instrument, and quality of the data should be evaluated. Fourth, analysis may need to normalize the preference weights, base-case and sensitivity analyses should be conducted, and the effect of preference heterogeneity analyzed. Finally, results should be communicated efficiently to decision makers and other stakeholders. In addition to detailed recommendations, we provide a checklist for reporting qBRAs developed through a Delphi process conducted with 34 experts.     

A Quantitative Benefit-Risk Analysis of Isoniazid for Treatment of Latent Tuberculosis Infection Using Incremental Benefit Framework

BackgroundWe undertook a quantitative benefit-risk analysis of a targeted isoniazid (INH) therapy for latent tuberculosis (TB) infection for different groups of contacts of active TB cases.MethodsWe developed a decision-analytic model to compare the treatment of latent TB infection in subgroups of contacts to no treatment over a 6-year time horizon in a Canadian setting. Contacts were stratified into 32 groups on the basis of five binary variables: type of contact (close or casual), tuberculin skin test (TST) results (positive or negative at 5 mm cutoff), Bacillus Calmette-Guérin vaccination status, place of birth (foreign- or Canadian-born), and age group (cutoff 35 years). Risk of TB reactivation was calculated for each subgroup from a longitudinal registry of contacts, adjusted for several potential confounders and comorbid conditions. We calculated the quality-adjusted life-years gained because of delayed or prevention of active TB via treatment of latent TB infection versus quality-adjusted life-years lost because of the adverse events to INH.ResultsA targeted policy based on adopting INH therapy only in subgroups with positive expected incremental net health benefit resulted in a different treatment decision than the current guidelines in five subgroups comprising 3.9% of the contacts. Namely, the targeted policy comprised no INH therapy in casual contacts with a positive vaccination history even with a positive TST result and INH therapy in foreign-born close contacts younger than 35 years even with a negative TST result.ConclusionsFrom a benefit-risk viewpoint, INH treatment of contacts should be tailored on the basis of risk assessment algorithms that consider a range of factors at the time of screening.     

A Framework for Conducting Risk Assessment

The new assessment framework does not mention the term 'risk', preferring to reframe it as need. Whilst it is clear that we do need to capture and process the multiplicity of need to determine the most appropriate response, abandoning the use of risk in our child care and child protection assessments is unhelpful and even dangerous. This paper represents an attempt to try and provide workers with an operational framework for conducting risk assessments by exploring the definition, components and parameters of the concept. The author argues that by using the concept of risk in its fullest sense, e.g. assessing for strengths and protective mechanisms as well as weaknesses, then it remains an important and central consideration in our work designed to safeguard the child. Risk assessments should remain the backbone of the child care system as we know it. This paper also seeks to explore the context and preoccupation with risk, before moving on to provide some practical guidelines for conducting risk assessments, generally and then specifically within the court process.     

A Conceptual Framework for Life-Cycle Health Technology Assessment

ObjectivesHealth technology assessment (HTA) uses evidence appraisal and synthesis with economic evaluation to inform adoption decisions. Standard HTA processes sometimes struggle to (1) support decisions that involve significant uncertainty and (2) encourage continued generation of and adaptation to new evidence. We propose the life-cycle (LC)-HTA framework, addressing these challenges by providing additional tools to decision makers and improving outcomes for all stakeholders.MethodsUnder the LC-HTA framework, HTA processes align to LC management. LC-HTA introduces changes in HTA methods to minimize analytic time while optimizing decision certainty. Where decision uncertainty exists, we recommend risk-based pricing and research-oriented managed access (ROMA). Contractual procurement agreements define the terms of reassessment and provide additional decision options to HTA agencies. LC-HTA extends value-of-information methods to inform ROMA agreements, leveraging routine, administrative data, and registries to reduce uncertainty.ResultsLC-HTA enables the adoption of high-value high-risk innovations while improving health system sustainability through risk-sharing and reducing uncertainty. Responsiveness to evolving evidence is improved through contractually embedded decision rules to simplify reassessment. ROMA allows conditional adoption to obtain additional information, with confidence that the net value of that adoption decision is positive.ConclusionsThe LC-HTA framework improves outcomes for patients, sponsors, and payers. Patients benefit through earlier access to new technologies. Payers increase the value of the technologies they invest in and gain mechanisms to review investments. Sponsors benefit through greater certainty in outcomes related to their investment, swifter access to markets, and greater opportunities to demonstrate value.     

Methods for Early Assessment of the Societal Value of Health Technologies: A Scoping Review and Proposal for Classification

ObjectivesEarly assessments of health technologies help to better align and integrate their development and assessment. Such assessments can take many forms and serve different purposes, hampering users in their selection of the most appropriate method for a specific goal. The aim of this scoping review was to structure the large set of methods according to their specific goal.MethodsA scoping review was conducted using PubMed and reference lists of retrieved articles, to identify review studies with a methodological focus. From the included reviews, all individual methods were listed. Based on additional literature and examples, we extracted the specific goal of each method. All goals were clustered to derive a set of subclasses and methods were grouped into these subclasses.ResultsOf the 404 screened, 5 reviews were included, and 1 was added when searching reference lists. The reviews described 56 methods, of which 43 (77%) were included and classified as methods to (1) explore the nature and magnitude of the problem, (2) estimate the nature and magnitude of the expected (societal) value, (3) identify conditions for the potential value to materialize, and (4) help develop and design the type of research that is needed.ConclusionsThe wide range of methods for exploring the societal value of health technologies at an early stage of development can be subdivided into a limited number of classes, distinguishing methods according to their specific objective. This facilitates selection of appropriate methods, depending on the specific needs and aims.     

公共卫生体系绩效评估的概念性框架

文章提出了一个评估公共卫生系统绩效的概念性框架,该框架将整个系统分为全国、地区两个层次,以Donabedian的理论为基础,将系统目标与持续性质量改善模式中的结构、过程、产出和结果及外部环境5个部分联系在一起.该评估框架可以用来监测公共卫生系统、具体的机构及项目的绩效.尽管目前测量公共卫生系统绩效还存在一定的困难,但相信随着评估技术的不断改进,公共卫生系统的绩效将会得以改善,最终改善人群的健康状况.     

不同方法评估人群营养状况作用比较

目的 比较体质指数（BMI），腰围。生物电阻抗法3种方法评估人群营养状况的作用。方法 上海某大学2001年秋季本专科入学新生940人（男生285人，女生655人），测身高，体重，脂肪含量，腰围等，计算三相关性；以BMI为参照指标。计算腰围法与生物电阻抗法的真实性，灵敏性。特异性等。结果 三间均显相关，按性别分组后，仍获相似结果。与BMI法比较，在检测体重过低发生率时，生物电阻抗法与腰围法均有良好的真实性和相当强的特异性，但两种方法的敏感性明显低于BMI法，在评价超重与肥胖状况时，两种方法的特异性与灵敏性比较平衡。因而均有较佳的真实性。结论 BMI作为人群普查的常用指标。是可靠怀适用的。如果可能，同果测量腰围更为理想，生物电阻抗法的高灵敏性使之适用于超重与肥胖人群筛查而非流行病学调查。     

Prospective Benefit-Risk Monitoring of New Drugs for Rapid Assessment of Net Favorability in Electronic Health Care Data

BackgroundBenefit-risk assessment (BRA) methods can combine measures of benefits and risks into a single value.ObjectivesTo examine BRA metrics for prospective monitoring of new drugs in electronic health care data.MethodsUsing two electronic health care databases, we emulated prospective monitoring of three drugs (rofecoxib vs. nonselective nonsteroidal anti-inflammatory drugs, prasugrel vs. clopidogrel, and denosumab vs. bisphosphonates) using a sequential propensity score–matched cohort design. We applied four BRA metrics: number needed to treat and number needed to harm; incremental net benefit (INB) with maximum acceptable risk; INB with relative-value–adjusted life-years; and INB with quality-adjusted life-years (QALYs). We determined whether and when the bootstrapped 99% confidence interval (CI) for each metric excluded zero, indicating net favorability of one drug over the other.ResultsFor rofecoxib, all four metrics yielded a negative value, suggesting net favorability of nonselective nonsteroidal anti-inflammatory drugs over rofecoxib, and the 99% CI for all but the number needed to treat and number needed to harm excluded the null during follow-up. For prasugrel, only the 99% CI for INB-QALY excluded the null, but trends in values over time were similar across the four metrics, suggesting overall net favorability of prasugrel versus clopidogrel. The 99% CI for INB-relative-value–adjusted life-years and INB-QALY excluded the null in the denosumab example, suggesting net favorability of denosumab over bisphosphonates.ConclusionsProspective benefit-risk monitoring can be used to determine net favorability of a new drug in electronic health care data. In three examples, existing BRA metrics produced qualitatively similar results but differed with respect to alert generation.     

垃圾减量分类方法的量化分析

运用线性回归分析方法得出了垃圾组分的相关性,采用5项激励措施进行减量分类研究,运用主成分分析方法进行因子分析,结果表明：物质奖励和垃圾分类收集袋措施对垃圾回收率成正相关;减免处理费和环保积分卡措施对垃圾分类效果正相关;更换先进设施措施可以有效减少垃圾数量.     

Evaluation of Therapeutic Strategies: A New Method for Balancing Risk and Benefit

OBJECTIVE: A patient-specific drug safety-efficacy index was developed that combined objective clinical trial information about dose-related efficacy and toxicity with subjective perspectives on efficacy-toxicity trades. METHODS: Patient preferences were systematically assessed using the probability tradeoff technique (PTT). Toxicity ranges over which a drug's efficacy exceeded the patient's minimally acceptable efficacy represented ranges of "surplus efficacy." These can be related to the dose interval in which a drug delivers this surplus efficacy. Seventy surplus efficacy functions (for 7 hypothetical drugs and 10 hypothetical preference curves) were simulated. RESULTS: The analysis showed that index values change markedly by dose and patient preference, suggesting that different patients will benefit from different drugs depending on the dose prescribed and each patient's subjective assessment of the efficacy/toxicity tradeoff. In most situations, drugs achieve positive surplus efficacy only over limited dose ranges. The model was sensitive to different preference curves and discriminated well among drugs with different efficacy or safety profiles. CONCLUSION: This index provides a new, systematic approach to choosing a specific therapeutic intervention and dosage, when known risks and benefits are reconciled against patient-specific preferences among an array of therapeutic alternatives.     

Assessment Methods for Patient-Reported Outcomes

The methods used to collect information on patient-reported outcomes (PROs) can affect the validity, reliability and sensitivity of the data. This review examines the influence of a variety of methodological issues that occur. In particular, when patients are asked to rely on their memory to aggregate and summarize their experience, a variety of inaccuracies and biases can affect the data. As the extent and magnitude of recall bias have become more well-known, researchers have increasingly turned toward collecting data in the field, closer in time to events and experiences of interest. Diary methods require patients to self-monitor their experiences, disease episodes and healthcare utilization over time. The collection of diary data from patients places a premium on adherence to the data collection protocol. Recent evidence suggests that patients routinely fail to adhere to diary protocols, thus introducing retrospective error and biases into the data. Reasons for patients’ non-adherence to data collection protocols include their desire to appear to be ‘good’ patients which can result in back-filling PRO entries before a site visit, simply forgetting, and data collection methods that fail to help patients be adherent to the data collection protocol. Principles that can be used to enhance patient adherence to PRO data collection methods include good patient training, creating simple and intuitive user interfaces for PRO assessments, and actively driving the protocol by helping patients remember to complete PRO measures as required by the protocol.     

Risk Attitudes and Personality Traits Predict Perceptions of Benefits and Risks for Medicinal Products: A Field Study of European Medical Assessors

Background: Risk attitudes and personality traits are known predictors of decision making among laypersons, but very little is known of their influence among experts participating in organizational decision making. Methods: Seventy-five European medical assessors were assessed in a field study using the Domain Specific Risk Taking scale and the Big Five Inventory scale. Assessors rated the risks and benefits for a mock “clinical dossier” specific to their area of expertise, and ordinal regression models were used to assess the odds of risk attitude or personality traits in predicting either the benefit or the risk ratings. Results: An increase in the “conscientiousness” score predicted an increase in the perception of the drug’s benefit, and male assessors gave higher scores for the drug’s benefit ratings than did female assessors. Extraverted assessors saw fewer risks, and assessors with a perceived neutral-averse or averse risk profile saw greater risks. Conclusions: Medical assessors perceive the benefits and risks of medicines via a complex interplay of the medical situation, their personality traits and even their gender. Further research in this area is needed to determine how these potential biases are managed within the regulatory setting.     

A Framework for Cumulative Risk Assessment: Exploring the Carcinogenic Effects of Chemical Mixtures

Open in a separate window

Humans are commonly exposed to mixtures of chemicals in their environments, rather than to one compound at a time. Accordingly, researchers have long been interested in the complex task of studying how mixtures affect disease end points.1 Cynthia Rider and colleagues recently proposed in Environmental Health Perspectives2 a research program on mixtures and cancer based on the key characteristics of carcinogenic chemicals.3“We wanted to apply what we have learned from studying outcomes with shorter exposure periods to a more complex scenario,” says Rider, a toxicologist at the National Institute of Environmental Health Sciences and first author of the new paper. “Because cancer results from the cumulative impact of multiple low-dose hits in several pathways during a long period of latency, the analysis of mixture effects is especially challenging.”Open in a separate windowAlthough people are exposed to many chemicals over their lifetimes—in food, water, personal care products, air, and other media—toxicology has historically studied agents individually. The authors of a new commentary recommend strategies for studying exposures to chemical mixtures. Image: © iStock/NoSystem Images.Based on literature reviews and expert workshops, the team, organized by Martyn Smith at the University of California, Berkeley, proposed to anchor mixture analyses to a single carcinogen with a known effect. This approach allows researchers to use in vitro or in vivo assays to test specific hypotheses about changes in the dose–response curve when an organism is exposed to other chemicals in combination with the known carcinogen.The proposal’s central tenet is that cancer results from a sequence of exposures to several carcinogens that affect the initiation, promotion, or progression of a slowly unfolding disease process.4^,5 Thus, an appropriate research plan should build on both the key characteristics of carcinogenic chemicals and the hallmarks of cancer. “Key characteristics” refers to qualities or capabilities of a carcinogen that can lead to cancer, such as the ability to induce DNA damage or chronic inflammation, cause genomic instability, or suppress the immune system.3 “Hallmarks” refers to properties of cancer cells that are acquired during tumor formation, such as the ability to proliferate limitlessly and control formation of new blood vessels.6The authors of the new paper proposed three strategies for studying how mixtures affect cancer outcomes. The first, a chemical screening, complements the other two approaches but may also stand alone. This approach involves mining existing data sets to select chemicals with some known link to cancer.The second strategy is a transgenic model-based approach, for which the authors propose the rasH2 mouse 26-week bioassay. Animal models allow researchers to include nonchemical stressors, such as a high-fat diet, and better mimic the complexity of human cancers. “The critical piece for mixture studies is knowing what to expect,” says Rider. “The rasH2 mouse is [a Food and Drug Administration]-approved model with rich historical data for multiple cancers that provides expected joint effects as a benchmark for observed data.”The third strategy, a disease-centered approach, may employ either an animal or in vitro organoid model. Organoid models, which may also be used in the screening approach, are well suited for testing how chemical mixtures affect cancer-relevant pathways or biomarkers.7^,8The authors also recommend experimental designs. For studying poorly understood cancers, they suggest starting with single-compound testing in an in vivo or in vitro screen, followed by high-throughput testing of multiple ratios of two chemicals along a single predicted response plane.9 For mixtures of more than two substances, the authors recommend the fixed ratio ray design,10 which focuses on a single fixed ratio of chemicals.Alan Boobis, an emeritus professor of toxicology at Imperial College London, applauds the authors for addressing a topic whose importance has long been recognized by researchers and funding agencies. He agrees with anchoring mixture tests to a known carcinogen but would not limit animal models to the rasH2 just because it has extensive historical data.“We have seen fantastic advances in systems biology and high-throughput screening technology,” says Boobis, who was not involved in the project. “I consider the rasH2 model a rather blunt instrument that may not provide sufficient power to detect low-dose effects and would prefer newer models that are better suited for proof-of-principle studies.”For Susan Tilton, an associate professor of environmental and molecular toxicology at Oregon State University, the rasH2 model is a reasonable starting point that can be followed up with other models, perhaps designed for nonchemical stressors. “Evaluating mixtures is a challenging task,” she says. “To tackle this complex problem, leveraging a data-rich environment of existing models and toxicity information for individual chemicals is important.” Tilton also was not involved in the project.Andreas Kortenkamp, a professor of human molecular toxicology at Brunel University London, hopes that carcinogenic effects of mixtures can eventually be predicted by modeling approaches alone, without further experiments. Kortenkamp, who also was not involved in the project, says, “This would be particularly attractive for risk assessment by regulatory agencies.”