Can we individualize the 'number needed to treat'? An empirical study of summary effect measures in meta-analyses

BACKGROUND: Meta-analyses summarize the magnitude of treatment effect using a number of measures of association, including the odds ratio (OR), risk ratio (RR), risk difference (RD) and/or number needed to treat (NNT). In applying the results of a meta-analysis to individual patients, some textbooks of evidence-based medicine advocate individualizing NNT, based on the RR and the patient's expected event rate (PEER). This approach assumes constant RR but no empirical study to date has examined the validity of this assumption. METHODS: We randomly selected a subset of meta-analyses from a recent issue of the Cochrane Library (1998, Issue 3). When a meta-analysis pooled more than three randomized controlled trials (RCT) to produce a summary measure for an outcome, we compared the OR, RR and RD of each RCT with the corresponding pooled OR, RR and RD from the meta-analysis of all the other RCT. Using the conventional P-value of 0.05, we calculated the percentage of comparisons in which there were no statistically significant differences in the estimates of OR, RR or RD, and refer to this percentage as the 'concordance rate'. RESULTS: For each effect measure, we made 1843 comparisons, extracted from 55 meta-analyses. The random effects model OR had the highest concordance rate, closely followed by the fixed effects model OR and random effects model RR. The minimum concordance rate for these indices was 82%, even when the baseline risk differed substantially. The concordance rates for RD, either fixed effects or random effects model, were substantially lower (54-65%). CONCLUSIONS: The fixed effects OR, random effects OR and random effects RR appear to be reasonably constant across different baseline risks. Given the interpretational and arithmetic ease of RR, clinicians may wish to rely on the random effects model RR and use the PEER to individualize NNT when they apply the results of a meta-analysis in their practice.     

Are active and passive smoking harmful? Determining causation

Assessing the evidence regarding any causal question involves examining the strength of the studies conducted and applying a series of "diagnostic tests" for causation. We have reviewed the strength of the evidence incriminating smoking as a cause of lung cancer, and passive smoking as a cause of respiratory illness and decreased pulmonary function in children. There are eight prospective studies of smoking and lung cancer which have consistently shown a strong relationship. These studies have confirmed the temporality of the association and demonstrated a dose-response gradient. The studies addressing the effects of passive smoking in children are considerably weaker. Although they are consistent in suggesting increased infections for children less than one year of age, neither increased risk nor a dose-response gradient is consistently found in older children and the effect size, when present, is small. The rules for assessing causation applied here can be used to integrate new information concerning the health hazards of smoking.     

Endoscopic therapy for acute nonvariceal upper gastrointestinal hemorrhage: a meta-analysis.

Endoscopic hemostatic therapy for upper gastrointestinal bleeding is gaining widespread acceptance despite often conflicting results of randomized controlled trials. To examine the effect of endoscopic therapy in acute nonvariceal upper gastrointestinal hemorrhage, a meta-analysis was performed using a computerized search of the English-language literature and a bibliographic review. The methodology, population, intervention, and outcomes of each relevant trial were evaluated by duplicate independent review. Thirty randomized controlled trials evaluating hemostatic endoscopic treatment were identified. Endoscopic therapy significantly reduced rates of further bleeding (odds ratio, 0.38; 95% confidence interval, 0.32-0.45), surgery (odds ratio, 0.36; 95% confidence interval, 0.28-0.45), and mortality (odds ratio, 0.55; 95% confidence interval, 0.40-0.76). When analyzed separately, thermal-contact devices (monopolar and bipolar electrocoagulation and heater probe), laser treatment, and injection therapy all significantly decreased further bleeding and surgery rates. The reductions in mortality were comparable for all three forms of therapy, but the decrease reached statistical significance only for laser therapy. Further examination of subgroups indicated that endoscopic treatment decreased rates of further bleeding, surgery, and mortality in patients with high-risk endoscopic features of active bleeding or nonbleeding visible vessels. Rebleeding was not reduced by endoscopic therapy in patients with ulcers containing flat pigmented spots or adherent clots. Endoscopic hemostatic therapy provides a clinically important reduction in morbidity and mortality in patients with acute nonvariceal upper gastrointestinal hemorrhage.     

Factors affecting physicians' decisions on caring for an incompetent elderly patient: an international study.

OBJECTIVES: To determine what treatment decisions physicians will make when faced with a hypothetical incompetent elderly patient with life-threatening gastrointestinal bleeding and to examine the relative importance of physician characteristics and factors (legal and ethical concerns, hospital costs, level of dementia, patient's age, physician's religion, patient's wishes and family's wishes) in making those decisions. DESIGN: Survey. SETTING: Family practice, medical and geriatrics rounds in academic medical centres and community hospitals in seven countries. PARTICIPANTS: Physicians who regularly cared for incompetent elderly patients. MAIN OUTCOME MEASURES: A self-administered questionnaire describing the elderly patient. Respondents were asked to choose one of four levels of care and to identify the level of importance factors had in making that decision. Older physicians, those less concerned about litigation, those for whom the level of dementia was important and those for whom the patient's age was important were expected to give less aggressive care than the other physicians. MAIN RESULTS: Supportive care was chosen by 8.1% of the respondents, limited therapeutic care by 41.5%, maximum therapeutic care without admission to the intensive care unit (ICU) by 32.2% and maximum care with admission to the ICU by 18.2%. The patient's wishes were reported by 91.0% as being extremely or very important in choosing the treatment. Stepwise logistic regression analysis revealed that the following variables independently predicted the level of treatment: level of dementia, country of residence, duration of practice, legal concerns, patient's age and ethical concerns. These factors were significantly correlated with the physicians' treatment choices (p less than 0.05). CONCLUSIONS: The importance that the physicians placed on the level of dementia was the strongest predictor of the level of care that would be provided. A societal consensus on the influence of cognitive function on the appropriate level of care as well as training of physicians in ethical issues are required.     

Mechanism of bronchodilator effect in chronic airflow limitation.

OBJECTIVE: To examine the mechanisms through which two bronchodilators (theophylline and salbutamol) influence dyspnea during daily activities. METHODS: Twenty-four patients with chronic airflow limitation participated in a multiple crossover, randomized, placebo-controlled trial. The effect of theophylline and salbutamol, alone or combined, on pulmonary function and dyspnea during daily activities was examined. Correlations of changes in forced expiratory volume in 1 second (FEV1) and maximum expiratory pressures (MIPs) (independent variables) and changes in dyspnea score during daily activities (dependent variable) were also examined. RESULTS: The two drugs proved to be beneficial the effects in general were additive rather than synergistic. The drugs improved the FEV1; theophylline significantly improved the MIPs. The correlation between the changes in FEV1 and those in dyspnea score, after adjustment for the changes in MIPs, was 0.55 (p less than 0.001). The correlation between the changes in MIPs and those in dyspnea score, after adjustment for the changes in FEV1, was 0.39 (p less than 0.001). CONCLUSIONS: Changes in airway calibre and in respiratory muscle strength play an independent and important role in dyspnea during daily activities in patients with chronic airflow limitation. Changes in airway calibre may be of greater importance.     

The standard gamble showed better construct validity than the time trade-off

BACKGROUND AND OBJECTIVE: There is little evidence for the relative cross-sectional validity of the standard gamble (SG) and time trade-off (TTO). We compared these preference-based instruments in patients with Irritable Bowel Syndrome (IBS). METHODS: Patients rated their own health on the SG and TTO and completed the disease-specific IBS questionnaire, the Brief Pain Inventory, the SF-36, the Sickness Impact Profile, and a global rating of disease severity. RESULTS: Mean scores of the 96 enrolled patients (mean age 39.5 years, 84.4% women) were 0.84 (standard deviation 0.16) for the SG and 0.76 (0.22) for the TTO. The correlation of the SG with the TTO was 0.36. For the SG, correlation coefficients with the IBS questionnaire domain scores ranged from 0.36 to 0.47, whereas those of the TTO were substantially lower (0.15-0.42). The SG also had higher correlations than the TTO with generic questionnaires (0.18-0.34 versus 0.13-0.26), Brief Pain Inventory (0.27 versus 0.11), global rating of disease severity (0.22 versus 0.10) as well as with SF-36-derived patient preferences (0.31-0.43 versus 0.27-0.31). CONCLUSIONS: The higher correlations of the SG with validation measures indicate that the SG better reflects health-related quality of life and patient preferences compared to the TTO.     

Prospectively screening for eligible patients was inaccurate in predicting patient recruitment of orthopedic randomized trials

Objective

To compare the accuracy of estimates of potential recruitment from a prospective 8-week screening study compared with a retrospective chart review across sites participating in two fracture management trials.

Study Design and Setting

During the planning phase of two large, multicenter, randomized controlled fracture management trials, 74 clinical sites provided estimates of the annual recruitment rate both retrospectively (based on chart reviews) and prospectively. The prospective estimate was generated by screening, for 8 weeks, all incoming patients for eligibility in the concerning trial, without actually enrolling any patient. We compared these prospective and retrospective estimates with one another (for 74 sites in the two trials) and with actual 1-year recruitment rates in the definitive trial (for nine sites in one trial).

Results

There was a median difference of four patients (interquartile range: −14 to 18 patients; P = 0.89) between a center's prospective estimate and its retrospective estimate. Both predictions were overestimations of recruitment in the definitive trial; only 31% (95% confidence interval [CI]: 28, 35) of retrospectively estimated patients, and 31% (95% CI: 27, 35) of prospectively estimated patients were recruited in the definitive trials.

Conclusion

Compared with relatively simple chart reviews, prospectively screening for eligible patients at clinical sites, which is associated with substantial costs, did not result in more accurate predictions of accrual in large, multicenter, randomized controlled trials.     

The impact of the VF-14 index,a perceived visual function measure,in the routine management of cataract patients

Background: Evidence about the impact of routine feedback of patient-reported outcomes is contradictory, and there is limited information regarding its use in the routine management of cataract patients. Methods: The VF-14 Index was used to assess the visual function of 833 consecutive cataract patients, attending 19 ophthalmologists from public and private hospitals and primary care practices in Spain, in 1999–2000. In this before/after trial, the intervention included (1) an educational session, and (2) the provision of the VF-14 scores of all subsequent patients to the ophthalmologist. Mixed effects linear and logistic models were constructed to assess the effect on the process (correlation between patients’ and physicians’ assessments of visual function, appropriateness of surgery recommendation) and the outcome of care (satisfaction). Results: The adjusted regression coefficient for the VF-14 score significantly increased after the intervention as a predictor of the ophthalmologist’s assessment of visual function (β coefficient: control 0.10 vs. intervention 0.35, p < 0.05). The intervention did not increase the probability of an appropriate medical decision (OR=0.90; 95% CI: 0.42; 2,69) and it did not change patient satisfaction with care. Conclusions: Routine provision of education and feedback on the patient’s VF-14 Index score significantly increases agreement between patients’ and physicians’ assessments of functional capacity. The lack of a beneficial effect on management or outcome suggests the need for a more intense intervention to change medical practice.The authors were done this work for the Systematic Use of Quality of Life Measures in the Clinical Practice Working GroupThe Systematic Use of Quality of Life Measures in the Clinical Practice Working Group are: Alonso J, Castells X, Espallargues M, Guyatt G, Prieto L, Valderas JM.