World health system performance revisited: the impact of varying the relative importance of health system goals

Background

In 2002, the World Health Organization published a health system performance ranking for 191 member countries. The ranking was based on five indicators, with fixed weights common to all countries.

Methods

We investigate the feasibility and desirability of using mathematical programming techniques that allow weights to vary across countries to reflect their varying circumstances and objectives.

Results

By global distributional measures, scores and ranks are found to be not very sensitive to changes in weights, although differences can be large for individual countries.

Conclusions

Building the flexibility of variable weights into calculation of the performance index is a useful way to respond to the debates and criticisms appearing since publication of the ranking.     

A generic model for the assessment of disease epidemiology: the computational basis of DisMod II

Reliable and comparable analysis of risks to health is key for preventing disease and injury. Causal attribution of morbidity and mortality to risk factors has traditionally been conducted in the context of methodological traditions of individual risk factors, often in a limited number of settings, restricting comparability.     

Population health metrics: crucial inputs to the development of evidence for health policy

Valid, reliable and comparable measures of the health states of individuals and of the health status of populations are critical components of the evidence base for health policy. We need to develop population health measurement strategies that coherently address the relationships between epidemiological measures (such as risk exposures, incidence, and mortality rates) and multi-domain measures of population health status, while ensuring validity and cross-population comparability.     

Breast and cervical cancer survival: making sense of ‘league tables’

During 1998, the Department of Health proposed to use survival rates of cervical and breast cancer in the 1989/90 incidence cohort as indicators of care. Valid interpretation was of concern within Trent and the Trent Cancer Registry responded by performing additional analyses. Trent Cancer Registry registrations for 1989/90 were re-analysed and the stability of districts' ranks for that cohort was investigated using random simulation techniques. Stability of ranks across more recent cohorts was investigated and attempts made to use all available information.The Department of Health's analyses were confirmed by our re-analysis of the 1989/90 cohort: Rotherham residents appeared to have the "worst" survival for cervical cancer, and Sheffield residents for breast cancer, although not statistically significantly so. Random simulations indicated that ranks based on a single cohort are not stable: for example Sheffield (ranked tenth for 1-y breast cancer survival) was ranked third or better in 6% of randomisations. Ranks were also unstable across cohorts: for example Rotherham 1-y cervical cancer survival was ranked tenth for 1989/90, fifth for 1991/92 and tenth for 1993/94. Analysis of 3-y running averages provided better information than the league table approach. Most districts improved over time, to different degrees, and similar sized gaps remained between the "best" and the "worst" districts. This analysis illustrates the need to be circumspect when interpreting "league tables" based on a single year or cohort analysis. League tables are based on ranks: clearly a large difference in rank may reflect only trivial (ie medically unimportant) differences in actual outcome. Lack of a statistically significant difference in survival between two districts does not mean their survival is equivalent. Even for a common cancer, like breast cancer, rankings were unstable from cohort to cohort. At the Registry we propose to perform these trend analyses routinely in future, adjusting, when possible, for the effects of deprivation and stage at diagnosis.