Parametric multistate survival models: Flexible modelling allowing transition‐specific distributions with application to estimating clinically useful measures of effect differences

Multistate models are increasingly being used to model complex disease profiles. By modelling transitions between disease states, accounting for competing events at each transition, we can gain a much richer understanding of patient trajectories and how risk factors impact over the entire disease pathway. In this article, we concentrate on parametric multistate models, both Markov and semi‐Markov, and develop a flexible framework where each transition can be specified by a variety of parametric models including exponential, Weibull, Gompertz, Royston‐Parmar proportional hazards models or log‐logistic, log‐normal, generalised gamma accelerated failure time models, possibly sharing parameters across transitions. We also extend the framework to allow time‐dependent effects. We then use an efficient and generalisable simulation method to calculate transition probabilities from any fitted multistate model, and show how it facilitates the simple calculation of clinically useful measures, such as expected length of stay in each state, and differences and ratios of proportion within each state as a function of time, for specific covariate patterns. We illustrate our methods using a dataset of patients with primary breast cancer. User‐friendly Stata software is provided.     

Multiple Sclerosis Progressive Courses: A Clinical Cohort Long-Term Disability Progression Study

ObjectivesImproving the understanding of multiple sclerosis (MS) mechanism and disability progression over time is essential to assess the value of healthcare interventions. Poor or no data on disability progression are available for progressive courses. This study aims to fill this gap.MethodsAn observational cohort study of patients with primary MS (PPMS) and secondary progressive MS (SPMS) was conducted on 2 Italian MS centers disease registries over an observational time of 34 years. Annual transition probabilities among Expanded Disability Status Scale (EDSS) states were estimated using continuous Markov models. A sensitivity analysis was performed in relation to clinical characteristic associated to disability progression.ResultsThe study cohort included 758 patients (274 PPMS and 434 SPMS) with a median follow-up of 8.2 years. Annual transition probability matrices of SPMS and PPMS reported different annual probabilities to move within EDSS levels. Excluding EDSS associated to relapse events or patient with relapses, the annual probability of staying stable in an EDSS level increased in both disease courses even not significantly.ConclusionsThis study provides estimates of annual disability progression as EDSS changes for PPMS and SPMS. These estimates could be a useful tool for healthcare decision makers and clinicians to properly assess impact of clinical interventions.     

Structured fusion lasso penalized multi‐state models

Multi‐state models generalize survival or duration time analysis to the estimation of transition‐specific hazard rate functions for multiple transitions. When each of the transition‐specific risk functions is parametrized with several distinct covariate effect coefficients, this leads to a model of potentially high dimension. To decrease the parameter space dimensionality and to work out a clear image of the underlying multi‐state model structure, one can either aim at setting some coefficients to zero or to make coefficients for the same covariate but two different transitions equal. The first issue can be approached by penalizing the absolute values of the covariate coefficients as in lasso regularization. If, instead, absolute differences between coefficients of the same covariate on different transitions are penalized, this leads to sparse competing risk relations within a multi‐state model, that is, equality of covariate effect coefficients. In this paper, a new estimation approach providing sparse multi‐state modelling by the aforementioned principles is established, based on the estimation of multi‐state models and a simultaneous penalization of the L₁‐norm of covariate coefficients and their differences in a structured way. The new multi‐state modelling approach is illustrated on peritoneal dialysis study data and implemented in the R package penMSM . Copyright © 2016 John Wiley & Sons, Ltd.     

State‐space size considerations for disease‐progression models

Markov models of disease progression are widely used to model transitions in patients' health state over time. Usually, patients' health status may be classified according to a set of ordered health states. Modelers lump together similar health states into a finite and usually small, number of health states that form the basis of a Markov chain disease‐progression model. This increases the number of observations used to estimate each parameter in the transition probability matrix. However, lumping together observably distinct health states also obscures distinctions among them and may reduce the predictive power of the model. Moreover, as we demonstrate, precision in estimating the model parameters does not necessarily improve as the number of states in the model declines. This paper explores the tradeoff between lumping error introduced by grouping distinct health states and sampling error that arises when there are insufficient patient data to precisely estimate the transition probability matrix. Copyright © 2013 John Wiley & Sons, Ltd.     

Estimation of markov chain transition probabilities and rates from fully and partially observed data: uncertainty propagation, evidence synthesis, and model calibration.

Markov transition models are frequently used to model disease progression. The authors show how the solution to Kolmogorov's forward equations can be exploited to map between transition rates and probabilities from probability data in multistate models. They provide a uniform, Bayesian treatment of estimation and propagation of uncertainty of transition rates and probabilities when 1) observations are available on all transitions and exact time at risk in each state (fully observed data) and 2) observations are on initial state and final state after a fixed interval of time but not on the sequence of transitions (partially observed data). The authors show how underlying transition rates can be recovered from partially observed data using Markov chain Monte Carlo methods in WinBUGS, and they suggest diagnostics to investigate inconsistencies between evidence from different starting states. An illustrative example for a 3-state model is given, which shows how the methods extend to more complex Markov models using the software WBDiff to compute solutions. Finally, the authors illustrate how to statistically combine data from multiple sources, including partially observed data at several follow-up times and also how to calibrate a Markov model to be consistent with data from one specific study.     

Multi‐state Markov models for disease progression in the presence of informative examination times: An application to hepatitis C

In many chronic diseases it is important to understand the rate at which patients progress from infection through a series of defined disease states to a clinical outcome, e.g. cirrhosis in hepatitis C virus (HCV)‐infected individuals or AIDS in HIV‐infected individuals. Typically data are obtained from longitudinal studies, which often are observational in nature, and where disease state is observed only at selected examinations throughout follow‐up. Transition times between disease states are therefore interval censored. Multi‐state Markov models are commonly used to analyze such data, but rely on the assumption that the examination times are non‐informative, and hence the examination process is ignorable in a likelihood‐based analysis. In this paper we develop a Markov model that relaxes this assumption through the premise that the examination process is ignorable only after conditioning on a more regularly observed auxiliary variable. This situation arises in a study of HCV disease progression, where liver biopsies (the examinations) are sparse, irregular, and potentially informative with respect to the transition times. We use additional information on liver function tests (LFTs), commonly collected throughout follow‐up, to inform current disease state and to assume an ignorable examination process. The model developed has a similar structure to a hidden Markov model and accommodates both the series of LFT measurements and the partially latent series of disease states. We show through simulation how this model compares with the commonly used ignorable Markov model, and a Markov model that assumes the examination process is non‐ignorable. Copyright © 2010 John Wiley & Sons, Ltd.     

A mixture of transition models for heterogeneous longitudinal ordinal data: with applications to longitudinal bacterial vaginosis data

Markov models used to analyze transition patterns in discrete longitudinal data are based on the limiting assumption that individuals follow the common underlying transition process. However, when one is interested in diseases with different disease or severity subtypes, explicitly modeling subpopulation‐specific transition patterns may be appropriate. We propose a model which captures heterogeneity in the transition process through a finite mixture model formulation and provides a framework for identifying subpopulations at different risks. We apply the procedure to longitudinal bacterial vaginosis study data and demonstrate that the model fits the data well. Further, we show that under the mixture model formulation, we can make the important distinction between how covariates affect transition patterns unique to each of the subpopulations and how they affect which subgroup a participant will belong to. Practically, covariate effects on subpopulation‐specific transition behavior and those on subpopulation membership can be interpreted as effects on short‐term and long‐term transition behavior. We further investigate models with higher‐order subpopulation‐specific transition dependence. Copyright © 2014 John Wiley & Sons, Ltd.     

A semi‐Markov model for stroke with piecewise‐constant hazards in the presence of left,right and interval censoring

This paper presents a parametric method of fitting semi‐Markov models with piecewise‐constant hazards in the presence of left, right and interval censoring. We investigate transition intensities in a three‐state illness–death model with no recovery. We relax the Markov assumption by adjusting the intensity for the transition from state 2 (illness) to state 3 (death) for the time spent in state 2 through a time‐varying covariate. This involves the exact time of the transition from state 1 (healthy) to state 2. When the data are subject to left or interval censoring, this time is unknown. In the estimation of the likelihood, we take into account interval censoring by integrating out all possible times for the transition from state 1 to state 2. For left censoring, we use an Expectation–Maximisation inspired algorithm. A simulation study reflects the performance of the method. The proposed combination of statistical procedures provides great flexibility. We illustrate the method in an application by using data on stroke onset for the older population from the UK Medical Research Council Cognitive Function and Ageing Study. Copyright © 2012 John Wiley & Sons, Ltd.     

A Bayesian semiparametric Markov regression model for juvenile dermatomyositis

Juvenile dermatomyositis (JDM) is a rare autoimmune disease that may lead to serious complications, even to death. We develop a 2‐state Markov regression model in a Bayesian framework to characterise disease progression in JDM over time and gain a better understanding of the factors influencing disease risk. The transition probabilities between disease and remission state (and vice versa) are a function of time‐homogeneous and time‐varying covariates. These latter types of covariates are introduced in the model through a latent health state function, which describes patient‐specific health over time and accounts for variability among patients. We assume a nonparametric prior based on the Dirichlet process to model the health state function and the baseline transition intensities between disease and remission state and vice versa. The Dirichlet process induces a clustering of the patients in homogeneous risk groups. To highlight clinical variables that most affect the transition probabilities, we perform variable selection using spike and slab prior distributions. Posterior inference is performed through Markov chain Monte Carlo methods. Data were made available from the UK JDM Cohort and Biomarker Study and Repository, hosted at the UCL Institute of Child Health.     

Analysis of transtheoretical model of health behavioral changes in a nutrition intervention study—a continuous time Markov chain model with Bayesian approach

Continuous time Markov chain (CTMC) models are often used to study the progression of chronic diseases in medical research but rarely applied to studies of the process of behavioral change. In studies of interventions to modify behaviors, a widely used psychosocial model is based on the transtheoretical model that often has more than three states (representing stages of change) and conceptually permits all possible instantaneous transitions. Very little attention is given to the study of the relationships between a CTMC model and associated covariates under the framework of transtheoretical model. We developed a Bayesian approach to evaluate the covariate effects on a CTMC model through a log‐linear regression link. A simulation study of this approach showed that model parameters were accurately and precisely estimated. We analyzed an existing data set on stages of change in dietary intake from the Next Step Trial using the proposed method and the generalized multinomial logit model. We found that the generalized multinomial logit model was not suitable for these data because it ignores the unbalanced data structure and temporal correlation between successive measurements. Our analysis not only confirms that the nutrition intervention was effective but also provides information on how the intervention affected the transitions among the stages of change. We found that, compared with the control group, subjects in the intervention group, on average, spent substantively less time in the precontemplation stage and were more/less likely to move from an unhealthy/healthy state to a healthy/unhealthy state. Copyright © 2015 John Wiley & Sons, Ltd.     

Applications of continuous time hidden Markov models to the study of misclassified disease outcomes

Disease progression in prospective clinical and epidemiological studies is often conceptualized in terms of transitions between disease states. Analysis of data from such studies can be complicated by a number of factors, including the presence of individuals in various prevalent disease states and with unknown prior disease history, interval censored observations of state transitions and misclassified measurements of disease states. We present an approach where the disease states are modelled as the hidden states of a continuous time hidden Markov model using the imperfect measurements of the disease state as observations. Covariate effects on transitions between disease states are incorporated using a generalized regression framework. Parameter estimation and inference are based on maximum likelihood methods and rely on an EM algorithm. In addition, techniques for model assessment are proposed. Applications to two binary disease outcomes are presented: the oral lesion hairy leukoplakia in a cohort of HIV infected men and cervical human papillomavirus (HPV) infection in a cohort of young women. Estimated transition rates and misclassification probabilities for the hairy leukoplakia data agree well with clinical observations on the persistence and diagnosis of this lesion, lending credibility to the interpretation of hidden states as representing the actual disease states. By contrast, interpretation of the results for the HPV data are more problematic, illustrating that successful application of the hidden Markov model may be highly dependent on the degree to which the assumptions of the model are satisfied.     

A model‐informed rank test for right‐censored data with intermediate states

The generalized Wilcoxon and log‐rank tests are commonly used for testing differences between two survival distributions. We modify the Wilcoxon test to account for auxiliary information on intermediate disease states that subjects may pass through before failure. For a disease with multiple states where patients are monitored periodically but exact transition times are unknown (e.g. staging in cancer), we first fit a multi‐state Markov model to the full data set; when censoring precludes the comparison of survival times between two subjects, we use the model to estimate the probability that one subject will have survived longer than the other given their censoring times and last observed status, and use these probabilities to compute an expected rank for each subject. These expected ranks form the basis of our test statistic. Simulations demonstrate that the proposed test can improve power over the log‐rank and generalized Wilcoxon tests in some settings while maintaining the nominal type 1 error rate. The method is illustrated on an amyotrophic lateral sclerosis data set. Copyright © 2015 John Wiley & Sons, Ltd.     

Competing‐risks duration models with correlated random effects: an application to dementia patients’ transition histories

Multi‐state transition models are widely applied tools to analyze individual event histories in the medical or social sciences. In this paper, we propose the use of (discrete‐time) competing‐risks duration models to analyze multi‐transition data. Unlike conventional Markov transition models, these models allow the estimated transition probabilities to depend on the time spent in the current state. Moreover, the models can be readily extended to allow for correlated transition probabilities. A further virtue of these models is that they can be estimated using conventional regression tools for discrete‐response data, such as the multinomial logit model. The latter is implemented in many statistical software packages and can be readily applied by empirical researchers. Moreover, model estimation is feasible, even when dealing with very large data sets, and simultaneously allowing for a flexible form of duration dependence and correlation between transition probabilities. We derive the likelihood function for a model with three competing target states and discuss a feasible and readily applicable estimation method. We also present the results from a simulation study, which indicate adequate performance of the proposed approach. In an empirical application, we analyze dementia patients’ transition probabilities from the domestic setting, taking into account several, partly duration‐dependent covariates. Copyright © 2014 John Wiley & Sons, Ltd.     

Multi-state models and outcome prediction in bone marrow transplantation.

Multi-state models have proved versatile and useful in the statistical analysis of the complicated course of events after bone marrow transplantation. Working from data from the International Bone Marrow Transplant Registry, we show that summary probability calculations may be useful to explore hypothetical scenarios where some transition intensities are set by the researcher. A multi-state Markov process model is specified with six states: the initial state 0; acute; chronic and both acute and chronic graft-versus-host disease A, C and AC; relapse R and death in remission D. Transition rates between the states are estimated using Nelson-Aalen estimators and Cox regression models and combined to transition probability estimators using Aalen-Johansen product integration. Besides the estimated transition probabilities to D and R we explore hypothetical probabilities obtained by artificially changing certain transition intensities, with the general purposes of getting summary views of the development for actual patients 'in this world' and of exploring the intrinsic information from real patients about consequences of various changed conditions.     

Analysis of interval‐censored disease progression data via multi‐state models under a nonignorable inspection process

Irreversible multi‐state models provide a convenient framework for characterizing disease processes that arise when the states represent the degree of organ or tissue damage incurred by a progressive disease. In many settings, however, individuals are only observed at periodic clinic visits and so the precise times of the transitions are not observed. If the life history and observation processes are not independent, the observation process contains information about the life history process, and more importantly, likelihoods based on the disease process alone are invalid. With interval‐censored failure time data, joint models are nonidentifiable and data analysts must rely on sensitivity analyses to assess the effect of the dependent observation times. This paper is concerned, however, with the analysis of data from progressive multi‐state disease processes in which individuals are scheduled to be seen at periodic pre‐scheduled assessment times. We cast the problem in the framework used for incomplete longitudinal data problems. Maximum likelihood estimation via an EM algorithm is advocated for parameter estimation. Simulation studies demonstrate that the proposed method works well under a variety of situations. Data from a cohort of patients with psoriatic arthritis are analyzed for illustration. Copyright © 2010 John Wiley & Sons, Ltd.     

Methods for testing the Markov condition in the illness–death model: a comparative study

Markov three‐state progressive and illness–death models are often used in biomedicine for describing survival data when an intermediate event of interest may be observed during the follow‐up. However, the usual estimators for Markov models (e.g., Aalen–Johansen transition probabilities) may be systematically biased in non‐Markovian situations. On the other hand, despite non‐Markovian estimators for transition probabilities and related curves are available, including the Markov information in the construction of the estimators allows for variance reduction. Therefore, testing for the Markov condition is a relevant issue in practice. In this paper, we discuss several characterizations of the Markov condition, with special focus on its equivalence with the quasi‐independence between left truncation and survival times in standard survival analysis. New methods for testing the Markovianity of an illness–death model are proposed and compared with existing ones by means of an intensive simulation study. We illustrate our findings through the analysis of a data set from stem cell transplant in leukemia. Copyright © 2016 John Wiley & Sons, Ltd.     

Multi-state Markov models for analysing incomplete disease history data with illustrations for hiv disease

Multi-state Markov models can be useful in analysing disease history data. We apply the general estimation methods of Kalbfleisch and Lawless to panel data in which individuals are viewed over only a portion of their life history and complete information about transition times between states is unavailable. Methods to assess goodness-of-fit are proposed. To illustrate the methods, we consider models of HIV disease relating important immunological marker measurements to the onset of AIDS.     

Fitting and interpreting continuous‐time latent Markov models for panel data

Multistate models characterize disease processes within an individual. Clinical studies often observe the disease status of individuals at discrete time points, making exact times of transitions between disease states unknown. Such panel data pose considerable modeling challenges. Assuming the disease process progresses accordingly, a standard continuous‐time Markov chain (CTMC) yields tractable likelihoods, but the assumption of exponential sojourn time distributions is typically unrealistic. More flexible semi‐Markov models permit generic sojourn distributions yet yield intractable likelihoods for panel data in the presence of reversible transitions. One attractive alternative is to assume that the disease process is characterized by an underlying latent CTMC, with multiple latent states mapping to each disease state. These models retain analytic tractability due to the CTMC framework but allow for flexible, duration‐dependent disease state sojourn distributions. We have developed a robust and efficient expectation–maximization algorithm in this context. Our complete data state space consists of the observed data and the underlying latent trajectory, yielding computationally efficient expectation and maximization steps. Our algorithm outperforms alternative methods measured in terms of time to convergence and robustness. We also examine the frequentist performance of latent CTMC point and interval estimates of disease process functionals based on simulated data. The performance of estimates depends on time, functional, and data‐generating scenario. Finally, we illustrate the interpretive power of latent CTMC models for describing disease processes on a dataset of lung transplant patients. We hope our work will encourage wider use of these models in the biomedical setting. Copyright © 2013 John Wiley & Sons, Ltd.     

乙型肝炎防治经济学评价——马尔科夫模型的构建

目的 构建乙型肝炎（乙肝）防治经济学评价马尔科夫模型。方法 依据马尔科夫链理论和方法,参考国内外相关文献,采用TreeAge Pro 2015软件构建不同特征及不同乙肝防治人群的马尔科夫模型（包括各种状态及其链接的设置、状态概率的确定）,并对该模型进行验证。结果 建立了新生儿乙肝免疫预防马尔科夫模型、围产期HBV感染后转归马尔科夫模型、成年人乙肝免疫预防马尔科夫模型、慢性乙肝抗病毒治疗和一般人群马尔科夫5类模型。新生儿模型是基础,包含10个Markov状态,分别为乙肝易感、免疫耐受、免疫清除、低复制、再活动、HBsAg清除、代偿和失代偿性肝硬化、肝细胞癌和死亡。围产期模型不包含乙肝易感,成年人模型忽略免疫耐受,均为9个Markov状态,一般人群马尔科夫模型只有健康和死亡2个Markov状态。5类模型共有起始状态9个,引入起始概率;Markov状态之间的转移概率共27个,根据我国乙肝防治现状确定。模拟验证显示,本研究构建的乙肝防治马尔科夫模型符合我国当前实际。结论 马尔科夫模型结构和参数具有动态不确定性,本研究构建的模型可以满足我国乙肝防治策略经济学评价需求。