A simulation study of sample size for multilevel logistic regression models

Background  

Many studies conducted in health and social sciences collect individual level data as outcome measures. Usually, such data have a hierarchical structure, with patients clustered within physicians, and physicians clustered within practices. Large survey data, including national surveys, have a hierarchical or clustered structure; respondents are naturally clustered in geographical units (e.g., health regions) and may be grouped into smaller units. Outcomes of interest in many fields not only reflect continuous measures, but also binary outcomes such as depression, presence or absence of a disease, and self-reported general health. In the framework of multilevel studies an important problem is calculating an adequate sample size that generates unbiased and accurate estimates.     

Representation primitives,process models and patient data in computer-interpretable clinical practice guidelines: a literature review of guideline representation models

Representation of clinical practice guidelines in a computer-interpretable format is a critical issue for guideline development, implementation, and evaluation. We studied 11 types of guideline representation models that can be used to encode guidelines in computer-interpretable formats. We have consistently found in all reviewed models that primitives for representation of actions and decisions are necessary components of a guideline representation model. Patient states and execution states are important concepts that closely relate to each other. Scheduling constraints on representation primitives can be modeled as sequences, concurrences, alternatives, and loops in a guideline's application process. Nesting of guidelines provides multiple views to a guideline with different granularities. Integration of guidelines with electronic medical records can be facilitated by the introduction of a formal model for patient data. Data collection, decision, patient state, and intervention constitute four basic types of primitives in a guideline's logic flow. Decisions clarify our understanding on a patient's clinical state, while interventions lead to the change from one patient state to another.     

The alarming problems of confounding equivalence using logistic regression models in the perspective of causal diagrams

Background

Confounders can produce spurious associations between exposure and outcome in observational studies. For majority of epidemiologists, adjusting for confounders using logistic regression model is their habitual method, though it has some problems in accuracy and precision. It is, therefore, important to highlight the problems of logistic regression and search the alternative method.

Methods

Four causal diagram models were defined to summarize confounding equivalence. Both theoretical proofs and simulation studies were performed to verify whether conditioning on different confounding equivalence sets had the same bias-reducing potential and then to select the optimum adjusting strategy, in which logistic regression model and inverse probability weighting based marginal structural model (IPW-based-MSM) were compared. The “do-calculus” was used to calculate the true causal effect of exposure on outcome, then the bias and standard error were used to evaluate the performances of different strategies.

Results

Adjusting for different sets of confounding equivalence, as judged by identical Markov boundaries, produced different bias-reducing potential in the logistic regression model. For the sets satisfied G-admissibility, adjusting for the set including all the confounders reduced the equivalent bias to the one containing the parent nodes of the outcome, while the bias after adjusting for the parent nodes of exposure was not equivalent to them. In addition, all causal effect estimations through logistic regression were biased, although the estimation after adjusting for the parent nodes of exposure was nearest to the true causal effect. However, conditioning on different confounding equivalence sets had the same bias-reducing potential under IPW-based-MSM. Compared with logistic regression, the IPW-based-MSM could obtain unbiased causal effect estimation when the adjusted confounders satisfied G-admissibility and the optimal strategy was to adjust for the parent nodes of outcome, which obtained the highest precision.

Conclusions

All adjustment strategies through logistic regression were biased for causal effect estimation, while IPW-based-MSM could always obtain unbiased estimation when the adjusted set satisfied G-admissibility. Thus, IPW-based-MSM was recommended to adjust for confounders set.

     

Practical approach to determine sample size for building logistic prediction models using high-throughput data

An empirical method of sample size determination for building prediction models was proposed recently. Permutation method which is used in this procedure is a commonly used method to address the problem of overfitting during cross-validation while evaluating the performance of prediction models constructed from microarray data. But major drawback of such methods which include bootstrapping and full permutations is prohibitively high cost of computation required for calculating the sample size.In this paper, we propose that a single representative null distribution can be used instead of a full permutation by using both simulated and real data sets. During simulation, we have used a dataset with zero effect size and confirmed that the empirical type I error approaches to 0.05. Hence this method can be confidently applied to reduce overfitting problem during cross-validation. We have observed that pilot data set generated by random sampling from real data could be successfully used for sample size determination. We present our results using an experiment that was repeated for 300 times while producing results comparable to that of full permutation method. Since we eliminate full permutation, sample size estimation time is not a function of pilot data size. In our experiment we have observed that this process takes around 30 min.With the increasing number of clinical studies, developing efficient sample size determination methods for building prediction models is critical. But empirical methods using bootstrap and permutation usually involve high computing costs. In this study, we propose a method that can reduce required computing time drastically by using representative null distribution of permutations. We use data from pilot experiments to apply this method for designing clinical studies efficiently for high throughput data.     

Cytopathology of malignant mesothelioma: a stepwise logistic regression analysis.

Twenty-four cytologic features, previously reported to be useful in the distinction of malignant mesothelioma, adenocarcinoma, and benign mesothelial proliferation in serous effusions were assessed. Forty-four cases of malignant mesotheliomas, 46 cases of metastatic adenocarcinomas, and 30 cases of benign mesothelial proliferations were examined for these parameters. When these cytologic features were subjected to a stepwise logistic regression analysis, five features were selected to distinguish malignant mesothelioma from adenocarcinoma. These were true papillary aggregates, multinucleation with atypia, cell-to-cell apposition, acinus-like structures, and balloon-like vacuolation, the latter two features being characteristic of adenocarcinoma. The four variables selected to distinguish malignant mesothelioma from benign mesothelial proliferations were nuclear pleomorphism, macronucleoli, cell-in-cell engulfment, and monolayer cell groups, the latter being a feature of benign proliferations. Using these selected variables, the logistic model correctly predicted 95.4% of cases of malignant mesothelioma versus 100% of adenocarcinoma and 100% of malignant mesotheliomas versus 90% of benign mesothelial proliferations. The results of regression analysis suggest that many of the previously described cytologic features are not important diagnostic discriminators.     

Stability analysis of sleep apnea time series using identified models: a case study

This paper investigates the use of identified nonlinear multivariable autonomous models in the classification of breathing patterns of a patient with sleep apnea. Details about the identification procedure are provided and the results reported for the case study at hand suggest that identified models could be useful in computer-based monitoring.     

Comparison of dimension reduction-based logistic regression models for case-control genome-wide association study: principal components analysis vs. partial least squares

With recent advances in biotechnology, genome-wide association study (GWAS) has been widely used to identify genetic variants that underlie human complex diseases and traits. In case-control GWAS, typical statistical strategy is traditional logistical regression (LR) based on single-locus analysis. However, such a single-locus analysis leads to the well-known multiplicity problem, with a risk of inflating type I error and reducing power. Dimension reduction-based techniques, such as principal component-based logistic regression (PC-LR), partial least squares-based logistic regression (PLS-LR), have recently gained much attention in the analysis of high dimensional genomic data. However, the performance of these methods is still not clear, especially in GWAS. We conducted simulations and real data application to compare the type I error and power of PC-LR, PLS-LR and LR applicable to GWAS within a defined single nucleotide polymorphism (SNP) set region. We found that PC-LR and PLS can reasonably control type I error under null hypothesis. On contrast, LR, which is corrected by Bonferroni method, was more conserved in all simulation settings. In particular, we found that PC-LR and PLS-LR had comparable power and they both outperformed LR, especially when the causal SNP was in high linkage disequilibrium with genotyped ones and with a small effective size in simulation. Based on SNP set analysis, we applied all three methods to analyze non-small cell lung cancer GWAS data.     

Quantifying ITV instabilities arising from 4DCT: a simulation study using patient data

Treatment planning for patients undergoing radiation therapy is often performed based on four-dimensional computed tomography (4DCT) when respiratory motion is present, as in lung cancer patients. 4DCT is used to define the internal target volume (ITV) that, ideally, incorporates all potential locations of the tumour. In this work, we use the locations of gold fiducial markers implanted in lung tumours of eight patients to represent tumour motion. These fiducial locations are used in a simulation of a four-slice CT scanner to generate the ITV for 10, 20 and 30 mm diameter model tumours. To demonstrate instabilities in the ITV definition based on 4DCT, the ITV calculation was repeated for the same patients for consecutive scan start times, staggered by 1 s. The volumetric difference in the ITV and the per cent of time that the ITV contains in the tumour are both evaluated. The ITV from a single patient was found to vary by 46%-127% for a tumour diameter of 10 mm. The ITV did not cover the entirety of the tumour 11%-74% of the time for a 10 mm tumour diameter.     

Comparison of artificial neural network and logistic regression models for prediction of mortality in head trauma based on initial clinical data

Background  

In recent years, outcome prediction models using artificial neural network and multivariable logistic regression analysis have been developed in many areas of health care research. Both these methods have advantages and disadvantages. In this study we have compared the performance of artificial neural network and multivariable logistic regression models, in prediction of outcomes in head trauma and studied the reproducibility of the findings.     

Cytologic features of mycobacterial pleuritis: logistic regression and statistical analysis of a blinded,case-controlled study

Tuberculous pleural effusions are characterized by lymphocytosis; the significance of mesothelial cells is uncertain, as are the cytologic features in concurrent human immunodeficiency virus (HIV) infection. This blinded study compared 38 culture-positive pleural fluids (6 HIV⁺) with 38 controls from benign exudative processes. Logistic regression analysis selected mature lymphocytes as most predictive of positive culture, and mesothelial cells and eosinophils as negative predictors. Mesothelial cells were scant (<10% of nucleated cells) in 36/38 cases with mycobacteria (sensitivity 95%); if these cells were <10%, tuberculosis was virtually ruled out in HIV⁻ patients. Specificity was maximized (82%) when mesothelial cells <10% were combined with lymphocytes <50%; positive predictive value with this combination was 76%, but was raised to 96% if moderate/marked cellularity was also identified. Among tuberculosis cases, reactive mesothelial cells differentiated HIV⁺ from HIV⁻ patients; there was no other significant difference. Diagn. Cytopathol. 1998;19:173–176. © 1998 Wiley-Liss, Inc.     

Risk factors for mortality after bereavement: a logistic regression analysis

A national sample of elderly widowed people was followed up for six years. Excess mortality was found for men aged 75 years and over in the first six months of bereavement compared with men of the same age in the general population. Logistic regression analysis, controlling for age and sex together, demonstrated that the best independent predictors of mortality among the elderly widowed were: interviewer assessment of low happiness level; interviewer assessed and self-reported problems with nerves and depression; and lack of telephone contacts. The general practitioner is well placed to assess levels of depression and unhappiness among the widowed and to check that they have adequate social support.     

Limitations of quantitative gene regulation models: a case study

Understanding the relationship between network structure and behavior is fundamental to the field of computational and systems biology. A particularly important distinction is the extent to which qualitative aspects of network performance are encoded in network topology as opposed to being determined through quantitative details, such as those of kinetics. Here, we develop a general and rigorous mathematical framework for the analysis of genetic networks and apply it to a family of synthetic gene networks. A key feature of our methodology involves determining network behavior that is insensitive to kinetic parameters such as rate constants and nonlinear functional dependencies of rates on molecular concentrations. Results indicate that behavior observed in some networks cannot be reconciled with standard gene expression and regulation models. We explore relaxing model assumptions to explain the observed behavior, allowing for both dynamic and stochastic phenomena, and propose an alternative model. Our alternative model includes the suggestion of a new mechanism by which the counterintuitive behavior could be achieved; central to the model is the assumption that the Clp protein degradation system, which is responsible for the regulatory proteins used in this study, becomes saturated.     

The use of continuous data versus binary data in MTC models: A case study in rheumatoid arthritis

The Computer Adaptive Test version of the Community Reintegration of Injured Service Members measure (CRIS-CAT) consists of three scales measuring Extent of, Perceived Limitations in, and Satisfaction with community integration. The CRIS-CAT was developed using item response theory methods. The purposes of this study were to assess the reliability, concurrent, known group and predictive validity and respondent burden of the CRIS-CAT. The CRIS-CAT was developed using item response theory methods. The purposes of this study were to assess the reliability, concurrent, known group and predictive validity and respondent burden of the CRIS-CAT. This was a three-part study that included a 1) a cross-sectional field study of 517 homeless, employed, and Operation Enduring Freedom / Operation Iraqi Freedom (OEF/OIF) Veterans; who completed all items in the CRIS item set, 2) a cohort study with one year follow-up study of 135 OEF/OIF Veterans, and 3) a 50-person study of CRIS-CAT administration. Conditional reliability of simulated CAT scores was calculated from the field study data, and concurrent validity and known group validity were examined using Pearson product correlations and ANOVAs. Data from the cohort were used to examine the ability of the CRIS-CAT to predict key one year outcomes. Data from the CRIS-CAT administration study were used to calculate ICC (2,1) minimum detectable change (MDC), and average number of items used during CAT administration. Reliability scores for all scales were above 0.75, but decreased at both ends of the score continuum. CRIS-CAT scores were correlated with concurrent validity indicators and differed significantly between the three Veteran groups (P < .001). The odds of having any Emergency Room visits were reduced for Veterans with better CRIS-CAT scores (Extent, Perceived Satisfaction respectively: OR = 0.94, 0.93, 0.95; P < .05). CRIS-CAT scores were predictive of SF-12 physical and mental health related quality of life scores at the 1 year follow-up. Scales had ICCs >0.9. MDCs were 5.9, 6.2, and 3.6, respectively for Extent, Perceived and Satisfaction subscales. Number of items (mn, SD) administered at Visit 1 were 14.6 (3.8) 10.9 (2.7) and 10.4 (1.7) respectively for Extent, Perceived and Satisfaction subscales. The CRIS-CAT demonstrated sound measurement properties including reliability, construct, known group and predictive validity, and it was administered with minimal respondent burden. These findings support the use of this measure in assessing community reintegration.     

Understanding data in clinical research: a simple graphical display for plotting data (up to four independent variables) after binary logistic regression analysis

In clinical research, suitable visualization techniques of data after statistical analysis are crucial for the researches' and physicians' understanding. Common statistical techniques to analyze data in clinical research are logistic regression models. Among these, the application of binary logistic regression analysis (LRA) has greatly increased during past years, due to its diagnostic accuracy and because scientists often want to analyze in a dichotomous way whether some event will occur or not. Such an analysis lacks a suitable, understandable, and widely used graphical display, instead providing an understandable logit function based on a linear model for the natural logarithm of the odds in favor of the occurrence of the dependent variable, Y. By simple exponential transformation, such a logit equation can be transformed into a logistic function, resulting in predicted probabilities for the presence of the dependent variable, P(Y-1/X). This model can be used to generate a simple graphical display for binary LRA. For the case of a single predictor or explanatory (independent) variable, X, a plot can be generated with X represented by the abscissa (i.e., horizontal axis) and P(Y-1/X) represented by the ordinate (i.e., vertical axis). For the case of multiple predictor models, I propose here a relief 3D surface graphic in order to plot up to four independent variables (two continuous and two discrete). By using this technique, any researcher or physician would be able to transform a lesser understandable logit function into a figure easier to grasp, thus leading to a better knowledge and interpretation of data in clinical research. For this, a sophisticated statistical package is not necessary, because the graphical display may be generated by using any 2D or 3D surface plotter.     

MELPREDICT: a logistic regression model to estimate CDKN2A carrier probability

Background

Heritable alterations in CDKN2A account for a subset of familial melanoma cases although no robust method exists to identify those at risk of being a mutation carrier.

Methods

We set out to construct a model for estimating CDKN2A mutation carrier probability using a cohort of 116 consecutive familial cutaneous melanoma patients evaluated at Massachusetts General Hospital Pigmented Lesion Center between April 2001 and September 2004. Germline CDKN2A and CDK4 status on the familial melanoma cases and clinical features associated with mutational status were then used to build a multiple logistic regression model to predict carrier probability and performance of model on external validation.

Results

From the 116 kindreds prone to melanoma in the Boston area, 13 CDKN2A mutation carriers were identified and 12 were subsequently used in the modeling. Proband age at diagnosis, number of proband primaries, and number of additional family primaries were most closely associated with germline mutations. The estimated probability of the proband being a mutation carrier based on the logistic regression model (MELPREDICT) is given bywhere L = 1.99+[0.92×(no. of proband primaries)]+[0.74×(no. of additional family primaries)]−[2.11×ln_(age)]. The mean estimated probabilities for subjects in the Boston dataset were 55.4% and 5.1% for the mutation carriers and non‐carriers respectively. In a receiver operator characteristic analysis, the area under the curve was 0.881 (95% confidence interval 0.739 to 1.000) for the Boston model set (n = 116) and 0.803 (0.729 to 0.877) for an external Toronto hereditary melanoma cohort (n = 143).

Conclusions

These results represent the first‐iteration logistic regression model to approximate CDKN2A carrier probability. Validation of this model with an external dataset revealed relatively robust performance.     

Creating genetics-based infusion centers: a case study of two models

In 1993, the first effective enzyme replacement therapy for a genetic disease, Ceredase (Genzyme Corporation, Cambridge, MA), was approved for use in patients with Gaucher disease. Over the next 13 years, enzyme replacement therapy became clinically available for the treatment of Fabry disease, mucopolysaccharidosis Type I, mucopolysaccharidosis Type II, mucopolysaccharidosis Type VI, and glycogen storage disease Type II. The development of enzyme replacement therapy to treat lysosomal storage diseases has resulted in an increasing number of genetic patients undergoing weekly or biweekly intravenous enzyme replacement therapy and an expanded role of the genetics team to include comprehensive care involving therapeutic intervention for lysosomal storage diseases. This article describes the development of two outpatient genetics-based infusion centers: the Northshore Genetics Infusion Clinic as part of the Children's Hospital of Wisconsin Lysosomal Diseases Treatment Center in conjunction with the Medical College of Wisconsin and the Emory Lysosomal Storage Disease Center for Genetic Infusions in the Emory University Department of Human Genetics.     

Outbreak detection algorithms for seasonal disease data: a case study using ross river virus disease

Background  

Detection of outbreaks is an important part of disease surveillance. Although many algorithms have been designed for detecting outbreaks, few have been specifically assessed against diseases that have distinct seasonal incidence patterns, such as those caused by vector-borne pathogens.     

Power of multifactor dimensionality reduction and penalized logistic regression for detecting gene-gene Interaction in a case-control study

Background  

There is a growing awareness that interaction between multiple genes play an important role in the risk of common, complex multi-factorial diseases. Many common diseases are affected by certain genotype combinations (associated with some genes and their interactions). The identification and characterization of these susceptibility genes and gene-gene interaction have been limited by small sample size and large number of potential interactions between genes. Several methods have been proposed to detect gene-gene interaction in a case control study. The penalized logistic regression (PLR), a variant of logistic regression with L ₂ regularization, is a parametric approach to detect gene-gene interaction. On the other hand, the Multifactor Dimensionality Reduction (MDR) is a nonparametric and genetic model-free approach to detect genotype combinations associated with disease risk.     

Spontaneous regression of intraoral mucosa-associated lymphoid tissue lymphoma: molecular study of a case

Mucosa-associated lymphoid tissue (MALT) lymphoma presentation in the oral cavity is very rare. Reported herein is a case of intraoral MALT lymphoma of the minor salivary gland in a 70-year-old woman with Sjogren's syndrome. Unexpectedly, a spontaneous clinically and histologically confirmed regression occurred 1 month after the tumor biopsy for diagnosis. Considering that salivary MALT lymphoma is associated with Sjogren's syndrome and that the chronic inflammation caused by Sjogren's syndrome persisted, it is hypothesized that the tumor clone might be present in the regressed lesion. Minimal residual tumor clone identical with the primary lesion was detected using the polymerase chain reaction (PCR) clonality assay for immunoglobulin heavy chain gene (IgH) rearrangement. No recurrence was clinically evident 38 months after the diagnosis. Spontaneous regression of MALT lymphoma should be examined at the molecular level in addition to clinical and histological evaluations. When minimal residual disease is detected, close follow up is necessary for early detection of the tumor relapse.     

Preliminary exploration of survival analysis using the OHDSI common data model: a case study of intrahepatic cholangiocarcinoma

Background

Data heterogeneity is a common phenomenon related to the secondary use of electronic health records (EHR) data from different sources. The Observational Health Data Sciences and Informatics (OHDSI) Common Data Model (CDM) organizes healthcare data into standard data structures using concepts that are explicitly and formally specified through standard vocabularies, thereby facilitating large-scale analysis. The objective of this study is to design, develop, and evaluate generic survival analysis routines built using the OHDSI CDM.

Methods

We used intrahepatic cholangiocarcinoma (ICC) patient data to implement CDM-based survival analysis methods. Our methods comprise the following modules: 1) Mapping local terms to standard OHDSI concepts. The analytical expression of variables and values related to demographic characteristics, medical history, smoking status, laboratory results, and tumor feature data. These data were mapped to standard OHDSI concepts through a manual analysis; 2) Loading patient data into the CDM using the concept mappings; 3) Developing an R interface that supports the portable survival analysis on top of OHDSI CDM, and comparing the CDM-based analysis results with those using traditional statistical analysis methods.

Results

Our dataset contained 346 patients diagnosed with ICC. The collected clinical data contains 115 variables, of which 75 variables were mapped to the OHDSI concepts. These concepts mainly belong to four domains: condition, observation, measurement, and procedure. The corresponding standard concepts are scattered in six vocabularies: ICD10CM, ICD10PCS, SNOMED, LOINC, NDFRT, and READ. We loaded a total of 25,950 patient data records into the OHDSI CDM database. However, 40 variables failed to map to the OHDSI CDM as they mostly belong to imaging data and pathological data.

Conclusions

Our study demonstrates that conducting survival analysis using the OHDSI CDM is feasible and can produce reusable analysis routines. However, challenges to be overcome include 1) semantic loss caused by inaccurate mapping and value normalization; 2) incomplete OHDSI vocabularies describing imaging data, pathological data, and modular data representation.