Generating automated kidney transplant biopsy reports combining molecular measurements with ensembles of machine learning classifiers

We previously reported a system for assessing rejection in kidney transplant biopsies using microarray‐based gene expression data, the Molecular Microscope^® Diagnostic System (MMDx). The present study was designed to optimize the accuracy and stability of MMDx diagnoses by replacing single machine learning classifiers with ensembles of diverse classifier methods. We also examined the use of automated report sign‐outs and the agreement between multiple human interpreters of the molecular results. Ensembles generated diagnoses that were both more accurate than the best individual classifiers, and nearly as stable as the best, consistent with expectations from the machine learning literature. Human experts had ≈93% agreement (balanced accuracy) signing out the reports, and random forest‐based automated sign‐outs showed similar levels of agreement with the human experts (92% and 94% for predicting the expert MMDx sign‐outs for T cell–mediated (TCMR) and antibody‐mediated rejection (ABMR), respectively). In most cases disagreements, whether between experts or between experts and automated sign‐outs, were in biopsies near diagnostic thresholds. Considerable disagreement with histology persisted. The balanced accuracies of MMDx sign‐outs for histology diagnoses of TCMR and ABMR were 73% and 78%, respectively. Disagreement with histology is largely due to the known noise in histology assessments (ClinicalTrials.gov NCT01299168).