Performance of computational methods for the evaluation of pericentriolar material 1 missense variants in CAGI‐5

The CAGI‐5 pericentriolar material 1 (PCM1) challenge aimed to predict the effect of 38 transgenic human missense mutations in the PCM1 protein implicated in schizophrenia. Participants were provided with 16 benign variants (negative controls), 10 hypomorphic, and 12 loss of function variants. Six groups participated and were asked to predict the probability of effect and standard deviation associated to each mutation. Here, we present the challenge assessment. Prediction performance was evaluated using different measures to conclude in a final ranking which highlights the strengths and weaknesses of each group. The results show a great variety of predictions where some methods performed significantly better than others. Benign variants played an important role as negative controls, highlighting predictors biased to identify disease phenotypes. The best predictor, Bromberg lab, used a neural‐network‐based method able to discriminate between neutral and non‐neutral single nucleotide polymorphisms. The CAGI‐5 PCM1 challenge allowed us to evaluate the state of the art techniques for interpreting the effect of novel variants for a difficult target protein.     

Are machine learning based methods suited to address complex biological problems? Lessons from CAGI‐5 challenges

In silico approaches are routinely adopted to predict the effects of genetic variants and their relation to diseases. The critical assessment of genome interpretation (CAGI) has established a common framework for the assessment of available predictors of variant effects on specific problems and our group has been an active participant of CAGI since its first edition. In this paper, we summarize our experience and lessons learned from the last edition of the experiment (CAGI‐5). In particular, we analyze prediction performances of our tools on five CAGI‐5 selected challenges grouped into three different categories: prediction of variant effects on protein stability, prediction of variant pathogenicity, and prediction of complex functional effects. For each challenge, we analyze in detail the performance of our tools, highlighting their potentialities and drawbacks. The aim is to better define the application boundaries of each tool.     

BRCA1‐ and BRCA2‐specific in silico tools for variant interpretation in the CAGI 5 ENIGMA challenge

BRCA1 and BRCA2 (BRCA1/2) germline variants disrupting the DNA protective role of these genes increase the risk of hereditary breast and ovarian cancers. Correct identification of these variants then becomes clinically relevant, because it may increase the survival rates of the carriers. Unfortunately, we are still unable to systematically predict the impact of BRCA1/2 variants. In this article, we present a family of in silico predictors that address this problem, using a gene‐specific approach. For each protein, we have developed two tools, aimed at predicting the impact of a variant at two different levels: Functional and clinical. Testing their performance in different datasets shows that specific information compensates the small number of predictive features and the reduced training sets employed to develop our models. When applied to the variants of the BRCA1/2 (ENIGMA) challenge in the fifth Critical Assessment of Genome Interpretation (CAGI 5) we find that these methods, particularly those predicting the functional impact of variants, have a good performance, identifying the large compositional bias towards neutral variants in the CAGI sample. This performance is further improved when incorporating to our prediction protocol estimates of the impact on splicing of the target variant.     

CAGI5: Objective performance assessments of predictions based on the Evolutionary Action equation

Many computational approaches estimate the effect of coding variants, but their predictions often disagree with each other. These contradictions confound users and raise questions regarding reliability. Performance assessments can indicate the expected accuracy for each method and highlight advantages and limitations. The Critical Assessment of Genome Interpretation (CAGI) community aims to organize objective and systematic assessments: They challenge predictors on unpublished experimental and clinical data and assign independent assessors to evaluate the submissions. We participated in CAGI experiments as predictors, using the Evolutionary Action (EA) method to estimate the fitness effect of coding mutations. EA is untrained, uses homology information, and relies on a formal equation: The fitness effect equals the functional sensitivity to residue changes multiplied by the magnitude of the substitution. In previous CAGI experiments (between 2011 and 2016), our submissions aimed to predict the protein activity of single mutants. In 2018 (CAGI5), we also submitted predictions regarding clinical associations, folding stability, and matching genomic data with phenotype. For all these diverse challenges, we used EA to predict the fitness effect of variants, adjusted to specifically address each question. Our submissions had consistently good performance, suggesting that EA predicts reliably the effects of genetic variants.     

Assessment of blind predictions of the clinical significance of BRCA1 and BRCA2 variants

Testing for variation in BRCA1 and BRCA2 (commonly referred to as BRCA1/2), has emerged as a standard clinical practice and is helping countless women better understand and manage their heritable risk of breast and ovarian cancer. Yet the increased rate of BRCA1/2 testing has led to an increasing number of Variants of Uncertain Significance (VUS), and the rate of VUS discovery currently outpaces the rate of clinical variant interpretation. Computational prediction is a key component of the variant interpretation pipeline. In the CAGI5 ENIGMA Challenge, six prediction teams submitted predictions on 326 newly‐interpreted variants from the ENIGMA Consortium. By evaluating these predictions against the new interpretations, we have gained a number of insights on the state of the art of variant prediction and specific steps to further advance this state of the art.     

Abnormal function of the UBA5 protein in a case of early developmental and epileptic encephalopathy with suppression‐burst

Early myoclonic epilepsy (EME) or Aicardi syndrome is one of the most severe epileptic syndromes affecting neonates. We performed whole exome sequencing in a sporadic case affected by EME and his parents. In the proband, we identified a homozygous missense variant in the ubiquitin‐like modifier activating enzyme 5 (UBA5) gene, encoding a protein involved in post‐translational modifications. Functional analysis of the UBA5 variant protein reveals that it is almost completely unable to perform its trans‐thiolation activity. Although recessive variants in UBA5 have recently been associated with epileptic encephalopathy, variants in this gene have never been reported to cause EME. Our results further demonstrate the importance of post‐translational modifications such as the addition of an ubiquitin‐fold modifier 1 (UFM1) to target proteins (ufmylation) for normal neuronal networks activity, and reveal that the dysfunction of the ubiquitous UBA5 protein is a cause of EME.     

Assessing predictions of the impact of variants on splicing in CAGI5

Precision medicine and sequence‐based clinical diagnostics seek to predict disease risk or to identify causative variants from sequencing data. The Critical Assessment of Genome Interpretation (CAGI) is a community experiment consisting of genotype‐phenotype prediction challenges; participants build models, undergo assessment, and share key findings. In the past, few CAGI challenges have addressed the impact of sequence variants on splicing. In CAGI5, two challenges (Vex‐seq and MaPSY) involved prediction of the effect of variants, primarily single‐nucleotide changes, on splicing. Although there are significant differences between these two challenges, both involved prediction of results from high‐throughput exon inclusion assays. Here, we discuss the methods used to predict the impact of these variants on splicing, their performance, strengths, and weaknesses, and prospects for predicting the impact of sequence variation on splicing and disease phenotypes.     

A patient with homozygous nonsense variants in two Leigh syndrome disease genes: Distinguishing a dual diagnosis from a hypomorphic protein‐truncating variant

Leigh syndrome is a mitochondrial disease caused by pathogenic variants in over 85 genes. Whole exome sequencing of a patient with Leigh‐like syndrome identified homozygous protein‐truncating variants in two genes associated with Leigh syndrome; a reported pathogenic variant in PDHX (NP_003468.2:p.(Arg446*)), and an uncharacterized variant in complex I (CI) assembly factor TIMMDC1 (NP_057673.2:p.(Arg225*)). The TIMMDC1 variant was predicted to truncate 61 amino acids at the C‐terminus and functional studies demonstrated a hypomorphic impact of the variant on CI assembly. However, the mutant protein could still rescue CI assembly in TIMMDC1 knockout cells and the patient's clinical phenotype was not clearly distinct from that of other patients with the same PDHX defect. Our data suggest that the hypomorphic effect of the TIMMDC1 protein‐truncating variant does not constitute a dual diagnosis in this individual. We recommend cautious assessment of variants in the C‐terminus of TIMMDC1 and emphasize the need to consider the caveats detailed within the American College of Medical Genetics and Genomics (ACMG) criteria when assessing variants.     

Using secondary structure to predict the effects of genetic variants on alternative splicing

Accurate interpretation of genomic variants that alter RNA splicing is critical to precision medicine. We present a computational framework, Prediction of variant Effect on Percent Spliced In (PEPSI), that predicts the splicing impact of coding and noncoding variants for the Fifth Critical Assessment of Genome Interpretation (CAGI5) “Vex‐seq” challenge. PEPSI is a random forest regression model trained on multiple layers of features associated with sequence conservation and regulatory sequence elements. Compared to other splicing defect prediction tools from the literature, our framework integrates secondary structure information in predicting variants that disrupt splicing regulatory elements (SREs). We applied our model to classify splice‐disrupting variants among 2,094 single‐nucleotide polymorphisms from the Exome Aggregation Consortium using model‐predicted changes in percent spliced in (ΔPSI) associated with tested variants. Benchmarking our model against widely used state‐of‐the‐art tools, we demonstrate that PEPSI achieves comparable performance in terms of sensitivity and precision. Moreover, we also show that using secondary structure context can help resolve several cases where changes in the counts of SREs do not correspond with the directionality of ΔPSI measured for tested variants.     

De novo variants in CELF2 that disrupt the nuclear localization signal cause developmental and epileptic encephalopathy

We report heterozygous CELF2 (NM_006561.3) variants in five unrelated individuals: Individuals 1–4 exhibited developmental and epileptic encephalopathy (DEE) and Individual 5 had intellectual disability and autistic features. CELF2 encodes a nucleocytoplasmic shuttling RNA‐binding protein that has multiple roles in RNA processing and is involved in the embryonic development of the central nervous system and heart. Whole‐exome sequencing identified the following CELF2 variants: two missense variants [c.1558C>T:p.(Pro520Ser) in unrelated Individuals 1 and 2, and c.1516C>G:p.(Arg506Gly) in Individual 3], one frameshift variant in Individual 4 that removed the last amino acid of CELF2 c.1562dup:p.(Tyr521Ter), possibly resulting in escape from nonsense‐mediated mRNA decay (NMD), and one canonical splice site variant, c.272‐1G>C in Individual 5, also probably leading to NMD. The identified variants in Individuals 1, 2, 4, and 5 were de novo, while the variant in Individual 3 was inherited from her mosaic mother. Notably, all identified variants, except for c.272‐1G>C, were clustered within 20 amino acid residues of the C‐terminus, which might be a nuclear localization signal. We demonstrated the extranuclear mislocalization of mutant CELF2 protein in cells transfected with mutant CELF2 complementary DNA plasmids. Our findings indicate that CELF2 variants that disrupt its nuclear localization are associated with DEE.     

NR5A1 gene variants repress the ovarian‐specific WNT signaling pathway in 46,XX disorders of sex development patients

Several recent reports have described a missense variant in the gene NR5A1 (c.274C>T; p.Arg92Trp) in a significant number of 46,XX ovotesticular or testicular disorders of sex development (DSDs) cases. The affected residue falls within the DNA‐binding domain of the NR5A1 protein, however the exact mechanism by which it causes testicular development in 46,XX individuals remains unclear. We have screened a cohort of 26 patients with 46,XX (ovo)testicular DSD and identified three unrelated individuals with this NR5A1 variant (p.Arg92Trp), as well as one patient with a novel NR5A1 variant (c.779C>T; p.Ala260Val). We examined the functional effect of these changes, finding that while protein levels and localization were unaffected, variant NR5A1 proteins repress the WNT signaling pathway and have less ability to upregulate the anti‐testis gene NR0B1. These findings highlight how NR5A1 variants impact ovarian differentiation across multiple pathways, resulting in a switch from ovarian to testis development in genetic females.     

Identifying mutation‐driven changes in gene functionality that lead to venous thromboembolism

Venous thromboembolism (VTE) is a common hematological disorder. VTE affects millions of people around the world each year and can be fatal. Earlier studies have revealed the possible VTE genetic risk factors in Europeans. The 2018 Critical Assessment of Genome Interpretation (CAGI) challenge had asked participants to distinguish between 66 VTE and 37 non‐VTE African American (AA) individuals based on their exome sequencing data. We used variants from AA VTE association studies and VTE genes from DisGeNET database to evaluate VTE risk via four different approaches; two of these methods were most successful at the task. Our best performing method represented each exome as a vector of predicted functional effect scores of variants within the known genes. These exome vectors were then clustered with k‐means. This approach achieved 70.8% precision and 69.7% recall in identifying VTE patients. Our second‐best ranked method had collapsed the variant effect scores into gene‐level function changes, using the same vector clustering approach for patient/control identification. These results show predictability of VTE risk in AA population and highlight the importance of variant‐driven gene functional changes in judging disease status. Of course, more in‐depth understanding of AA VTE pathogenicity is still needed for more precise predictions.     

Classifying MLH1 and MSH2 variants using bioinformatic prediction,splicing assays,segregation, and tumor characteristics

Reliable methods for predicting functional consequences of variants in disease genes would be beneficial in the clinical setting. This study was undertaken to predict, and confirm in vitro, splicing aberrations associated with mismatch repair (MMR) variants identified in familial colon cancer patients. Six programs were used to predict the effect of 13 MLH1 and 6 MSH2 gene variants on pre‐mRNA splicing. mRNA from cycloheximide‐treated lymphoblastoid cell lines of variant carriers was screened for splicing aberrations. Tumors of variant carriers were tested for microsatellite instability and MMR protein expression. Variant segregation in families was assessed using Bayes factor causality analysis. Amino acid alterations were examined for evolutionary conservation and physicochemical properties. Splicing aberrations were detected for 10 variants, including a frameshift as a minor cDNA product, and altered ratio of known alternate splice products. Loss of splice sites was well predicted by splice‐site prediction programs SpliceSiteFinder (90%) and NNSPLICE (90%), but consequence of splice site loss was less accurately predicted. No aberrations correlated with ESE predictions for the nine exonic variants studied. Seven of eight missense variants had normal splicing (88%), but only one was a substitution considered neutral from evolutionary/physicochemical analysis. Combined with information from tumor and segregation analysis, and literature review, 16 of 19 variants were considered clinically relevant. Bioinformatic tools for prediction of splicing aberrations need improvement before use without supporting studies to assess variant pathogenicity. Classification of mismatch repair gene variants is assisted by a comprehensive approach that includes in vitro, tumor pathology, clinical, and evolutionary conservation data. Hum Mutat 0, 1–14, 2009. © 2009 Wiley‐Liss, Inc.     

Future directions for high‐throughput splicing assays in precision medicine

Classification of variants of unknown significance is a challenging technical problem in clinical genetics. As up to one‐third of disease‐causing mutations are thought to affect pre‐mRNA splicing, it is important to accurately classify splicing mutations in patient sequencing data. Several consortia and healthcare systems have conducted large‐scale patient sequencing studies, which discover novel variants faster than they can be classified. Here, we compare the advantages and limitations of several high‐throughput splicing assays aimed at mitigating this bottleneck, and describe a data set of ~5,000 variants that we analyzed using our Massively Parallel Splicing Assay (MaPSy). The Critical Assessment of Genome Interpretation group (CAGI) organized a challenge, in which participants submitted machine learning models to predict the splicing effects of variants in this data set. We discuss the winning submission of the challenge (MMSplice) which outperformed existing software. Finally, we highlight methods to overcome the limitations of MaPSy and similar assays, such as tissue‐specific splicing, the effect of surrounding sequence context, classifying intronic variants, synthesizing large exons, and amplifying complex libraries of minigene species. Further development of these assays will greatly benefit the field of clinical genetics, which lack high‐throughput methods for variant interpretation.     

Biallelic variant in AGTPBP1 causes infantile lower motor neuron degeneration and cerebellar atrophy

Infantile hereditary lower motor neuron disorders beyond 5q–spinal muscular atrophy (5q‐SMA) are usually caused by mutations other than deletions or mutations in SMN1. In addition to motor neuron degeneration, further neurologic or multisystemic pathologies in non‐5q‐SMAs are not seldom. Some of the non‐5q‐SMA phenotypes, such as pontocerebellar hypoplasia (PCH1), have been classified later as a different disease group due to distinctive primary pathologies. Likewise, a novel phenotype, childhood‐onset neurodegeneration with cerebellar atrophy (CONDCA) has been described recently in individuals with lower motor neuron disorder and cerebellar atrophy due to biallelic loss‐of‐function variants in AGTPBP1 that encodes cytosolic carboxypeptidase 1 (CCP1). Here we present two individuals with CONDCA in whom a biallelic missense AGTPBP1 variant (NM_001330701.1:c.2396G>T, p.Arg799Leu) was identified by whole exome sequencing. Affected individuals in this report correspond to the severe infantile spectrum of the disease and underline the severe pathogenic effect of this missense variant. This report is the second in the literature that delineates the pathogenic effects of biallelic AGTPBP1 variants presenting the recently described CONDCA disease.     

Functional characterization of novel NR5A1 variants reveals multiple complex roles in disorders of sex development

Variants in the NR5A1 gene encoding SF1 have been described in a diverse spectrum of disorders of sex development (DSD). Recently, we reported the use of a targeted gene panel for DSD where we identified 15 individuals with a variant in NR5A1, nine of which are novel. Here, we examine the functional effect of these changes in relation to the patient phenotype. All novel variants tested had reduced trans‐activational activity, while several had altered protein level, localization, or conformation. In addition, we found evidence of new roles for SF1 protein domains including a region within the ligand binding domain that appears to contribute to SF1 regulation of Müllerian development. There was little correlation between the severity of the phenotype and the nature of the NR5A1 variant. We report two familial cases of NR5A1 deficiency with evidence of variable expressivity; we also report on individuals with oligogenic inheritance. Finally, we found that the nature of the NR5A1 variant does not inform patient outcomes (including pubertal androgenization and malignancy risk). This study adds nine novel pathogenic NR5A1 variants to the pool of diagnostic variants. It highlights a greater need for understanding the complexity of SF1 function and the additional factors that contribute.