Machine Learning and Computational Mathematics

Neural network-based machine learning is capable of approximating functions in very high dimension with unprecedented efficiency and accuracy. This has opened up many exciting new possibilities, not just in traditional areas of artificial intelligence, but also in scientific computing and computational science. At the same time, machine learning has also acquired the reputation of being a set of "black box" type of tricks, without fundamental principles. This has been a real obstacle for making further progress in machine learning.In this article, we try to address the following two very important questions: (1) How machine learning has already impacted and will further impact computational mathematics, scientific computing and computational science? (2) How computational mathematics, particularly numerical analysis, can impact machine learning? We describe some of the most important progress that has been made on these issues. Our hope is to put things into a perspective that will help to integrate machine learning with computational mathematics.     

An Augmented Lagrangian Deep Learning Method for Variational Problems with Essential Boundary Conditions

This paper is concerned with a novel deep learning method for variational problems with essential boundary conditions. To this end, we first reformulate the original problem into a minimax problem corresponding to a feasible augmented Lagrangian, which can be solved by the augmented Lagrangian method in an infinite dimensional setting. Based on this, by expressing the primal and dual variables with two individual deep neural network functions, we present an augmented Lagrangian deep learning method for which the parameters are trained by the stochastic optimization method together with a projection technique. Compared to the traditional penalty method, the new method admits two main advantages: i) the choice of the penalty parameter is flexible and robust, and ii) the numerical solution is more accurate in the same magnitude of computational cost. As typical applications, we apply the new approach to solve elliptic problems and (nonlinear) eigenvalue problems with essential boundary conditions, and numerical experiments are presented to show the effectiveness of the new method.     

Numerical Entropy and Adaptivity for Finite Volume Schemes

We propose an a-posteriori error/smoothness indicator for standard semi-discrete finite volume schemes for systems of conservation laws, based on the numerical production of entropy. This idea extends previous work by the first author limited to central finite volume schemes on staggered grids. We prove that the indicator converges to zero with the same rate of the error of the underlying numerical scheme on smooth flows under grid refinement. We construct and test an adaptive scheme for systems of equations in which the mesh is driven by the entropy indicator. The adaptive scheme uses a single nonuniform grid with a variable timestep. We show how to implement a second order scheme on such a space-time non uniform grid, preserving accuracy and conservation properties. We also give an example of a p-adaptive strategy.     

Convolution Neural Network Shock Detector for Numerical Solution of Conservation Laws

We propose a universal discontinuity detector using convolution neural network (CNN) and apply it in conjunction of solving nonlinear conservation laws in both 1D and 2D. The CNN detector is trained offline with synthetic data. The training data are generated using randomly constructed piecewise functions, which are then processed using randomized linear advection solver to count for the cases of numerical errors in practice. The detector is then paired with high-order numerical solvers. In particular, we combined high-order WENO in troubled cells with high-order central difference in smooth region. Extensive numerical examples are presented. We observe that the proposed method produces notably sharper and cleaner signals near the discontinuities, when compared to other well known troubled cell detector methods.     

Troubled-Cell Indicator Based on Mean Value Property for Hybrid WENO Schemes

In this paper, we introduce a new type of troubled-cell indicator to improve hybrid weighted essentially non-oscillatory (WENO) schemes for solving the hyperbolic conservation laws. The hybrid WENO schemes selectively adopt the high-order linear upwind scheme or the WENO scheme to avoid the local characteristic decompositions and calculations of the nonlinear weights in smooth regions. Therefore, they can reduce computational cost while maintaining non-oscillatory properties in non-smooth regions. Reliable troubled-cell indicators are essential for efficient hybrid WENO methods. Most of troubled-cell indicators require proper parameters to detect discontinuities precisely, but it is very difficult to determine the parameters automatically. We develop a new troubled-cell indicator derived from the mean value theorem that does not require any variable parameters. Additionally, we investigate the characteristics of indicator variable; one of the conserved properties or the entropy is considered as indicator variable. Detailed numerical tests for 1D and 2D Euler equations are conducted to demonstrate the performance of the proposed indicator. The results with the proposed troubled-cell indicator are in good agreement with pure WENO schemes. Also the new indicator has advantages in the computational cost compared with the other indicators.     

A Modeling Framework for Coupling Plasticity with Species Diffusion

This paper presents a modeling framework—mathematical model and computational framework—to study the response of a plastic material due to the presence and transport of a chemical species in the host material. Such a modeling framework is important to a wide variety of problems ranging from Li-ion batteries, moisture diffusion in cementitious materials, hydrogen diffusion in metals, to consolidation of soils under severe loading-unloading regimes. The mathematical model incorporates experimental observations reported in the literature on how (elastic and plastic) material properties change because of the presence and transport of a chemical species. Also, the model accounts for one-way (transport affects the deformation but not vice versa) and two-way couplings between deformation and transport subproblems. The resulting coupled equations are not amenable to analytical solutions; so, we present a robust computational framework for obtaining numerical solutions. Given that popular numerical formulations do not produce nonnegative solutions, the computational framework uses an optimized-based nonnegative formulation that respects physical constraints (e.g., nonnegative concentrations). For completeness, we also show the effect and propagation of the negative concentrations, often produced by contemporary transport solvers, into the overall predictions of deformation and concentration fields. Notably, anisotropy of the diffusion process exacerbates these unphysical violations. Using representative numerical examples, we discuss how the concentration field affects plastic deformations of a degrading solid. Based on these numerical examples, we also discuss how plastic zones spread because of material degradation. To illustrate how the proposed computational framework performs, we report various performance metrics such as optimization iterations and time-to-solution.     

High Order Fixed-Point Sweeping WENO Methods for Steady State of Hyperbolic Conservation Laws and Its Convergence Study

Fixed-point iterative sweeping methods were developed in the literature to efficiently solve static Hamilton-Jacobi equations. This class of methods utilizes the Gauss-Seidel iterations and alternating sweeping strategy to achieve fast convergence rate. They take advantage of the properties of hyperbolic partial differential equations (PDEs) and try to cover a family of characteristics of the corresponding Hamilton-Jacobi equation in a certain direction simultaneously in each sweeping order. Different from other fast sweeping methods, fixed-point iterative sweeping methods have the advantages such as that they have explicit forms and do not involve inverse operation of nonlinear local systems. In principle, it can be applied to solving very general equations using any monotone numerical fluxes and high order approximations easily. In this paper, based on the recently developed fifth order WENO schemes which improve the convergence of the classical WENO schemes by removing slight post-shock oscillations, we design fifth order fixed-point sweeping WENO methods for efficient computation of steady state solution of hyperbolic conservation laws. Especially, we show that although the methods do not have linear computational complexity, they converge to steady state solutions much faster than regular time-marching approach by stability improvement for high order schemes with a forward Euler time-marching.     

Simulations of Compressible Two-Medium Flow by Runge-Kutta Discontinuous Galerkin Methods with the Ghost Fluid Method

The original ghost fluid method (GFM) developed in [13] and the modified GFM (MGFM) in [26] have provided a simple and yet flexible way to treat two-medium flow problems. The original GFM and MGFM make the material interface ”invisible” during computations and the calculations are carried out as for a single medium such that its extension to multi-dimensions becomes fairly straightforward. The Runge-Kutta discontinuous Galerkin (RKDG) method for solving hyperbolic conservation laws is a high order accurate finite element method employing the useful features from high resolution finite volume schemes, such as the exact or approximate Riemann solvers, TVD Runge-Kutta time discretizations, and limiters. In this paper, we investigate using RKDG finite element methods for two-medium flow simulations in one and two dimensions in which the moving material interfaces are treated via nonconservative methods based on the original GFM and MGFM. Numerical results for both gas-gas and gas-water flows are provided to show the characteristic behaviors of these combinations.     

Approximate Riemann Solvers and Robust High-Order Finite Volume Schemes for Multi-Dimensional Ideal MHD Equations

We design stable and high-order accurate finite volume schemes for the ideal MHD equations in multi-dimensions. We obtain excellent numerical stability due to some new elements in the algorithm. The schemes are based on three- and five-wave approximate Riemann solvers of the HLL-type, with the novelty that we allow a varying normal magnetic field. This is achieved by considering the semi-conservative Godunov-Powell form of the MHD equations. We show that it is important to discretize the Godunov-Powell source term in the right way, and that the HLL-type solvers naturally provide a stable upwind discretization. Second-order versions of the ENO- and WENO-type reconstructions are proposed, together with precise modifications necessary to preserve positive pressure and density. Extending the discrete source term to second order while maintaining stability requires non-standard techniques, which we present. The first- and second-order schemes are tested on a suite of numerical experiments demonstrating impressive numerical resolution as well as stability, even on very fine meshes.     

Deep Nitsche Method: Deep Ritz Method with Essential Boundary Conditions

We propose a new method to deal with the essential boundary conditions encountered in the deep learning-based numerical solvers for partial differential equations. The trial functions representing by deep neural networks are non-interpolatory, which makes the enforcement of the essential boundary conditions a nontrivial matter. Our method resorts to Nitsche's variational formulation to deal with this difficulty, which is consistent, and does not require significant extra computational costs. We prove the error estimate in the energy norm and illustrate the method on several representative problems posed in at most 100 dimension.     

High Order Conservative Semi-Lagrangian Scheme for the BGK Model of the Boltzmann Equation

In this paper, we present a conservative semi-Lagrangian finite-difference scheme for the BGK model. Classical semi-Lagrangian finite difference schemes, coupled with an L-stable treatment of the collision term, allow large time steps, for all the range of Knudsen number [17, 27, 30]. Unfortunately, however, such schemes are not conservative. Lack of conservation is analyzed in detail, and two main sources are identified as its cause. Firstly, when using classical continuous Maxwellian, conservation error is negligible only if velocity space is resolved with sufficiently large number of grid points. However, for a small number of grid points in velocity space such error is not negligible, because the parameters of the Maxwellian do not coincide with the discrete moments. Secondly, the non-linear reconstruction used to prevent oscillations destroys the translation invariance which is at the basis of the conservation properties of the scheme. As a consequence, the schemes show a wrong shock speed in the limit of small Knudsen number. To treat the first problem and ensure machine precision conservation of mass, momentum and energy with a relatively small number of velocity grid points, we replace the continuous Maxwellian with the discrete Maxwellian introduced in [22]. The second problem is treated by implementing a conservative correction procedure based on the flux difference form as in [26]. In this way we can construct conservative semi-Lagrangian schemes which are Asymptotic Preserving (AP) for the underlying Euler limit, as the Knudsen number vanishes. The effectiveness of the proposed scheme is demonstrated by extensive numerical tests.     

Learning curve using robotic surgery

The da Vinci (Intuitive Surgical, Inc., Sunnyvale, CA) surgical system is being used by an increasing number of surgeons across several surgical specialties. The robotic interface is different not only to open surgery, but also to laparoscopy because it involves remote surgical control, stereoscopic vision, and lack of haptic feedback. As the transition is made from traditional open to robotic surgery, factors such as learning of robotic skills, assessment of pro.ciency in robotics, and structured training for urologists in practice and residents assumes importance. Understanding how the robotic surgical technique is learned and how such learning can be best assessed will enable us to de.ne protocols for training and set standards for pro.ciency. Learning curve and surgical dexterity are two parameters that are used to compare surgical learning and training. This article presents the current gold standard for assessing skill training and compares surgical skill acquisition and pro.ciency using conventional laparoscopy and robotic interfaces.     

On Universal Osher-Type Schemes for General Nonlinear Hyperbolic Conservation Laws

This paper is concerned with a new version of the Osher-Solomon Riemann solver and is based on a numerical integration of the path-dependent dissipation matrix. The resulting scheme is much simpler than the original one and is applicable to general hyperbolic conservation laws, while retaining the attractive features of the original solver: the method is entropy-satisfying, differentiable and complete in the sense that it attributes a different numerical viscosity to each characteristic field, in particular to the intermediate ones, since the full eigenstructure of the underlying hyperbolic system is used. To illustrate the potential of the proposed scheme we show applications to the following hyperbolic conservation laws: Euler equations of compressible gasdynamics with ideal gas and real gas equation of state, classical and relativistic MHD equations as well as the equations of nonlinear elasticity. To the knowledge of the authors, apart from the Euler equations with ideal gas, an Osher-type scheme has never been devised before for any of these complicated PDE systems. Since our new general Riemann solver can be directly used as a building block of high order finite volume and discontinuous Galerkin schemes we also show the extension to higher order of accuracy and multiple space dimensions in the new framework of P_NP_M schemes on unstructured meshes recently proposed in [9].     

人工智能对胃肠疾病诊疗的推动作用

高速发展的计算机技术给日常生活及工作带来巨大变化。人工智能是计算机科学的一个分支,是让计算机去行使通常情况下具备智能生命才可能行使的活动,广义的人工智能涵盖机器学习和机器人等等,本文主要聚焦于机器学习与相关的医学领域,深度学习是机器学习中的人工神经网络,卷积神经网络(CNN)是深度神经网络的一种,是在深度神经网络基础上,进一步模仿大脑的视觉皮层构造和视觉活动原理而开发;目前在医疗大数据分析中应用的机器学习方式主要为CNN。在未来数年内,人工智能作为常规工具进入医学图像解读相关的科室是发展趋势。本文主要分享人工智能与生物医学的融合进展,并结合实际案例,重点介绍CNN在胃肠道疾病的病理诊断、影像学诊断及内镜诊断等方面的应用研究现状。     

A Neural Network Approach to Sampling Based Learning Control for Quantum System with Uncertainty

Robust control design for quantum systems with uncertainty is a key task for developing practical quantum technology. In this paper, we apply neural networks to learn the control of a quantum system with uncertainty. By exploiting the auto differentiation function developed for neural network models, our method avoids the manual computation of the gradient of the cost function as required in traditional methods. We implement our method using two algorithms. One uses neural networks to learn both the states and the controls and one uses neural networks to learn only the controls but solve the states by finite difference methods. Both algorithms incorporate the sampling-based learning process into the training of the networks. The performance of the algorithms is evaluated on a practical numerical example, followed by a detailed discussion about the advantage and trade-offs between our method and the other numerical schemes.     

Artificial intelligence for adult spinal deformity: current state and future directions

As we experience a technological revolution unlike any other time in history, spinal surgery as a discipline is poised to undergo a dramatic transformation. As enormous amounts of data become digitized and more readily available, medical professionals approach a critical juncture with respect to how advanced computational techniques may be incorporated into clinical practices. Within neurosurgery, spinal disorders in particular, represent a complex and heterogeneous disease entity that can vary dramatically in its clinical presentation and how it may impact patients’ lives. The spectrum of pathologies is extremely diverse, including many different etiologies such as trauma, oncology, spinal deformity, infection, inflammatory conditions, and degenerative disease among others. The decision to perform spine surgery, especially complex spine surgery, involves several nuances due to the interplay of biomechanical forces, bony composition, neurologic deficits, and the patient's desired goals. Adult spinal deformity as an example is one of the most complex, given its involvement of not only the spine, but rather the entirety of the skeleton in order to appreciate radiographic completeness. With the vast array of variables contributing to spinal disorders, treatment algorithms can vary significantly, and it is very difficult for surgeons to predict how patients will respond to surgery. As such, it will become imperative for spine surgeons to utilize the burgeoning availability of advanced computational tools to process unprecedented amounts of data and provide novel insights into spinal disease. These tools range from predictive models built using machine learning algorithms, to deep learning methods for imaging analysis, to natural language processing that can mine text from electronic medical records or transcribed patient visits – all to better treat the intricacies of spinal disorders. The adoption of such techniques will empower patients and propel spine surgeons into the era of personalized medicine, by allowing clinical plans to be tailored to address individual patients’ needs. This paper, which exists in the context of a larger body of literatutre, provides a comprehensive review of the current state and future of artificial intelligence and machine learning with a particular emphasis on Adult spinal deformity surgery.     

Convergence Rate Analysis for Deep Ritz Method

Using deep neural networks to solve PDEs has attracted a lot of attentions recently. However, why the deep learning method works is falling far behind its empirical success. In this paper, we provide a rigorous numerical analysis on deep Ritz method (DRM) [47] for second order elliptic equations with Neumann boundary conditions. We establish the first nonasymptotic convergence rate in $H^1$ norm for DRM using deep networks with ${\rm ReLU}^2$ activation functions. In addition to providing a theoretical justification of DRM, our study also shed light on how to set the hyperparameter of depth and width to achieve the desired convergence rate in terms of number of training samples. Technically, we derive bound on the approximation error of deep ${\rm ReLU}^2$ network in $C^1$ norm and bound on the Rademacher complexity of the non-Lipschitz composition of gradient norm and ${\rm ReLU}^2$ network, both of which are of independent interest.     

A New Approach of High Order Well-Balanced Finite Volume WENO Schemes and Discontinuous Galerkin Methods for a Class of Hyperbolic Systems with Source Terms

Hyperbolic balance laws have steady state solutions in which the flux gradients are nonzero but are exactly balanced by the source terms. In our earlier work [31–33], we designed high order well-balanced schemes to a class of hyperbolic systems with separable source terms. In this paper, we present a different approach to the same purpose: designing high order well-balanced finite volume weighted essentially non-oscillatory (WENO) schemes and RungeKutta discontinuous Galerkin (RKDG) finite element methods. We make the observation that the traditional RKDG methods are capable of maintaining certain steady states exactly, if a small modification on either the initial condition or the flux is provided. The computational cost to obtain such a well balanced RKDG method is basically the same as the traditional RKDG method. The same idea can be applied to the finite volume WENO schemes. We will first describe the algorithms and prove the well balanced property for the shallow water equations, and then show that the result can be generalized to a class of other balance laws. We perform extensive one and two dimensional simulations to verify the properties of these schemes such as the exact preservation of the balance laws for certain steady state solutions, the non-oscillatory property for general solutions with discontinuities, and the genuine high order accuracy in smooth regions.     

Continuous-Variable Deep Quantum Neural Networks for Flexible Learning of Structured Classical Information

Quantum computation using optical modes has been well-established in its ability to construct deep neural networks. These networks have been shown to be flexible both architecturally as well as in terms of the type of data being processed. We leverage this property of the Continuous-Variable (CV) model to construct stacked single mode networks that are shown to learn structured classical information, while placing no restrictions on the size of the network, and at the same time maintaining its complexity. The hallmark of the CV model is its ability to forge non-linear functions using a set of gates that allows it to remain completely unitary. The proposed model exemplifies that the appropriate photonic hardware can be integrated with present day optical communication systems to meet our information processing requirements. In this paper, using the Strawberry Fields software library on the MNIST dataset of hand-written digits, we demonstrate the adaptability of the network to learn classical information in a multitude of machine learning tasks to very large fidelities.     

Artificial Intelligence and Machine Learning in Lower Extremity Arthroplasty: A Review

BackgroundDriven by the rapid development of big data and processing power, artificial intelligence and machine learning (ML) applications are poised to expand orthopedic surgery frontiers. Lower extremity arthroplasty is uniquely positioned to most dramatically benefit from ML applications given its central role in alternative payment models and the value equation.MethodsIn this report, we discuss the origins and model specifics behind machine learning, consider its progression into healthcare, and present some of its most recent advances and applications in arthroplasty.ResultsA narrative review of artificial intelligence and ML developments is summarized with specific applications to lower extremity arthroplasty, with specific lessons learned from osteoarthritis gait models, joint-specific imaging analysis, and value-based payment models.ConclusionThe advancement and employment of ML provides an opportunity to provide data-driven, high performance medicine that can rapidly improve the science, economics, and delivery of lower extremity arthroplasty.