An Implicit Algorithm of Solving Nonlinear Filtering Problems

Nonlinear filter problems arise in many applications such as communications and signal processing. Commonly used numerical simulation methods include Kalman filter method, particle filter method, etc. In this paper a novel numerical algorithm is constructed based on samples of the current state obtained by solving the state equation implicitly. Numerical experiments demonstrate that our algorithm is more accurate than the Kalman filter and more stable than the particle filter.     

Nonlinear convex and concave relaxations for the solutions of parametric ODEs

Convex and concave relaxations for the parametric solutions of ordinary differential equations (ODEs) are central to deterministic global optimization methods for nonconvex dynamic optimization and open‐loop optimal control problems with control parametrization. Given a general system of ODEs with parameter dependence in the initial conditions and right‐hand sides, this work derives sufficient conditions under which an auxiliary system of ODEs describes convex and concave relaxations of the parametric solutions, pointwise in the independent variable. Convergence results for these relaxations are also established. A fully automatable procedure for constructing an appropriate auxiliary system has been developed previously by the authors. Thus, the developments here lead to an efficient, automatic method for computing convex and concave relaxations for the parametric solutions of a very general class of nonlinear ODEs. The proposed method is presented in detail for a simple example problem. Copyright © 2012 John Wiley & Sons, Ltd.     

Probabilistic High Order Numerical Schemes for Fully Nonlinear Parabolic PDEs

In this paper, we are concerned with probabilistic high order numerical schemes for Cauchy problems of fully nonlinear parabolic PDEs. For such parabolic PDEs, it is shown by Cheridito, Soner, Touzi and Victoir [4] that the associated exact solutions admit probabilistic interpretations, i.e., the solution of a fully nonlinear parabolic PDE solves a corresponding second order forward backward stochastic differential equation (2FBSDEs). Our numerical schemes rely on solving those 2FBSDEs, by extending our previous results [W. Zhao, Y. Fu and T. Zhou, SIAM J. Sci. Comput., 36 (2014), pp. A1731-A1751.]. Moreover, in our numerical schemes, one has the flexibility to choose the associated forward SDE, and a suitable choice can significantly reduce the computational complexity. Various numerical examples including the HJB equations are presented to show the effectiveness and accuracy of the proposed numerical schemes.     

MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs

In this paper, we propose a machine learning approach via model-operator-data network (MOD-Net) for solving PDEs. A MOD-Net is driven by a model to solve PDEs based on operator representation with regularization from data. For linear PDEs, we use a DNN to parameterize the Green’s function and obtain the neural operator to approximate the solution according to the Green’s method. To train the DNN, the empirical risk consists of the mean squared loss with the least square formulation or the variational formulation of the governing equation and boundary conditions. For complicated problems, the empirical risk also includes a few labels, which are computed on coarse grid points with cheap computation cost and significantly improves the model accuracy. Intuitively, the labeled dataset works as a regularization in addition to the model constraints. The MOD-Net solves a family of PDEs rather than a specific one and is much more efficient than original neural operator because few expensive labels are required. We numerically show MOD-Net is very efficient in solving Poisson equation and one-dimensional radiative transfer equation. For nonlinear PDEs, the nonlinear MOD-Net can be similarly used as an ansatz for solving nonlinear PDEs, exemplified by solving several nonlinear PDE problems, such as the Burgers equation.     

Quasi-Optimized Overlapping Schwarz Waveform Relaxation Algorithm for PDEs with Time-Delay

Schwarz waveform relaxation (SWR) algorithm has been investigated deeply and widely for regular time dependent problems. But for time delay problems, complete analysis of the algorithm is rare. In this paper, by using the reaction diffusion equations with a constant discrete delay as the underlying model problem, we investigate the convergence behavior of the overlapping SWR algorithm with Robin transmission condition. The key point of using this transmission condition is to determine a free parameter as well as possible and it is shown that the best choice of the parameter is determined by the solution of a min-max problem, which is more complex than the one arising for regular problems without delay. We propose new notion to solve the min-max problem and obtain a quasi-optimized choice of the parameter, which is shown efficient to accelerate the convergence of the SWR algorithm. Numerical results are provided to validate the theoretical conclusions.     

A Sylvester-Based IMEX Method via Differentiation Matrices for Solving Nonlinear Parabolic Equations

In this paper we describe a new pseudo-spectral method to solve numerically two- and three-dimensional nonlinear diffusion equations over unbounded domains, taking Hermite functions, sinc functions, and rational Chebyshev polynomials as basis functions. The idea is to discretize the equations by means of differentiation matrices and to relate them to Sylvester-type equations by means of a fourth-order implicit-explicit scheme, being of particular interest the treatment of three-dimensional Sylvester equations that we make. The resulting method is easy to understand and express, and can be implemented in a transparent way by means of a few lines of code. We test numerically the three choices of basis functions, showing the convenience of this new approach, especially when rational Chebyshev polynomials are considered.     

An Iterative Discontinuous Galerkin Method for Solving the Nonlinear Poisson Boltzmann Equation

An iterative discontinuous Galerkin (DG) method is proposed to solve the nonlinear Poisson Boltzmann (PB) equation. We first identify a function space in which the solution of the nonlinear PB equation is iteratively approximated through a series of linear PB equations, while an appropriate initial guess and a suitable iterative parameter are selected so that the solutions of linear PB equations are monotone within the identified solution space. For the spatial discretization we apply the direct discontinuous Galerkin method to those linear PB equations. More precisely, we use one initial guess when the Debye parameter λ=O(1), and a special initial guess for λ≪1 to ensure convergence. The iterative parameter is carefully chosen to guarantee the existence, uniqueness, and convergence of the iteration. In particular, iteration steps can be reduced for a variable iterative parameter. Both one and two-dimensional numerical results are carried out to demonstrate both accuracy and capacity of the iterative DG method for both cases of λ=O(1) and λ≪1. The (m+1)th order of accuracy for L² and mth order of accuracy for H¹for P^m elements are numerically obtained.     

A Multi-Scale DNN Algorithm for Nonlinear Elliptic Equations with Multiple Scales

Algorithms based on deep neural networks (DNNs) have attracted increasing attention from the scientific computing community. DNN based algorithms areeasy to implement, natural for nonlinear problems, and have shown great potential toovercome the curse of dimensionality. In this work, we utilize the multi-scale DNN-based algorithm (MscaleDNN) proposed by Liu, Cai and Xu (2020) to solve multi-scaleelliptic problems with possible nonlinearity, for example, the p-Laplacian problem.We improve the MscaleDNN algorithm by a smooth and localized activation function.Several numerical examples of multi-scale elliptic problems with separable or non-separable scales in low-dimensional and high-dimensional Euclidean spaces are usedto demonstrate the effectiveness and accuracy of the MscaleDNN numerical scheme.     

Novel Symplectic Discrete Singular Convolution Method for Hamiltonian PDEs

This paper explores the discrete singular convolution method for Hamiltonian PDEs. The differential matrices corresponding to two delta type kernels of the discrete singular convolution are presented analytically, which have the properties of high-order accuracy, band-limited structure and thus can be excellent candidates for the spatial discretizations for Hamiltonian PDEs. Taking the nonlinear Schrödinger equation and the coupled Schrödinger equations for example, we construct two symplectic integrators combining this kind of differential matrices and appropriate symplectic time integrations, which both have been proved to satisfy the square conservation laws. Comprehensive numerical experiments including comparisons with the central finite difference method, the Fourier pseudospectral method, the wavelet collocation method are given to show the advantages of the new type of symplectic integrators.     

The Bulk-Surface Virtual Element Method for Reaction-Diffusion PDEs: Analysis and Applications

Bulk-surface partial differential equations (BS-PDEs) are prevalent in manyapplications such as cellular, developmental and plant biology as well as in engineering and material sciences. Novel numerical methods for BS-PDEs in three space dimensions (3D) are sparse. In this work, we present a bulk-surface virtual elementmethod (BS-VEM) for bulk-surface reaction-diffusion systems, a form of semilinearparabolic BS-PDEs in 3D. Unlike previous studies in two space dimensions (2D), the3D bulk is approximated with general polyhedra, whose outer faces constitute a flatpolygonal approximation of the surface. For this reason, the method is restricted tothe lowest order case where the geometric error is not dominant. The BS-VEM guarantees all the advantages of polyhedral methods such as easy mesh generation andfast matrix assembly on general geometries. Such advantages are much more relevantthan in 2D. Despite allowing for general polyhedra, general nonlinear reaction kineticsand general surface curvature, the method only relies on nodal values without needing additional evaluations usually associated with the quadrature of general reactionkinetics. This latter is particularly costly in 3D. The BS-VEM as implemented in thisstudy retains optimal convergence of second order in space.     

A Kernel-Free Boundary Integral Method for Variable Coefficients Elliptic PDEs

This work proposes a generalized boundary integral method for variable coefficients elliptic partial differential equations (PDEs), including both boundary value and interface problems. The method is kernel-free in the sense that there is no need to know analytical expressions for kernels of the boundary and volume integrals in the solution of boundary integral equations. Evaluation of a boundary or volume integral is replaced with interpolation of a Cartesian grid based solution, which satisfies an equivalent discrete interface problem, while the interface problem is solved by a fast solver in the Cartesian grid. The computational work involved with the generalized boundary integral method is essentially linearly proportional to the number of grid nodes in the domain. This paper gives implementation details for a second-order version of the kernel-free boundary integral method in two space dimensions and presents numerical experiments to demonstrate the efficiency and accuracy of the method for both boundary value and interface problems. The interface problems demonstrated include those with piecewise constant and large-ratio coefficients and the heterogeneous interface problem, where the elliptic PDEs on two sides of the interface are of different types.     

An Energy-Preserving Wavelet Collocation Method for General Multi-Symplectic Formulations of Hamiltonian PDEs

In this paper, we develop a novel energy-preserving wavelet collocation method for solving general multi-symplectic formulations of Hamiltonian PDEs. Based on the autocorrelation functions of Daubechies compactly supported scaling functions, the wavelet collocation method is conducted for spatial discretization. The obtained semi-discrete system is shown to be a finite-dimensional Hamiltonian system, which has an energy conservation law. Then, the average vector field method is used for time integration, which leads to an energy-preserving method for multi-symplectic Hamiltonian PDEs. The proposed method is illustrated by the nonlinear Schrödinger equation and the Camassa-Holm equation. Since differentiation matrix obtained by the wavelet collocation method is a cyclic matrix, we can apply Fast Fourier transform to solve equations in numerical calculation. Numerical experiments show the high accuracy, effectiveness and conservation properties of the proposed method.     

An Adaptive ANOVA-Based Data-Driven Stochastic Method for Elliptic PDEs with Random Coefficient

In this paper, we present an adaptive, analysis of variance (ANOVA)-based data-driven stochastic method (ANOVA-DSM) to study the stochastic partial differential equations (SPDEs) in the multi-query setting. Our new method integrates the advantages of both the adaptive ANOVA decomposition technique and the data-driven stochastic method. To handle high-dimensional stochastic problems, we investigate the use of adaptive ANOVA decomposition in the stochastic space as an effective dimension-reduction technique. To improve the slow convergence of the generalized polynomial chaos (gPC) method or stochastic collocation (SC) method, we adopt the data-driven stochastic method (DSM) for speed up. An essential ingredient of the DSM is to construct a set of stochastic basis under which the stochastic solutions enjoy a compact representation for a broad range of forcing functions and/or boundary conditions.Our ANOVA-DSM consists of offline and online stages. In the offline stage, the original high-dimensional stochastic problem is decomposed into a series of low-dimensional stochastic subproblems, according to the ANOVA decomposition technique. Then, for each subproblem, a data-driven stochastic basis is computed using the Karhunen-Loève expansion (KLE) and a two-level preconditioning optimization approach. Multiple trial functions are used to enrich the stochastic basis and improve the accuracy. In the online stage, we solve each stochastic subproblem for any given forcing function by projecting the stochastic solution into the data-driven stochastic basis constructed offline. In our ANOVA-DSM framework, solving the original high-dimensional stochastic problem is reduced to solving a series of ANOVA-decomposed stochastic subproblems using the DSM. An adaptive ANOVA strategy is also provided to further reduce the number of the stochastic subproblems and speed up our method. To demonstrate the accuracy and efficiency of our method, numerical examples are presented for one- and two-dimensional elliptic PDEs with random coefficients.     

Calculus of variations approach for state and parameter estimation in switched 1D hyperbolic PDEs

This paper proposes the use of calculus of variations to solve the problem of state and parameter estimation for a class of switched 1‐dimensional hyperbolic partial differential equations coupled with an ordinary differential equation. The term “switched” here refers to a system changing its characteristics according to a switching rule, which may depend on time, parameters of the system, and/or state of the system. The estimation method is based on a smooth approximation of the system dynamics and the use of variational calculus on an augmented Lagrangian cost functional to get the sensitivity with respect to the initial state and some (possibly distributed) parameters of the system. Those sensitivities or variations, together with related adjoint systems, are used as inputs for an optimization algorithm to identify the values of the variables to be estimated. Two examples are provided to demonstrate the effectiveness of the proposed method. The first one is concerned with a switched overland flow model, developed from Saint‐Venant equations and Green‐Ampt law; the second example deals with a switched free traffic flow model based on the Lighthill‐Whitham‐Richards representation, modified by the presence of a relief route.     

On the Convergence of Physics Informed Neural Networks for Linear Second-Order Elliptic and Parabolic Type PDEs

Physics informed neural networks (PINNs) are deep learning based techniques for solving partial differential equations (PDEs) encountered in computational science and engineering. Guided by data and physical laws, PINNs find a neural network that approximates the solution to a system of PDEs. Such a neural network is obtained by minimizing a loss function in which any prior knowledge of PDEs and data are encoded. Despite its remarkable empirical success in one, two or three dimensional problems, there is little theoretical justification for PINNs.As the number of data grows, PINNs generate a sequence of minimizers which correspond to a sequence of neural networks. We want to answer the question: Does the sequence of minimizers converge to the solution to the PDE? We consider two classes of PDEs: linear second-order elliptic and parabolic. By adapting the Schauder approach and the maximum principle, we show that the sequence of minimizers strongly converges to the PDE solution in $C^0$. Furthermore, we show that if each minimizer satisfies the initial/boundary conditions, the convergence mode becomes $H^1$. Computational examples are provided to illustrate our theoretical findings. To the best of our knowledge, this is the first theoretical work that shows the consistency of PINNs.     

The Recursive Formulation of Particular Solutions for Some Elliptic PDEs with Polynomial Source Functions

In this paper we develop an efficient meshless method for solving inhomogeneous elliptic partial differential equations. We first approximate the source function by Chebyshev polynomials. We then focus on how to find a polynomial particular solution when the source function is a polynomial. Through the principle of the method of undetermined coefficients and a proper arrangement of the terms for the polynomial particular solution to be determined, the coefficients of the particular solution satisfy a triangular system of linear algebraic equations. Explicit recursive formulas for the coefficients of the particular solutions are derived for different types of elliptic PDEs. The method is further incorporated into the method of fundamental solutions for solving inhomogeneous elliptic PDEs. Numerical results show that our approach is efficient and accurate.     

A Nonlinear PIC Algorithm for High Frequency Waves in Magnetized Plasmas Based on Gyrocenter Gauge Kinetic Theory

Numerical methods based on gyrocenter gauge kinetic theory are suitable for first principle simulations of high frequency waves in magnetized plasmas. The δf gyrocenter gauge PIC simulation for linear rf wave has been previously realized. In this paper we further develop a full-f nonlinear PIC algorithm appropriate for the nonlinear physics of high frequency waves in magnetized plasmas. Numerical cases of linear rf waves are calculated as a benchmark for the nonlinear GyroGauge code, meanwhile, nonlinear rf-wave phenomena are studied. The technique and advantage of the reduction of the numerical noise in this full-f gyrocenter gauge PIC algorithm are also discussed.     

A Rate of Convergence of Physics Informed Neural Networks for the Linear Second Order Elliptic PDEs

In recent years, physical informed neural networks (PINNs) have been shown to be a powerful tool for solving PDEs empirically. However, numerical analysis of PINNs is still missing. In this paper, we prove the convergence rate to PINNs for the second order elliptic equations with Dirichlet boundary condition, by establishing the upper bounds on the number of training samples, depth and width of the deep neural networks to achieve desired accuracy. The error of PINNs is decomposed into approximation error and statistical error, where the approximation error is given in $C^2$ norm with ReLU$^3$ networks (deep network with activation function max$\{0,x^3\}$) and the statistical error is estimated by Rademacher complexity. We derive the bound on the Rademacher complexity of the non-Lipschitz composition of gradient norm with ReLU$^3$ network, which is of immense independent interest.     

Nonlinear controller for ventricular assist devices

This paper presents the design of a gain-scheduled proportional integral (PI) feedback controller for ventricular assist devices to maintain physiologically motivated perfusion. The selected control objective is to maintain an average differential pressure deltaP between the left ventricle and the aorta. Computer simulations for different pathological conditions, ranging from the normal heart to left heart asystole, and a wide range of physiological scenarios, ranging from rest to strenuous exercise, were used to validate the performance of the controller and the effectiveness of the selected control objective in ensuring physiologically adequate perfusion under different clinical and cardiac demand conditions.     

An Interface-Capturing Regularization Method for Solving the Equations for Two-Fluid Mixtures

Many problems in biology involve gels which are mixtures composed of a polymer network permeated by a fluid solvent (water). The two-fluid model is a widely used approach to described gel mechanics, in which both network and solvent coexist at each point of space and their relative abundance is described by their volume fractions. Each phase is modeled as a continuum with its own velocity and constitutive law. In some biological applications, free boundaries separate regions of gel and regions of pure solvent, resulting in a degenerate network momentum equation where the network volume fraction vanishes. To overcome this difficulty, we develop a regularization method to solve the two-phase gel equations when the volume fraction of one phase goes to zero in part of the computational domain. A small and constant network volume fraction is temporarily added throughout the domain in setting up the discrete linear equations and the same set of equation is solved everywhere. These equations are very poorly conditioned for small values of the regularization parameter, but the multigrid-preconditioned GMRES method we use to solve them is efficient and produces an accurate solution of these equations for the full range of relevant regularization parameter values.