Abstract: One of the main challenges in using deep learning-based methods for simulating physical systems and solving partial differential equations (PDEs) is formulating physics-based data in the desired structure for neural networks. Graph neural networks (GNNs) have gained popularity in this area since graphs offer a natural way of modeling particle interactions and provide a clear way of discretizing the continuum models. However, the graphs constructed for approximating such tasks usually ignore long-range interactions due to unfavorable scaling of the computational complexity with respect to the number of nodes. The errors due to these approximations scale with the discretization of the system, thereby not allowing for generalization under mesh-refinement. Inspired by the classical multipole methods, we purpose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Our multi-level formulation is equivalent to recursively adding inducing points to the kernel matrix, unifying GNNs with multi-resolution matrix factorization of the kernel. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.

Publication: arXiv
ID: CaltechAUTHORS:20201106-120222366

]]>

Abstract: These lecture notes highlight the mathematical and computational structure relating to the formulation of, and development of algorithms for, the Bayesian approach to inverse problems in differential equations. This approach is fundamental in the quantification of uncertainty within applications in volving the blending of mathematical models with data. The finite dimensional situation is described first, along with some motivational examples. Then the development of probability measures on separable Banach space is undertaken, using a random series over an infinite set of functions to construct draws; these probability measures are used as priors in the Bayesian approach to inverse problems. Regularity of draws from the priors is studied in the natural Sobolev or Besov spaces implied by the choice of functions in the random series construction, and the Kolmogorov continuity theorem is used to extend regularity considerations to the space of Hölder continuous functions. Bayes’ theorem is de rived in this prior setting, and here interpreted as finding conditions under which the posterior is absolutely continuous with respect to the prior, and determining a formula for the Radon-Nikodym derivative in terms of the likelihood of the data. Having established the form of the posterior, we then describe various properties common to it in the infinite dimensional setting. These properties include well-posedness, approximation theory, and the existence of maximum a posteriori estimators. We then describe measure-preserving dynamics, again on the infinite dimensional space, including Markov chain-Monte C arlo and sequential Monte Carlo methods, and measure-preserving reversible stochastic differential equations. By formulating the theory and algorithms on the underlying infinite dimensional space, we obtain a framework suitable for rigorous analysis of the accuracy of reconstructions, of computational complexity, as well as naturally constructing algorithms which perform well under mesh refinement, since they are inherently well-defined in infinite dimensions.

ID: CaltechAUTHORS:20161111-104641272

]]>

Abstract: There are many applications where it is desirable to fit reduced stochastic descriptions (e.g. SDEs) to data. These include molecular dynamics (Schlick (2000), Frenkel and Smit (2002)), atmosphere/ocean science (Majda and Kramer (1999)), cellular biology (Alberts et al. (2002)) and econometrics (Dacorogna, Gençay, Miiller, Olsen, and Pictet (2001)). The data arising in these problems often has a multiscale character and may not be compatible with the desired diffusion at small scales (see Givon, Kupferman, and Stuart (2004), Majda, Timofeyev, and Vanden-Eijnden (1999), Kepler and Elston (2001), Zhang, Mykland, and Aft-Sahalia (2005) and Olhede, Sykulski, and Pavliotis (2009)). The question then arises as to how to optimally employ such data to find a useful diffusion approximation.

ID: CaltechAUTHORS:20161111-105740878

]]>

Abstract: {No abstract]

ID: CaltechAUTHORS:20161111-111346824

]]>

Abstract: The need to blend observational data and mathematical models arises in many applications and leads naturally to inverse problems. Parameters appearing in the model, such as constitutive tensors, initial conditions, boundary conditions, and forcing can be estimated on the basis of observed data. The resulting inverse problems are usually ill-posed and some form of regularization is required. These notes discuss parameter estimation in situations where the unknown parameters vary across multiple scales. We illustrate the main ideas using a simple model for groundwater flow. We will highlight various approaches to regularization for inverse problems, including Tikhonov and Bayesian methods. We illustrate three ideas that arise when considering inverse problems in the multiscale context. The first idea is that the choice of space or set in which to seek the solution to the inverse problem is intimately related to whether a homogenized or full multiscale solution is required. This is a choice of regularization. The second idea is that, if a homogenized solution to the inverse problem is what is desired, then this can be recovered from carefully designed observations of the full multiscale system. The third idea is that the theory of homogenization can be used to improve the estimation of homogenized coefficients from multiscale data.

No.: 83
ID: CaltechAUTHORS:20161111-110328030

]]>

Abstract: Applied mathematics is concerned with developing models with predictive capability, and with probing those models to obtain qualitative and quantitative insight into the phenomena being modelled. Statistics is data-driven and is aimed at the development of methodologies to optimize the information derived from data. The increasing complexity of phenomena that scientists and engineers wish to model, together with our increased ability to gather, store and interrogate data, mean that the subjects of applied mathematics and statistics are increasingly required to work in conjunction in order to significantly progress understanding.This article is concerned with a research program at the interface between these two disciplines, aimed at problems in differential equations where profusion of data and the sophisticated model combine to produce the mathematical problem of obtaining information from a probability measure on function space. In this context there is an array of problems with a common mathematical structure, namely that the probability measure in question is a change of measure from a Gaussian. We illustrate the wide-ranging applicability of this structure. For problems whose solution is determined by a probability measure on function space, information about the solution can be obtained by sampling from this probability measure. One way to do this is through the use of Markov chain Monte-Carlo (MCMC) methods. We show how the common mathematical structure of the aforementioned problems can be exploited in the design of effective MCMC methods.

ID: CaltechAUTHORS:20170612-093904953

]]>

Abstract: For many practical problems it is useful to be able to sample conditioned diffusions on a computer (e.g. in filtering/ smoothing to sample from the conditioned distribution of the unknown signal given the known observations). We present a recently developed, SPDE-based method to tackle this problem. The method is an infinite-dimensional generalization of the Langevin sampling technique.

No.: 353
ID: CaltechAUTHORS:20170614-080019284

]]>

Abstract: This article contains an overview of the literature concerning the computational complexity of Metropolis-Hastings based MCMC methods for sampling probability measures on ℝ^d, when the dimension d is large. The material is structured in three parts addressing, in turn, the following questions: (i) what are sensible assumptions to make on the family of probability measures indexed by d? (ii) what is known concerning computational complexity for Metropolis-Hastings methods applied to these families? (iii) what remains open in this area?

ID: CaltechAUTHORS:20170612-102025036

]]>

Abstract: We describe a new numerical technique to estimate Green’s functions of elliptic differential operators on bounded open sets. The algorithm utilizes SPDE based function space sampling techniques in conjunction with Metropolis-Hastings MCMC. The key idea is that neither the proposal nor the acceptance probability require the evaluation of a Dirac measure. The method allows Green’s functions to be estimated via ergodic averaging. Numerical examples in both 1D and 2D, with second and fourth order elliptic PDE’s, are presented to validate this methodology.

ID: CaltechAUTHORS:20161111-113156226

]]>

Abstract: The transport of inertial particles in incompressible flows and subject to molecular diffusion is studied through direct numerical simulations. It was shown in recent work [9, 15] that the long time behavior of inertial particles, with motion governed by Stokes’ law in a periodic velocity field and in the presence of molecular diffusion, is diffusive. The effective diffusivity is defined through the solution of a degenerate elliptic partial differential equation. In this paper we study the dependence of the effective diffusivity on the non-dimensional parameters of the problem, as well as on the streamline topology, for a class of two dimensional periodic incompressible flows.

ID: CaltechAUTHORS:20170613-140425580

]]>

Abstract: The theory of (random) dynamical systems is a framework for the analysis of large time behaviour of time-evolving systems (driven by noise). These notes contain an elementary introduction to the theory of both dynamical and random dynamical systems. The subject matter is made accessible by means of very simple examples and highlights relationships between the deterministic and the random theories.

No.: 75
ID: CaltechAUTHORS:20170613-125750895

]]>

Abstract: The study of numerical methods for initial value problems by considering their approximation properties from a dynamical systems viewpoint is now a well-established field; a substantial body of knowledge, developed over the past two decades, can be found in the literature. Nonetheless many open questions remain concerning the meaning of long-time simulations performed by approximating dynamical systems. In recent years various attempts to analyse the statistical content of these long-time simulations have emerged, and the purpose of this article is to review some of that work. The subject area is far from complete; nonetheless a certain unity can be seen in what has been achieved to date and it is therefore of value to give an overview of the field. Some mathematical background concerning the propagation of probability measures by discrete and continuous time dynamical systems or Markov chains will be given. In particular the Frobenius-Perron and Fokker-Planck operators will be described. Using the notion of ergodicity two different approaches, direct and indirect, will be outlined. The majority of the review is concerned with indirect methods, where the initial value problem is simulated from a single initial condition and the statistical content of this trajectory studied. Three classes of problems will be studied: deterministic problems in fixed finite dimension, stochastic problems in fixed finite dimension, and deterministic problems with random data in dimension n → ∞; in the latter case ideas from statistical mechanics can be exploited to analyse or interpret numerical schemes. Throughout, the ideas are illustrated by simple numerical experiments. The emphasis is on understanding underlying concepts at a high level and mathematical detail will not be given a high priority in this review.

No.: 284
ID: CaltechAUTHORS:20170614-073434784

]]>

Abstract: The study of the running times of algorithms in computer science can be broken down into two broad types: worst-case and average-case analyses. For many problems this distinction is very important as the orders of magnitude (in terms of some measure of the problem size) of the running times may differ significantly in each case, providing useful information about the merits of the algorithm. Historically average-case analyses were first done with respect to a measure on the input data; to counter the argument that it is often difficult to find a natural measure on the data, randomised algorithms were then developed. In this paper similar questions are studied for adaptive software used to integrate initial value problems for ODEs. In worst case these algorithms may fail completely giving O (1) errors. We consider the probability of failure for generic vector fields with random initial data chosen from a ball and perform average-case and worst-case analyses.We then perform a different average-case analysis where, having fixed the initial data, it is the algorithm that is chosen at random from some suitable class.This last analysis suggests a modified deterministic algorithm which cannot fail for generic vector fields.

No.: 118 ISSN: 0940-6573

ID: CaltechAUTHORS:20170613-142949630

]]>

Abstract: In this article we give an overview of the application of theories from dynamical systems to the analysis of numerical methods for initial-value problems. We start by describing the classical viewpoints of numerical analysis and of dynamical systems and then indicate how the two viewpoints can be merged to provide a framework for both the interpretation of data obtained from numerical simulations and the design of efficient numerical methods. This is done in Section 2.

ID: CaltechAUTHORS:20170613-141546199

]]>

Abstract: When considering the effect of perturbations on initial value problems over long time intervals it is not possible, in general, to uniformly approximate individual trajectories. This is because well-posed initial value problems allow exponential divergence of trajectories and this fact is reflected in the error bound relating trajectories of the perturbed and unperturbed problems. In order to interpret data obtained from numerical simulations over long time intervals, and from other forms of perturbations, it is hence often necessary to ask different questions concerning the behavior as the approximation is refined. One possibility, which we concentrate on in this review, is to study the effect of perturbation on sets which are invariant under the evolution equation. Such sets include equilibria, periodic solutions, stable and unstable manifolds, phase portraits, inertial manifolds and attractors; they are crucial to the understanding of long-time dynamics. An abstract semilinear evolution equation in a Hilbert space X is considered, yielding a semigroup S(t) actlng on a subspace V of X. A general class of perturped semigroups S^h(t) are considered which are C^1 close to S(t) uniformly on bounded subsets of V and time intervals [t_1, t_2] with 0 < t_1 < t_2 < ∞. A variety of perturbed problems are shown to satisfy these approximation properties. Examples include a Galerkin method based on the eigenfunctions of the linear part of the abstract sectorial evolution equation, a backward Euler approximation of the same equation and a singular perturbation of the Cahn-Hilliard equation arising from the phase-field model of phase transitions. The invariant sets of S(t) and S^h(t) are compared and convergence properties established.

No.: 4
ID: CaltechAUTHORS:20170613-133150018

]]>

Abstract: In this article the numerical analysis of dissipative semilinear evolution equations with sectorial linear part is reviewed. In particular the approximation theory for such equations over long time intervals is discussed. Emphasis is placed on studying the effect of approximation on certain invariant objects which play an important role in understanding long time dynamics. Specifically the existence of absorbing sets, the upper and lower semicontinuity of global attractors and the existence and convergence of attractive invariant manifolds, such as the inertial manifold and unstable manifolds of equilibrium points, is studied.

No.: 303
ID: CaltechAUTHORS:20170612-152618052

]]>

Abstract: The effect of temporal discretisation on dissipative differential equations is analysed. We discuss the effect of discretisation on the global attractor and survey some recent results in the area. The advantage of concentrating on ω and α limit sets (which are contained in the global attractor) is described. An analysis of spurious bifurcations in the ω and α limit sets is presented for linear multistep methods, using the time-step Δtas the bifurcation parameter. The results arising from application of local bifurcation theory are shown to hold globally and a necessary and sufficient condition is derived for the non-existence of a particular class of spurious solutions, for allΔt> 0. The class of linear multistep methods satisfying this condition is fairly restricted since the underlying theory is very general and takes no account of any inherent structure in the underlying differential equations. Hence a method complementary to the bifurcation analysis is described, the aim being to construct methods for which spurious solutions do not exist forΔt sufficiently small; for infinite dimensional dynamical systems the method relies on examining steady boundary value problems (which govern the existence of spurious solutions) in the singular limit corresponding to Δt→ 0_+. The analysis we describe is helpful in the design of schemes for long-time simulations

No.: 313
ID: CaltechAUTHORS:20170612-140046210

]]>

Abstract: Two partial differential equations arising from the theory of porous medium combustion are examined. While both equations possess a trivial steady solution, the form of the reaction rate, which is discontinuous as a function of the dependent variable, precludes bifurcation of non-trivial steady solutions from the branch of trivial solutions. A constructive approach to the existence theory for non-trivial global solution branches is developed. The method relies on finding an appropriate set of solution dependent transformations which render the problems in a form to which local bifurcation theory is directly applicable. Specifically, by taking a singular limit of the (solution dependent) transformation, an artificial trivial solution (or set of solutions) of the transformed problem is created. The (solution dependent) mapping is not invertible when evaluated at the trivial solution(s) of the transformed problem; however, for non-trivial solutions which exist arbitrarily close to the artificial trivial solution, the mapping is invertible. By applying local bifurcation theory to the transformed problem and mapping back to the original problem, a series expansion for the non-trivial solution branch is obtained.

No.: 13
ID: CaltechAUTHORS:20170612-132031395

]]>