Abstract: We present a Gaussian process (GP) approach, called Gaussian process hydrodynamics (GPH) for approximating the solution to the Euler and Navier-Stokes (NS) equations. Similar to smoothed particle hydrodynamics (SPH), GPH is a Lagrangian particle-based approach that involves the tracking of a finite number of particles transported by a flow. However, these particles do not represent mollified particles of matter but carry discrete/partial information about the continuous flow. Closure is achieved by placing a divergence-free GP prior ξ on the velocity field and conditioning it on the vorticity at the particle locations. Known physics (e.g., the Richardson cascade and velocity increment power laws) is incorporated into the GP prior by using physics-informed additive kernels. This is equivalent to expressing ξ as a sum of independent GPs ξl, which we call modes, acting at different scales (each mode ξl self-activates to represent the formation of eddies at the corresponding scales). This approach enables a quantitative analysis of the Richardson cascade through the analysis of the activation of these modes, and enables us to analyze coarse-grain turbulence statistically rather than deterministically. Because GPH is formulated by using the vorticity equations, it does not require solving a pressure equation. By enforcing incompressibility and fluid-structure boundary conditions through the selection of a kernel, GPH requires significantly fewer particles than SPH. Because GPH has a natural probabilistic interpretation, the numerical results come with uncertainty estimates, enabling their incorporation into an uncertainty quantification (UQ) pipeline and adding/removing particles (quanta of information) in an adapted manner. The proposed approach is suitable for analysis because it inherits the complexity of state-of-the-art solvers for dense kernel matrices and results in a natural definition of turbulence as information loss. Numerical experiments support the importance of selecting physics-informed kernels and illustrate the major impact of such kernels on the accuracy and stability. Because the proposed approach uses a Bayesian interpretation, it naturally enables data assimilation and predictions and estimations by mixing simulation data and experimental data.

Publication: Applied Mathematics and MechanicsISSN: 0253-4827

ID: CaltechAUTHORS:20230502-759894900.1

]]>

Abstract: This technical note presents an extension of kernel model decomposition (KMD) for detecting critical transitions in some fast–slow random dynamical systems. The approach rests upon modifying KMD for reconstructing an observable by using a novel data-based time-frequency-phase kernel that allows to approximate signals with critical transitions. In particular, we apply the developed method for approximating the solution and detecting critical transitions in some prototypical slow–fast SDEs with critical transitions. We also apply it to detecting seizures in a multi-scale mesoscale nine-dimensional SDE model of brain activity.

Publication: Physica A Vol.: 616ISSN: 0378-4371

ID: CaltechAUTHORS:20230605-334829000.12

]]>

Abstract: We introduce a Gaussian Process (GP) generalization of ResNets (with unknown functions of the network replaced by GPs and identified via MAP estimation), which includes ResNets (trained with L₂ regularization on weights and biases) as a particular case (when employing particular kernels). We show that ResNets (and their warping GP regression extension) converge, in the infinite depth limit, to a generalization of image registration variational algorithms. In this generalization, images are replaced by functions mapping input/output spaces to a space of unexpressed abstractions (ideas), and material points are replaced by data points. Whereas computational anatomy aligns images via warping of the material space, this generalization aligns ideas (or abstract shapes as in Plato’s theory of forms) via the warping of the Reproducing Kernel Hilbert Space (RKHS) of functions mapping the input space to the output space. While the Hamiltonian interpretation of ResNets is not new, it was based on an Ansatz. We do not rely on this Ansatz and present the first rigorous proof of convergence of ResNets with trained weights and biases towards a Hamiltonian dynamics driven flow. Since our proof is constructive and based on discrete and continuous mechanics, it reveals several remarkable properties of ResNets and their GP generalization. ResNets regressors are kernel regressors with data-dependent warping kernels. Minimizers of L₂ regularized ResNets satisfy a discrete least action principle implying the near preservation of the norm of weights and biases across layers. The trained weights of ResNets with scaled/strong L² regularization can be identified by solving an autonomous Hamiltonian system. The trained ResNet parameters are unique up to (a function of) the initial momentum, and the initial momentum representation of those parameters is generally sparse. The kernel (nugget) regularization strategy provides a provably robust alternative to Dropout for ANNs. We introduce a functional generalization of GPs and show that pointwise GP/RKHS error estimates lead to probabilistic and deterministic generalization error estimates for ResNets. When performed with feature maps, the proposed analysis identifies the (EPDiff) mean fields limit of trained ResNet parameters as the number of data points goes to infinity. The search for good architectures can be reduced to that of good kernels, and we show that the composition of warping regression blocks with reduced equivariant multichannel kernels (introduced here) recovers and generalizes CNNs to arbitrary spaces and groups of transformations.

Publication: Physica D Vol.: 444ISSN: 0167-2789

ID: CaltechAUTHORS:20230307-205876300.5

]]>

Abstract: We consider the problem of learning Stochastic Differential Equations of the form dXₜ = f (Xₜ)dₜ + σ(Xₜ)dWₜ from one sample trajectory. This problem is more challenging than learning deterministic dynamical systems because one sample trajectory only provides indirect information on the unknown functions f, σ, and stochastic process dWₜ representing the drift, the diffusion, and the stochastic forcing terms, respectively. We propose a method that combines Computational Graph Completion [1] and data adapted kernels learned via a new variant of cross validation. Our approach can be decomposed as follows: (1) Represent the time-increment map Xₜ → X_(t+dt) as a Computational Graph in which f, σ and dWₜ appear as unknown functions and random variables. (2) Complete the graph (approximate unknown functions and random variables) via Maximum a Posteriori Estimation (given the data) with Gaussian Process (GP) priors on the unknown functions. (3) Learn the covariance functions (kernels) of the GP priors from data with randomized cross-validation. Numerical experiments illustrate the efficacy, robustness, and scope of our method.

Publication: Physica D Vol.: 444ISSN: 0167-2789

ID: CaltechAUTHORS:20230307-205876300.8

]]>

Abstract: A simple and interpretable way to learn a dynamical system from data is to interpolate its vector-field with a kernel. In particular, this strategy is highly efficient (both in terms of accuracy and complexity) when the kernel is data-adapted using Kernel Flows (KF) (Owhadi and Yoo, 2019) (which uses gradient-based optimization to learn a kernel based on the premise that a kernel is good if there is no significant loss in accuracy if half of the data is used for interpolation). Despite its previous successes, this strategy (based on interpolating the vector field driving the dynamical system) breaks down when the observed time series is not regularly sampled in time. In this work, we propose to address this problem by approximating a generalization of the flow map of the dynamical system by incorporating time differences between observations in the (KF) data-adapted kernels. We compare our approach with the classical one over different benchmark dynamical systems and show that it significantly improves the forecasting accuracy while remaining simple, fast, and robust.

Publication: Physica D Vol.: 443ISSN: 0167-2789

ID: CaltechAUTHORS:20221122-564647900.3

]]>

Abstract: This paper introduces algorithms to select/design kernels in Gaussian process regression/kriging surrogate modeling techniques. We adopt the setting of kernel method solutions in ad hoc functional spaces, namely Reproducing Kernel Hilbert Spaces (RKHS), to solve the problem of approximating a regular target function given observations of it, i.e. supervised learning. A first class of algorithms is kernel flow, which was introduced in the context of classification in machine learning. It can be seen as a cross-validation procedure whereby a "best" kernel is selected such that the loss of accuracy incurred by removing some part of the dataset (typically half of it) is minimized. A second class of algorithms is called spectral kernel ridge regression, and aims at selecting a "best" kernel such that the norm of the function to be approximated is minimal in the associated RKHS. Within Mercer's theorem framework, we obtain an explicit construction of that "best" kernel in terms of the main features of the target function. Both approaches of learning kernels from data are illustrated by numerical examples on synthetic test functions, and on a classical test case in turbulence modeling validation for transonic flows about a two-dimensional airfoil.

Publication: Journal of Computational Physics Vol.: 470ISSN: 0021-9991

ID: CaltechAUTHORS:20221020-727675300.1

]]>

Abstract: Training a residual neural network with L^2 regularization on weights and biases is equivalent to minimizing a discrete least action principle and to controlling a discrete Hamiltonian system representing the propagation of input data across layers. The kernel/feature map analysis of this Hamiltonian system suggests a mean-field limit for trained weights and biases as the number of data points goes to infinity. The purpose of this paper is to investigate this mean-field limit and illustrate its existence through numerical experiments and analysis (for simple kernels).

Publication: Dolomites Research Notes on Approximation Vol.: 15 No.: 3 ISSN: 2035-6803

ID: CaltechAUTHORS:20230603-041912328

]]>

Abstract: We introduce a framework for generating, organizing, and reasoning with computational knowledge. It is motivated by the observation that most problems in Computational Sciences and Engineering (CSE) can be described as that of completing (from data) a computational graph (or hypergraph) representing dependencies between functions and variables. In that setting nodes represent variables and edges (or hyperedges) represent functions (or functionals). Functions and variables may be known, unknown, or random. Data come in the form of observations of distinct values of a finite number of subsets of the variables of the graph (satisfying its functional dependencies). The underlying problem combines a regression problem (approximating unknown functions) with a matrix completion problem (recovering unobserved variables in the data). Replacing unknown functions by Gaussian processes and conditioning on observed data provides a simple but efficient approach to completing such graphs. Since the proposed framework is highly expressive, it has a vast potential application scope. Since the completion process can be automatized, as one solves √√2+√3 on a pocket calculator without thinking about it, one could, with the proposed framework, solve a complex CSE problem by drawing a diagram. Compared to traditional regression/kriging, the proposed framework can be used to recover unknown functions with much scarcer data by exploiting interdependencies between multiple functions and variables. The computational graph completion (CGC) problem addressed by the proposed framework could therefore also be interpreted as a generalization of that of solving linear systems of equations to that of approximating unknown variables and functions with noisy, incomplete, and nonlinear dependencies. Numerous examples illustrate the flexibility, scope, efficacy, and robustness of the CGC framework and show how it can be used as a pathway to identifying simple solutions to classical CSE problems. These examples include the seamless CGC representation of known methods (for solving/learning PDEs, surrogate/multiscale modeling, mode decomposition, deep learning) and the discovery of new ones (digital twin modeling, dimension reduction).

Publication: Research in the Mathematical Sciences Vol.: 9 No.: 2 ISSN: 2522-0144

ID: CaltechAUTHORS:20220511-647768900

]]>

Abstract: We introduce a simple, rigorous, and unified framework for solving nonlinear partial differential equations (PDEs), and for solving inverse problems (IPs) involving the identification of parameters in PDEs, using the framework of Gaussian processes. The proposed approach: (1) provides a natural generalization of collocation kernel methods to nonlinear PDEs and IPs; (2) has guaranteed convergence for a very general class of PDEs, and comes equipped with a path to compute error bounds for specific PDE approximations; (3) inherits the state-of-the-art computational complexity of linear solvers for dense kernel matrices. The main idea of our method is to approximate the solution of a given PDE as the maximum a posteriori (MAP) estimator of a Gaussian process conditioned on solving the PDE at a finite number of collocation points. Although this optimization problem is infinite-dimensional, it can be reduced to a finite-dimensional one by introducing additional variables corresponding to the values of the derivatives of the solution at collocation points; this generalizes the representer theorem arising in Gaussian process regression. The reduced optimization problem has the form of a quadratic objective function subject to nonlinear constraints; it is solved with a variant of the Gauss–Newton method. The resulting algorithm (a) can be interpreted as solving successive linearizations of the nonlinear PDE, and (b) in practice is found to converge in a small number of iterations (2 to 10), for a wide range of PDEs. Most traditional approaches to IPs interleave parameter updates with numerical solution of the PDE; our algorithm solves for both parameter and PDE solution simultaneously. Experiments on nonlinear elliptic PDEs, Burgers' equation, a regularized Eikonal equation, and an IP for permeability identification in Darcy flow illustrate the efficacy and scope of our framework.

Publication: Journal of Computational Physics Vol.: 447ISSN: 0021-9991

ID: CaltechAUTHORS:20210719-210146136

]]>

Abstract: In statistical decision theory, a model is said to be Pareto optimal (or admissible) if no other model carries less risk for at least one state of nature while presenting no more risk for others. How can you rationally aggregate/combine a finite set of Pareto optimal models while preserving Pareto efficiency? This question is nontrivial because weighted model averaging does not, in general, preserve Pareto efficiency. This paper presents an answer in four logical steps: (1) A rational aggregation rule should preserve Pareto efficiency (2) Due to the complete class theorem, Pareto optimal models must be Bayesian, i.e., they minimize a risk where the true state of nature is averaged with respect to some prior. Therefore each Pareto optimal model can be associated with a prior, and Pareto efficiency can be maintained by aggregating Pareto optimal models through their priors. (3) A prior can be interpreted as a preference ranking over models: prior π prefers model A over model B if the average risk of A is lower than the average risk of B. (4) A rational/consistent aggregation rule should preserve this preference ranking: If both priors π and π′ prefer model A over model B, then the prior obtained by aggregating π and π′ must also prefer A over B. Under these four steps, we show that all rational/consistent aggregation rules are as follows: Give each individual Pareto optimal model a weight, introduce a weak order/ranking over the set of Pareto optimal models, aggregate a finite set of models S as the model associated with the prior obtained as the weighted average of the priors of the highest-ranked models in S. This result shows that all rational/consistent aggregation rules must follow a generalization of hierarchical Bayesian modeling. Following our main result, we present applications to Kernel smoothing, time-depreciating models, and voting mechanisms.

Publication: arXiv
ID: CaltechAUTHORS:20220524-180318744

]]>

Abstract: A simple and interpretable way to learn a dynamical system from data is to interpolate its vector-field with a kernel. In particular, this strategy is highly efficient (both in terms of accuracy and complexity) when the kernel is data-adapted using Kernel Flows (KF) [34] (which uses gradient-based optimization to learn a kernel based on the premise that a kernel is good if there is no significant loss in accuracy if half of the data is used for interpolation). Despite its previous successes, this strategy (based on interpolating the vector field driving the dynamical system) breaks down when the observed time series is not regularly sampled in time. In this work, we propose to address this problem by directly approximating the vector field of the dynamical system by incorporating time differences between observations in the (KF) data-adapted kernels. We compare our approach with the classical one over different benchmark dynamical systems and show that it significantly improves the forecasting accuracy while remaining simple, fast, and robust.

Publication: arXiv
ID: CaltechAUTHORS:20220524-180315371

]]>

Abstract: A natural notion of rationality/consistency for aggregating models is that, for all (possibly aggregated) models A and B, if the output of model A is f(A) and if the output model B is f(B), then the output of the model obtained by aggregating A and B must be a weighted average of f(A) and f(B). Similarly, a natural notion of rationality for aggregating preferences of ensembles of experts is that, for all (possibly aggregated) experts A and B, and all possible choices x and y, if both A and B prefer x over y, then the expert obtained by aggregating A and B must also prefer x over y. Rational aggregation is an important element of uncertainty quantification, and it lies behind many seemingly different results in economic theory: spanning social choice, belief formation, and individual decision making. Three examples of rational aggregation rules are as follows. (1) Give each individual model (expert) a weight (a score) and use weighted averaging to aggregate individual or finite ensembles of models (experts). (2) Order/rank individual model (expert) and let the aggregation of a finite ensemble of individual models (experts) be the highest-ranked individual model (expert) in that ensemble. (3) Give each individual model (expert) a weight, introduce a weak order/ranking over the set of models/experts, aggregate A and B as the weighted average of the highest-ranked models (experts) in A or B. Note that (1) and (2) are particular cases of (3). In this paper, we show that all rational aggregation rules are of the form (3). This result unifies aggregation procedures across different economic environments. Following the main representation, we show applications and extensions of our representation in various separated economics topics such as belief formation, choice theory, and social welfare economics.

Publication: arXiv
ID: CaltechAUTHORS:20220524-180312022

]]>

Abstract: We introduce a new regularization method for Artificial Neural Networks (ANNs) based on the Kernel Flow (KF) algorithm. The algorithm was introduced in Owhadi and Yoo (2019) as a method for kernel selection in regression/kriging based on the minimization of the loss of accuracy incurred by halving the number of interpolation points in random batches of the dataset. Writing f_θ(x) = (f^((n))_(θn)∘f^((n−1))_(θn−1)∘⋯∘f^((1))_(θ₁))(x) for the functional representation of compositional structure of the ANN (where θ_i are the weights and biases of the layer i), the inner layers outputs h^((i))(x) = (f^((i))_(θi)∘f^((i−1))_(θi−1)∘⋯∘f^((1))_(θ1))(x) define a hierarchy of feature maps and a hierarchy of kernels k^((i))(x,x′) = exp(−γ_i∥h^((i))(x)−h^((i))(x′)∥²₂). When combined with a batch of the dataset, these kernels produce KF losses e(i)₂ (defined as the L² regression error incurred by using a random half of the batch to predict the other half) depending on the parameters of the inner layers θ₁,…,θ_i (and γ_i). The proposed method simply consists of aggregating (as a weighted sum) a subset of these KF losses with a classical output loss (e.g., cross-entropy). We test the proposed method on Convolutional Neural Networks (CNNs) and Wide Residual Networks (WRNs) without alteration of their structure nor their output classifier and report reduced test errors, decreased generalization gaps, and increased robustness to distribution shift without a significant increase in computational complexity relative to standard CNN and WRN training (with Drop Out and Batch Normalization). We suspect that these results might be explained by the fact that while conventional training only employs a linear functional (a generalized moment) of the empirical distribution defined by the dataset and can be prone to trapping in the Neural Tangent Kernel regime (under over-parameterizations), the proposed loss function (defined as a nonlinear functional of the empirical distribution) effectively trains the underlying kernel defined by the CNN beyond regressing the data with that kernel.

Publication: Physica D: Nonlinear Phenomena Vol.: 426ISSN: 0167-2789

ID: CaltechAUTHORS:20201110-075343797

]]>

Abstract: There are essentially three kinds of approaches to Uncertainty Quantification (UQ): (A) robust optimization, (B) Bayesian, (C) decision theory. Although (A) is robust, it is unfavorable with respect to accuracy and data assimilation. (B) requires a prior, it is generally brittle and posterior estimations can be slow. Although (C) leads to the identification of an optimal prior, its approximation suffers from the curse of dimensionality and the notion of risk is one that is averaged with respect to the distribution of the data. We introduce a 4th kind which is a hybrid between (A), (B), (C), and hypothesis testing. It can be summarized as, after observing a sample x, (1) defining a likelihood region through the relative likelihood and (2) playing a minmax game in that region to define optimal estimators and their risk. The resulting method has several desirable properties (a) an optimal prior is identified after measuring the data, and the notion of risk is a posterior one, (b) the determination of the optimal estimate and its risk can be reduced to computing the minimum enclosing ball of the image of the likelihood region under the quantity of interest map (which is fast and not subject to the curse of dimensionality). The method is characterized by a parameter in [0,1] acting as an assumed lower bound on the rarity of the observed data (the relative likelihood). When that parameter is near 1, the method produces a posterior distribution concentrated around a maximum likelihood estimate with tight but low confidence UQ estimates. When that parameter is near 0, the method produces a maximal risk posterior distribution with high confidence UQ estimates. In addition to navigating the accuracy-uncertainty tradeoff, the proposed method addresses the brittleness of Bayesian inference by navigating the robustness-accuracy tradeoff associated with data assimilation.

Publication: arXiv
ID: CaltechAUTHORS:20220524-180308552

]]>

Abstract: Modelling geophysical processes as low-dimensional dynamical systems and regressing their vector field from data is a promising approach for learning emulators of such systems. We show that when the kernel of these emulators is also learned from data (using kernel flows, a variant of cross-validation), then the resulting data-driven models are not only faster than equation-based models but are easier to train than neural networks such as the long short-term memory neural network. In addition, they are also more accurate and predictive than the latter. When trained on geophysical observational data, for example the weekly averaged global sea-surface temperature, considerable gains are also observed by the proposed technique in comparison with classical partial differential equation-based models in terms of forecast computational cost and accuracy. When trained on publicly available re-analysis data for the daily temperature of the North American continent, we see significant improvements over classical baselines such as climatology and persistence-based forecast techniques. Although our experiments concern specific examples, the proposed approach is general, and our results support the viability of kernel methods (with learned kernels) for interpretable and computationally efficient geophysical forecasting for a large diversity of processes.

Publication: Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences Vol.: 477 No.: 2252 ISSN: 1364-5021

ID: CaltechAUTHORS:20210719-210203260

]]>

Abstract: Regressing the vector field of a dynamical system from a finite number of observed states is a natural way to learn surrogate models for such systems. We present variants of cross-validation (Kernel Flows (Owhadi and Yoo, 2019) and its variants based on Maximum Mean Discrepancy and Lyapunov exponents) as simple approaches for learning the kernel used in these emulators.

Publication: Physica D Vol.: 421ISSN: 0167-2789

ID: CaltechAUTHORS:20210208-140557660

]]>

Abstract: Gaussian process regression has proven very powerful in statistics, machine learning and inverse problems. A crucial aspect of the success of this methodology, in a wide range of applications to complex and real-world problems, is hierarchical modeling and learning of hyperparameters. The purpose of this paper is to study two paradigms of learning hierarchical parameters: one is from the probabilistic Bayesian perspective, in particular, the empirical Bayes approach that has been largely used in Bayesian statistics; the other is from the deterministic and approximation theoretic view, and in particular the kernel flow algorithm that was proposed recently in the machine learning literature. Analysis of their consistency in the large data limit, as well as explicit identification of their implicit bias in parameter learning, are established in this paper for a Matérn-like model on the torus. A particular technical challenge we overcome is the learning of the regularity parameter in the Matérn-like field, for which consistency results have been very scarce in the spatial statistics literature. Moreover, we conduct extensive numerical experiments beyond the Matérn-like model, comparing the two algorithms further. These experiments demonstrate learning of other hierarchical parameters, such as amplitude and lengthscale; they also illustrate the setting of model misspecification in which the kernel flow approach could show superior performance to the more traditional empirical Bayes approach.

Publication: Mathematics of Computation Vol.: 90ISSN: 0025-5718

ID: CaltechAUTHORS:20201109-141002843

]]>

Abstract: We propose to compute a sparse approximate inverse Cholesky factor L of a dense covariance matrix Θ by minimizing the Kullback--Leibler divergence between the Gaussian distributions N(0,Θ) and N(0,L^(−⊤)L⁻¹), subject to a sparsity constraint. Surprisingly, this problem has a closed-form solution that can be computed efficiently, recovering the popular Vecchia approximation in spatial statistics. Based on recent results on the approximate sparsity of inverse Cholesky factors of Θ obtained from pairwise evaluation of Green's functions of elliptic boundary-value problems at points {x_i}_(1≤i≤N) ⊂ ℝ^d, we propose an elimination ordering and sparsity pattern that allows us to compute ϵ-approximate inverse Cholesky factors of such Θ in computational complexity O(N log(N/ϵ)^d) in space and O(N log(N/ϵ)^(2d)) in time. To the best of our knowledge, this is the best asymptotic complexity for this class of problems. Furthermore, our method is embarrassingly parallel, automatically exploits low-dimensional structure in the data, and can perform Gaussian-process regression in linear (in N) space complexity. Motivated by its optimality properties, we propose applying our method to the joint covariance of training and prediction points in Gaussian-process regression, greatly improving stability and computational cost. Finally, we show how to apply our method to the important setting of Gaussian processes with additive noise, compromising neither accuracy nor computational complexity.

Publication: SIAM Journal on Scientific Computing Vol.: 43 No.: 3 ISSN: 1064-8275

ID: CaltechAUTHORS:20201109-155534680

]]>

Abstract: Dense kernel matrices Θ∈R^(N×N) obtained from point evaluations of a covariance function G at locations {x_i}_(1≤i≤N)⊂ ℝ^d arise in statistics, machine learning, and numerical analysis. For covariance functions that are Green's functions of elliptic boundary value problems and homogeneously distributed sampling points, we show how to identify a subset S⊂{1,…,N}², with #S=O(N log(N)log^d(N/ϵ)), such that the zero fill-in incomplete Cholesky factorization of the sparse matrix Θ_(i,j)1_((i,j)∈S) is an ϵ-approximation of Θ. This factorization can provably be obtained in complexity O(N log(N) log^d(N/ϵ)) in space and O(N log²(N) log^(2d))(N/ϵ)) in time, improving upon the state of the art for general elliptic operators; we further present numerical evidence that d can be taken to be the intrinsic dimension of the data set rather than that of the ambient space. The algorithm only needs to know the spatial configuration of the x_i and does not require an analytic representation of G. Furthermore, this factorization straightforwardly provides an approximate sparse PCA with optimal rate of convergence in the operator norm. Hence, by using only subsampling and the incomplete Cholesky factorization, we obtain, at nearly linear complexity, the compression, inversion, and approximate PCA of a large class of covariance matrices. By inverting the order of the Cholesky factorization we also obtain a solver for elliptic PDE with complexity O (N log^d(N/ϵ)) in space and O (N log^(2d)(N/ϵ)) in time, improving upon the state of the art for general elliptic operators.

Publication: Multiscale Modeling and Simulation Vol.: 19 No.: 2 ISSN: 1540-3459

ID: CaltechAUTHORS:20170710-083743909

]]>

Abstract: The design and testing of supervised machine learning models combine two fundamental distributions: (1) the training data distribution (2) the testing data distribution. Although these two distributions are identical and identifiable when the data set is infinite; they are imperfectly known (and possibly distinct) when the data is finite (and possibly corrupted) and this uncertainty must be taken into account for robust Uncertainty Quantification (UQ). We present a general decision-theoretic bootstrapping solution to this problem: (1) partition the available data into a training subset and a UQ subset (2) take m subsampled subsets of the training set and train m models (3) partition the UQ set into n sorted subsets and take a random fraction of them to define n corresponding empirical distributions μ_j (4) consider the adversarial game where Player I selects a model i∈{1,…,m}, Player II selects the UQ distribution μ_j and Player I receives a loss defined by evaluating the model i against data points sampled from μ_j (5) identify optimal mixed strategies (probability distributions over models and UQ distributions) for both players. These randomized optimal mixed strategies provide optimal model mixtures and UQ estimates given the adversarial uncertainty of the training and testing distributions represented by the game. The proposed approach provides (1) some degree of robustness to distributional shift in both the distribution of training data and that of the testing data (2) conditional probability distributions on the output space forming aleatory representations of the uncertainty on the output as a function of the input variable.

Publication: arXiv
ID: CaltechAUTHORS:20210323-130821498

]]>

Abstract: Modeling geophysical processes as low-dimensional dynamical systems and regressing their vector field from data is a promising approach for learning emulators of such systems. We show that when the kernel of these emulators is also learned from data (using kernel flows, a variant of cross-validation), then the resulting data-driven models are not only faster than equation-based models but are easier to train than neural networks such as the long short-term memory neural network. In addition, they are also more accurate and predictive than the latter. When trained on geophysical observational data, for example, the weekly averaged global sea-surface temperature, considerable gains are also observed by the proposed technique in comparison to classical partial differential equation-based models in terms of forecast computational cost and accuracy. When trained on publicly available re-analysis data for the daily temperature of the North-American continent, we see significant improvements over classical baselines such as climatology and persistence-based forecast techniques. Although our experiments concern specific examples, the proposed approach is general, and our results support the viability of kernel methods (with learned kernels) for interpretable and computationally efficient geophysical forecasting for a large diversity of processes.

Publication: arXiv
ID: CaltechAUTHORS:20220524-180305206

]]>

Abstract: We show that ResNets converge, in the infinite depth limit, to a generalization of image registration algorithms. In this generalization, images are replaced by abstractions (ideas) living in high dimensional RKHS spaces, and material points are replaced by data points. Whereas computational anatomy aligns images via deformations of the material space, this generalization aligns ideas by via transformations of their RKHS. This identification of ResNets as idea registration algorithms has several remarkable consequences. The search for good architectures can be reduced to that of good kernels, and we show that the composition of idea registration blocks with reduced equivariant multi-channel kernels (introduced here) recovers and generalizes CNNs to arbitrary spaces and groups of transformations. Minimizers of L2 regularized ResNets satisfy a discrete least action principle implying the near preservation of the norm of weights and biases across layers. The parameters of trained ResNets can be identified as solutions of an autonomous Hamiltonian system defined by the activation function and the architecture of the ANN. Momenta variables provide a sparse representation of the parameters of a ResNet. The registration regularization strategy provides a provably robust alternative to Dropout for ANNs. Pointwise RKHS error estimates lead to deterministic error estimates for ANNs.

Publication: arXiv
ID: CaltechAUTHORS:20201109-155524397

]]>

Abstract: Regressing the vector field of a dynamical system from a finite number of observed states is a natural way to learn surrogate models for such systems. We present variants of cross-validation (Kernel Flows [31] and its variants based on Maximum Mean Discrepancy and Lyapunov exponents) as simple approaches for learning the kernel used in these emulators.

Publication: arXiv
ID: CaltechAUTHORS:20201109-155527819

]]>

Abstract: Constrained competitive optimization involves multiple agents trying to minimize conflicting objectives, subject to constraints. This is a highly expressive modeling language that subsumes most of modern machine learning. In this work we propose competitive mirror descent (CMD): a general method for solving such problems based on first order information that can be obtained by automatic differentiation. First, by adding Lagrange multipliers, we obtain a simplified constraint set with an associated Bregman potential. At each iteration, we then solve for the Nash equilibrium of a regularized bilinear approximation of the full problem to obtain a direction of movement of the agents. Finally, we obtain the next iterate by following this direction according to the dual geometry induced by the Bregman potential. By using the dual geometry we obtain feasible iterates despite only solving a linear system at each iteration, eliminating the need for projection steps while still accounting for the global nonlinear structure of the constraint set. As a special case we obtain a novel competitive multiplicative weights algorithm for problems on the positive cone.

Publication: arXiv
ID: CaltechAUTHORS:20201106-120218966

]]>

Abstract: We present a method for the fast computation of the eigenpairs of a bijective positive symmetric linear operator L. The method is based on a combination of operator adapted wavelets (gamblets) with hierarchical subspace correction. First, gamblets provide a raw but fast approximation of the eigensubspaces of L by block-diagonalizing L into sparse and well-conditioned blocks. Next, the hierarchical subspace correction method computes the eigenpairs associated with the Galerkin restriction of L to a coarse (low-dimensional) gamblet subspace and then corrects those eigenpairs by solving a hierarchy of linear problems in the finer gamblet subspaces (from coarse to fine, using multigrid iteration). The proposed algorithm is robust to the presence of multiple (a continuum of) scales and is shown to be of near-linear complexity when L is an (arbitrary local, e.g., differential) operator mapping H^s₀(Ω) to H^(−s)(Ω) (e.g., an elliptic PDE with rough coefficients).

Publication: SIAM Journal on Numerical Analysis Vol.: 57 No.: 6 ISSN: 0036-1429

ID: CaltechAUTHORS:20190923-154306921

]]>

Abstract: Donoho and Johnstone (Ann Stat 26(3):879–921, 1998) proposed a method from reconstructing an unknown smooth function u from noisy data u+ζ by translating the empirical wavelet coefficients of u+ζ towards zero. We consider the situation where the prior information on the unknown function u may not be the regularity of u but that of Lu where L is a linear operator (such as a PDE or a graph Laplacian). We show that the approximation of u obtained by thresholding the gamblet (operator adapted wavelet) coefficients of u+ζ is near minimax optimal (up to a multiplicative constant), and with high probability, its energy norm (defined by the operator) is bounded by that of u up to a constant depending on the amplitude of the noise. Since gamblets can be computed in O(NpolylogN) complexity and are localized both in space and eigenspace, the proposed method is of near-linear complexity and generalizable to nonhomogeneous noise.

Publication: Statistics and Computing Vol.: 29 No.: 6 ISSN: 0960-3174

ID: CaltechAUTHORS:20190923-104545857

]]>

Abstract: In this paper, we introduce a hierarchical construction of material-adapted refinable basis functions and associated wavelets to offer efficient coarse-graining of linear elastic objects. While spectral methods rely on global basis functions to restrict the number of degrees of freedom, our basis functions are locally supported; yet, unlike typical polynomial basis functions, they are adapted to the material inhomogeneity of the elastic object to better capture its physical properties and behavior. In particular, they share spectral approximation properties with eigenfunctions, offering a good compromise between computational complexity and accuracy. Their construction involves only linear algebra and follows a fine-to-coarse approach, leading to a block-diagonalization of the stiffness matrix where each block corresponds to an intermediate scale space of the elastic object. Once this hierarchy has been precomputed, we can simulate an object at runtime on very coarse resolution grids and still capture the correct physical behavior, with orders of magnitude speedup compared to a fine simulation. We show on a variety of heterogeneous materials that our approach outperforms all previous coarse-graining methods for elasticity.

Publication: ACM Transactions on Graphics Vol.: 38 No.: 6 ISSN: 0730-0301

ID: CaltechAUTHORS:20191111-134055447

]]>

Abstract: Fractures should be simulated accurately given their significant effects on whole flow patterns in porous media. But such high-resolution simulations impose severe computational challenges to numerical methods in the applications. Therefore, the demand for accurate and efficient coarse-graining techniques is increasing. In this work, a near-linear complexity multiresolution operator decomposition method is proposed for solving and coarse graining flow problems in fractured porous media. We use the Discrete Fracture Model (DFM) to describe fractures, in which the fractures are explicitly represented as -dimensional elements. Using operator adapted wavelets, the solution space is decomposed into subspaces where DFM subsolutions can be computed by solving sparse and well-conditioned linear systems. By keeping only the coarse-scale part of the solution space, we furthermore obtain a reduced order model. We provide numerical experiments that investigate the accuracy of the reduced order model for different resolutions and different choices of medium.

Publication: Journal of Computational Physics Vol.: 391ISSN: 0021-9991

ID: CaltechAUTHORS:20190109-090349439

]]>

Abstract: Mode decomposition is a prototypical pattern recognition problem that can be addressed from the (a priori distinct) perspectives of numerical approximation, statistical inference and deep learning. Could its analysis through these combined perspectives be used as a Rosetta stone for deciphering mechanisms at play in deep learning? Motivated by this question we introduce programmable and interpretable regression networks for pattern recognition and address mode decomposition as a prototypical problem. The programming of these networks is achieved by assembling elementary modules decomposing and recomposing kernels and data. These elementary steps are repeated across levels of abstraction and interpreted from the equivalent perspectives of optimal recovery, game theory and Gaussian process regression (GPR). The prototypical mode/kernel decomposition module produces an optimal approximation (w₁,w₂,⋯,w_m) of an element (v₁,v₂,…,v_m) of a product of Hilbert subspaces of a common Hilbert space from the observation of the sum v:=v₁+⋯+v_m. The prototypical mode/kernel recomposition module performs partial sums of the recovered modes w_i based on the alignment between each recovered mode w_i and the data v. We illustrate the proposed framework by programming regression networks approximating the modes v_i=a_i(t)y_i(θ_i(t)) of a (possibly noisy) signal ∑_iv_i when the amplitudes a_i, instantaneous phases θ_i and periodic waveforms y_i may all be unknown and show near machine precision recovery under regularity and separation assumptions on the instantaneous amplitudes a_i and frequencies θ_i. The structure of some of these networks share intriguing similarities with convolutional neural networks while being interpretable, programmable and amenable to theoretical analysis.

Publication: arXiv
ID: CaltechAUTHORS:20190923-153747161

]]>

Abstract: Learning can be seen as approximating an unknown function by interpolating the training data. Although Kriging offers a solution to this problem, it requires the prior specification of a kernel and it is not scalable to large datasets. We explore a numerical approximation approach to kernel selection/construction based on the simple premise that a kernel must be good if the number of interpolation points can be halved without significant loss in accuracy (measured using the intrinsic RKHS norm ∥·∥ associated with the kernel). We first test and motivate this idea on a simple problem of recovering the Green's function of an elliptic PDE (with inhomogeneous coefficients) from the sparse observation of one of its solutions. Next we consider the problem of learning non-parametric families of deep kernels of the form K_1(F_n(x), F_n(x')) with F_(n+1) = (I_d + ϵG_(n+1)) ◦ F_n and G_(n+1) ∈ span{K_1(F_n(x_i), ·)}. With the proposed approach constructing the kernel becomes equivalent to integrating a stochastic data driven dynamical system, which allows for the training of very deep (bottomless) networks and the exploration of their properties. These networks learn by constructing flow maps in the kernel and input spaces via incremental data-dependent deformations/perturbations (appearing as the cooperative counterpart of adversarial examples) and, at profound depths, they (1) can achieve accurate classification from only one data point per class (2) appear to learn archetypes of each class (3) expand distances between points that are in different classes and contract distances between points in the same class. For kernels parameterized by the weights of Convolutional Neural Networks, minimizing approximation errors incurred by halving random subsets of interpolation points, appears to outperform training (the same CNN architecture) with relative entropy and dropout.

Publication: Journal of Computational Physics Vol.: 389ISSN: 0021-9991

ID: CaltechAUTHORS:20190328-180953225

]]>

Abstract: We introduce in this paper an operator-adapted multiresolution analysis for finite-element differential forms. From a given continuous, linear, bijective, and self-adjoint positive-definite operator L, a hierarchy of basis functions and associated wavelets for discrete differential forms is constructed in a fine-to-coarse fashion and in quasilinear time. The resulting wavelets are L-orthogonal across all scales, and can be used to derive a Galerkin discretization of the operator such that its stiffness matrix becomes block-diagonal, with uniformly well-conditioned and sparse blocks. Because our approach applies to arbitrary differential p-forms, we can derive both scalar-valued and vector-valued wavelets block-diagonalizing a prescribed operator. We also discuss the generality of the construction by pointing out that it applies to various types of computational grids, offers arbitrary smoothness orders of basis functions and wavelets, and can accommodate linear differential constraints such as divergence-freeness. Finally, we demonstrate the benefits of the corresponding operator-adapted multiresolution decomposition for coarse-graining and model reduction of linear and non-linear partial differential equations.

Publication: Journal of Computational Physics Vol.: 388ISSN: 0021-9991

ID: CaltechAUTHORS:20190315-100824617

]]>

Abstract: The practical implementation of Bayesian inference requires numerical approximation when closed-form expressions are not available. What types of accuracy (convergence) of the numerical approximations guarantee robustness and what types do not? In particular, is the recursive application of Bayes’ rule robust when subsequent data or posteriors are approximated? When the prior is the push forward of a distribution by the map induced by the solution of a PDE, in which norm should that solution be approximated? Motivated by such questions, we investigate the sensitivity of the distribution of posterior distributions (i.e. of posterior distribution-valued random variables, randomized through the data) with respect to perturbations of the prior and data-generating distributions in the limit when the number of data points grows towards infinity.

Publication: ESAIM: Probability and Statistics Vol.: 21ISSN: 1292-8100

ID: CaltechAUTHORS:20160223-152453383

]]>

Abstract: Implicit schemes are popular methods for the integration of time dependent PDEs such as hyperbolic and parabolic PDEs. However the necessity to solve corresponding linear systems at each time step constitutes a complexity bottleneck in their application to PDEs with rough coefficients. We present a generalization of gamblets introduced in [62] enabling the resolution of these implicit systems in near-linear complexity and provide rigorous a-priori error bounds on the resulting numerical approximations of hyperbolic and parabolic PDEs. These generalized gamblets induce a multiresolution decomposition of the solution space that is adapted to both the underlying (hyperbolic and parabolic) PDE (and the system of ODEs resulting from space discretization) and to the time-steps of the numerical scheme.

Publication: Journal of Computational Physics Vol.: 347ISSN: 0021-9991

ID: CaltechAUTHORS:20170707-092240450

]]>

Abstract: The energy that is needed for operating a self-powered device is provided by the energy excess in the system in the form of kinetic energy, or a combination of regenerative and renewable energy. This paper addresses the energy exchange issues pertaining to regenerative and renewable energy in the development of a self-powered dynamic system. A rigorous framework that explores the supply and demand of energy for self-powered systems is developed, which considers uncertainties and optimal bounds, in the context of optimal uncertainty quantification. Examples of regenerative and solar-powered systems are given, and the analysis of self-powered feedback control for developing a fully self-powered dynamic system is discussed.

Publication: Journal of Dynamic Systems, Measurement, and Control Vol.: 139 No.: 9 ISSN: 0022-0434

ID: CaltechAUTHORS:20170825-144657432

]]>

Abstract: The past century has seen a steady increase in the need of estimating and predicting complex systems and making (possibly critical) decisions with limited information. Although computers have made possible the numerical evaluation of sophisticated statistical models, these models are still designed by humans because there is currently no known recipe or algorithm for dividing the design of a statistical model into a sequence of arithmetic operations. Indeed enabling computers to think as humans, especially when faced with uncertainty, is challenging in several major ways: (1) Finding optimal statistical models remains to be formulated as a well-posed problem when information on the system of interest is incomplete and comes in the form of a complex combination of sample data, partial knowledge of constitutive relations and a limited description of the distribution of input random variables. (2) The space of admissible scenarios along with the space of relevant information, assumptions, and/or beliefs, tends to be infinite dimensional, whereas calculus on a computer is necessarily discrete and finite. With this purpose, this paper explores the foundations of a rigorous framework for the scientific computation of optimal statistical estimators/models and reviews their connections with decision theory, machine learning, Bayesian inference, stochastic optimization, robust optimization, optimal uncertainty quantification, and information-based complexity.

ID: CaltechAUTHORS:20160224-064915023

]]>

Abstract: We show how the discovery of robust scalable numerical solvers for arbitrary bounded linear operators can be automated as a Game Theory problem by reformulating the process of computing with partial information and limited resources as that of playing underlying hierarchies of adversarial information games. When the solution space is a Banach space B endowed with a quadratic norm ∥⋅∥, the optimal measure (mixed strategy) for such games (e.g. the adversarial recovery of u ∈ B, given partial measurements [ϕ_i,u] with ϕ_i ∈ B^∗, using relative error in ∥⋅∥-norm as a loss) is a centered Gaussian field ξ solely determined by the norm ∥⋅∥, whose conditioning (on measurements) produces optimal bets. When measurements are hierarchical, the process of conditioning this Gaussian field produces a hierarchy of elementary bets (gamblets). These gamblets generalize the notion of Wavelets and Wannier functions in the sense that they are adapted to the norm ∥⋅∥ and induce a multi-resolution decomposition of B that is adapted to the eigensubspaces of the operator defining the norm ∥⋅∥. When the operator is localized, we show that the resulting gamblets are localized both in space and frequency and introduce the Fast Gamblet Transform (FGT) with rigorous accuracy and (near-linear) complexity estimates. As the FFT can be used to solve and diagonalize arbitrary PDEs with constant coefficients, the FGT can be used to decompose a wide range of continuous linear operators (including arbitrary continuous linear bijections from H^s_0 to H^(−s) or to L^2) into a sequence of independent linear systems with uniformly bounded condition numbers and leads to O(NpolylogN) solvers and eigenspace adapted Multiresolution Analysis (resulting in near linear complexity approximation of all eigensubspaces).

ID: CaltechAUTHORS:20170710-085210757

]]>

Abstract: We demonstrate that a reproducing kernel Hilbert or Banach space of functions on a separable absolute Borel space or an analytic subset of a Polish space is separable if it possesses a Borel measurable feature map.

Publication: Proceedings of the American Mathematical Society Vol.: 145 No.: 5 ISSN: 0002-9939

ID: CaltechAUTHORS:20160224-070508936

]]>

Abstract: Can the theory that reality is a simulation be tested? We investigate this question based on the assumption that if the system performing the simulation is finite (i.e. has limited resources), then to achieve low computational complexity, such a system would, as in a video game, render content (reality) only at the moment that information becomes available for observation by a player and not at the moment of detection by a machine (that would be part of the simulation and whose detection would also be part of the internal computation performed by the Virtual Reality server before rendering content to the player). Guided by this principle we describe conceptual wave/particle duality experiments aimed at testing the simulation theory.

ID: CaltechAUTHORS:20190109-112815344

]]>

Abstract: We introduce a near-linear complexity (geometric and meshless/algebraic) multigrid/multiresolution method for PDEs with rough (L∞) coefficients with rigorous a priori accuracy and performance estimates. The method is discovered through a decision/game theory formulation of the problems of (1) identifying restriction and interpolation operators, (2) recovering a signal from incomplete measurements based on norm constraints on its image under a linear operator, and (3) gambling on the value of the solution of the PDE based on a hierarchy of nested measurements of its solution or source term. The resulting elementary gambles form a hierarchy of (deterministic) basis functions of H^1_0 (Ω) (gamblets) that (1) are orthogonal across subscales/subbands with respect to the scalar product induced by the energy norm of the PDE, (2) enable sparse compression of the solution space in H^1_0 (Ω), and (3) induce an orthogonal multiresolution operator decomposition. The operating diagram of the multigrid method is that of an inverted pyramid in which gamblets are computed locally (by virtue of their exponential decay) and hierarchically (from fine to coarse scales) and the PDE is decomposed into a hierarchy of independent linear systems with uniformly bounded condition numbers. The resulting algorithm is parallelizable both in space (via localization) and in bandwidth/subscale (subscales can be computed independently from each other). Although the method is deterministic, it has a natural Bayesian interpretation under the measure of probability emerging (as a mixed strategy) from the information game formulation, and multiresolution approximations form a martingale with respect to the filtration induced by the hierarchy of nested measurements.

Publication: SIAM Review Vol.: 59 No.: 1 ISSN: 0036-1445

ID: CaltechAUTHORS:20160223-133809979

]]>

Abstract: We discuss the development of Uncertainty Quantification framework founded upon a combination of game/decision theory and information based complexity. We suggest that such a framework could be used not only to guide decisions in presence of epistemic uncertainties and complexity management capabilities constraints but also to automate the process of discovery in (1) model form uncertainty quantification and design (2) model reduction (3) the design of fast, robust and scalable numerical solvers. Although these applications appear dissimilar, they are all based on the efficient processing of incomplete information with limited computational resources: (1) model form UQ and design require the management and processing of epistemic uncertainties and limited data (2) model reduction requires the approximation of the full state of a complex system through operations performed on a few (coarse/reduced) variables (3) fast and robust computation requires computation with partial information. The core idea of the proposed framework is to reformulate the process of computing with partial information and limited resources as that of playing underlying hierarchies of adversarial information games characterizing the adversarial and nested processing of hierarchies of partial/missing information.

ID: CaltechAUTHORS:20190816-144341264

]]>

Abstract: We show that, for the space of Borel probability measures on a Borel subset of a Polish metric space, the extreme points of the Prokhorov, Monge–Wasserstein and Kantorovich metric balls about a measure whose support has at most n points, consist of measures whose supports have at most n+2 points. Moreover, we use the Strassen and Kantorovich–Rubinstein duality theorems to develop representations of supersets of the extreme points based on linear programming, and then develop these representations towards the goal of their efficient computation.

Publication: Communications in Mathematical Sciences Vol.: 15 No.: 1 ISSN: 1539-6746

ID: CaltechAUTHORS:20160223-151629237

]]>

Abstract: Animals are products of nature and have evolved over millions of years to perform better in their activities. Engineering research and development can benefit greatly by looking into nature and finding engineering solutions by learning from animals’ evolution and biological systems. Another relevant factor in the present context is highlighted by the statement of the Nobel laureate Richard Smalley: “Energy is the single most important problem facing humanity today.” This paper focuses on how the research and education in the area of Dynamic Systems can be geared towards these two considerations. In particular, recent advances in self-powered dynamic systems and bio-inspired dynamic systems are highlighted. Self-powered dynamic systems benefit by capturing wasted energy in a dynamic system and converting it into useful energy in the mode of a regenerative system, possibly in conjunction with renewable energies. Examples of solar-powered vehicles, regenerative vibration control, and energy harvesting are presented in the paper. Particularly, development of solar-powered quadrotor, octocopter, and tricopter airships are presented, a self-powered vibration control of a mass-spring system using electromagnetic actuators/generators, and piezoelectric flutter energy harvesting using bi-stable material are discussed. As examples of bioinspired dynamic systems, flapping wing flying robots, vertical axis wind turbines inspired by fish schooling, propulsion inspired by jellyfish, and Psi Intelligent Control are given. In particular, various design and developments of bird-inspired and insect-inspired flapping wings with the piezoelectric and electromagnetic actuation mechanisms, a scaled vertical axis wind turbine farm consist of 4 turbines and the corresponding wind tunnel testing, jellyfish-inspired pulsing jet and experimenting the increase in efficiency of energy consumption, and a multi-agent/robotic based predictive control scheme inspired by Psi precognition (event or state not yet experienced). Examples of student projects and research carried out at Brunel University and the experimental rigs built (in all the mentioned areas) are discussed, as an integrated research and educational activity. For the analysis and understanding of the behavior of self-powered and bio-inspired systems, Optimal Uncertainty Quantification (OUQ) is used. OUQ establishes a unified analysis framework in obtaining optimized solutions of the dynamic systems responses, which takes into account uncertainties and incomplete information in the simulation of these systems.

Vol.: 4A
ID: CaltechAUTHORS:20170602-084400992

]]>

Abstract: We show that symplectic and linearly implicit integrators proposed by Zhang & Skeel (1997, Cheap implicit symplectic integrators. Appl. Numer. Math., 25, 297–302) are variational linearizations of Newmark methods. When used in conjunction with penalty methods (i.e., methods that replace constraints by stiff potentials), these integrators permit coarse time-stepping of holonomically constrained mechanical systems and bypass the resolution of nonlinear systems. Although penalty methods are widely employed, an explicit link to Lagrange multiplier approaches appears to be lacking; such a link is now provided (in the context of two-scale flow convergence (Tao, M., Owhadi, H. & Marsden, J. E. (2010) Nonintrusive and structure-preserving multiscale integration of stiff ODEs, SDEs and Hamiltonian systems with hidden slow dynamics via flow averaging. Multiscale Model. Simul., 8, 1269–1324). The variational formulation also allows efficient simulations of mechanical systems on Lie groups.

Publication: IMA Journal of Numerical Analysis Vol.: 36 No.: 1 ISSN: 0272-4979

ID: CaltechAUTHORS:20160317-083434102

]]>

Abstract: Technology, in common with many other activities, tends toward avoidance of risks by investors. Uncertainty is ruled out if possible. People generally prefer the predictable. Few recognize how destructive this can be, how it imposes severe limits on variability and thus makes whole populations fatally vulnerable to the shocking ways our universe can throw the dice.

ID: CaltechAUTHORS:20170615-123732747

]]>

Abstract: The incorporation of priors in the Optimal Uncertainty Quantification (OUQ) framework reveals brittleness in Bayesian inference; a model may share an arbitrarily large number of finite-dimensional marginals with, or be arbitrarily close (in Prokhorov or total variation metrics) to, the data-generating distribution and still make the largest possible prediction error after conditioning on an arbitrarily large number of samples. The initial purpose of this paper is to unwrap this brittleness mechanism by providing (i) a quantitative version of the Brittleness Theorem of and (ii) a detailed and comprehensive analysis of its application to the revealing example of estimating the mean of a random variable on the unit interval [0, 1] using priors that exactly capture the distribution of an arbitrarily large number of Hausdorff moments. However, in doing so, we discovered that the free parameter associated with Markov and Kreĩn’s canonical representations of truncated Hausdorff moments generates reproducing kernel identities corresponding to reproducing kernel Hilbert spaces of polynomials. Furthermore, these reproducing identities lead to biorthogonal systems of Selberg integral formulas. This process of discovery appears to be generic: whereas Karlin and Shapley used Selberg’s integral formula to first compute the volume of the Hausdorff moment space (the polytope defined by the first n moments of a probability measure on the interval [0, 1]), we observe that the computation of that volume along with higher order moments of the uniform measure on the moment space, using different finite-dimensional representations of subsets of the infinite-dimensional set of probability measures on [0, 1] representing the first n moments, leads to families of equalities corresponding to classical and new Selberg identities.

ID: CaltechAUTHORS:20160224-073833523

]]>

Abstract: We consider the temporal homogenization of linear ODEs of the form ẋ =Ax+ϵP(t)x+f(t), where P(t) is periodic and ϵ is small. Using a 2-scale expansion approach, we obtain the long-time approximation x(t)≈exp(At) (Ω(t)+∫^t_0exp(−Aτ)f(τ)dτ), where Ω solves the cell problem Ω=ϵBΩ+ϵF(t) with an effective matrix B and an explicitly-known F(t). We provide necessary and sufficient conditions for the accuracy of the approximation (over a O(ϵ^(−1)) time-scale), and show how B can be computed (at a cost independent of ϵ). As a direct application, we investigate the possibility of using RLC circuits to harvest the energy contained in small scale oscillations of ambient electromagnetic fields (such as Schumann resonances). Although a RLC circuit parametrically coupled to the field may achieve such energy extraction via parametric resonance, its resistance R needs to be smaller than a threshold κ proportional to the fluctuations of the field, thereby limiting practical applications. We show that if n RLC circuits are appropriately coupled via mutual capacitances or inductances, then energy extraction can be achieved when the resistance of each circuit is smaller than nκ. Hence, if the resistance of each circuit has a non-zero fixed value, energy extraction can be made possible through the coupling of a sufficiently large number n of circuits (n≈1000 for the first mode of Schumann resonances and contemporary values of capacitances, inductances and resistances). The theory is also applied to the control of the oscillation amplitude of a (damped) oscillator.

Publication: Archive for Rational Mechanics and Analysis Vol.: 220 No.: 1 ISSN: 0003-9527

ID: CaltechAUTHORS:20160224-071627521

]]>

Abstract: For a Gaussian measure on a separable Hilbert space with covariance operator C, we show that the family of conditional measures associated with conditioning on a closed subspace S^⊥ are Gaussian with covariance operator the short S(C) of the operator C to S. We provide two proofs. The first uses the theory of Gaussian Hilbert spaces and a characterization of the shorted operator by Andersen and Trapp. The second uses recent developments by Corach, Maestripieri and Stojanoff on the relationship between the shorted operator and C-symmetric oblique projections onto S^⊥. To obtain the assertion when such projections do not exist, we develop an approximation result for the shorted operator by showing, for any positive operator A, how to construct a sequence of approximating operators A^n which possess A^n- symmetric oblique projections onto S^⊥ such that the sequence of shorted operators S(A^n) converges to S(A) in the weak operator topology. This result combined with the martingale convergence of random variables associated with the corresponding approximations C^n establishes the main assertion in general. Moreover, it in turn strengthens the approximation theorem for shorted operator when the operator is trace class; then the sequence of shorted operators S(A^n) converges to S(A) in trace norm.

ID: CaltechAUTHORS:20160224-065740350

]]>

Abstract: Numerical homogenization, i.e., the finite-dimensional approximation of solution spaces of PDEs with arbitrary rough coefficients, requires the identification of accurate basis elements. These basis elements are oftentimes found after a laborious process of scientific investigation and plain guesswork. Can this identification problem be facilitated? Is there a general recipe/decision framework for guiding the design of basis elements? We suggest that the answer to the above questions could be positive based on the reformulation of numerical homogenization as a Bayesian inference problem in which a given PDE with rough coefficients (or multiscale operator) is excited with noise (random right-hand side/source term) and one tries to estimate the value of the solution at a given point based on a finite number of observations. We apply this reformulation to the identification of bases for the numerical homogenization of arbitrary integro-differential equations and show that these bases have optimal recovery properties. In particular we show how rough polyharmonic splines can be rediscovered as the optimal solution of a Gaussian filtering problem.

Publication: Multiscale Modeling and Simulation Vol.: 13 No.: 3 ISSN: 1540-3459

ID: CaltechAUTHORS:20151023-105214463

]]>

Abstract: Optimal uncertainty quantification (OUQ) is a framework for numerical extreme-case analysis of stochastic systems with imperfect knowledge of the underlying probability distribution. This paper presents sufficient conditions under which an OUQ problem can be reformulated as a finite-dimensional convex optimization problem, for which efficient numerical solutions can be obtained. The sufficient conditions include that the objective function is piecewise concave and the constraints are piecewise convex. In particular, we show that piecewise concave objective functions may appear in applications where the objective is defined by the optimal value of a parameterized linear program.

Publication: SIAM Journal of Optimization Vol.: 25 No.: 3 ISSN: 1052-6234

ID: CaltechAUTHORS:20151030-084615377

]]>

Abstract: We derive, in the classical framework of Bayesian sensitivity analysis, optimal lower and upper bounds on posterior values obtained from Bayesian models that exactly capture an arbitrarily large number of finite-dimensional marginals of the data-generating distribution and/or that are as close as desired to the data-generating distribution in the Prokhorov or total variation metrics; these bounds show that such models may still make the largest possible prediction error after conditioning on an arbitrarily large number of sample data measured at finite precision. These results are obtained through the development of a reduction calculus for optimization problems over measures on spaces of measures. We use this calculus to investigate the mechanisms that generate brittleness/robustness and, in particular, we observe that learning and robustness are antagonistic properties. It is now well understood that the numerical resolution of PDEs requires the satisfaction of specific stability conditions. Is there a missing stability condition for using Bayesian inference in a continuous world under finite information?

Publication: Electronic Journal of Statistics Vol.: 9 No.: 1 ISSN: 1935-7524

ID: CaltechAUTHORS:20160108-101302184

]]>

Abstract: With the advent of high-performance computing, Bayesian methods are becoming increasingly popular tools for the quantification of uncertainty throughout science and industry. Since these methods can impact the making of sometimes critical decisions in increasingly complicated contexts, the sensitivity of their posterior conclusions with respect to the underlying models and prior beliefs is a pressing question to which there currently exist positive and negative answers. We report new results suggesting that, although Bayesian methods are robust when the number of possible outcomes is finite or when only a finite number of marginals of the data-generating distribution are unknown, they could be generically brittle when applied to continuous systems (and their discretizations) with finite information on the data-generating distribution. If closeness is defined in terms of the total variation (TV) metric or the matching of a finite system of generalized moments, then (1) two practitioners who use arbitrarily close models and observe the same (possibly arbitrarily large amount of) data may reach opposite conclusions; and (2) any given prior and model can be slightly perturbed to achieve any desired posterior conclusion. The mechanism causing brittleness/robustness suggests that learning and robustness are antagonistic requirements, which raises the possibility of a missing stability condition when using Bayesian inference in a continuous world under finite information.

Publication: SIAM Review Vol.: 57 No.: 4 ISSN: 0036-1445

ID: CaltechAUTHORS:20160108-105121903

]]>

Abstract: We present an optimal uncertainty quantification (OUQ) protocol for systems that are characterized by an existing physics-based model and for which only legacy data is available, i.e., no additional experimental testing of the system is possible. Specifically, the OUQ strategy developed in this work consists of using the legacy data to establish, in a probabilistic sense, the level of error of the model, or modeling error, and to subsequently use the validated model as a basis for the determination of probabilities of outcomes. The quantification of modeling uncertainty specifically establishes, to a specified confidence, the probability that the actual response of the system lies within a certain distance of the model. Once the extent of model uncertainty has been established in this manner, the model can be conveniently used to stand in for the actual or empirical response of the system in order to compute probabilities of outcomes. To this end, we resort to the OUQ reduction theorem of Owhadi et al. (2013) in order to reduce the computation of optimal upper and lower bounds on probabilities of outcomes to a finite-dimensional optimization problem. We illustrate the resulting UQ protocol by means of an application concerned with the response to hypervelocity impact of 6061-T6 Aluminum plates by Nylon 6/6 impactors at impact velocities in the range of 5–7 km/s. The ability of the legacy OUQ protocol to process diverse information on the system and its ability to supply rigorous bounds on system performance under realistic—and less than ideal—scenarios demonstrated by the hypervelocity impact application is remarkable.

Publication: Journal of the Mechanics and Physics of Solids Vol.: 72ISSN: 0022-5096

ID: CaltechAUTHORS:20141201-081327318

]]>

Abstract: We introduce a new variational method for the numerical homogenization of divergence form elliptic, parabolic and hyperbolic equations with arbitrary rough (L^∞) coefficients. Our method does not rely on concepts of ergodicity or scale-separation but on compactness properties of the solution space and a new variational approach to homogenization. The approximation space is generated by an interpolation basis (over scattered points forming a mesh of resolution H) minimizing the L^2 norm of the source terms; its (pre-)computation involves minimizing O(H^(-d)) quadratic (cell) problems on (super-)localized sub-domains of size O(H ln(1/H)). The resulting localized linear systems remain sparse and banded. The resulting interpolation basis functions are biharmonic for d ≤ 3, and polyharmonic for d ≥ 4, for the operator -div(a∇.) and can be seen as a generalization of polyharmonic splines to differential operators with arbitrary rough coefficients. The accuracy of the method (O(H)) in energy norm and independent from aspect ratios of the mesh formed by the scattered points) is established via the introduction of a new class of higher-order Poincaré inequalities. The method bypasses (pre-)computations on the full domain and naturally generalizes to time dependent problems, it also provides a natural solution to the inverse problem of recovering the solution of a divergence form elliptic equation from a finite number of point measurements.

Publication: ESAIM-Mathematical Modelling and Numerical Analysis Vol.: 48 No.: 2 ISSN: 0764-583X

ID: CaltechAUTHORS:20160223-143844457

]]>

Abstract: The mathematical study of “multiscale problems” has grown remarkably since the seventies beyond the asymptotic analysis of PDE’s governing the behavior of heterogeneous media. The search for sharp bounds on the effective moduli of composites and homogenization approximation errors has led investigators to derive as much information as possible about fields in composites, and the behavior of correctors in periodic and stochastic environments.

Publication: ESAIM: Mathematical Modelling and Numerical Analysis Vol.: 48 No.: 2 ISSN: 0764-583X

ID: CaltechAUTHORS:20160224-085030779

]]>

Abstract: We consider the problem of providing optimal uncertainty quantification (UQ) – and hence rigorous certification – for partially-observed functions. We present a UQ framework within which the observations may be small or large in number, and need not carry information about the probability distribution of the system in operation. The UQ objectives are posed as optimization problems, the solutions of which are optimal bounds on the quantities of interest; we consider two typical settings, namely parameter sensitivities (McDiarmid diameters) and output deviation (or failure) probabilities. The solutions of these optimization problems depend non-trivially (even non-monotonically and discontinuously) upon the specified legacy data. Furthermore, the extreme values are often determined by only a few members of the data set; in our principal physically-motivated example, the bounds are determined by just 2 out of 32 data points, and the remainder carry no information and could be neglected without changing the final answer. We propose an analogue of the simplex algorithm from linear programming that uses these observations to offer efficient and rigorous UQ for high-dimensional systems with high-cardinality legacy data. These findings suggest natural methods for selecting optimal (maximally informative) next experiments.

Publication: ESAIM-Mathematical Modelling and Numerical Analysis Vol.: 47 No.: 6 ISSN: 0764-583X

ID: CaltechAUTHORS:20131119-093417975

]]>

Abstract: Imaging and simulation methods are typically constrained to resolutions much coarser than the scale of physical microstructures present in body tissues or geological features. Mathematical homogenization and numerical homogenization address this practical issue by identifying and computing appropriate spatial averages that result in accuracy and consistency between the macroscales we observe and the underlying microscale models we assume. Among the various applications benefiting from homogenization, electrical impedance tomography (EIT) images the electrical conductivity of a body by measuring electrical potentials consequential to electric currents applied to the exterior of the body. EIT is routinely used in breast cancer detection and cardiopulmonary imaging, where current flow in fine‐scale tissues underlies the resulting coarse‐scale images.

ID: CaltechAUTHORS:20210224-145143008

]]>

Abstract: We present a novel approach for the analysis and design of self-supporting simplicial masonry structures. A finite-dimensional formulation of their compressive stress field is derived, offering a new interpretation of thrust networks through numerical homogenization theory. We further leverage geometric properties of the resulting force diagram to identify a set of reduced coordinates characterizing the equilibrium of simplicial masonry. We finally derive computational form-finding tools that improve over previous work in efficiency, accuracy, and scalability.

Publication: ACM Transactions on Graphics Vol.: 32 No.: 4 ISSN: 0730-0301

ID: CaltechAUTHORS:20130829-153504635

]]>

Abstract: In this contribution, we develop a variational integrator for the simulation of (stochastic and multiscale) electric circuits. When considering the dynamics of an electric circuit, one is faced with three special situations: 1. The system involves external (control) forcing through external (controlled) voltage sources and resistors. 2. The system is constrained via the Kirchhoff current (KCL) and voltage laws (KVL). 3. The Lagrangian is degenerate. Based on a geometric setting, an appropriate variational formulation is presented to model the circuit from which the equations of motion are derived. A time-discrete variational formulation provides an iteration scheme for the simulation of the electric circuit. Dependent on the discretization, the intrinsic degeneracy of the system can be canceled for the discrete variational scheme. In this way, a variational integrator is constructed that gains several advantages compared to standard integration tools for circuits; in particular, a comparison to BDF methods (which are usually the method of choice for the simulation of electric circuits) shows that even for simple LCR circuits, a better energy behavior and frequency spectrum preservation can be observed using the developed variational integrator.

Publication: Journal of Computational Physics Vol.: 242ISSN: 0021-9991

ID: CaltechAUTHORS:20130628-141442082

]]>

Abstract: How does one evaluate the performance of a stochastic system in the absence of a perfect model (i.e. probability distribution)? We address this question under the framework of optimal uncertainty quantification (OUQ), which is an information-based approach for worst-case analysis of stochastic systems. We are able to generalize previous results and show that the OUQ problem can be solved using convex optimization when the function under evaluation can be expressed in a polytopic canonical form (PCF). We also propose iterative methods for scaling the convex formulation to larger systems. As an application, we study the problem of storage placement in power grids with renewable generation. Numerical simulation results for simple artificial examples as well as an example using the IEEE 14-bus test case with real wind generation data are presented to demonstrate the usage of OUQ analysis.

ID: CaltechCDSTR:2012.002

]]>

Abstract: We propose a rigorous framework for uncertainty quantification (UQ) in which the UQ objectives and its assumptions/information set are brought to the forefront. This framework, which we call optimal uncertainty quantification (OUQ), is based on the observation that, given a set of assumptions and information about the problem, there exist optimal bounds on uncertainties: these are obtained as values of well-defined optimization problems corresponding to extremizing probabilities of failure, or of deviations, subject to the constraints imposed by the scenarios compatible with the assumptions and information. In particular, this framework does not implicitly impose inappropriate assumptions, nor does it repudiate relevant information. Although OUQ optimization problems are extremely large, we show that under general conditions they have finite-dimensional reductions. As an application, we develop optimal concentration inequalities (OCI) of Hoeffding and McDiarmid type. Surprisingly, these results show that uncertainties in input parameters, which propagate to output uncertainties in the classical sensitivity analysis paradigm, may fail to do so if the transfer functions (or probability distributions) are imperfectly known. We show how, for hierarchical structures, this phenomenon may lead to the nonpropagation of uncertainties or information across scales. In addition, a general algorithmic framework is developed for OUQ and is tested on the Caltech surrogate model for hypervelocity impact and on the seismic safety assessment of truss structures, suggesting the feasibility of the framework for important complex systems. The introduction of this paper provides both an overview of the paper and a self-contained minitutorial on the basic concepts and issues of UQ.

Publication: SIAM Review Vol.: 55 No.: 2 ISSN: 0036-1445

ID: CaltechAUTHORS:20130618-075057070

]]>

Abstract: This paper introduces a geometric method for proving ergodicity of degenerate noise driven stochastic processes. The driving noise is assumed to be an arbitrary Levy process with non-degenerate diffusion component (but that may be applied to a single degree of freedom of the system). The geometric conditions are the approximate controllability of the process the fact that there exists a point in the phase space where the interior of the image of a point via a secondarily randomized version of the driving noise is non void. The paper applies the method to prove ergodicity of a sliding disk governed by Langevin-type equations (a simple stochastic rigid body system). The paper shows that a key feature of this Langevin process is that even though the diffusion and drift matrices associated to the momentums are degenerate, the system is still at uniform temperature.

ID: CaltechAUTHORS:20160224-103320707

]]>

Abstract: We study the internal resonance, energy transfer, activation mechanism, and control of a model of DNA division via parametric resonance. While the system is robust to noise, this study shows that it is sensitive to specific fine scale modes and frequencies that could be targeted by low intensity electro-magnetic fields for triggering and controlling the division. The DNA model is a chain of pendula in a Morse potential. While the (possibly parametrically excited) system has a large number of degrees of freedom and a large number of intrinsic time scales, global and slow variables can be identified by (1) first reducing its dynamic to two modes exchanging energy between each other and (2) averaging the dynamic of the reduced system with respect to the phase of the fastest mode. Surprisingly, the global and slow dynamic of the system remains Hamiltonian (despite the parametric excitation) and the study of its associated effective potential shows how parametric excitation can turn the unstable open state into a stable one. Numerical experiments support the accuracy of the time-averaged reduced Hamiltonian in capturing the global and slow dynamic of the full system.

Publication: Chaos Vol.: 23 No.: 1 ISSN: 1054-1500

ID: CaltechAUTHORS:20130502-133827036

]]>

Abstract: This work is concerned with establishing the feasibility of a data-on-demand (DoD) uncertainty quantification (UQ) protocol based on concentration-of-measure inequalities. Specific aims are to establish the feasibility of the protocol and its basic properties, including the tightness of the predictions afforded by the protocol. The assessment is based on an application to terminal ballistics and a specific system configuration consisting of 6061-T6 aluminum plates struck by spherical S-2 tool steel projectiles at ballistic impact speeds. The system's inputs are the plate thickness and impact velocity and the perforation area is chosen as the sole performance measure of the system. The objective of the UQ analysis is to certify the lethality of the projectile, i.e., that the projectile perforates the plate with high probability over a prespecified range of impact velocities and plate thicknesses. The net outcome of the UQ analysis is an M/U ratio, or confidence factor, of 2.93, indicative of a small probability of no perforation of the plate over its entire operating range. The high-confidence (>99.9%) in the successful operation of the system afforded the analysis and the small number of tests (40) required for the determination of the modeling-error diameter, establishes the feasibility of the DoD UQ protocol as a rigorous yet practical approach for model-based certification of complex systems.

Publication: Journal of the Mechanics and Physics of Solids Vol.: 60 No.: 5 ISSN: 0022-5096

ID: CaltechAUTHORS:20120502-091106236

]]>

Abstract: This Part II of this series is concerned with establishing the feasibility of an extended data-on-demand (XDoD) uncertainty quantification (UQ) protocol based on concentration-of-measure inequalities and martingale theory. Specific aims are to establish the feasibility of the protocol and its basic properties, including the tightness of the predictions afforded by the protocol. The assessment is based on an application to terminal ballistics and a specific system configuration consisting of 6061-T6 aluminum plates struck by spherical 440c stainless steel projectiles at ballistic impact speeds in the range of 2.4–2.8 km/s. The system's inputs are the plate thickness, plate obliquity and impact velocity. The perforation area is chosen as the sole performance measure of the system. The objective of the UQ analysis is to certify the lethality of the projectile, i.e., that the projectile perforates the plate with high probability over a prespecified range of impact velocities, plate thicknesses and plate obliquities. All tests were conducted at Caltech's Small Particle Hypervelocity Range (SPHIR), which houses a two-stage gas gun. A feature of this facility is that the impact velocity, while amenable to precise measurement, cannot be controlled precisely but varies randomly according to a known probability density function. In addition, due to a competition between petalling and plugging mechanisms for the material system under consideration, the measured perforation area exhibits considerable scatter. The analysis establishes the feasibility of the XDoD UQ protocol as a rigorous yet practical approach for model-based certification of complex systems characterized by uncontrollable inputs and noisy experimental data.

Publication: Journal of the Mechanics and Physics of Solids Vol.: 60 No.: 5 ISSN: 0022-5096

ID: CaltechAUTHORS:20120430-133945935

]]>

Abstract: We have recently proposed a rigorous framework for Uncertainty Quantification (UQ) in which UQ objectives and assumption/information set are brought into the forefront, providing a framework for the communication and comparison of UQ results. In particular, this framework does not implicitly impose inappropriate assumptions nor does it repudiate relevant information. This framework, which we call Optimal Uncertainty Quantification (OUQ), is based on the observation that given a set of assumptions and information, there exist bounds on uncertainties obtained as values of optimization problems and that these bounds are optimal. It provides a uniform environment for the optimal solution of the problems of validation, certification, experimental design, reduced order modeling, prediction, extrapolation, all under aleatoric and epistemic uncertainties. OUQ optimization problems are extremely large, and even though under general conditions they have finite-dimensional reductions, they must often be solved numerically. This general algorithmic framework for OUQ has been implemented in the mystic optimization framework. We describe this implementation, and demonstrate its use in the context of the Caltech surrogate model for hypervelocity impact.

ID: CaltechAUTHORS:20160224-080348129

]]>

Abstract: We construct finite-dimensional approximations of solution spaces of divergence-form operators with L^∞-coefficients. Our method does not rely on concepts of ergodicity or scale-separation, but on the property that the solution space of these operators is compactly embedded in H^1 if source terms are in the unit ball of L^2 instead of the unit ball of H^(−1). Approximation spaces are generated by solving elliptic PDEs on localized subdomains with source terms corresponding to approximation bases for H^2. The H^1-error estimates show that O(h^(−d))-dimensional spaces with basis elements localized to subdomains of diameter O(hα ln math) (with α ∊ [½,1)) result in an O(h^(2−2α)) accuracy for elliptic, parabolic, and hyperbolic problems. For high-contrast media, the accuracy of the method is preserved, provided that localized subdomains contain buffer zones of width O(h^α ln 1/4), where the contrast of the medium remains bounded. The proposed method can naturally be generalized to vectorial equations (such as elasto-dynamics).

Publication: Multiscale Modeling and Simulation Vol.: 9 No.: 4 ISSN: 1540-3459

ID: CaltechAUTHORS:20120126-112528320

]]>

Abstract: We describe a rigorous approach for certifying the safe operation of complex systems that bypasses the need for integral testing. We specifically consider systems that have a modular structure. These systems are composed of subsystems, or components, that interact through unidirectional interfaces. We show that, for systems that have the structure of an acyclic graph, it is possible to obtain rigorous upper bounds on the probability of failure of the entire system from an uncertainty analysis of the individual components and their interfaces and without the need for integral testing. Certification is then achieved if the probability of failure upper bound is below an acceptable failure tolerance. We demonstrate the approach by means of an example concerned with the performance of a fractal electric circuit.

Publication: Reliability Engineering and System Safety Vol.: 96 No.: 9 ISSN: 0951-8320

ID: CaltechAUTHORS:20110822-140452319

]]>

Abstract: We propose a method for the approximation of solutions of PDEs with stochastic coefficients based on the direct, i.e., non-adapted, sampling of solutions. This sampling can be done by using any legacy code for the deterministic problem as a black box. The method converges in probability (with probabilistic error bounds) as a consequence of sparsity and a concentration of measure phenomenon on the empirical correlation between samples. We show that the method is well suited for truly high-dimensional problems.

Publication: Journal of Computational Physics Vol.: 230 No.: 8 ISSN: 0021-9991

ID: CaltechAUTHORS:20110414-085411000

]]>

Abstract: We consider uncertainty quantification in the context of certification, i.e. showing that the probability of some ‘failure’ event is acceptably small. In this paper, we derive a new method for rigorous uncertainty quantification and conservative certification by combining McDiarmid's inequality with input domain partitioning and a new concentration-of-measure inequality. We show that arbitrarily sharp upper bounds on the probability of failure can be obtained by partitioning the input parameter space appropriately; in contrast, the bound provided by McDiarmid's inequality is usually not sharp. We prove an error estimate for the method (Proposition 3.2); we define a codimension-one recursive partitioning scheme and prove its convergence properties (Theorem 4.1); finally, we apply a new concentration-of-measure inequality to give confidence levels when empirical means are used in place of exact ones (Section 5).

Publication: International Journal for Numerical Methods in Engineering Vol.: 85 No.: 12 ISSN: 0029-5981

ID: CaltechAUTHORS:20110315-151154130

]]>

Abstract: We present a new class of integrators for stiff PDEs. These integrators are generalizations of FLow AVeraging integratORS (FLAVORS) for stiff ODEs and SDEs introduced in [32] with the following properties: (i) Multiscale: they are based on flow averaging and have a computational cost determined by mesoscopic steps in space and time instead of microscopic steps in space and time; (ii) Versatile: the method is based on averaging the flows of the given PDEs (which may have hidden slow and fast processes). This bypasses the need for identifying explicitly (or numerically) the slow variables or reduced effective PDEs; (iii) Nonintrusive: A pre-existing numerical scheme resolving the microscopic time scale can be used as a black box and easily turned into one of the integrators in this paper by turning the large coefficients on over a microscopic timescale and off during a mesoscopic timescale; (iv) Convergent over two scales: strongly over slow processes and in the sense of measures over fast ones; (v) Structure-preserving: for stiff Hamiltonian PDEs (possibly on manifolds), they can be made to be multi-symplectic, symmetry-preserving (symmetries are group actions that leave the system invariant) in all variables and variational.

Publication: Dynamics of Partial Differential Equations Vol.: 8 No.: 1 ISSN: 1548-159X

ID: CaltechAUTHORS:20110502-134102771

]]>

Abstract: We present a multiscale integrator for Hamiltonian systems with slowly varying quadratic stiff potentials that uses coarse timesteps (analogous to what the impulse method uses for constant quadratic stiff potentials). This method is based on the highly-non-trivial introduction of two efficient symplectic schemes for exponentiations of matrices that only require O(n) matrix multiplications operations at each coarse time step for a preset small number n. The proposed integrator is shown to be (i) uniformly convergent on positions; (ii) symplectic in both slow and fast variables; (iii) well adapted to high dimensional systems. Our framework also provides a general method for iteratively exponentiating a slowly varying sequence of (possibly high dimensional) matrices in an efficient way.

Publication: Applied Mathematics Research eXpress Vol.: 2011 No.: 2 ISSN: 1687-1200

ID: CaltechAUTHORS:20120215-101206163

]]>

Abstract: We present a simple algorithm for the simulation of stiff, discrete-space, continuous-time Markov processes. The algorithm is based on the concept of flow averaging for the integration of stiff ordinary and stochastic differential equations and ultimately leads to a straightforward variation of the the well-known stochastic simulation algorithm (SSA). The speedup that can be achieved by the present algorithm [flow averaging integrator SSA (FLAVOR-SSA)] over the classical SSA comes naturally at the expense of its accuracy. The error of the proposed method exhibits a cutoff phenomenon as a function of its speed-up, allowing for optimal tuning. Two numerical examples from chemical kinetics are provided to illustrate the efficiency of the method.

Publication: Journal of Chemical Physics Vol.: 133 No.: 24 ISSN: 0021-9606

ID: CaltechAUTHORS:20110318-145129123

]]>

Abstract: We consider linear divergence-form scalar elliptic equations and vectorial equations for elasticity with rough (L∞(Ω)Ω⊂R^d) coefficients a(x) that, in particular, model media with non-separated scales and high contrast in material properties. While the homogenization of PDEs with periodic or ergodic coefficients and well separated scales is now well understood, we consider here the most general case of arbitrary bounded coefficients. For such problems, we introduce explicit and optimal finite dimensional approximations of solutions that can be viewed as a theoretical Galerkin method with controlled error estimates, analogous to classical homogenization approximations. In particular, this approach allows one to analyze a given medium directly without introducing the mathematical concept of an ε family of media as in classical homogenization. We define the flux norm as the L^2 norm of the potential part of the fluxes of solutions, which is equivalent to the usual H^1-norm. We show that in the flux norm, the error associated with approximating, in a properly defined finite-dimensional space, the set of solutions of the aforementioned PDEs with rough coefficients is equal to the error associated with approximating the set of solutions of the same type of PDEs with smooth coefficients in a standard space (for example, piecewise polynomial). We refer to this property as the transfer property. A simple application of this property is the construction of finite dimensional approximation spaces with errors independent of the regularity and contrast of the coefficients and with optimal and explicit convergence rates. This transfer property also provides an alternative to the global harmonic change of coordinates for the homogenization of elliptic operators that can be extended to elasticity equations. The proofs of these homogenization results are based on a new class of elliptic inequalities. These inequalities play the same role in our approach as the div-curl lemma in classical homogenization.

Publication: Archive for Rational Mechanics and Analysis Vol.: 198 No.: 2 ISSN: 0003-9527

ID: CaltechAUTHORS:20101025-093609352

]]>

Abstract: We construct finite-dimensional approximations of solution spaces of divergence form operators with L^∞-coefficients. Our method does not rely on concepts of ergodicity or scale-separation, but on the property that the solution of space of these operators is compactly embedded in H^1 if source terms are in the unit ball of L^2 instead of the unit ball of H^−1. Approximation spaces are generated by solving elliptic PDEs on localized sub-domains with source terms corresponding to approximation bases for H^2. The H^1-error estimates show that O(h^−d)-dimensional spaces with basis elements localized to sub-domains of diameter O(h^∞ ln 1/h) (with α ∈ [1/2 , 1)) result in an O(h^(2−2α) accuracy for elliptic, parabolic and hyperbolic problems. For high-contrast media, the accuracy of the method is preserved provided that localized sub-domains contain buffer zones of width O(h^α ln 1/h ) where the contrast of the medium remains bounded. The proposed method can naturally be generalized to vectorial equations (such as elasto-dynamics).

No.: 2010-04
ID: CaltechAUTHORS:20111012-113719601

]]>

Abstract: We consider a random variable X that takes values in a (possibly infinite-dimensional) topological vector space X. We show that, with respect to an appropriate "normal distance" on X, concentration inequalities for linear and non-linear functions of X are equivalent. This normal distance corresponds naturally to the concentration rate in classical concentration results such as Gaussian concentration and concentration on the Euclidean and Hamming cubes. Under suitable assumptions on the roundness of the sets of interest, the concentration inequalities so obtained are asymptotically optimal in the high-dimensional limit.

ID: CaltechAUTHORS:20160224-082333411

]]>

Abstract: We propose a rigorous framework for Uncertainty Quantification (UQ) in which the UQ objectives and the assumptions/information set are brought to the forefront. This framework, which we call Optimal Uncertainty Quantification (OUQ), is based on the observation that, given a set of assumptions and information about the problem, there exist optimal bounds on uncertainties: these are obtained as extreme values of well-defined optimization problems corresponding to extremizing probabilities of failure, or of deviations, subject to the constraints imposed by the scenarios compatible with the assumptions and information. In particular, this framework does not implicitly impose inappropriate assumptions, nor does it repudiate relevant information. Although OUQ optimization problems are extremely large, we show that under general conditions, they have finite-dimensional reductions. As an application, we develop Optimal Concentration Inequalities (OCI) of Hoeffding and McDiarmid type. Surprisingly, contrary to the classical sensitivity analysis paradigm, these results show that uncertainties in input parameters do not necessarily propagate to output uncertainties. In addition, a general algorithmic framework is developed for OUQ and is tested on the Caltech surrogate model for hypervelocity impact, suggesting the feasibility of the framework for important complex systems.

No.: 2010-03
ID: CaltechAUTHORS:20111012-113158874

]]>

Abstract: We introduce a new class of integrators for stiff ODEs as well as SDEs. Examples of subclasses of systems that we treat are ODEs and SDEs that are sums of two terms, one of which has large coefficients. These integrators are as follows: (i) Multiscale: They are based on flow averaging and thus do not fully resolve the fast variables and have a computational cost determined by slow variables. (ii) Versatile: The method is based on averaging the flows of the given dynamical system (which may have hidden slow and fast processes) instead of averaging the instantaneous drift of assumed separated slow and fast processes. This bypasses the need for identifying explicitly (or numerically) the slow or fast variables. (iii) Nonintrusive: A pre-existing numerical scheme resolving the microscopic time scale can be used as a black box and easily turned into one of the integrators in this paper by turning the large coefficients on over a microscopic time scale and off during a mesoscopic time scale. (iv) Convergent over two scales: They converge strongly over slow processes and in the sense of measures over fast ones. We introduce the related notion of two-scale flow convergence and analyze the convergence of these integrators under the induced topology. (v) Structure preserving: They inherit the structure preserving properties of the legacy integrators from which they are derived. Therefore, for stiff Hamiltonian systems (possibly on manifolds), they can be made to be symplectic, time-reversible, and symmetry preserving (symmetries are group actions that leave the system invariant) in all variables. They are explicit and applicable to arbitrary stiff potentials (that need not be quadratic). Their application to the Fermi–Pasta–Ulam problems shows accuracy and stability over four orders of magnitude of time scales. For stiff Langevin equations, they are symmetry preserving, time-reversible, Boltzmann–Gibbs-reversible, quasi-symplectic on all variables, and conformally symplectic with isotropic friction.

Publication: Multiscale Modeling and Simulation Vol.: 8 No.: 4 ISSN: 1540-3459

ID: CaltechAUTHORS:20100927-105248361

]]>

Abstract: This paper is concerned with tuning friction and temperature in Langevin dynamics for fast sampling from the canonical ensemble. We show that near-optimal acceleration is achieved by choosing friction so that the local quadratic approximation of the Hamiltonian is a critical damped oscillator. The system is also over-heated and cooled down to its final temperature. The performances of different cooling schedules are analyzed as functions of total simulation time.

ID: CaltechAUTHORS:20160224-090906859

]]>

Abstract: We present an optimal control methodology, which we refer to as concentration-of-measure optimal control (COMOC), that seeks to minimize a concentration-of-measure upper bound on the probability of failure of a system. The systems under consideration are characterized by a single performance measure that depends on random inputs through a known response function. For these systems, concentration-of-measure upper bound on the probability of failure of a system can be formulated in terms of the mean performance measure and a system diameter that measures the uncertainty in the operation of the system. COMOC then seeks to determine the optimal controls that maximize the confidence in the safe operation of the system, defined as the ratio of the design margin, which is measured by the difference between the mean performance and the design threshold, to the system uncertainty, which is measured by the system diameter. This strategy has been assessed in the case of a robot-arm maneuver for which the performance measure of interest is assumed to be the placement accuracy of the arm tip. The ability of COMOC to significantly increase the design confidence in that particular example of application is demonstrated.

Publication: Journal of Computational and Nonlinear Dynamics Vol.: 5 No.: 3 ISSN: 1555-1423

ID: CaltechAUTHORS:20100709-105908209

]]>

Abstract: In [LOO08], it was proposed that a concentration-of-measure inequality known as Mc-Diarmid’s inequality [McD89] be used to provide upper bounds on the failure probability of a system of interest, the response of which depends on a collection of independent random inputs. McDiarmid’s inequality has the advantage of providing an upper bound in terms of only the mean response of the system, the failure threshold, and measures of system spread known as the McDiarmid subdiameters. A disadvantage of McDiarmid’s inequality is that it that takes a global view of the response function: even if the response function exhibits large plateaus of success with only small, localized regions of failure, McDiarmid’s inequality is unable to use this to any advantage. We propose a partitioning algorithm that uses McDiarmid diameters to generate “good” sequences of partitions, on which McDiarmid’s inequality can be applied to each partition element, yielding arbitrarily tight upper bounds. We also investigate some new concentration-of-measure inequalities that arise if mean performance is known only through sampling.

Publication: Procedia - Social and Behavioral Sciences Vol.: 2 No.: 6 ISSN: 1877-0428

ID: CaltechAUTHORS:20190109-104739732

]]>

Abstract: Impulse methods are generalized to a family of integrators for Langevin systems with quadratic stiff potentials and arbitrary soft potentials. Uniform error bounds (independent from stiff parameters) are obtained on integrated positions allowing for coarse integration steps. The resulting integrators are explicit and structure preserving (quasi-symplectic for Langevin systems).

ID: CaltechAUTHORS:20160224-085934570

]]>

Abstract: This paper studies mathematical models of biopolymer supramolecular aggregates that are formed by the self-assembly of single monomers. We develop a new multiscale numerical approach to model the structural properties of such aggregates. This theoretical approach establishes micro-macro relations between the geometrical and mechanical properties of the monomers and supramolecular aggregates. Most atomistic-to-continuum methods are constrained by a crystalline order or a periodic setting and therefore cannot be directly applied to modeling of soft matter. By contrast, the energy matching method developed in this paper does not require crystalline order and, therefore, can be applied to general microstructures with strongly variable spatial correlations. In this paper we use this method to compute the shape and the bending stiffness of their supramolecular aggregates from known chiral and amphiphilic properties of the short chain peptide monomers. Numerical implementation of our approach demonstrates consistency with results obtained by molecular dynamics simulations.

Publication: Multiscale Modeling & Simulation Vol.: 8 No.: 5 ISSN: 1540-3459

ID: CaltechAUTHORS:20110526-090344006

]]>

Abstract: This paper presents a Lie–Trotter splitting for inertial Langevin equations (geometric Langevin algorithm) and analyzes its long-time statistical properties. The splitting is defined as a composition of a variational integrator with an Ornstein–Uhlenbeck flow. Assuming that the exact solution and the splitting are geometrically ergodic, the paper proves the discrete invariant measure of the splitting approximates the invariant measure of inertial Langevin equations to within the accuracy of the variational integrator in representing the Hamiltonian. In particular, if the variational integrator admits no energy error, then the method samples the invariant measure of inertial Langevin equations without error. Numerical validation is provided using explicit variational integrators with first-, second-, and fourth-order accuracy.

Publication: SIAM Journal on Numerical Analysis Vol.: 48 No.: 1 ISSN: 0036-1429

ID: CaltechAUTHORS:20100609-111014824

]]>

Abstract: n this paper, we study multiple target detection using Bayesian learning. The main aim of the paper is to present a computationally efficient way to compute the belief map update exactly and efficiently using results from the theory of symmetric polynomials. In order to illustrate the idea, we consider a simple search scenario with multiple search agents and an unknown but fixed number of stationary targets in a given region that is divided into cells. To estimate the number of targets, a belief map for number of targets is also propagated. The belief map is updated using Bayes' theorem and an optimal reassignment of vehicles based on the values of the current belief map is adopted. Exact computation of the belief map update is combinatorial in nature and often an approximation is needed. We show that the Bayesian update can be exactly computed in an efficient manner using Newton's identities and the detection history in each cell.

ID: CaltechAUTHORS:20170328-173555191

]]>

Abstract: We introduce a new geometric approach for the homogenization and inverse homogenization of the divergence form elliptic operator with rough conductivity coefficients σ(x) in dimension two. We show that conductivity coefficients are in one-to-one correspondence with divergence-free matrices and convex functions s(x) over the domain Ω. Although homogenization is a non-linear and non-injective operator when applied directly to conductivity coefficients, homogenization becomes a linear interpolation operator over triangulations of Ω when re-expressed using convex functions, and is a volume averaging operator when re-expressed with divergence-free matrices. We explicitly give the transformations which map conductivity coefficients into divergence-free matrices and convex functions, as well as their respective inverses. Using optimal weighted Delaunay triangulations for linearly interpolating convex functions, we apply this geometric framework to obtain an optimally robust homogenization algorithm for arbitrary rough coefficients, extending the global optimality of Delaunay triangulations with respect to a discrete Dirichlet energy to weighted Delaunay triangulations. Next, we consider inverse homogenization, that is, the recovery of the microstructure from macroscopic information, a problem which is known to be both non-linear and severly ill-posed. We show how to decompose this reconstruction into a linear ill-posed problem and a well-posed non-linear problem. We apply this new geometric approach to Electrical Impedance Tomography (EIT) in dimension two. It is known that the EIT problem admits at most one isotropic solution. If an isotropic solution exists, we show how to compute it from any conductivity having the same boundary Dirichlet-to-Neumann map. This is of practical importance since the EIT problem always admits a unique solution in the space of divergence-free matrices and is stable with respect to G-convergence in that space (this property fails for isotropic matrices). As such, we suggest that the space of convex functions is the natural space to use to parameterize solutions of the EIT problem.

No.: 2009-02
ID: CaltechAUTHORS:20111011-163848887

]]>

Abstract: We consider linear divergence-form scalar elliptic equations and vectorial equations for elasticity with rough (L^∞(Ω), Ω ⊂ ℝ^d ) coefficients a(x) that, in particular, model media with non-separated scales and high contrast in material properties. While the homogenization of PDEs with periodic or ergodic coefficients and well separated scales is now well understood, we consider here the most general case of arbitrary bounded coefficients. For such problems we introduce explicit finite dimensional approximations of solutions with controlled error estimates, which we refer to as homogenization approximations. In particular, this approach allows one to analyze a given medium directly without introducing the mathematical concept of an ∈ family of media as in classical periodic homogenization. We define the flux norm as the L^2 norm of the potential part of the fluxes of solutions, which is equivalent to the usual H^1-norm. We show that in the flux norm, the error associated with approximating, in a properly defined finite-dimensional space, the set of solutions of the aforementioned PDEs with rough coefficients is equal to the error associated with approximating the set of solutions of the same type of PDEs with smooth coefficients in a standard space (e.g., piecewise polynomial). We refer to this property as the transfer property. A simple application of this property is the construction of finite dimensional approximation spaces with errors independent of the regularity and contrast of the coefficients and with optimal and explicit convergence rates. This transfer property also provides an alternative to the global harmonic change of coordinates for the homogenization of elliptic operators that can be extended to elasticity equations. The proofs of these homogenization results are based on a new class of elliptic inequalities which play the same role in our approach as the div-curl lemma in classical homogenization.

No.: 2009-03
ID: CaltechAUTHORS:20111012-105135181

]]>

Abstract: We introduce a new class of integrators for stiff ODEs as well as SDEs. An example of subclass of systems that we treat are ODEs and SDEs that are sums of two terms one of which has large coefficients. These integrators are (i) Multiscale: they are based on ow averaging and so do not resolve the fast variables but rather employ step-sizes determined by slow variables (ii) Basis: the method is based on averaging the ow of the given dynamical system (which may have hidden slow and fast processes) instead of averaging the instantaneous drift of assumed separated slow and fast processes. This bypasses the need for identifying explicitly (or numerically) the slow or fast variables. (iii) Non intrusive: A pre-existing numerical scheme resolving the microscopic time scale can be used as a black box and turned into one of the integrators in this paper by simply turning the large coefficients on over a microscopic timescale and off during a mesoscopic timescale. (iv) Convergent over two scales: strongly over slow processes and in the sense of measures over fast ones. We introduce the related notion of two scale ow convergence and analyze the convergence of these integrators under the induced topology. (v) Structure preserving: For stiff Hamiltonian systems (possibly on manifolds), they are symplectic, time-reversible, and symmetric (under the group action leaving the Hamiltonian invariant) in all variables. They are explicit and apply to arbitrary stiff potentials (that need not be quadratic). Their application to the Fermi-Pasta-Ulam problems shows accuracy and stability over 4 orders of magnitude of time scales. For stiff Langevin equations, they are symmetric (under a group action), time-reversible and Boltzmann-Gibbs reversible, quasi-symplectic on all variables and conformally symplectic with isotropic friction.

No.: 2009-04
ID: CaltechAUTHORS:20111012-110532817

]]>

Abstract: We propose an approach for efficiently simulating elastic objects made of non-homogeneous, non-isotropic materials. Based on recent developments in homogenization theory, a methodology is introduced to approximate a deformable object made of arbitrary fine structures of various linear elastic materials with a dynamically-similar coarse model. This numerical coarsening of the material properties allows for simulation of fine, heterogeneous structures on very coarse grids while capturing the proper dynamics of the original dynamical system, thus saving orders of magnitude in computational time. Examples including inhomogeneous and/or anisotropic materials can be realistically simulated in realtime with a numerically-coarsened model made of a few mesh elements.

Publication: ACM Transactions on Graphics Vol.: 28 No.: 3 ISSN: 0730-0301

ID: CaltechAUTHORS:20090911-153604312

]]>

Abstract: We propose an approach for efficiently simulating elastic objects made of non-homogeneous, non-isotropic materials. Based on recent developments in homogenization theory, a methodology is introduced to approximate a deformable object made of arbitrary fine structures of various linear elastic materials with a dynamicallysimilar coarse model. This numerical coarsening of the material properties allows for simulation of fine, heterogeneous structures on very coarse grids while capturing the proper dynamics of the original dynamical system, thus saving orders of magnitude in computational time. Examples including inhomogeneous and/or anisotropic materials can be realistically simulated in realtime with a numerically-coarsened model made of a few mesh elements.

ID: CaltechAUTHORS:20161116-134645591

]]>

Abstract: This paper presents a continuous and discrete Lagrangian theory for stochastic Hamiltonian systems on manifolds, akin to the Ornstein–Uhlenbeck theory of Brownian motion in a force field. The main result is to derive governing SDEs for such systems from a critical point of a stochastic action. Using this result, the paper derives Langevin-type equations for constrained mechanical systems and implements a stochastic analogue of Lagrangian reduction. These are easy consequences of the fact that the stochastic action is intrinsically defined. Stochastic variational integrators (SVIs) are developed using a discrete variational principle. The paper shows that the discrete flow of an SVI is almost surely symplectic and in the presence of symmetry almost surely momentum-map preserving. A first-order mean-squared convergent SVI for mechanical systems on Lie groups is introduced. As an application of the theory, SVIs are exhibited for multiple, randomly forced and torqued rigid bodies interacting via a potential.

Publication: IMA Journal of Numerical Analysis Vol.: 29 No.: 2 ISSN: 0272-4979

ID: CaltechAUTHORS:20090713-115112523

]]>

Abstract: In this paper, we consider numerical homogenization of acoustic wave equations with heterogeneous coefficients, namely, when the bulk modulus and the density of the medium are only bounded. We show that under a Cordes type condition the second order derivatives of the solution with respect to harmonic coordinates are L^2 (instead H^-1 with respect to Euclidean coordinates) and the solution itself is in L∞(0,T,H^2(Ω)) (instead of L∞(0,T,H^1(Ω)) with respect to Euclidean coordinates). Then, we propose an implicit time stepping method to solve the resulted linear system on coarse spatial scales, and present error estimates of the method. It follows that by pre-computing the associated harmonic coordinates, it is possible to numerically homogenize the wave equation without assumptions of scale separation or ergodicity.

Publication: Computer Methods in Applied Mechanics and Engineering Vol.: 198 No.: 3-4 ISSN: 0045-7825

ID: CaltechAUTHORS:OWHcmame08

]]>

Abstract: We apply concentration-of-measure inequalities to the quantification of uncertainties in the performance of engineering systems. Specifically, we envision uncertainty quantification in the context of certification, i.e., as a tool for deciding whether a system is likely to perform safely and reliably within design specifications. We show that concentration-of-measure inequalities rigorously bound probabilities of failure and thus supply conservative certification criteria. In addition, they supply unambiguous quantitative definitions of terms such as margins, epistemic and aleatoric uncertainties, verification and validation measures, confidence factors, and others, as well as providing clear procedures for computing these quantities by means of concerted simulation and experimental campaigns. We also investigate numerically the tightness of concentration-of-measure inequalities with the aid of an imploding ring example. Our numerical tests establish the robustness and viability of concentration-of-measure inequalities as a basis for certification in that particular example of application.

Publication: Computer Methods in Applied Mechanics and Engineering Vol.: 197 No.: 51-52 ISSN: 0045-7825

ID: CaltechAUTHORS:LUCcmame08

]]>

Abstract: In the last century, mercury levels in the global environment have tripled as a result of increased pollution from industrial, occupational, medicinal and domestic uses. Glutathione is known to be the main agent responsible for the excretion of mercury (Refs. 2 to 4). It has also been shown that mercury inhibits glutathione synthetase (an enzyme acting in the synthesization of glutathione), therefore leading to decreased glutathione levels (Refs. 5 to 7). Mercury also interferes with the production of heme in the porphyrin pathway. Heme is needed for biological energy production and ability to detox organic toxins via the P450 enzymes. The purpose of this paper is to show that the body’s response to mercury exposure is hysteretic, i.e. when this feedback of mercury on its main detoxifying agents is strong enough, then mercury body burden has two points of equilibrium: one with normal abilities to detoxify and low levels of mercury and one with inhibited abilities to detoxify and high levels of mercury. Furthermore, a small increase of the body’s mercury burden may not be sufficient to trigger observable neurotoxic effects but it may be sufficient to act as a switch leading to an accumulation of mercury in the body through environmental exposure until its toxicity is manifested.

Publication: Journal of Biological Systems Vol.: 16 No.: 1 ISSN: 0218-3390

ID: CaltechAUTHORS:20160226-104801314

]]>

Abstract: This paper addresses the issue of the homogenization of linear divergence form parabolic operators in situations where no ergodicity and no scale separation in time or space are available. Namely, we consider divergence form linear parabolic operators in $\Omega \subset \mathbb{R}^n$ with $L^\infty(\Omega \times (0,T))$-coefficients. It appears that the inverse operator maps the unit ball of $L^2(\Omega\times (0,T))$ into a space of functions which at small (time and space) scales are close in $H^1$ norm to a functional space of dimension $n$. It follows that once one has solved these equations at least $n$ times it is possible to homogenize them both in space and in time, reducing the number of operation counts necessary to obtain further solutions. In practice we show under a Cordes-type condition that the first order time derivatives and second order space derivatives of the solution of these operators with respect to caloric coordinates are in $L^2$ (instead of $H^{-1}$ with Euclidean coordinates). If the medium is time-independent, then it is sufficient to solve $n$ times the associated elliptic equation in order to homogenize the parabolic equation.

Publication: SIAM Journal on Numerical Analysis Vol.: 46 No.: 1 ISSN: 0036-1429

ID: CaltechAUTHORS:OWHsiamjna07

]]>

Abstract: A paradigm for isothermal, mechanical rectification of stochastic uctuations is introduced in this paper. The central idea is to transform energy injected by random perturbations into rigid-body rotational kinetic energy. The prototype considered in this paper is a mechanical system consisting of a set of rigid bodies in interaction through magnetic fields. The system is stochastically forced by white noise and dissipative through mechanical friction. The Gibbs-Boltzmann distribution at a specific temperature defines the unique invariant measure under the flow of this stochastic process and allows us to define "the temperature" of the system. This measure is also ergodic and strongly mixing. Although the system does not exhibit global directed motion, it is shown that global ballistic motion is possible (the mean-squared displacement grows like t^2). More precisely, although work cannot be extracted from thermal energy by the second law of thermodynamics, it is shown that ballistic transport from thermal energy is possible. In particular, the dynamics is characterized by a meta-stable state in which the system exhibits directed motion over random time scales. This phenomenon is caused by interaction of three attributes of the system: a non at (yet bounded) potential energy landscape, a rigid body effect (coupling translational momentum and angular momentum through friction) and the degeneracy of the noise/friction tensor on the momentums (the fact that noise is not applied to all degrees of freedom).

ID: CaltechAUTHORS:20160224-101438724

]]>

Abstract: Stochastic variational integrators for constrained, stochastic mechanical systems are developed in this paper. The main results of the paper are twofold: an equivalence is established between a stochastic Hamilton-Pontryagin (HP) principle in generalized coordinates and constrained coordinates via Lagrange multipliers, and variational partitioned Runge-Kutta (VPRK) integrators are extended to this class of systems. Among these integrators are first and second-order strongly convergent RATTLE-type integrators. We prove strong order of accuracy of the methods provided. The paper also reviews the deterministic treatment of VPRK integrators from the HP viewpoint.

ID: CaltechAUTHORS:20160224-100431574

]]>

Abstract: We consider divergence form elliptic operators in dimension n ≥ 2 with L∞ coefficients. Although solutions of these operators are only Hölder continuous, we show that they are differentiable (C1,α) with respect to harmonic coordinates. It follows that numerical homogenization can be extended to situations where the medium has no ergodicity at small scales and is characterized by a continuum of scales by transferring a new metric in addition to traditional averaged (homogenized) quantities from subgrid scales into computational scales and error bounds can be given. This numerical homogenization method can also be used as a compression tool for differential operators.

ID: CaltechAUTHORS:20160224-094627612

]]>

Abstract: Over the past years an intense work has been undertaken to understand the origin of the crashes and bubbles of financial markets. The explanations of these crashes have been grounded on the hypothesis of behavioral and social correlations between the agents in interacting particle models or on a feedback of the stock prices on trading behaviors in mean-field models (here bubbles and crashes are seen as collective hysteria). In this paper, we will introduce a market model as a particle system with no other interaction between the agents than the fact that to be able to sell, somebody must be willing to buy and no feedback of the price on their trading behavior. We will show that this model crashes in finite estimable time. Although the age of the market does not appear in the price dynamic the population of traders taken as a whole system is maturing towards collapse. The wealth distribution among the agents follows the second law of thermodynamics and with probability one an agent (or a minority of agents) will accumulate a large portion of the total wealth, at some point this disproportion in the wealth distribution becomes unbearable for the market leading to its collapse. We believe that the origin of the collapse in our model could be of some relevance in understanding long-term economic cycles such as the Kondratiev cycle.

Publication: Physica A Vol.: 343ISSN: 0378-4371

ID: CaltechAUTHORS:20190109-105904110

]]>

Abstract: In this paper we analyze the transport of passive tracers by deterministic stationary incompressible flows which can be decomposed over an infinite number of spatial scales without separation between them. It appears that a low order dynamical system related to local Peclet numbers can be extracted from these flows and it controls their transport properties. Its analysis shows that these flows are strongly self-averaging and super-diffusive: the delay τ (r) for any finite number of passive tracers initially close to separate till a distance r is almost surely anomalously fast (τ (r)∼ r^(2–ν) , with ν > 0). This strong self-averaging property is such that the dissipative power of the flow compensates its convective power at every scale. However as the circulation increases in the eddies the transport behavior of the flow may (discontinuously) bifurcate and become ruled by deterministic chaos: the self-averaging property collapses and advection dominates dissipation. When the flow is anisotropic a new formula describing turbulent conductivity is identified.

Publication: Communications in Mathematical Physics Vol.: 247 No.: 3 ISSN: 0010-3616

ID: CaltechAUTHORS:20160224-131618930

]]>

Abstract: This paper is concerned with the asymptotic behavior of solutions of stochastic differential equations $dy_t=d\omega_t -\nabla V(y_t)\, dt$, $y_0=0$. When $d=1$ and V is not periodic but obtained as a superposition of an infinite number of periodic potentials with geometrically increasing periods [$V(x) = \sum_{k=0}^\infty U_k(x/R_k)$, where $U_k$ are smooth functions of period 1, $U_k(0)=0$, and $R_k$ grows exponentially fast with k] we can show that $y_t$ has an anomalous slow behavior and we obtain quantitative estimates on the anomaly using and developing the tools of homogenization. Pointwise estimates are based on a new analytical inequality for subharmonic functions. When $d\geq 1$ and V is periodic, quantitative estimates are obtained on the heat kernel of $y_t$, showing the rate at which homogenization takes place. The latter result proves Davies' conjecture and is based on a quantitative estimate for the Laplace transform of martingales that can be used to obtain similar results for periodic elliptic generators.

Publication: Annals of Probability Vol.: 31 No.: 4 ISSN: 0091-1798

ID: CaltechAUTHORS:OWHaop03

]]>

Abstract: This paper is concerned with the approximation of the effective conductivity σ(A, μ) associated to an elliptic operator ∇_xA(x,η)∇_x where for xЄℝ^d,d ≥ 1, A(x,η) is a bounded elliptic random symmetric d×d matrix and η takes value in an ergodic probability space (X, μ). Writing A^N (x, η) the periodization of A(x, η) on the torus T^d_N of dimension d and side N we prove that for μ-almost all η lim ^(N→+∞) σ(A^N, η) = σ(A,μ) We extend this result to non-symmetric operators ∇_x (a+E(x, η))∇_x corresponding to diffusions in ergodic divergence free flows (a is d×d elliptic symmetric matrix and E(x, η) an ergodic skew-symmetric matrix); and to discrete operators corresponding to random walks on ℤ^d with ergodic jump rates. The core of our result is to show that the ergodic Weyl decomposition associated to L^2(X,μ) can almost surely be approximated by periodic Weyl decompositions with increasing periods, implying that semi-continuous variational formulae associated to L^2(X,μ) can almost surely be approximated by variational formulae minimizing on periodic potential and solenoidal functions.

Publication: Probability Theory and Related Fields Vol.: 125 No.: 2 ISSN: 0178-8051

ID: CaltechAUTHORS:20160224-124326510

]]>

Abstract: We show that the effective diffusivity matrix D(V^n) for the heat operator ∂_t − (Δ/2 − ∇V^n∇) in a periodic potential V^n = Σ^n_(k=0)U_k(x/R_k) obtained as a superposition of Hölder-continuous periodic potentials U_k (of period T^d:= ℝ^d/ℤ^d, d ∈ ℕ^*, U_k(0) = 0) decays exponentially fast with the number of scales when the scale ratios R_(k+1)/R_k are bounded above and below. From this we deduce the anomalous slow behavior for a Brownian motion in a potential obtained as a superposition of an infinite number of scales, dy_t = dω_t − ∇V^∞(yt)dt.

Publication: Communications on Pure and Applied Mathematics Vol.: 56 No.: 1 ISSN: 0010-3640

ID: CaltechAUTHORS:20160224-121708361

]]>

Abstract: This paper is concerned with the asymptotic behavior solutions of stochastic differential equations dy_t =dω_ t −∇Γ(y_t ) dt, y_0=0 and d=2. Γ is a 2 x 2 skew-symmetric matrix associated to a shear flow characterized by an infinite number of spatial scales Γ_(12) = −Γ_(21) = h(x_1), with h(x_1) = ∑_(n =0)^∞γ_n h^n (x_1/R_n ), where h^n are smooth functions of period 1, h^n (0)=0, γ_ n and R_n grow exponentially fast with n. We can show that y_t has an anomalous fast behavior (?[|y_t |^2]∼t^(1+ν) with ν > 0) and obtain quantitative estimates on the anomaly using and developing the tools of homogenization.

Publication: Communications in Mathematical Physics Vol.: 227 No.: 2 ISSN: 0010-3616

ID: CaltechAUTHORS:20160224-110658762

]]>

Abstract: This note reviews, compares and contrasts three notions of "distance" or "size" that arise often in concentration-of-measure inequalities. We review Talagrand′s convex distance and McDiarmid′s diameter, and consider in particular the normal distance on a topological vector space

Publication: International Journal for Uncertainty Quantification Vol.: 2 No.: 1 ISSN: 2152-5080

ID: CaltechAUTHORS:20170408-142731157

]]>