MATH4UQ Seminar


YouTube Channel

We are now on YouTube. You can find recordings of previous seminar talks on the MATH4UQ channel.


The MATH4UQ seminar features talks by internal and external colleagues and collaborators as well as guests visiting the chair. Everybody interested is welcome to attend.

Please subscribe to our MATH4UQ seminar mailing list to receive notifications about upcoming seminars. Recordings of several previous talks can also be found on our MATH4UQ YouTube channel .


Upcoming and past talks

Upcoming talks

  • 13.02.2024, Tuesday, 12:00 (CET)Dr. Robert Kunsch, RWTH Aachen
    • Title: Monte Carlo quadrature with optimal confidence
    • Abstract: 

      We study the numerical integration of smooth functions using finitely many function evaluations within randomized algorithms, aiming for the smallest possible error guarantees with high probability (the so-called confidence level). There are different strategies for constructing robust estimators, for example, taking the median of several repeated realizations of a basic method where the number of required repetitions depends on the desired confidence level. For Sobolev classes of continuous functions, however, we can find linear integration methods that have optimal error bounds for all confidence levels. Numerical experiments show that the tails of the error distribution are significantly smaller compared to previously known methods. 

    • Short Bio:
      From 2009 until 2017 I studied mathematics at the Friedrich Schiller University Jena. My doctorate studies from 2014 until 2017 have been supervised by Erich Novak who introduced me to Monte Carlo methods in the framework of information-based complexity, the dissertation was titled "High-dimensional function approximation: Breaking the curse with Monte Carlo methods". After two years as a PostDoc at Osnabrück University I came to RWTH Aachen in 2019, being a member of the Chair of "Mathematics of Information Processing" since then. In summer 2023 I spend a semester at TU Chemnitz as a substitute professor teaching courses on Monte Carlo methods and inverse problems. My research interest lies in the search for optimal (randomized) methods in different algorithmic settings where we only have access to partial information about problem instances, for example, finitely many function values in approximation or integration.

Previous talks (2023)

  • 05.12.2023, Tuesday, 13:30 (CET)Prof. Björn Sprungk, TU Bergakademie Freiberg
    • Title: Metropolis-adjusted interacting particle sampling.
    • Abstract: 

      In recent years, several interacting particle samplers have been proposed in order to sample approximately from a complicated target distributions such as posterior measures occuring in Bayesian inverse problems. These interacting particle samplers use an ensemble of interacting particles moving in the product state space according to coupled stochastic differential equations. In practice, we have to apply numerical time stepping to simulate these systems, such as the Euler-Maruyama scheme. However, the time discretization affects the invariance of the particle system with respect to the target distribution and, thus, introduces a bias. In order to correct for this we study the application of a Metropolisation step similar to the Metropolis-adjusted Langevin algorithm. We discuss ensemble- and particle-wise Metropolization and state the basic convergence of the resulting ensemble Markov chain to the product target distribution. We show the benefits of this correction in numerical examples for common interacting particle samplers such as the affine invariant interacting Langevin dynamics (ALDI), consensus-based sampling, and stochastic Stein variational gradient descent.

  • 28.11.2023, Tuesday, 12:00 (CET): Prof. Sebastian Reich,  University of Potsdam
    • Title:  An application of the EnKF to optimal control.
    • Abstract: 

      Stochastic optimal control problems are typically phrased in terms of the
      associated Hamilton-Jacobi-Bellman equation. Solving such partial differential equations remains 
      challenging. In this talk, an alternative approach involving forward and backward mean-field
      evolution equations will be considered. In its simplest incarnation, these equations become 
      equivalent to the formulations used in score generative modelling. General optimal control problems
      lead to more complex forward-backward mean-field equations which require further approximations for which
      we will employ variants of the ensemble Kalman filter. 

  • 21.11.2023, Tuesday, 12:00 (CET) Prof. David Pardo,  University of the Basque Country
    • Title: Multimodal Variational Autoencoder for Inverse Problems in Geophysics.
    • Abstract: 

      Estimating subsurface properties from geophysical measurements is a common challenge in inverse problems. In numerous geophysical applications, there is no unique solution; instead, there are multiple plausible solutions. In this context, some Bayesian methods are designed to solve geophysical inverse problems and their associated uncertainties. 

      In this presentation, based on [1], we describe a novel approach to solve geophysical inverse problems with uncertainty quantification (UQ): a multimodal variational autoencoder (MVAE) model that employs a mixture of truncated Gaussian densities to provide multiple solutions to the inverse problem. These solutions contain information about their probability of occurrence and their UQ.

      The MVAE model consists of two main components: an encoder and a decoder. The encoder employs a neural network to generate a mixture of truncated Gaussian densities, representing the distribution to the inverse problem solution. Conversely, the decoder computes the numerical solution to the forward problem based on geophysical principles.

      MVAE allows for a more comprehensive exploration of potential solutions to geophysical inverse problems, accommodating their inherent UQ and providing a more affluent understanding of the solutions space.

      The objective of the talk is (a) to describe the need for UQ in geophysical exploration methods, (b) introduce neural networks as a family of methods that introduces new opportunities toward achieving UQ, (c) explain MVAE as an example of a method that provides UQ, and more importantly, (d) open a collaboration space to develop new practical UQ methods in geophysics based on neural networks.

       [1] Rodriguez, O., Taylor, J. M., & Pardo, D. (2023). Multimodal variational autoencoder for inverse problems in geophysics: application to a 1-D Magnetotelluric problem. Geophysical Journal International, 235(3), 2598-2613.

  • 14.11.2023, Tuesday, 12:00 (CET) Xinzhu Liang, University of Manchester
    • Title: A randomized multi-index Monte Carlo method
    • Abstract: 

      We consider the problem of estimating expectations with respect to a target distribution with an unknown normalizing constant, and where even the unnormalized target needs to be approximated at finite resolution. Under such an assumption, we extend a recently introduced multi-index Sequential Monte Carlo (SMC) ratio estimator, which provably enjoys the complexity improvements of multi-index Monte Carlo (MIMC) and the efficiency of SMC for inference. The present work leverages a randomization strategy to remove bias entirely, which simplifies estimation substantially, particularly in the MIMC context, where the choice of index set is otherwise important. With theoretical results, the proposed method provably achieves the same canonical complexity of MSE-1 under appropriate assumptions as the original method, but without discretization bias. It is illustrated on examples of Bayesian inverse and spatial statistics problems.

  • 07.11.2023, Tuesday, 12:00 (CET) Prof. Jonas Latz, University of Manchester
    • Title: Subsampling in Continuous Time: from Optimisation to Sampling.
    • Abstract: 

      The Stochastic Gradient Langevin Dynamics (SGLD) are popularly used to approximate Bayesian posterior distributions in statistical learning procedures with large-scale data. As opposed to many usual Markov chain Monte Carlo (MCMC) algorithms, SGLD is not stationary with respect to the posterior distribution; two sources of error appear: The first error is introduced by an Euler--Maruyama discretisation of a Langevin diffusion process, the second error comes from the data subsampling that enables its use in large-scale data settings. In this work, we consider an idealised version of SGLD to analyse the method's pure subsampling error that we then see as a best-case error for diffusion-based subsampling MCMC methods. Based on a recent continuous-time formulation of the Stochastic Gradient Descent algorithm, we introduce and study the Stochastic Gradient Langevin Diffusion (SGLDiff). This dynamical system is a continuous-time Markov process that follows the Langevin diffusion corresponding to a data subset and switches this data subset after exponential waiting times. There, we show that the Wasserstein distance between the posterior and the limiting distribution of SGLDiff is bounded above by a fractional power of the mean waiting time. Importantly, this fractional power does not depend on the dimension of the state space. We bring our results into context with other analyses of SGLD.

  • 24.10.2023, Tuesday, 12:00 (CEST) Sophia Wiechert, RWTH Aachen University
    • Title: Generic Importance Sampling via Optimal Control for Stochastic Reaction Networks.
    • Abstract: 

      Stochastic reaction networks (SRNs) are a class of continuous-time discrete-state Markov processes describing the random interaction of d species through reactions. SRNs are commonly used for modeling diverse phenomena such as (bio)chemical reactions, epidemics, risk theory, queuing, and supply chain networks. In this context, we propose two alternative importance sampling (IS) approaches to improve the efficiency of Monte Carlo (MC) estimators for various statistical quantities. Our special interest lays in estimating rare event probabilities in high dimensional SRNs, in which d>>1. The challenge in IS is to choose an appropriate change of probability measure to achieve a substantial variance reduction. We propose an automated approach to obtain a highly efficient path-dependent measure change based on an original connection between finding optimal IS parameters and solving a variance minimization problem via a stochastic optimal control formulation. We pursue two alternative approaches to mitigate the curse of dimensionality when solving the resulting  backward Hamilton-Jacobi-Bellman (HJB) equation. In the first approach [1], we propose a learning-based method to approximate the value function using an ansatz function (e.g. a neural network), where the parameters are determined via a stochastic optimization algorithm. As an alternative, we present in [2] a dimension reduction method, based on mapping the problem to a significantly lower dimensional space via the Markovian projection (MP) idea. The output of this model reduction technique is a much lower dimensional SRN that preserves at all time steps the marginal distribution of the original high-dimensional problem. By solving the HJB equation for a projected lower dimensional process, we get projected IS parameters, which can be mapped back to the original $d$ dimensional SRN. Our analysis and numerical experiments verify that both proposed IS (learning based and MP-HJB) approaches substantially reduce the MC estimator’s variance, resulting in a lower computational complexity in the rare event regime than standard MC estimators.

      [1] Ben Hammouda, C., Ben Rached, N., Tempone, R., & Wiechert, S. (2023). Learning-based importance sampling via stochastic optimal control for stochastic reaction networks. Statistics and Computing, 33(3), 58.

      [2] Hammouda, C. B., Rached, N. B., Tempone, R., & Wiechert, S. (2023). Automated Importance Sampling via Optimal Control for Stochastic Reaction Networks: A Markovian Projection-based Approach. arXiv preprint arXiv:2306.02660.

  • 17.10.2023, Tuesday, 12:00 (CEST) Arved Bartuska, RWTH Aachen University
    • Title: Efficient nested integration estimators for expected information gain.
    • Abstract: 

      Nested integration problems consist of one outer and one inner integral separated by a nonlinear function. Estimating these integrals poses significant computational challenges. We employ the randomized quasi-Monte Carlo method to estimate both the outer and inner integrals, yielding a double-loop quasi-Monte Carlo estimator.  Furthermore, we derive asymptotic error bounds to obtain the optimal number of outer and inner samples, depending on a specified error tolerance. This estimator is then applied to find the expected information gain (EIG) of an experiment, allowing for efficient data collection. 

      Mathematical models of experiments often contain nuisance uncertainty, i.e., uncertainty about parameters that are not of interest to the experimenter directly. If these uncertainties are to be quantified rigorously, a second inner integral is added to the nested structure of the EIG. We utilize the Laplace approximation to derive two novel and efficient estimators that consider nuisance uncertainty and demonstrate the effectiveness of these estimators via three numerical examples.

  • 10.10.2023, Tuesday, 12:00 (CEST) Prof.  Ahmed Kebaier, University of Evry-University Paris-Saclay
    • Title: The interpolated drift implicit Euler scheme Multilevel Monte Carlo method for pricing Barrier options and applications to the CIR and CEV models.
    • Abstract: 

      Recently, Giles et al. [14] proved that the efficiency of the Multilevel Monte Carlo (MLMC) method for evaluating Down-and-Out barrier options for a diffusion process (Xt) t∈[0,T] with globally Lipschitz coefficients, can be improved by combining a Brownian bridge technique and a conditional Monte Carlo method provided that the running minimum inf t∈[0,T] Xt has a bounded density in the vicinity of the barrier. 
      In the present work, thanks to the Lamperti transformation technique and using a Brownian interpolation of the drift implicit Euler scheme of Alfonsi [2], we show that the efficiency of the MLMC can be also improved for the evaluation of barrier options for models with non-Lipschitz diffusion coefficients under certain moment constraints. We study two example models: the Cox-Ingersoll-Ross (CIR) and the Constant of Elasticity of Variance (CEV) processes for which we show that the conditions of our theoretical framework are satisfied under certain restrictions on the models parameters. In particular, we develop semi-explicit formulas for the densities of the running minimum and running maximum of both CIR and CEV processes which are of independent interest. Finally, numerical tests are processed to illustrate our results.

      [14] M. B. Giles, K. Debrabant, and A. Rössler. 'Analysis of multilevel Monte Carlo path simulation using the Milstein discretisation.' Discrete Contin. Dyn. Syst. Ser. B, 24(8):3881–3903, 2019.

      [2] A. Alfonsi. 'Strong order one convergence of a drift implicit euler scheme: Application to the cir process. Statistics & Probability Letters, 83(2):602–607, 2013.

  • 09.05.2023, Tuesday, 15:00 (CEST) Prof.  Michael Herty, IGPM, RWTH Aachen University
    • Title: UQ for hyperbolic problems
    • Abstract: 

      We are interested in quantifying uncertainties that appear in nonlinear hyperbolic partial differential equations arising in a variety of applications from fluid flow to traffic modeling. A common approach to treat the stochastic components of the solution is by using generalized polynomial chaos expansions. This method was successfully applied in particular for general elliptic and parabolic PDEs as well as linear hyperbolic stochastic equations. More recently, gPC methods have been successfully applied to particular hyperbolic PDEs using the explicit form of nonlinearity or the particularity of the studied system structure as, e.g., in the p-system. While such models arise in many applications, e.g., in atmospheric flows, fluid flows under uncertain gas compositions and shallow water flows, a general gPC theory with corresponding numerical methods are still at large. Typical analytical and numerical challenges that appear for the gPC expanded systems are loss of hyperbolicity and positivity of solutions (like gas density or water depth). Any of those effects might trigger severe instabilities within classical finite-volume or discontinuous Galerkin methods. We will discuss properties and conditions to guarantee stability and present numerical results on selected examples.

  • 02.05.2023, Tuesday, 15:00 (CEST)  Dr. Burcu Aydogan, RWTH Aachen University
    • Title: Optimal investment strategies under the relative performance in jump-diffusion markets 
    • Abstract: 

      We work on a portfolio management problem for a representative agent and a group of people, forming a market under relative performance concerns in a continuous-time setting. Herein, we define two wealth dynamics: the agent’s and the market’s wealth. The wealth dynamics appear in jump-diffusion markets. In our setting, we measure the performances of the market and the individual agent with preferences linked to the market performance. Therefore, we have two classical Merton problems to determine what the market does and the agent’s optimal strategy relative to the market performance. Furthermore, our framework assumes that the agent’s utility performance does not affect the market, while the market affects the agent’s utility. We explore the optimal investment strategies for both the agent and the market.

      This is a joint work with Mogens Steffensen in the University of Copenhagen.

  • 18.04.2023, Tuesday, 15:00 (CEST) Prof. Ian Hugh Sloan, University of New South Wales (UNSW Australia)
    • Title: High dimensional approximation – avoiding the curse of dimensionality, doubling the proven convergence rate.
    • Abstract: 

      High dimensional approximation problems commonly arise from parametric PDE problems in which the parametric input depends on very many independent univariate random variables.  Typically (as in the method of “generalized polynomial chaos”, or GPC) the dependence on these variables is modelled by multivariate polynomials, leading to exponentially increasing difficulty and cost (expressed as the “curse of dimensionality”) as the dimension increases.  For this reason sparsity of coefficients is a major focus in implementations of GPC.

      In this lecture we develop a different approach to one version of GPC.  In  this method there is no need for sparsification, and no curse of dimensionality.    The method, proposed in a 2022 paper with Frances Kuo, Vesa Kaarnioja, Yoshihito Kazashi and Fabio Nobile, uses kernel interpolation with periodic kernels, with the kernels located at lattice points, as advocated long ago by Hickernell and colleagues.

      The lattice points and the kernels depend on parameters called “weights”.  In the 2022 paper the recommended weights were “SPOD” weights, leading to a cost growing as the square of the number of lattice points.   A newer 2023 paper with Kuo and Kaarnioja introduced “serendipitous” weights, for which the cost grows only linearly with both dimension and number of lattice points, allowing practical computations in as many as 1,000 dimensions.

      The rate of convergence proved in the above papers was of the order $n^{-\alpha/2}$, for interpolation using the reproducing kernel of a space with mixed smoothness of order $\alpha$.  A new result with Frances Kuo doubles the proven convergence rate to $n^{-\alpha}$.

  • 11.04.2023, Tuesday, 15:00 (CEST) Yang Liu, King Abdullah University of Science and Technology
    • Title: Goal-oriented adaptive finite element multilevel Monte Carlo with convergence rates
    • Abstract: 

      We propose our Adaptive Multilevel Monte Carlo (AMLMC) [Beck, Joakim, et al. "Goal-oriented adaptive finite element multilevel Monte Carlo with convergence rates." Computer Methods in Applied Mechanics and Engineering (2022)] method to solve an elliptic partial differential equation (PDE) with lognormal random input data, where the PDE model is subject to geometry-induced singularity. 

      The previous work [Moon, K-S., et al. "Convergence rates for an adaptive dual weighted residual finite element algorithm." BIT Numerical Mathematics 46.2 (2006)] developed convergence rates for a goal-oriented adaptive algorithm based on isoparametric d-linear quadrilateral finite element approximations and the dual weighted residual error representation in the deterministic setting. This algorithm refines the mesh based on the error contribution to the QoI. 

      This work aims to combine MLMC and the adaptive finite element solver. Contrary to the standard Multilevel Monte Carlo methods, where each sample is computed using a discretization-based numerical method, whose resolution is linked to the level, our AMLMC algorithm uses a sequence of tolerances as the levels. Specifically, for a given realization of the input coefficient and a given accuracy level, the AMLMC constructs its approximate sample as the ones using the first mesh in the sequence of deterministic, non-uniform meshes generated by the above-mentioned adaptive algorithm that satisfies the sample-dependent bias constraint.

  • 28.03.2023, Tuesday, 15:00 (CEST) Dr. André Carlon, KAUST
    • Title:  Bayesian quasi-Newton method for stochastic optimization
    • Abstract: 

      Stochastic optimization problems arise in many fields, like data sciences, reliability engineering, and finance. The stochastic gradient descent (SGD) method is a cheap approach to solving such problems, relying on noisy gradient estimates to converge to local optima. However, in the case of μ-convex, L-smooth problems, the convergence is deeply affected by the condition number of the problem, L/μ. Here, we propose a Bayesian approach to find a suitable matrix to pre-condition gradient estimates in stochastic optimization that reduces the effect of large conditioning numbers. We show that maximizing the posterior distribution to find a suitable pre-conditioning matrix is a constrained deterministic strongly convex problem that can be solved efficiently using the Newton-CG method with a path-following approach. Numerical results on stochastic problems with large condition numbers show that our Bayesian quasi-Newton pre-conditioner improves the convergence of SGD.

  • 21.03.2023, Tuesday, 15:00 (CET) Prof. Antonis Papapantoleon, the Delft Institute of Applied Mathematics,  Institute of Applied and Computational Mathematics, FORTH
    • Title:A splitting deep Ritz method for multi-asset option pricing in Lévy models
    • Abstract: 

      Solving high-dimensional differential equations is still a challenging field for researchers. In recent years, many works have been presented that provide approximation by training neural networks using loss functions based on the differential operator of the equation at hand, as well as its initial/terminal and boundary conditions. In this work, we use a machine learning approach for pricing European (basket) options written with respect to a set of correlated underlyings whose dynamics undertake random jumps. We approximate the solution of the corresponding partial integro-differential equation using a variant of the deep Ritz method that splits the differential operator into symmetric and asymmetric parts. The method is driven by a modified version of the neural network introduced in the deep Galerkin method. The structure of the proposed neural network ensures the asymptotic behavior of the solution for large values of the underlyings. Moreover, it leads the outputs of the network to be consistent with the prior known qualitative properties of the solution. We present results on the Merton jump-diffusion model.

  • 14.03.2023, Tuesday, 15:00 (CET)Prof. Dr. Markus Bachmayr, RWTH Aachen University
    • Title: Optimality of adaptive stochastic Galerkin methods for affine-parametric elliptic PDEs
    • Abstract: 

      We consider the computational complexity of approximating elliptic PDEs with random coefficients by sparse product polynomial expansions. Except for special cases (for instance, when the spatial discretisation limits the achievable overall convergence rate), previous approaches for a posteriori selection of polynomial terms and corresponding spatial discretizations do not guarantee optimal complexity in the sense of computational costs scaling linearly in the number of degrees of freedom. We show that one can achieve optimality of an adaptive Galerkin scheme for discretizations by spline wavelets in the spatial variable when a multiscale representation of the affinely parameterized random coefficients is used. This is joint work with Igor Voulis.

      M. Bachmayr and I. Voulis, An adaptive stochastic Galerkin method based on multilevel expansions of random fields: Convergence and optimality, ESAIM:M2AN 56(6), pp. 1955-1992, 2022. Preprint: arXiv:2109:09136.

  • 07.03.2023, Tuesday, 15:00 (CET)Prof. dr. ir. C.W. (Kees) Oosterlee, Utrecht University
    • Title: On the application of machine learning to enhance algorithms in computational finance
    • Abstract: In this presentation, we will first give a brief overview of our experiences with the use of artificial neural networks (ANNs) in finance. We'll give an example of supervised, unsupervised and reinforcement learning.

      After this we will outline the use of neural networks for the calibration of a financial asset price model in the context of financial option pricing. To provide an efficient calibration framework, a data-driven approach is proposed to learn the solutions of financial models and to reduce the corresponding computation time significantly. 
      Specifically, fitting model parameters is formulated as training hidden neurons within a machine-learning framework.
      The rapid on-line computation of ANNs combined with a flexible optimization method (i.e. Differential Evolution) provides us fast calibration without getting stuck in local minima.

  • 21.02.2023, Tuesday, 15:00 (CET)Shyam Mohan Subbiah Pillai, RWTH Aachen University
    • Title: Importance sampling for McKean-Vlasov stochastic differential equation
    • Abstract: 

      We are interested in Monte Carlo (MC) methods for estimating probabilities of rare events associated with solutions to the McKean-Vlasov stochastic differential equation (MV-SDE). MV-SDEs arise in the mean-field limit of stochastic interacting particle systems, which have many applications in pedestrian dynamics, collective animal behaviour and financial mathematics. Importance sampling (IS) is used to reduce high relative variance in MC estimators of rare event probabilities. Optimal change of measure is methodically derived from variance minimisation, yielding a high-dimensional partial differential control equation which is cumbersome to solve. This problem is circumvented by using a decoupling approach, resulting in a lower dimensional control PDE. The decoupling approach necessitates the use of a double Loop Monte Carlo (DLMC) estimator. We further combine IS with a novel multilevel DLMC estimator which not only reduces complexity from O(TOL-4) to O(TOL-3) but also drastically reduces associated constant, enabling computationally feasible estimation of rare event probabilities.

  • 14.02.2023, Tuesday, 15:00 (CET)Christian Bayer, Weierstrass Institute for Applied Analysis and Stochastics
    • Title: Motivated by the challenges related to the calibration of financial models, we consider the problem of solving numerically a singular McKean-Vlasov equation
    • Abstract: 

      d S_t= \sigma(t,S_t) S_t \frac{\sqrt v_t}{\sqrt {\mathbb{E}[v_t|S_t]}}dW_t,
      where $W$ is a Brownian motion and $v$ is an adapted diffusion process. This equation can be considered as a singular local stochastic volatility model.
      Whilst such models are quite popular among practitioners, unfortunately, its well-posedness has not been fully understood yet and, in general, is possibly not guaranteed at all.
      We develop a novel regularization approach based on the reproducing kernel Hilbert space (RKHS) technique and show that  the regularized  model is well-posed.  Furthermore, we prove propagation of chaos. We demonstrate numerically  that a thus regularized model is able to perfectly replicate option prices due to typical local volatility models. Our results are also applicable to more general McKean--Vlasov equations.

      (Joint work with Denis Belomestny, Oleg Butkovsky, and John Schoenmakers.)

  • 07.02.2023, Tuesday, 15:00 (CET)Prof. Per-Christian Hansen, Technical University of Denmark
    • Title:   Edge-Preserving Computed Tomography (CT) with Uncertain View Angles.
    • Abstract: In computed tomography, data consist of measurements of the attenuation of X-rays passing through an object. The goal is to reconstruct an image of the linear attenuation coefficient of the object's interior. For each position of the X-ray source, characterized by its angle with respect to a fixed coordinate system, one measures a set of data referred to as a view. A common assumption is that these view angles are known, but in some applications they are known with imprecision.

      We present a Bayesian inference approach to solving the joint inverse problem for the image and the view angles, while also providing uncertainty estimates. For the image, we impose a Laplace difference prior enabling the representation of sharp edges in the image; this prior has connections to total variation regularization. For the view angles, we use a von Mises prior which is a 2π-periodic continuous probability distribution.
      Numerical results show that our algorithm can jointly identify the image and the view angles, while also providing uncertainty estimates of both. We demonstrate our method with simulations of a 2D X-ray computed tomography problems using fan beam configurations.

      This is joint work with N. A. B. Riis and Y. Dong, Technical University of Denmark; F. Uribe, LUT University, Finland; and J. M. Bardsley, University of Montana.


Previous talks (2022)

  • 06.12.2022, Tuesday, 15:00 (CET)Prof.  Michael Feischl, TU Wien (Institute for Analysis and Scientific Computing)
    • Title: A quasi-Monte Carlo data compression algorithm for machine learning.
    • Abstract:  We present an algorithm to reduce large data sets using so-called digital nets, which are well distributed point sets in the unit cube. The algorithm efficiently scans the data and computes certain data dependent weights. Those weights are used to approximately represent the data, without making any assumptions on the distribution of the data points. Under smoothness assumptions on the model, we then show that this can be used to reduce the computational effort needed in finding good parameters in machine learning problems which aim to minimize standard loss functions. While the principal idea of the approximation might also work with other point sets, the particular structural properties of digital nets can be exploited to make the computation of the necessary weights extremely fast.  
  • 29.11.2022, Tuesday, 15:00 (CET) Dr. Truong-Vinh  Hoang, Chair of Mathematics for Uncertainty Quantification at RWTH Aachen University
    • Title:  A likelihood-free nonlinear filtering approach using the machine-learning-based approximation of conditional expectation. 
    • Abstract: We discuss the machine learning-based ensemble conditional mean filter (ML-EnCMF) developed  for the nonlinear data assimilation based on the orthogonal projection of the conditional mean. The updated mean of the filter matches that of the posterior. Moreover, we show that the filter's updated covariance coincides with the expected conditional covariance. Implementing the EnCMF requires computing the conditional mean. A likelihood-based estimator is prone to significant errors for small ensemble sizes, causing filter divergence. We develop a systematical methodology for integrating machine learning into the EnCMF using the conditional expectation's orthogonal projection
      property. First, we use a combination of an artificial neural network (ANN) and a linear function, obtained based on the ensemble Kalman filter (EnKF), to approximate the conditional mean, enabling the ML-EnCMF to inherit EnKF's advantages. Secondly, we apply a suitable variance reduction technique to reduce statistical errors when estimating loss function. Lastly, we propose a model selection procedure for element-wisely selecting the applied filter. We demonstrate the ML-EnCMF performance using the Lorenz-63 and Lorenz-96 systems and show that the ML-EnCMF outperforms the EnKF and the likelihood based EnCMF.
  • 22.11.2022, Tuesday, 15:00 (CET)Prof. Raúl Tempone, RWTH Aachen University and KAUST
    • Title:  A simple approach to proving the existence, uniqueness, and strong and weak convergence rates for a broad class of McKean-Vlasov equations.
    • Abstract: By employing a system of interacting stochastic particles as an approximation of the McKean–Vlasov equation and utilizing classical stochastic analysis tools, namely Itô’s formula and Kolmogorov–Chentsov continuity theorem, we prove the existence and uniqueness of strong solutions for a broad class of McKean–Vlasov equations as a limit of the conditional expectation of exchangeable particles. Considering an increasing number of particles in the approximating stochastic particle system, we also prove the Lp  strong convergence rate and derive the weak convergence rates using the Kolmogorov backward equation and variations of the stochastic particle system. Our convergence rates were verified by numerical experiments which also indicate that the assumptions made here and in the literature can be relaxed.
  • 15.11.2022, Tuesday, 15:00 (CET) Dr. Aretha Teckentrup, University of Edinburgh
    • Title:  Gaussian process regression in inverse problems and Markov chain Monte Carlo.
    • Abstract: We are interested in the inverse problem of estimating unknown parameters in a mathematical model from observed data. We follow the Bayesian approach, in which the solution to the inverse problem is the distribution of the unknown parameters conditioned on the observed data, the so-called posterior distribution. We are particularly interested in the case where the mathematical model is non-linear and expensive to simulate, for example, given by a partial differential equation. A major challenge in the application of sampling methods such as Markov chain Monte Carlo is then the large computational cost associated with simulating the model for a given parameter value. To overcome this issue, we consider using Gaussian process regression to approximate the likelihood of the data. This results in an approximate posterior distribution, to which sampling methods can be applied with feasible computational cost. In this talk, we will show how the uncertainty estimate provided by Gaussian process regression can be incorporated into the approximate Bayesian posterior distribution to avoid overconfident predictions and present efficient Markov chain Monte Carlo methods in this context.
  • 08.11.2022, Tuesday, 15:00 (CET)  Prof.  Mike Giles, Mathematical Institute, University of Oxford
    • Title:  MLMC techniques for discontinuous functions
    • Abstract: The Multilevel Monte Carlo (MLMC) approach usually works well when estimating the expected value of a quantity which is a Lipschitz function of intermediate quantities, but if it is a discontinuous function it can lead to a much slower decay in the variance of the MLMC correction.  In this talk I will review the existing literature on techniques which can be used to overcome this challenge in a variety of different contexts, and discuss recent developments using either a branching diffusion or adaptive sampling.
  • 25.10.2022, Tuesday, 15:00 (CEST) Dr. Alexander Litvinenko, RWTH Aachen University
    • Title:  Computing f-Divergences and Distances of High-Dimensional Probability Density Functions
    • Abstract: Very often, in the course of uncertainty quantification tasks or data analysis, one has to deal with high-dimensional random variables (RVs).  Just like any other RV, a high-dimensional RV can be described by its probability density (pdf) and/or by the corresponding probability characteristic functions (pcf),  or a more general representation as a function of other, known, random variables. Here the interest is mainly to compute characterisations like the entropy, the Kullback-Leibler, or more general $f$-divergences. These are all computed from the \pdf, which is often not available directly,  and it is a computational challenge to even represent it in a numerically feasible fashion in case the dimension $d$ is even moderately large.   It is an even stronger numerical challenge to then actually compute said characterisations in the high-dimensional case.  In this regard, in order to achieve a computationally feasible task, we propose to approximate density by a low-rank tensor. This allows us to reduce the computational complexity and storage cost from exponential to almost linear.

      The characterisations such as entropy or the $f$-divergences need the possibility to compute point-wise functions of the \pdf.  This normally rather trivial task becomes more difficult when the \pdf is approximated in a low-rank tensor format, as the point values are not directly accessible anymore. The data is considered as an element of a high order tensor space.   All we require is that the data can be considered as an element of an associative, commutative algebra with an inner product. Such an algebra is isomorphic to a commutative sub-algebra of the usual matrix algebra, allowing the use of matrix algorithms to accomplish the mentioned tasks. The algorithms to be described are iterative methods or truncated series expansions for particular functions of the tensor, which will then exhibit the desired result.

  • 18.10.2022, Tuesday, 15:00 (CEST) Sankarasubramanian Ragunathan, RWTH Aachen University
    • Title:  Higher-order adaptive methods for computing exit times of Itô diffusions
    • Abstract: We construct a higher-order adaptive method for strong approximations of exit times of Itô stochastic differential equations (SDE). The method employs a strong Itô--Taylor scheme for simulating SDE paths, and adaptively decreases the step-size in the numerical integration as the solution approaches the boundary of the domain. These techniques turn out to complement each other nicely: adaptive time-stepping improves the accuracy of the exit time by reducing the magnitude of the overshoot of the numerical solution when it exits the domain, and higher-order schemes improve the approximation of the state of the diffusion process. We present two versions of the higher-order adaptive method. The first one uses the Milstein scheme as the numerical integrator and two step-sizes for adaptive time-stepping. The second method is an extension of the first one using the strong It\^o--Taylor scheme of order 1.5 as the numerical integrator and three step-sizes for adaptive time-stepping. For any $\xi>0$, we prove that the strong error is bounded by $O(h^{1-\xi})$ and $O(h^{3/2-\xi})$ for the first and second methods, respectively. Under some conditions, we show that the expected computational cost of both methods is bounded by $O(h^{-1} \abs{\log(h)})$, indicating that both methods are tractable.  The theoretical results are supported by numerical examples, and we discuss the potential for extensions that improve the strong convergence rate even further.
  • 11.10.2022, Tuesday, 15:00 (CEST) Prof. Dr. Christoph Belak, Technische Universität Berlin
    • Titel:  Convergence of Deep Solvers for Semilinear PDEs.
    • Zusammenfassung: We derive convergence rates for a deep solver for semilinear partial differential equations which is based on a Feynman-Kac representation in terms of an uncoupled forward-backward stochastic differential equation and a discretization in time. We show that the error of the deep solver is bounded in terms of its loss functional, hence yielding a direct measure to judge the quality in numerical applications, and that the loss functional converges sufficiently fast to zero to guarantee that the approximation error vanishes in the limit. As a consequence of these results, we show that the deep solver has a strong convergence rate of order 1/2.


  • 04.10.2022, Tuesday, 15:00 (CEST) Prof. Peter Tankov,  ENSAE (National School of Statistics and Economic Administration), Institut Polytechnique de Paris.
    • Title:  Asset pricing under transition scenario uncertainty.
    • Abstract: The uncertain impact of environmental transition on the economy and the financial system may be quantified through integrated assessment model scenarios, published by international bodies such as the International Energy Agency or the Network for Greening the Financial System. However, these scenarios are very rarely updated and therefore not suitable for, dynamic risk management, pricing, and hedging. In this work, we introduce dynamic climate scenario uncertainty through Bayesian learning, by assuming that the economic agent acquires information about the scenario progressively by observing a signal, such as the carbon price or the CO2 emissions level. The framework is illustrated with two financial engineering applications: valuing an energy asset through the real options approach, and pricing a corporate bond subject to transition risk.
  • 28.06.2022, 16:00: Prof. Dr.  Michael Multerer USI Lugano.
    • Titel: Shape Uncertainty Quantification.
    • Zusammenfassung: The numerical simulation of physical phenomena is well understood given that the input data are known exactly. In practice, however, the collection of these data is usually subject to measurement errors. The goal of uncertainty quantification is to assess these errors and their impact on simulation results. In this talk, we address different numerical aspects of uncertainty quantification in partial differential equations on random domains.  Starting from the modeling of random domains via random vector fields, we discuss how the corresponding Karhunen-Loeve expansion can efficiently be computed.  In particular, we provide a means to rigorously control the approximation error. Moreover, we will discuss how measurement data can be incorporated into the model by means of Bayesian inversion. We provide numerical results to illustrate the presented approaches.
  • 21.06.2022, 13:00: Dr. Zdravko Botev, UNSW.
    • Titel: Efficient Truncated Multivariate Student Computations with Applications.
    • Zusammenfassung: We consider computations with the multivariate student density, truncated on a set described by a linear system of inequalities. Our goal is to both simulate from this truncated density, as well as to estimate its normalizing constant. We propose a bounded relative error estimator, as well as a reject-regenerate sampling method. We present applications and some extensions of the method.
  • 14.06.2022, 16:00: Prof. Michael Tretyakov, University of Nottingham.
    • Titel: Uncertainty Quantification in Composites Manufacturing.
    • Zusammenfassung: Fibre-reinforced composite materials in aerospace, automotive, green energy and marine industries and other areas has seen a significant growth over the last two decades. One of the main types of manufacturing processes for producing advanced composites is the Liquid Composite Moulding (LCM) which, in particular, includes Resin Transfer Moulding (RTM). The following UQ aspects related to RTM or more generally LCM will be discussed: examples of the forward models; practical Bayesian inversion to estimate local permeability and porosity of fibrous reinforcements and to discover defects using measured values of resin pressure and flow front positions during resin injection; how densely located sensors should be to ensure inferring positions and shapes of defects of a given size and defects’ severity; possibilities for in-process control of the flow front propagation. The talk is based on joint works with Marco Iglesias, Mikhail Matveev, Andreas Endruweit, Andy Long, Mihno Park.
  • 07.06.2022, 16:00: Dr.  Mihai Anitescu Argonne National Laboratory and University of Chicago.
    • Titel: Scalable Physics-based Maximum Likelihood Estimation using Hierarchical Matrices.
    • Zusammenfassung: Physics-based covariance models provide a systematic way to construct covariance models that are consistent with the underlying physical laws in Gaussian process analysis. The unknown parameters in the covariance models can be estimated using maximum likelihood estimation, but direct construction of the covariance matrix and classical strategies of computing with it requires $n$ physical model runs,  $n^2$ storage complexity, and $n^3$ computational complexity. To address such challenges, we propose to approximate the discretized covariance function using hierarchical matrices. By utilizing randomized range sketching for individual off-diagonal blocks, the construction process of the hierarchical covariance approximation requires $O(\log{n})$ physical model applications and the maximum likelihood computations require $O(n\log^2{n})$ effort per iteration. We propose a new approach to compute exactly the trace of products of hierarchical matrices which results in the expected Fischer information matrix being computable in $O(n\log^2{n})$  as well. The construction is totally matrix-free and the derivatives of the covariance matrix can then be approximated in the same hierarchical structure by differentiating the whole process. Numerical results are provided to demonstrate the effectiveness, accuracy, and efficiency of the proposed method.
  • 10.05.2022, 16:00: Dr.  Alen Alexanderian Department of Mathematics, North Carolina State University.
    • Titel: Design of large-scale Bayesian inverse problems governed by PDEs under uncertainty.
    • Zusammenfassung: We will discuss methods for optimal experimental design for Bayesian inverse problems governed by partial differential equations (PDEs) with infinite-dimensional parameters. Specifically, we will consider problems where one seeks to optimize the placement of measurement points (e.g., sensors), at which measurement data are collected, in such a way that the uncertainty in the estimated parameters is minimized. We will focus on the cases where the governing PDEs include uncertain parameters, in addition to those being estimated. We will discuss design of such inverse problems as well as sensitivity analysis with respect to the additional model uncertainties. 
  • 03.05.2022, 16:00: Dr.  Jonas Latz,  Heriot-Watt University, Maxwell Institute for Mathematical Sciences.
    • Titel: Stochastic gradient descent in continuous time: discrete and continuous data.
    • Zusammenfassung: Optimisation problems with discrete and continuous data appear in statistical estimation, machine learning, functional data science, robust optimal control, and variational inference. The 'full' target function in such an optimisation problem is given by the integral over a family of parameterised target functions with respect to a discrete or continuous probability measure. Such problems can often be solved by stochastic optimisation methods: performing optimisation steps with respect to the parameterised target function with randomly switched parameter values. In this talk, we discuss a continuous-time variant of the stochastic gradient descent algorithm. This so-called stochastic gradient process couples a gradient flow minimising a parameterised target function and a continuous-time 'index' process which determines the parameter.We first briefly introduce the stochastic gradient processes for finite, discrete data which uses pure jump index processes. Then, we move on to continuous data. Here, we allow for very general index processes: reflected diffusions, pure jump processes, as well as other Lévy processes on compact spaces. Thus, we study multiple sampling patterns for the continuous data space. We show that the stochastic gradient process can approximate the gradient flow minimising the full target function at any accuracy. Moreover, we give convexity assumptions under which the stochastic gradient process with constant learning rate is geometrically ergodic. In the same setting, we also obtain ergodicity and convergence to the minimiser of the full target function when the learning rate decreases over time sufficiently slowly.
  • 26.04.2022, 16:00: Prof. Dr.  Arnulf Jentzen, The Chinese University of Hong Kong and University of Münster.
    • Titel: Overcoming the curse of dimensionality: from nonlinear Monte Carlo to deep learning.
    • Zusammenfassung:   Partial differential equations (PDEs) are among the most universal tools used in modelling problems in nature and man-made complex systems. For example, stochastic PDEs are a fundamental ingredient in models for nonlinear filtering problems in chemical engineering and weather forecasting, deterministic Schroedinger PDEs describe the wave function in a quantum physical system, deterministic Hamiltonian-Jacobi-Bellman PDEs are employed in operations research to describe optimal control problems where companys aim to minimise their costs, and deterministic Black-Scholes-type PDEs are highly employed in portfolio optimization models as well as in state-of-the-art pricing and hedging models for financial derivatives. The PDEs appearing in such models are often high-dimensional as the number of dimensions, roughly speaking, corresponds to the number of all involved interacting substances, particles, resources, agents, or assets in the model. For instance, in the case of the above mentioned financial engineering models the dimensionality of the PDE often corresponds to the number of financial assets in the involved hedging portfolio. Such PDEs can typically not be solved explicitly and it is one of the most challenging tasks in applied mathematics to develop approximation algorithms which are able to approximatively compute solutions of high-dimensional PDEs. Nearly all approximation algorithms for PDEs in the literature suffer from the so-called "curse of dimensionality" in the sense that the number of required computational operations of the approximation algorithm to achieve a given approximation accuracy grows exponentially in the dimension of the considered PDE. With such algorithms it is impossible to approximatively compute solutions of high-dimensional PDEs even when the fastest currently available computers are used. In the case of linear parabolic PDEs and approximations at a fixed space-time point, the curse of dimensionality can be overcome by means of Monte Carlo approximation algorithms and the Feynman-Kac formula. In this talk we prove that suitable deep neural network approximations do indeed overcome the curse of dimensionality in the case of a general class of semilinear parabolic PDEs and we thereby prove, for the first time, that a general semilinear parabolic PDE can be solved approximatively without the curse of dimensionality.
  • 05.04.2022, 16:00: Prof. Dr.  Björn Sprungk, TU Bergakademie Freiberg.
    • Titel: Stability of Uncertainty Quantification and Bayesian Inverse Problems.
    • Zusammenfassung: For partial differential equations with random coefficients we investigate the sensitivity of the distribution of the random solution with respect to perturbations in the input distribution for the unknown data. We prove a local Lipschitz continuity with respect to total variation as well as Wasserstein distance and extend our sensitivity analysis also to quantities of interest of the solution as well as risk functionals applied to such quantities. Here, we provide a novel result for the sensitivity of coherent risk functionals with respect to the underlying probability distribution. Besides these sensitivity results for the propagation of uncertainty, we also investigate the inverse problem, i. e., Bayesian inference for the unknown coefficients given noisy observations of the solution. Although well-posedness of Bayesian inverse problems is well-known, we extend the local Lipschitz stability of the posterior to pertubations of the prior. Again we consider stability in the Wasserstein distance as well as with respect to several other common metrics for probability measures. However, our explicit bounds indicate a growing sensitivity of Bayesian inference for increasingly informative observational data.
  • 29.03.2022, 16:00: Dr. Christian Bayer, Weierstrass Institute of Applied Analysis and Stochastics in Berlin.
    • Titel: Optimal stopping with signatures.
    • Zusammenfassung: We propose a new method for solving optimal stopping problems (such as American option pricing in finance) under minimal assumptions on the underlying stochastic process $X$. We consider classic and randomized stopping times represented by linear and non-linear functionals of the rough path signature $\X^{<\infty}$ associated to $X$, and prove that maximizing over these classes of signature stoppingtimes, in fact, solves the original optimal stopping problem. Using the algebraic properties of the signature, we can then recast the problem as a (deterministic) optimization problem depending only on the (truncated) expected signature $\E\left[ \X^{\le N}_{0,T} \right]$. By applying a deep neural network approach to approximate the non-linear signature functionals, we can efficiently solve the optimal stoppingproblem numerically. The only assumption on the process $X$ is that it is a continuous (geometric) random rough path. Hence, the theory encompasses processes such as fractional Brownian motion, which fail to be either semi-martingales or Markov processes, and can be used, in particular, for American-type option pricing in fractional models, e.g. on financial or electricity markets. (Joint work with Paul Hager, Sebastian Riedel, and John Schoenmakers.)
  • 22.03.2022, 16:00: Dr. Peng Cheng, The University of Texas at Austin
    • Titel: Fast and scalable methods for high-dimensional learning and optimization under uncertainty. 
    • Zusammenfassung: In this talk, I will present some recent work on high-dimensional learning of complex models and model-constrained optimization (of control, design, and experiment) under uncertainty. Tremendous computational challenges are faced for such problems when (1) the models (e.g., described by partial differential equations) are expensive to solve and have to be solved many times or in real time; and (2) the data, optimization, and uncertain variables are high-dimensional, bringing the curse of dimensionality for most conventional methods. We tackle these challenges by exploiting both data and model informed properties, such as smoothness, sparsity, correlation, intrinsic low-dimensionality or low-rankness, etc. I will present several new computational methods that achieve significant computational reduction (fast) and break the curse of dimensionality (scalable), including structure-exploiting model reduction, randomized high-order tensor decomposition, derivative informed deep learning, projected transport map, and functional Taylor approximations. I will also briefly talk about some applications of these methods in learning and optimal mitigation of infectious disease (COVID-19), optimal control of turbulent combustion, optimal design of stellarator for plasma fusion, and optimal experimental design for sensor placement.
  • 15.03.2022, 16:00: Prof. Robert Gramacy, Virginia Polytechnic and State University.
    • Titel: Entropy-based adaptive design for contour finding and estimating reliability.
    • Zusammenfassung: In reliability analysis, methods used to estimate failure probability are often limited by the costs associated with model evaluations. Many of these methods, such as multifidelity importance sampling (MFIS), rely upon a computationally efficient, surrogate model like a Gaussian process (GP) to quickly generate predictions. The quality of the GP fit, particularly in the vicinity of the failure region(s), is instrumental in supplying accurately predicted failures for such strategies. We introduce an entropy-based GP adaptive design that, when paired with MFIS, provides more accurate failure probability estimates and with higher confidence. We show that our greedy data acquisition strategy better identifies multiple failure regions compared to existing contour-finding schemes. We then extend the method to batch selection, without sacrificing accuracy. Illustrative examples are provided on benchmark data as well as an application to an impact damage simulator for National Aeronautics and Space Administration (NASA) spacesuits. 
  • 08.03.2022, 16:00: Dr. Alexander Litvinenko, RWTH Aachen.
    • Titel: Computing f-Divergences and Distances of High-Dimensional Probability Density Functions.
    • Zusammenfassung:Very often, in the course of uncertainty quantification tasks or data analysis, one has to deal with high-dimensional random variables (RVs)   (with values in $\Rd$).  Just like any other  random variable, a high-dimensional RV can be described by its probability density (pdf) and/or by the corresponding probability characteristic functions (pcf), or some more general representation as a function of other, known, random variables. Here the interest is mainly to compute characterisations like the entropy, or relations between two distributions, like their Kullback-Leibler divergence, or more general measures such as $f$-divergences, among others.  These are all computed from the pdf, which is often not available directly, and it is a computational challenge to even represent it in a numerically feasible fashion in case the dimension $d$ is even moderately large.   It is an even stronger numerical challenge to then actually compute said characterisations in the high-dimensional case.  In this regard, in order to achieve a computationally feasible task, we propose to represent the density by a high order tensor product, and approximate this in a low-rank format. See for more details reconstruction.
  • 01.03.2022, 16:00: Dr. Dmitry Kabanov, RWTH Aachen.
    • Titel: Physics-informed neural networks for discrete Helmholtz—Hodge decomposition.
    • Zusammenfassung: In recent years, neural networks have been increasingly used to solve scientific problems. Particularly, physics-informed neural networks—networks augmented with the knowledge of physical constraints—have been applied to problems in mechanics to reconstruct fields of state variables with higher accuracy, than purely data-driven neural networks can achieve. Physical constraints are usually added to the training loss function as additional terms and they serve as regularizers that improve generalization error. In this work, we construct a physics-informed neural network for the problem of discrete Helmholtz—Hodge decomposition, namely, decomposition of a vector field on potential (curl-free) and solenoidal (divergence-free) subfields, where the vector field is given as a finite set of measurements. We demonstrate that the information on orthogonality of the subfields helps the physics-informed neural network learn better reconstruction.
  • 22.02.2022, 16:00: Arved Bartuska, RWTH Aachen.
    • Titel: Small-noise approximation for Bayesian optimal experimental design with nuisance uncertainty.
    • Zusammenfassung: Optimal experimental design aims at maximizing the amount of information gained from an experiment. If the experimental model contains nuisance parameters that cannot, or do not need to, be recovered, this task involves the computation of one outer and two inner nested integrals. We then propose a small-noise approximation for the inner integral stemming from the nuisance uncertainty, thus allowing for the application of previously derived Monte Carlo estimators without increasing their computational cost. In this talk, we present the optimal setting for the small-noise Monte Carlo Laplace estimator and the small-noise double-loop Monte Carlo estimator with importance sampling. We then showcase the computational constraints and a physical application of our estimators via an example from electrical impedance tomography.
  • 15.02.2022, 16:00: Sophia Wiechert, RWTH Aachen.
    • Titel: Efficient Importance Sampling via Stochastic Optimal Control for Stochastic Reaction Networks.
    • Zusammenfassung: We are interested in the efficient estimation of statistical quantities, particularly rare event probabilities, for stochastic reaction networks (SRNs). To this aim, we propose a novel importance sampling (IS) approach to improve the efficiency of Monte Carlo (MC) estimators when based on an approximate tau-leap (TL) scheme. In IS, the crucial step is to choose an appropriate change of probability measure to achieve a substantial variance reduction. Based on an original connection between finding the optimal IS parameters, within a class of probability measures, and a stochastic optimal control (SOC) formulation, we propose an automated approach to derive a highly efficient path-dependent measure change. Given that it is challenging to solve this SOC problem analytically, we propose a numerical dynamic programming algorithm to approximate the optimal control parameters. In the one-dimensional case, our numerical results show that the variance of our proposed estimator decays with rate O(dt) for a step size of dt, compared to being O(1) for the standard MC estimator. To mitigate the curse of dimensionality in the multidimensional case, we propose an alternative learning-based method that approximates the value function by a neural network whose parameters are determined via a stochastic optimization algorithm. Our numerical experiments demonstrate that our learning-based IS approach substantially reduces the MC estimator's variance.
  • 08.02.2022, 16:00: Dr. Hakon Hoel, RWTH Aachen.
    • Titel: Construction and analysis of numerical methods for stochastic conservation laws.
    • Zusammenfassung: Stochastic conservation laws (SCL) with quasilinear multiplicative rough path dependence in the flux arise in modeling of mean field games. We present computable numerical methods for pathwise solutions of scalar SCL with, for instance, rough paths in the form of Wiener processes. For strictly convex flux functions, we show that rough path oscillations lead to cancellations in the flow map of the SCL dynamics, and we exploit this property to develop efficient numerical methods.

Previous talks (2021)

  • 30.11.2021, 16:00: Alexander NovikovDeepMind.
    • Title: Tensor methods and machine learning applications
    • Abstract: Tensor decompositions generalize matrix low-rank decomposition to arrays of dimensionality larger than 2 (tensors). Tensor decompositions allow one to efficiently represent exponentially large tensors and perform operations on them. In this talk, I’m going to showcase two machine learning models where tensors arise naturally and by using appropriate tensor decomposition one can gain speed and/or memory benefits. Namely, we are going to discuss how one can compress fully connected layers of neural networks and speed up gaussian processes – a probabilistic model that is often used for optimizing noisy functions that are expensive to evaluate (e.g. choosing hyperparameters of a neural network).
      Since low-rank tensors form a smooth (Riemannian) manifold, when training machine learning models parameterized by factorized tensors, people often consider Riemannian optimization. In the second half of the talk, I’m going to present a method for Riemannian automatic differentiation that allows one to compute Riemannian gradients automatically and with optimal complexity.
  • 23.11.2021, 16:00: Emil Loevbak, PhD Fellow, KU Leuven.
    • Title: Stochastic optimization for tokamak fusion reactor divertor design.
    • Abstract: Nuclear fusion energy has the potential to be a clean, reliable source of energy for the future. Constructing a well working reactor has however proven to contain significant challenges. One such challenge is the designing the divertor, a component that removes waste particles from the reactor. The divertor comes into contact with a dense plasma, modeled as a fluid, as well as lower density neutral particles, modeled as a kinetic process. The B2-EIRENE research code for divertor design, simulates the coupled plasma-neutral model through a combination of finite volume and Monte Carlo particle methods. The design is iteratively refined in an adjoint based optimization routine, i.e., gradients are computed by simulating the adjoint model with a similar discretization. These Monte Carlo simulations are currently too expensive for divertor design to be feasible. At KU Leuven we have developed a variety of techniques for accelerating these codes. We have developed a new class of asymptotic-preserving Multilevel Monte Carlo schemes for accelerating both the forward and adjoint simulations. We have also developed a discrete adjoint approach in which we use the same stochastic paths for the forward and adjoint Monte Carlo simulation, with the goal of reducing the number of iterations required in the optimization routine. In this talk, I will introduce these techniques, as well as our current work on challenges remaining on the path towards valorization in the B2-EIRENE code.
  • 16.11.2021, 16:00:  Dr. Jonas Kiessling, H-Ai AB and KTH, Stockholm, Sweden.
    • Title: A Neural Network approach to survival estimation after elective repair of Abdominal Aortic Aneurysm.
    • Abstract: Abdominal Aortic Aneurysm (AAA) is a common health problem, and increasing with an older and more obsese population. Untreated it can lead to rupture of the abdominal aorta, resulting in massive internal bleeding and a high risk of death. Elective treatment of AAA falls into two categories; open surgical repair (OSR) and endovascular repair (EVAR). The long term relative merits of OSR and EVAR is an open and important question in the medical field of cardio-vascular surgery. In this seminar I will outline how supervised learning can be applied to estimate the survival probability after elective repair of AAA. I will report on results from a recent study where we compare the prediction accuracy of the standard Kaplan-Meier model with that of a neural network based model. Finally I will show what the more accurate neural network based model predicts on the relative merits of EVAR vs. OSR.
  • 09.11.2021, 16:00:  Dr. Luis Espath, RWTH Aachen, Germany.
    • Title: Statistical Learning for Fluid Flows: Sparse Fourier divergence-free approximations
    • Abstract: We reconstruct the velocity field of incompressible flows given a finite set of measurements. For the spatial approximation, we introduce the Sparse Fourier divergence-free (SFdf) approximation based on a discrete $L^2$ projection. Within this physics-informed type of statistical learning framework, we adaptively build a sparse set of Fourier basis functions with corresponding coefficients by solving a sequence of minimization problems where the set of basis functions is augmented greedily at each optimization problem. We regularize our minimization problems with the seminorm of the fractional Sobolev space in a Tikhonov fashion. In the Fourier setting, the incompressibility (divergence-free) constraint becomes a finite set of linear algebraic equations. We couple our spatial approximation with the truncated Singular Value Decomposition (SVD) of the flow measurements for temporal compression. Our computational framework thus combines supervised and unsupervised learning techniques. We assess the capabilities of our method in various numerical examples arising in fluid mechanics.
  • 02.11.2021, 16:00: Prof.  Dr.  Evelyn Buckwar, Johannes Kepler University, Linz, Austria
    • Title: Splitting methods in Approximate Bayesian Computation for partially observed diffusion processes.
    • Abstract: Approximate Bayesian Computation (ABC) has become one of the major tools of likelihood-free statistical inference in complex mathematical models. Simultaneously, stochastic differential equations (SDEs) have developed as an established tool for modelling time dependent, real world phenomena with underlying random effects. When applying ABC to stochastic models, two major difficulties arise. First, the derivation of effective summary statistics and proper distances is particularly challenging, since simulations from the stochastic process under the same parameter configuration result in different trajectories. Second, exact simulation schemes to generate trajectories from the stochastic model are rarely available, requiring the derivation of suitable numerical methods for the synthetic data generation. In this talk we consider SDEs having an invariant density and apply measure-preserving splitting schemes for the synthetic data generation. We illustrate the results of the parameter estimation with the corresponding ABC algorithm with simulated data.This talk is based on joined work with Massimiliano Tamborrino, University of Warwick, and Irene Tubikanec, Johannes Kepler University Linz.
  • 26.10.2021, 16:00:  Dr.  Charles-Edouard Bréhier, CNRS & Université Lyon 1, France.
    • Title: Asymptotic Preserving schemes for multiscale SDEs.
    • Abstract: I will present the construction of asymptotic preserving schemes for slow-fast SDE (and SPDE) systems, where the fast component is an Ornstein-Uhlenbeck process. Naive schemes fail to capture either averaged coefficients or correction interpretation of the noise. I will present examples of asymptotic preserving schemes which overcome these issues. This is based on joint work with Shmuel Rakotonirina-Ricquebourg.
  • 19.10.2021, 16:00:  Prof. Dr. Michael Beer, Leibniz Universität Hannover, Germany
    • Title: Efficient reliability analysis with imprecise probabilities
    • Abstract: An efficient analysis of our engineered systems and structures is a key requirement for their proper design and operation. This requirement, however, is challenging engineers to come up with innovative solutions that can cope with the increasing complexity of our systems and structures and with the uncertainties involved. Imprecise probabilities have shown useful conceptual features to facilitate a modelling at a reasonable level of detail and capturing the remaining epistemic uncertainty in a set-valued manner. This approach allows for an optimal balance between model detailedness and imprecision of results to still derive useful decisions. However, it is also associated with some extensive numerical cost when applied in a crude way. This seminar will provide selected solutions for efficient numerical analysis with imprecise probabilities, specifically for reliability analysis, to attack high-dimensional and nonlinear problems. After an introductory overview on conceptual pathways for solution one intrusive and three non-intrusive specific developments will be discussed. These solutions include operator norm theory to solve first passage problems by linear algebra, intervening variables to moderate nonlinearities for linearized approximate solutions, the exploitation of topological properties of the reliability problem associated with line sampling, and the utilization of high dimensional model representation of the failure probability for non-intrusive efficient sampling. Engineering examples are presented to demonstrate the capabilities of the approaches and concepts.
  • 12.10.2021, 16:00: Dr. John Jackeman,  Sandia National Laboratories, Albuquerque, NM USA.
    • Title: Surrogate Modeling for Efficiently, Accurately and Conservatively Estimating Measures of Risk
    • Abstract: This talk will present a surrogate-based framework for conservatively estimating risk from limited evaluations of an expensive physical experiment or simulation. Focus will be given to the computation of risk measures that quantify tail statistics of the loss, such as Average Value at Risk (AVaR). Monte Carlo (MC) sampling can be used to approximate such risk measures, however MC requires a large number of model simulations, which can make accurately estimating risk intractable for computationally expensive models.Given a set of samples surrogates are constructed such that the estimate of risk, obtained from the surrogate, is always greater than the empirical estimate obtained from the training data. These surrogates not only limit over-confidence in model reliability, but produce estimates of risk that converge much faster to the true risk, than purely sampled based estimates. The first part of the talk will discuss how to use the risk quadrangle, which rigorously connects stochastic optimization and statistical estimation, to construct conservative surrogates that can be tailored to the specific risk preferences of the model stakeholder. Surrogates constructed using least squares and quantile regression are specific cases of this framework. The second part of the talk will then present an approach, based upon stochastic orders, for constructing surrogates that are conservative with respect to families of risk measures, which is useful when risk preferences are  difficult to elicit. This approach uses first and second order stochastic dominance to respectively enforce that the surrogate over-estimates probability of failture and AVaR for a finite set of thresholds. The conservative surrogates constructed introduce a bias that allows them to conservatively estimate risk. Theoretical results will be provided that show that for orthonormal models such as polynomial chaos expansions, this bias decays at the same rate as the root mean squared  error in the surrogate. Numerous numerical examples will be used to confirm that that risk-aware surrogates do indeed over estimate risk while converging at the expected rate.
  • 05.10.2021, 16:00: Prof. Ibrahim Hoteit, KAUST.
    • Title: Forward and Inverse Tracking of Oil Spills Under Ensemble DA Uncertainties with Real World Applications in the Red Sea
    • Abstract: The talk will present our efforts to develop the next generation operational system for the Red Sea, as part of Aramco’s resolution toward the Fourth Industrial Revolution. This integrated system has been built around state-of-the-art ocean-atmosphere-wave general circulation models that have been specifically developed for the Red Sea region and nested within the global weather systems. It is now fully operational, routinely running on the KAUST supercomputer Shaheen. I will showcase the supporting real-time online visualization-analytics tools and servers that are currently being developed to provide a user-friendly interface to analyze the large datasets outputted by the system. The second part of the talk will outline some of our ongoing research activities to continue enhancing the system performance and equipping it with new capabilities and features, discussing in particular: (i) improving the forecasting skills through new ensemble data assimilation schemes accounting and providing information about the system uncertainties, and (ii) developing new tools for exploiting the ensemble uncertainties in the system outputs in forward and inverse tracking of oil spills
  • 28.09.2021, 16:00: Dr. Nadhir Ben Rached, RWTH Aachen University.
    • Title: Efficient Importance Sampling for Large Sums of IID Random Variables
    • Abstract: We discuss estimating the probability that the sum of nonnegative independent and identically distributed random variables falls below a given threshold, i.e., $\mathbb{P}(\sum_{i=1}^{N}{X_i} \leq \gamma)$, via importance sampling (IS).  We are particularly interested in the rare event regime when  $N$ is large and/or $\gamma$ is small.  The exponential twisting is a popular technique for similar problems that, in most cases, compares favorably to other estimators. However, it has some limitations: i) it assumes the knowledge of the moment generating function of $X_i$ and ii) sampling under the new IS PDF is not straightforward and might be expensive. The aim of this work is to propose an alternative IS PDF that approximately yields, for certain classes of distributions and in the rare event regime,  at least the same performance as the exponential twisting technique and,  at the same time, does not introduce serious limitations. The first class includes distributions whose probability density functions (PDFs) are asymptotically equivalent, as $x \rightarrow 0$, to $bx^{p}$, for $p>-1$ and $b>0$. For this class of distributions, the Gamma IS PDF with appropriately chosen parameters retrieves approximately, in the rare event regime corresponding to small values of $\gamma$ and/or large values of $N$, the same performance of the estimator based on the use of the exponential twisting technique. In the second class, we consider the Log-normal setting, whose  PDF at zero vanishes faster than any polynomial, and we show numerically that a Gamma IS PDF with optimized parameters clearly outperforms the exponential twisting IS PDF. Numerical experiments validate the efficiency of the proposed estimator in delivering a highly accurate estimate in the regime of large $N$ and/or small $\gamma$
  • 21.09.2021, 16:00: Prof.Serge Prudhomme, Polytechnique Montréal.
    • Title: Construction of reduced-order models for structural dynamics applications
    • Abstract: The talk is concerned with the construction of proper-generalized decomposition formulations that preserve the structure and stability of the original system, such as its symplectic properties. The formulations are based on the  Hamiltonian and will be shown to be more stable than classical approaches. The formalism also allows one to define an optimization problem with constraints on the error in goal functionals in order to construct reduced models capable of delivering accurate approximation of quantities of interest. Numerical examples dealing with the dynamical behavior of beam structures will be presented in order to demonstrate the efficiency of the proposed approach.
  • 27.07.2021, 15:00: Prof. Youssef Marzouk, MIT.
    • Title: Density estimation and conditional simulation using triangular transport.
    • Abstract: Triangular transformations of measures, such as the Knothe–Rosenblatt rearrangement, underlie many new computational approaches for density estimation and conditional simulation. This talk will discuss several aspects of such constructions. First we present new approximation results for triangular maps, focusing on analytic densities with bounded support. We show that there exist sparse polynomial approximations and deep ReLU network approximations of such maps that converge exponentially, where error is assessed on the pushforward distributions of such maps in a variety of distances and divergences. We also give an explicit a priori description of the polynomial ansatz space guaranteeing this convergence rate. Second, we discuss the problem of estimating a triangular transformation given a sample from a distribution of interest—and hence, transport-driven density estimation. We present a general functional framework for representing monotone triangular maps between distributions on unbounded domains, and analyze properties of maximum likelihood estimation in this framework. We demonstrate that the associated optimization problem is smooth and, under appropriate conditions, has no spurious local minima. This result provides a foundation for a greedy semi-parametric estimation procedure. Time permitting, we may also discuss a conditional simulation method that employs a specific composition of maps, derived from the Knothe–Rosenblatt rearrangement, to push forward a joint distribution to any desired conditional. We show that this composed-map approach reduces variability in conditional density estimates and reduces the bias associated with any approximate map representation. For context, and as a pointer to an interesting application domain, we elucidate links between conditional simulation with composed maps and the ensemble Kalman filter used in geophysical data assimilation. This is joint work with Ricardo Baptista (MIT), Olivier Zahm (INRIA), and Jakob Zech (Heidelberg).
  • 20.07.2021, 16:00: Juan Pablo Madrigal Cianci,CSQI lab, EPFL.
    • Title: Multi-level Markov chain Monte Carlo with maximally coupled proposals.
    • Abstract:  In this work we present a novel class of Multi-Level Markov chain Monte Carlo (ML-MCMC) algorithms based on maximally coupled proposals, and apply them in the context of Bayesian inverse problems. In this context, the likelihood function involves a complex differential model, which is then approximated on a sequence of increasingly accurate discretizations. The key point of this algorithm is to construct highly coupled Markov chains together with the standard Multi-level Monte Carlo argument to obtain a better cost-tolerance complexity than a single level MCMC algorithm. Our approach generates these highly coupled chains by sampling from a maximal coupling of the proposals for each marginal Markov chain. By doing this, we are allowed to create novel ML-MCMC methods for which, contrary to previously used models, the proposals at each iteration can depend on the current state of this chain, while at the same time, creating chains that are highly correlated. The presented method is tested on an array of academic examples which evidence how our extended ML-MCMC method is robust when targeting some pathological posteriors, for which some of the previously proposed ML-MCMC algorithms fail.
  • 13.07.2021, 16:00:  Prof. Dr. Helmut Harbrecht, Universität Basel.
    • Title: Multilevel approximation of Gaussian random fields.
    • Abstract: Centered Gaussian random fields are determined by their covariance operators. We consider such fields to be given sample-wise as variational  solutions to colouring operator equations driven by spatial white  noise, with pseudodifferential colouring operator being elliptic, self-adjoint and positive from the H\"ormander class. Especially, this  includes the Mat\'ern class of covariance operators. By using wavelet matrix compression, we prove that the precision and covariance operators,  respectively, may be identified with bi-infinite matrices and finite sections may be diagonally preconditioned rendering the condition number  independent of the dimension $N$ of this section. This results in several powerful algorithms like fast sampling, multilevel Monte Carlo oracles  for covariance estimation, and kriging. Besides the plain theory and related numerical experiments, we also present application driven  algorithm which extend the theoretical findings.
  • 06.07.2021, 16:00:  Md Nurtaj Hossain, Indian Institute of Science, Bangalore.
    • Title: Uncertainty quantification of dynamical systems using adaptively trained reduced order models.
    • Abstract: Uncertainty quantification (UQ) of dynamical systems requires multiple solutions of the governing differential equation. Therefore, for a large dynamical system involving a million degrees of freedom, the task of UQ becomes computationally very expensive. This expensive task can be alleviated by replacing the large dynamical system with a computationally cheaper reduced order model (ROM), such as proper orthogonal decomposition (POD) based ROM. However, POD-based ROM suffers from a lack of robustness and accuracy in the absence of a diverse and large number of training points. To circumvent these issues, first, a posteriori error estimators are derived for the ROMs of both linear and nonlinear dynamical systems. Then these error estimators are used in conjunction with a modified greedy search algorithm to develop methods for adaptive training of POD-based ROMs. After adequate training, the adaptively trained ROMs are employed for UQ. Different numerical studies are performed to demonstrate the efficacy of the error estimators and developed algorithms. Usage of the adaptively trained ROMs in the estimation failure probability leads to a speed-up of more than two orders of magnitude.
  • 29.06.2021, 16:00:   Dr.  Yoshihito Kazashi,  CSQI, École polytechnique fédérale de Lausanne.
    • Title: Density estimation in RKHS with application to Korobov spaces in high dimensions.
    • Abstract: In this talk, we will consider a kernel method for estimating a probability density function (pdf) from an i.i.d. sample drawn from such density. Our estimator is a linear combination of kernel functions, the coefficients of which are determined by a linear equation. We will present an error analysis for the mean integrated squared error in a general reproducing kernel Hilbert space setting. We will discuss how this theory can be applied to estimate pdfs belonging to weighted Korobov spaces, and show a dimension independent convergence rate. Under a suitable smoothness assumption, our method attains a rate arbitrarily close to the optimal rate.
      This is joint work with Fabio Nobile (CSQI, EPFL).
  • 22.06.2021, 16:00:  Prof. Pierre Del Moral, INRIA, France.
    • Title: Stability and uniform fluctuation analysis of Ensemble Kalman-Bucy filters.
    • Abstract: This talk is concerned with the long time behavior of particle filters and Ensemble Kalman filters. These filters can be interpreted as mean field type particle interpretation of the filtering equation and the Kalman recursion.  We present a series of old and new results on the stability properties of these filters. We initiate a comparison between these particle samplers and discuss some open research questions.
  • 15.06.2021, 16:00:  Prof. Ahmed Kebaier, University Sorbonne Paris Nord (USPN).
    • Title: Approximation of Stochastic Volterra Equations.
    • Abstract: In this talk, we present a multi-factor approximation for Stochastic Volterra Equations with Lipschitz coefficients and kernels of completely monotone type that may be singular.  Our approach consists in truncating and then discretizing the integral defining the kernel, which corresponds to a classical Stochastic Differential Equation. We prove strong convergence results for this approximation. For the particular rough kernel case with Hurst parameter lying in $(0,1/2)$, we propose various discretization procedures and give their precise rates of convergence. We illustrate the efficiency of our approximation schemes with numerical tests for the rough Bergomi model.
  • 08.06.2021, 16:00:  Loris Di Cairano, PhD student at RWTH Aachen University and FZ Jülich.
    • Title: An Effective Continuum Model for studying Protein Diffusion in Lipid Membrane. 
    • Abstract:  A model for describing protein diffusion in lipid membrane starting from first principles is presented. The lipid membrane is modelled as a linear viscoelastic fluid (Stokes equation with an explicit time-dependence in the Cauchy's stress tensor), the protein by a sphere (First Euler’s law) and, finally, a constitutive equation, namely, a relation between the Cauchy’ stress tensor and strain tensor and it describes the response of the fluid to a given deformation. Thus, the common model for linear viscoelastic fluid is to consider the instantaneous and non instantaneous responses of fluid in an additive way and the latter is model by an appropriate time-dependent response function. Through an elimination procedure of the membrane degrees of freedom (one formally solves the membrane Stokes’ equation), one gets an only equation for the protein degrees of freedom which takes the form of a GLE. The kernel function entering the GLE is exactly the response function appearing in the Cauchy stress tensor. B exploiting methods used in the literature, one can solve the GLE, namely, write an analytical solution for the correlation function (for example, mean squared displacement) in terms of the kernel function in GLE. By choosing a 3-paramenter Mittag-Leffler, one can compare MD data with model obtaining a remarkable agreement.
  • 01.06.2021, 16:00:  Dr.  Michael F. Herbst, RWTH Aachen University.
    • Title: Uncertainty quantification in electronic-structure theory: Status and directions for future research.
    • Abstract: The aim of electronic structure theory is to describe the behaviour of electrons in matter on a quantum-mechanical level. On the one hand related to a rich set of applications in chemistry, pharmacy and materials science, the field is on the other hand a fruitful source of numerically and mathematically interesting problems. One aspect that is only barely explored so far is the question of estimating error and uncertainties in simulation results. In practice this means that people select numerical parameters and physical models based on prior experience and rely on their physical intuition to verify the reliability of obtained results. However, this approach does not properly scale to the recently emerging high-throughput approaches where tens of thousands (or more) chemical systems need to be tackled in a computational study. More rigorous mathematical approaches to understand errors and uncertainties are therefore desperately needed to increase efficiency and automatisability. In my talk I will sketch the physical and numerical procedures used in the field and I will summarise the currently employed ideas for error estimation and uncertainty quantification. I will conclude my talk with some open questions and angles of research where input from uncertainty quantification would be needed.
  • 25.05.2021, 16:00:  Prof. Ajay Jasra, CEMSE, KAUST.
    • Title: On unbiased score estimation for partially observed diffusions.
    • Abstract: We consider the problem of statistical inference for a class of partially-observed diffusion processes, with discretely-observed data and finite-dimensional parameters. We construct unbiased estimators of the score function, i.e. the gradient of the log-likelihood function with respect to parameters, with no time-discretization bias. These estimators can be straightforwardly employed within stochastic gradient methods to perform maximum likelihood estimation or Bayesian inference. As our proposed methodology only requires access to a time-discretization scheme such as the Euler-Maruyama method, it is applicable to a wide class of diffusion processes and observation models. Our approach is based on a representation of the score as a smoothing expectation using Girsanov theorem, and a novel adaptation of the randomization schemes developed in Mcleish [2011], Rhee and Glynn [2015], Jacob et al. [2020a]. This allows one to remove the time-discretization bias and burn-in bias when computing smoothing expectations using the conditional particle filter of Andrieu et al. [2010]. Central to our approach is the development of new couplings of multiple conditional particle filters. We prove under assumptions that our estimators are unbiased and have finite variance. The methodology is illustrated on several challenging applications from population ecology and neuroscience.
  • 18.05.2021, 16:00:  Prof. Dr. Elisabeth Ullmann, Department of Mathematics, Technical University of Munich.
    • Title: Rare event estimation with PDE-based models.
    • Abstract: The estimation of the probability of rare events is an important task in reliability and risk assessment of critical societal systems, for example, groundwater flow and transport, and engineering structures. In this talk we consider rare events that are expressed in terms of a limit state function which depends on the solution of a partial differential equation (PDE). We present recent progress on mathematical and computational aspects of this problem: (1) the impact of the PDE approximation error on the failure probability estimate, and (2) the use of the Ensemble Kalman Filter for the estimation of failure probabilities.  

      This is joint work with Fabian Wagner (TUM), Iason Papaioannou (TUM) and Jonas Latz (Cambridge).

  • 11.05.2021, 16:00:  Prof. Bernard Ghanem, KAUST.
    • Title: On Adversarial Network Attacks, Robustness, and Certification.
    • Abstract: The outstanding performance of deep neural networks (DNNs), for visual recognition tasks in particular, has been demonstrated on many large-scale benchmarks. This performance has immensely strengthened the line of research that aims to understand and analyze the driving reasons behind the effectiveness of these networks. One important aspect of this analysis has gained much attention, namely the sensitivity of a DNN to perturbations. This has spawned a thrust in the research community, which focuses on developing adversarial attacks that fool a DNN, training strategies that make DNNs more robust against such attacks, as well as methods to certify the behavior of a DNN irrespective of the attack.  In this talk, I will introduce this exciting research thrust and span some of its landscape from synthesizing adversarial attacks all the way to randomized smoothing for DNN certification. I will also give an update on the latest research progress in this direction from the Image and Video Understanding Lab (IVUL) at KAUST. 
  • 04.05.2021, 16:00:  Prof. Dr.  Carsten Hartmann, Institut für Mathematik, BTU Cottbus-Senftenberg.
    • Title: Optimal control of the underdamped Langevin sampler.
    • Abstract: The underdamped Langevin equation is a popular computational model in various fields of science (e.g. molecular dynamics, meteorology, or machine learning) that is used to sample from complicated multimodal probability distributions by a combination of (dissipative) Hamiltonian dynamics and diffusion. I will explain how control theory can help to speed up the convergence of an underdamped Langevin equation to its equilibrium probability distribution, specifically in situations in which the diffusion coefficient is small (i.e. low temperature) and the asymptotic properties of the dynamics are dominated by metastability and poor convergence. I will discuss different limits as the temperature ratio goes to infinity and prove convergence to a limit dynamics. It turns out that, depending on whether the lower ("target") or the higher ("simulation") temperature is fixed, the controlled dynamics converges either to the overdamped Langevin equation or to a deterministic gradient flow.
  • 27.04.2021, 16:00:  Marco Ballesio, KAUST
    • Title: Unbiased Estimation of Log-Likelihood Gradient for a Class of Continuous-Time State-Space Models
    • Abstract: We consider static parameter estimation for a class of continuous-time state-space models. Our goal is to obtain an unbiased estimate of the log-likelihood gradient, which is an estimate that is unbiased even if the stochastic processes involved in the model must be discretized in time. To achieve this goal, we apply a doubly randomized scheme, that involves a novel coupled conditional particle filter (CCPF) on the second level of randomization. Our novel estimate helps facilitate the application of gradient-based estimation algorithms, such as stochastic gradient descent (SGD). We illustrate our methodology in the context of SGD in several numerical examples and compare with the Rhee &Glynn estimator. Results show consistency of the method.  
  • 20.04.2021, 16:00:  Eike Cramer Forschungszentrum Jülich GmbH and RWTH Aachen University, Germany.
    • Title: A principal component normalizing flow for manifold density estimation.
    • Abstract: Recent developments in machine learning have enabled 'model-free' distribution learning using so-called generative neural networks. In particular, normalizing flows have proven effective as they are trained through direct log-likelihood maximization, in contrast to other generative networks such as generative adversarial networks and variational autoencoders. However, normalizing flows fail to fit distributions of data manifolds correctly, leading to severe overfitting and generation of data that does not follow the target distribution. To leverage the modeling power of normalizing flows for manifold data, we propose an injective mapping based on principal component analysis. The resulting principal component flow (PCF) model enables density estimation in latent space and avoids complications due to manifold data. We show that the dimensionality reduction does not interfere with the density estimation procedure. As an illustrative case study, we apply the PCF model to learn the distribution of the daily total photovoltaic power generation in Germany. The PCF model is able to generate new data by sampling from a significantly lower latent dimensionality than the full data dimensionality and trains more effectively than full space flow models. The PCF performs particularly well in identifying parts of the data with zero variance, while preserving critical features of the original time series like the probability and power spectral density.
  • 13.04.2021, 16:00:  Dr. Bruno Tuffin INRIA Rennes Bretagne-Atlantique, France.
    • Title: Estimating by Simulation the Mean and Distribution of Hitting Times of Rare Events.
    • Abstract: Rare events occur by definition with a very small probability but are important to analyze because of potential catastrophic consequences. During this talk, we will focus on rare event for so-called regenerative processes, that are basically processes such that portions of the process are statistically independent of each other. For many complex and/or large models, simulation is the only tool at hand but requires specific implementations to get an accurate answer in a reasonable time. There are two main families of rare-event simulation techniques: importance sampling (IS) and splitting. 
      We will (somewhat arbitrarily) devote most of the talk to IS. 

      We will then focus on the estimation of the mean hitting time of a rarely visited set. A natural and direct estimator consists in averaging independent and identically distributed copies of simulated hitting times, but an alternative standard estimator uses the regenerative structure allowing to represent the mean as a ratio of quantities. We will see that in the setting of crude simulation, the two estimators are actually asymptotically identical in a rare-event context, but inefficient for different, even if related, reasons:  the direct estimator requires a large average computational time of a single run whereas the ratio estimator faces a small probability computation. We then explain that the ratio estimator is advised when using IS.

      In the third part of the talk, we will discuss the estimation of the distribution, not just the mean,  of the hitting time to a rarely visited set of states. We will exploit the property that  the distribution of the hitting time divided by its expectation converges weakly to an exponential as the target set probability decreases to zero. The problem then reduces to the extensively studied estimation of the mean described previously. It leads to simple estimators of a quantile and conditional tail expectation of the hitting time. Some variants will be presented and the accuracy of the estimators illustrated on numerical examples.

      This talk is mostly based on collaborative works with Peter W. Glynn and Marvin K. Nakayama.
  • 06.04.2021, 16:00:  Dr. Christian Bayer Weierstraß Institute for Applied Analysis and Stochastics.
    • Title: A pricing BSPDE for rough volatility.
    • Abstract: In this talk, we study the option pricing problems for rough volatility models. As the framework is non-Markovian, the value function for a European option is not deterministic; rather, it is random and satisfies a backward stochastic partial differential equation (BSPDE). The existence and uniqueness of weak solutions is proved for general nonlinear BSPDEs with unbounded random leading coefficients whose connections with certain forward-backward stochastic differential equations are derived as well. These BSPDEs are then used to approximate American option prices. A deep learning-based method is also investigated for the numerical approximations to such BSPDEs and associated non-Markovian pricing problems. Finally, examples of rough Bergomi type are numerically computed for both European and American options.
  • 30.03.2021, 16:00:  Dr. André Gustavo Carlon , KAUST.
    • Title: Multi-Iteration Stochastic Optimizers.
    • Abstract: Stochastic optimization problems are of great importance for many fields ranging from engineering to machine learning. Stochastic gradient descent methods (SGD) are the main class of methods to solve these problems. However, standard SGD converges sublinearly and is not easily parallelizable. Multi-Iteration Stochastic Optimizers are a novel class of first-order stochastic optimizers where the gradient is estimated using the Multi-Iteration stochastiC Estimator (MICE). The MICE estimator controls the coefficient of variation of the mean gradient approximation using successive control variates along the path of iterations. The SGD-MICE optimizer converges linearly in the class of strongly-convex and L-smooth functions. The performances of multi-iteration stochastic optimizers are evaluated in numerical examples, validating our analysis, and converging faster than well-known stochastic optimization methods for the same number of gradient evaluations.
  • 23.03.2021, 16:00:  Marco Ballesio , KAUST.
    • Title:  Multilevel Particle Filters.
    • Abstract: We consider the filtering problem for partially observed diffusions, which are regularly observed at discrete times. We are concerned with the case when one must resort to time-discretization of the diffusion process if the transition density is not available in an appropriate form. In such cases, one must resort to advanced numerical algorithms such as particle filters to consistently estimate the filter. It is also well known that the particle filter can be enhanced by considering hierarchies of discretizations and the multilevel Monte Carlo (MLMC) method, in the sense of reducing the computational effort to achieve a given mean square error (MSE). A variety of multilevel particle filters (MLPF) have been suggested in the literature, e.g., in Jasra et al., SIAM J, Numer. Anal., 55, 3068–3096. Here we introduce a new alternative that involves a resampling step based on the optimal Wasserstein coupling. We prove a central limit theorem (CLT) for the new method. On considering the asymptotic variance, we establish that in some scenarios, there is a reduction, relative to the approach in the aforementioned paper by Jasra et al., in computational effort to achieve a given MSE. These findings are confirmed in numerical examples. We also consider filtering diffusions with unstable dynamics; we empirically show that in such cases a change of measure technique seems to be required to maintain our findings.
  • 09.03.2021, 16:00:  Dr. Eric Hall, University of Dundee, Scotland.
    • Title:  Weak Error Rates for Option Pricing under the Rough Bergomi Model.
    • Abstract: Modeling the volatility structure of underlying assets is a key component in the pricing of options. Rough stochastic volatility models, such as the rough Bergomi model [Bayer, Friz, Gatheral, Quantitative Finance 16(6), 887-904, 2016], seek to fit observed market data based on the observation that the log-realized variance behaves like a fractional Brownian motion with small Hurst parameter, H < 1/2, over reasonable timescales. In fact, both time series data of asset prices and option derived price data indicate that H often takes values close to 0.1 or even smaller, i.e. rougher than Brownian Motion. The non-Markovian nature of the driving fractional Brownian motion in the rough Bergomi model, however, poses a challenge for numerical options pricing. Indeed, while the explicit Euler method is known to converge to the solution of the rough Bergomi model, the strong rate of convergence is only H ([Neuenkirch and Shalaiko, arXiv:1606.03854]). We prove rate H + 1/2 for the weak convergence of the Euler method and, in the case of quadratic payoff functions, we obtain rate one. Indeed, the problem is very subtle; we provide examples demonstrating that the rate of convergence for payoff functions well approximated by second-order polynomials, as weighted by the law of the fractional Brownian motion, may be hard to distinguish from rate one empirically. Our proof relies on Taylor expansions and an affine Markovian representation of the underlying.
  • 02.03.2021, 15:00:  Juan Pablo Madrigal Cianci  CSQI laboratory in EPFL.
    • Title:  Generalized parallel tempering for Bayesian inverse problems.
    • Abstract: In the current work we present two generalizations of the Parallel Tempering algorithm, inspired by the so-called continuous-time Infinite Swapping algorithm, which found  its origins in the molecular dynamics community, and can be understood as  the continuous-time limit of a Parallel Tempering algorithm with state-dependent swapping rates.  In the current work, we extend this idea to the context of time-discrete Markov chains and present two Markov chain Monte Carlo algorithms that follow the same paradigm as the   continuous-time infinite swapping procedure. We present results on the reversibility and ergodicity properties of our generalized PT algorithms. Numerical results on sampling from different target distributions originating from Bayesian inverse problems, show that the proposed methods significantly improve sampling efficiency over more traditional sampling algorithms such as Random Walk Metropolis and  (standard)  Parallel Tempering.
  • 23.02.2021, 16:00:  Felix Terhag.
    • Title:  Dropout for Uncertainty Estimation in Neural Networks
    • Abstract: The recent success of neural networks has led to applications in evermore domains. While they yield good results on many problems one has to be especially careful in safety critical applications, as classical approaches do not contain accurate quantifications of uncertainty. In this talk, I want to give an introduction into the topic of uncertainty estimation in deep neural networks. To do so, I will show, along an illustrative example, that neural networks are prone to predicting with high confidence even in samples, which do not belong to the training distribution. In the following I will give an introduction into a Bayesian approach to neural network and for that purpose introduce Gal and Ghahramani’s Dropout as Bayesian Approximation, which is a good starting point into the topic due to its clarity. Going back to the motivating example I will show the improvements gained by this method.
  • 16.02.2021, 16:00:  Dr.  Abdul-Lateef Haji-Ali, Assistant professor, Heriot-Watt University.
    • Title:  Multilevel Monte Carlo for Computing Probabilities.
    • Abstract: In this talk, I will discuss the challenges in computing probabilities of the form $\prob{X \in \Omega}$ where $X$ is a random variable and $\Omega$ is a d-dimensional set. Computing such probabilities is important in many contexts, e.g., risk assessment and finance. A frequent challenge is encountered when only a costly approximation of the random variable $X$ can be sampled. For example, when $X$ depends on an inner expectation that has to be approximated with Monte Carlo or when $X$ depends on a non-trivial (stochastic) differential equations and a numerical discretization must be employed. A naive Monte Carlo method has a prohibitive complexity that compounds the slow convergence of Monte Carlo with the complexity of approximation. Instead, I will present a variant of Multilevel Monte Carlo (MLMC) with adaptive levels that can, under certain conditions, have a complexity that is independent of the complexity of approximation.
  • 09.02.2021, 16:00:  Dr.  Chiara Piazzola, CNR-IMATI, Pavia, Italy.
    • Title: Comparing Multi-Index Stochastic Collocation and Radial Basis Function Surrogates for Ship Resistance Uncertainty Quantification
    • Abstract: A comparison of two methods for the forward Uncertainty Quantification (UQ) of complex industrial problems is presented. Specifically, the performance of Multi-Index Stochastic Collocation (MISC) and multi-fidelity Stochastic Radial Basis Functions (SRBF) surrogates is assessed for the UQ of a roll-on/roll-off passengers ferry advancing in calm water and subject to two operational uncertainties, namely the ship speed and draught. The estimation of the expected value, standard deviation, and probability density function of the (model-scale) resistance is presented and discussed.  Both methods need to repeatedly solve the free-surface Navier-Stokes equations for different configurations of the operational parameters.  The required CFD simulations are obtained by a multi-grid Reynolds Averaged Navier-Stokes (RANS) equations solver. Both MISC and SRBF use as fidelity levels the intermediate grids employed by the RANS solver
  • 02.02.2021, 16:00: Dr. Joakim Beck, Stochastic Numerics Research Group, CEMSE Division, KAUST
    • Title: Multi-index stochastic collocation using isogeometric analysis for random PDEs
    • Abstract: We consider the forward uncertainty quantification (UQ) problem of solving partial differential equations (PDEs) with random coefficients in domains of more challenging shape than hyperrectangles. We present an extension of multi-index stochastic collocation (MISC) that uses isogeometric analysis (IGA) instead of conventional finite element analysis. MISC uses tensorized PDE solvers, and this allows IGA solvers as they build on tensorization of univariate splines. A feature of IGA that enables its solvers to handle non-standard domains is that the basis functions that describe the domain geometry, typically standard B-splines or non-uniform rational B-splines (NURBS), are used as the basis in approximating the PDE solution on the domain. We numerically demonstrate the computational efficiency of IGA-based MISC for linear elliptic PDEs with random coefficients in domains of complicated shape.
  • 26.01.2021, 16:00: Dr. Zaid Sawlan, KAUST
    • Title: Statistical and Bayesian methods for fatigue life prediction.
    • Abstract: Predicting fatigue in mechanical components is extremely important for preventing hazardous situations. In this work, we calibrate several plausible probabilistic stress-lifetime (S-N) models using fatigue experiments on unnotched specimens. To generate accurate fatigue life predictions, competing S-N models are ranked according to several classical information-based measures. A different set of predictive information criteria is then used to compare the candidate Bayesian models. Moreover, we propose a spatial stochastic model to generalize S-N models to fatigue crack initiation in general geometries. The model is based on a spatial Poisson process with an intensity function that combines the S-N curves with an averaged effective stress that is computed from the solution of the linear elasticity equations. The resulting model can predict the initiation of cracks in specimens made from the same material with new geometries.
  • 19.01.2021, 16:00: Giacomo Garegnani, EPFL
    • Title: Filtering the data: An alternative to subsampling for drift estimation of multiscale diffusions
    • Abstract:  We present a novel technique for estimating the effective drift of two-scale diffusion processes. We set ourselves in a semi-parametric framework and fit to data a single-scale equation of the overdamped Langevin type. If data is given in the form of a continuous time series, a pre-processing technique is needed for unbiasedness. Oftentimes, this is achieved by subsampling the data at an appropriate rate, which lies between the two characteristic time scales. We avoid subsampling and process the data with an appropriate low-pass filter, thus proposing maximum likelihood estimators and Bayesian techniques which are based on the filtered process. We show that our methodology is asymptotically unbiased and demonstrate numerically an enhanced robustness with respect to subsampling on several test cases.

Previous talks (2020)

  • 15.12.2020, 16:00: Prof. Sebastian Krumscheid, RWTH Aachen University.
    • Title: Adaptive Stratified Sampling for Non-smooth Problems
    • Abstract: Sampling based variance reduction techniques, such as multilevel Monte Carlo methods, have been established as a general-purpose procedure for quantifying uncertainties in computational models. It is known however, that these techniques may not provide performance gains when there is a non-smooth parameter dependence. Moreover, in many applications (e.g. transport problems in fractured porous media of relevance to carbon storage and wastewater injection) the key idea of multilevel Monte Carlo cannot be fully exploited since no hierarchy of computational models can be constructed. An alternative means to obtain variance reduction in these cases is offered by stratified sampling methods. In this talk we will discuss various ideas on adaptive stratified sampling methods tailored to applications with a discontinuous parameter dependence. Specifically, we will build upon ideas from adaptive PDE mesh refinement strategies applied to the stochastic instead of the physical domain. That is, the stochastic domain is adaptively stratified using local sensitivity estimates, and the samples are sequentially allocated to the strata for asymptotically optimal variance reduction. The proposed methodology is demonstrated on geomechanics in fractured reservoirs, and computational speedup compared to standard Monte Carlo is obtained. This is joint work with Per Pettersson (NORCE)
  • 08.12.2020, 16:00: Dr. Jonas Kiessling, RWTH Aachen University, and KTH.
    • Title: Deep Residual Neural Networks: Generalization Error and Parameterization.
    • Abstract: In this talk I will consider the supervised learning problem of reconstructing a target function from noisy data using a deep residual neural network. I will give an overview of residual networks and present estimates of optimal generalization errors. I will touch on the ever important question of "why deep and not shallow neural networks" and show that under certain circumstances, deep residual networks have better approximation capacity than shallow networks with similar number of free parameters. I will also discuss a layer-by-layer training algorithm and show some results on simulated data. This is joint work with A. Kammonen, P. Plechac, M. Sandberg, A. Szepessy and R. Tempone. The seminar is based on Smaller generalization error derived for a deep residualneural network compared to shallow networks, available on arXiv.
  • 01.12.2020, 16:00: Dr. Marco Scavino, Instituto de Estadística, Universidad de la República, Uruguay.
    • Title: A SDE model with derivative tracking for wind power forecast error: inference and application.
    • Abstract: Reliable wind power generation forecasting is crucial for many applications in the electricity market. We propose a data-driven modelbased on parametric Stochastic Differential Equations (SDEs) to capture the real asymmetric dynamics of wind power forecast errors. Our SDE framework features time-derivative tracking of the forecast, time-varying mean-reversion parameter, and an improved state-dependent diffusion term. The statistical inference methods we developed and applied allows the simulation of future wind power production paths and to obtain sharp empirical confidence bands. All the procedures are agnostic of the forecasting technology, and they enable comparisons between different forecast providers. We apply the model to historical Uruguayan wind power production data and forecasts between April and December 2019.
      This talk is based on the work: Renzo Caballero, Ahmed Kebaier, Marco Scavino, Raúl Tempone (2020). A Derivative Tracking Model for Wind Power Forecast Error (
  • 24.11.2020, 16:00: Dr. Ben Mansour Dia, College of Petroleum Engineering & Geosciences, KFUPM, KSA
    • Title: Continuation Bayesian inference
    • Abstract: We present a continuation method that entails generating a sequence of transition probability density functions  from the prior to the posterior in the context of Bayesian inference for parameter estimation problems.  The characterization of transition distributions, by tempering the likelihood function, results in a homogeneous nonlinear  partial integro-differential equation for which existence and uniqueness of solutions are addressed.  The posterior probability distribution comes as the interpretation of the final state of a path of transition distributions.  A computationally stable scaling domain for the likelihood is explored for the approximation of the expected deviance,  where we manage to restrict the evaluations of the forward predictive model at the prior stage. It follows the computational  tractability of the posterior distribution and opens access to the posterior distribution for direct sampling.  To get a solution formulation of the expected deviance, we derive a partial differential equation governing the moment generating  function of the log-likelihood. We show also that a spectral formulation of the expected deviance can be obtained  for low-dimensional problems under certain conditions. The effectiveness of the proposed method is demonstrated through three  numerical examples that focus on analyzing the computational bias generated by the method, assessing the continuation method  in the Bayesian inference with non-Gaussian noise, and evaluating its ability to invert a multimodal parameter of interest.
  • 17.11.2020, 16:00: Dr. Jonas Kiessling, RWTH Aachen and KTH.
    • Title: Adaptive Random Fourier Features with Metropolis Sampling.
  • 10.11.2020, 16:00: Dr. Alexander Litvinenko, RWTH Aachen University.
    • Title: Solution of the density-driven groundwater flow problem with uncertain porosity and permeability.
    • Abstract: The pollution of groundwater, essential for supporting populations and agriculture, can have catastrophic consequences. Thus, accurate modeling of water pollution at the surface and in groundwater aquifers is vital.  Here, we consider a density-driven groundwater flow problem with uncertain porosity and permeability. Addressing this problem is relevant for geothermal reservoir simulations, natural saline-disposal basins, modeling of contaminant plumes and subsurface flow predictions.  This strongly nonlinear time-dependent problem describes the convection of a two-phase flow, whereby a liquid flows and propagates into groundwater reservoirs under the force of gravity to form so-called ``fingers'’.  To achieve an accurate numerical solution, fine spatial resolution with an unstructured mesh and, therefore, high computational resources are required.  Here we run a parallelized simulation toolbox UG4 with a geometric multigrid solver on a parallel cluster, and the parallelization is carried out in physical and stochastic spaces.  Additionally, we demonstrate how the UG4 toolbox can be run in a black-box fashion for testing different scenarios in the density-driven flow.  As a benchmark, we solve the Elder-like problem in a 3D domain. For approximations in the stochastic space, we use the generalized polynomial chaos expansion. We compute the mean, variance, and exceedance probabilities for the mass fraction. We use the solution obtained from the quasi-Monte Carlo method as a reference solution.
  • 03.11.2020, 16:00: Dr. Lorenzo Tamellini, CNR-IMATI Pavia.
  • Title: Uncertainty quantification and identifiability of SIR-like dynamical systems for epidemiology.
    • Abstract: In this talk, we provide an overview of the methods that can be used for prediction under uncertainty and data fitting of dynamical systems, and of the fundamental challenges that arise in this context. The focus is on SIR-like models, that are being commonly used when attempting to predict the trend of the COVID-19 pandemic. In particular, we raise a warning flag about identifiability of the parameters of SIR-like models; often, it might be hard to infer the correct values of the parameters from data, even for very simple models, making it non-trivial to use these models for meaningful predictions. Most of the points that we touch upon are actually generally valid for inverse problems in more general setups.
  • 30.10.2020, 13:00: Sophia Franziska Wiechert, RWTH Aachen.
    • Title: Continuous Time Markov Decision Processes with Finite Time Horizon.
    • Abstract: One can derive a Markov Decision Process (MDP) by adding an input to a continuous-time Markov Process. These inputs, also called actions, allow us to change the states' transition rates, hence, to "control" the Markov Process. By adding a reward dependent on the current state and action, one can formulate the MDP's optimal control problem. In this talk, we restrict ourselves to finite horizon problems. The aim is to find the optimal actions, which maximize the reward over a finite time horizon. The Hamilton-Jacobi-Bellman equation gives an analytic solution of the optimal control problem. Solving this system of ordinary differential equations is difficult in general. Therefore, the problem is discretized in time and solved as a discrete-time Markov decision chain by a simple algorithm that iterates backward in time. We illustrate this approach through the example of salmon farming.
  • 27.10.2020, 16:00: Emil Loevbak, KU Leuven, Belgium.
    • Title: Asymptotic-preserving multilevel Monte Carlo particle methods for diffusively scaled kinetic equations.
    • Abstract: In many applications it is necessary to compute the time-dependent distribution of an ensemble of particles subject to transport and collision phenomena. Kinetic equations are PDEs that model such particles in a position-velocity phase space. In the low collisional regime explicit particle-based Monte Carlo methods simulate these high dimensional equations efficiently, but, as the collision rate increases, these methods suffer from severe time-step constraints.

      Asymptotic-preserving particle schemes are able to avoid these time-step constraints by explicitly including information from models describing the infinite collision rate case. However, these schemes produce biased results when used with large simulation time steps. In recent years, we have shown that the multilevel Monte Carlo method can be used to reduce this bias by combining simulations with large and small time steps, computing accurate results with greatly reduced simulation cost. In this talk, I will present the current state of the art for this newly developed asymptotic-preserving multilevel Monte Carlo approach. This includes an overview of existing methods and numerical results. I will then conclude with a view on future prospects for these methods.

  • 20.10.2020, 16:00: Dr. Luis Espath (RWTH Aachen)
    • Title: Multilevel Double Loop Monte Carlo Method with Importance Sampling for Bayesian Optimal Experimental Design
    • Abstract: An optimal experimental set-up maximizes the value of data for statistical inferences. The efficiency of strategies for finding optimal experimental set-ups is particularly important for experiments that are time-consuming or expensive to perform. When the experiments are modeled by Partial Differential Equations (PDEs), multilevel methods have been proven to reduce the computational complexity of their single-level counterparts when estimating expected values. For a setting where PDEs can model experiments, we propose a multilevel method for estimating the widespread criterion known as the Expected Information Gain (EIG) in Bayesian optimal experimental design. We propose a Multilevel Double Loop Monte Carlo (MLDLMC), where the Laplace approximation is used for importance sampling in the inner expectation. The method's efficiency is demonstrated by estimating EIG for inference of the fiber orientation in composite laminate materials from an electrical impedance tomography experiment.
  • 13.10.2020, 16:00: Dr. Neil Chada (KAUST)
    • Title: Consistency analysis of data-driven bilevel learning in inverse problems
    • Abstract: One fundamental problem when solving inverse problems is how to find regularization parameters. This talk considers solving this problem using data-driven bilevel optimization. This approach can be interpreted as solving an empirical risk minimization problem, and its performance with large data sample size can be studied in general nonlinear settings. To reduce the associated computational cost, online numerical schemes are derived using the stochastic gradient method. The convergence of these numerical schemes can also be analyzed under suitable assumptions. Numerical experiments are presented illustrating the theoretical results and demonstrating the applicability and efficiency of the proposed approaches for various linear and nonlinear inverse problems, including Darcy flow, the eikonal equation, and an image denoising example.
  • 06.10.2020, 16:00: Arved Bartuska (RWTH)
    • Title: Laplace approximation for Bayesian experimental design
    • Abstract: The problem of finding the optimal design of an experiment in a Bayesian setting via the expected information gain (EIG) leads to the computation of two nested integrals that are usually not given in closed-form. The standard approach uses a double loop Monte Carlo estimator, which can still be very costly in many cases. Two alternative estimators based on the Laplace approximation will be presented in this talk, followed by a numerical example from the field of electrical impedance tomography (EIT).
  • 29.09.2020, 16:00: Jonas Kiessling, Emanuel Ström, Magnus Tronstad
    • Title: Wind Field Reconstruction from Historical Weather Data
    • Abstract: In this talk we will present ongoing work in wind field reconstruction from historical weather measurements. We draw on techniques from Machine Learning (ML) and Fourier Analysis, and show how standard ML models can be improved by including physically motivated penalty terms. Our model is tested on public historical weather data from Sweden, and benchmarked against a range of other published models, including Kriging, Nearest Neighbour and Average Inverse Distance. This is a joint work with Andreas Enblom, Luis Espath, Dmitry Kabanov and Raul Tempone.
  • 22.09.2020, 16:00: Dr. Nadhir Ben Rached (RWTH)
    • Title: Dynamic splitting method for rare events simulation
    • Abstract: We propose a unified rare-event estimator based on the multilevel splitting algorithm. In its original form, the splitting algorithm cannot be applied to time-independent problems because splitting requires an underlying continuous-time Markov process whose trajectories can be split. We embed the time-independent problem within a continuous-time Markov process so that the target static distribution corresponds to the distribution of the Markov process at a given time instant. To illustrate the large scope of applicability of the proposed approach, we apply it to the problem of estimating the cumulative distribution function (CDF) of sums of random variables (RVs), the CDF of partial sums of ordered RVs, the CDF of ratios of RVs, and the CDF of weighted sums of  Poisson RVs. We investigate the computational efficiency of the proposed estimator via a number of simulation studies and find that it compares favorably with existing estimators.
  • 09.06.2020, 14:00: Prof. Benjamin Berkels (RWTH)
    • Webinar: zoom meeting link will be circulated through the MATH4UQ seminar mailing list.
    • Title: Image registration and segmentation using variational methods
    • Abstract: Image segmentation and registration are two of the fundamental image processing problems arising in many different application areas.
      Registration is the task of transforming two or more images into a common coordinate system. After a short introduction to variational image registration, we demonstrate that non-rigid registration techniques can be used to achieve sub-picometer precision measurements of atom positions from a series of scanning transmission electron microscopy images at atomic scale. Particular challenges here are input data with low signal-to-noise ratio and periodic structures, as well as initialization bias of the resulting iterative optimization strategies for the non-convex objective.
      Segmentation is to decompose an image into disjoint regions that are roughly homogeneous in a suitable sense. If three or more regions are sought, one speaks of multi-phase segmentation. We first review how to find global minimizers of the non-convex binary Mumford-Shah model to solve the classical two-phase segmentation problem and show segmentation problems from different application areas. Then, we propose a flexible framework for multi-phase segmentation based on the Mumford-Shah model and high-dimensional local feature vectors.
  • 13.03.2020, 14:00: Prof. Fabio Nobile (EPFL)
    • Title: A multilevel stochastic gradient algorithm for PDE-constrained optimal control problems under uncertainty
    • Abstract: We consider an optimal control problem for an elliptic PDE with random coefficients. The control function is a deterministic, distributed forcing term that minimizes an expected quadratic regularized loss functional. For its numerical treatment we propose and analyze a multilevel stochastic gradient (MLSG) algorithm which uses at each iteration a full, or randomized, multilevel Monte Carlo estimator of the expected gradient, build on a hierarchy of finite element approximations of the underlying PDE. The algorithm requires choosing proper rates at which the finite element discretization is refined and the Monte Carlo sample size increased over the iterations.  We present complexity bounds for such algorithm. In particular, we show that if the refinement rates are properly chosen, in certain cases the asymptotic complexity of the full MLSG algorithm in computing the optimal control is the same as the complexity of computing the expected loss functional for one given control by a standard multilevel Monte Carlo estimator. This is joint work with Matthieu Martin (CRITEO, Grenoble), Panagiotis Tsilifis (General Electric), Sebastian Krumscheid (RWTH Aachen).
  • 05.03.2020, 14:00: Prof. Fabio Nobile (EPFL)
    • Title: Dynamical Low Rank approximation of random time dependent PDEs
    • Abstract: In this talk we consider time dependent PDEs with random parameters and seek for an approximate solution in separable form that can be written at each time instant as a linear combination of a fixed number of linearly independent spatial functions multiplied by linearly independent random variables (low rank approximation). Since the optimal deterministic and stochastic modes can significantly change over time, we consider a dynamical approach where those modes are computed on the fly as solutions of suitable evolution equations. We discuss the construction of the method, present an existence result for the low rank approximate solution of a random semi-linear evolutionary equation of diffusion type, and introduce an operator splitting numerical discretization of the low rank equations for which we can prove (conditional) stability. This is joint work with Yoshihito Kazashi, Eleonora Musharbash, Eva Vidlicková.
  • 27.02.2020, 14:15: Chiheb Ben Hammouda (KAUST)
    • Title: Adaptive sparse grids and quasi-Monte Carlo for option pricing under the rough Bergomi model
    • Abstract: The rough Bergomi (rBergomi) model, introduced recently in (Bayer, Friz,  Gatheral,  2016), is a promising rough volatility model in quantitative finance. It is a parsimonious model depending on only three parameters, and yet remarkably fits with empirical implied volatility surfaces. In the absence of analytical European option pricing methods for the model, and due to the non Markovian nature of the fractional driver, the prevalent option is to use the Monte Carlo (MC) simulation for pricing. Despite recent advances in the MC method in this context, pricing under the rBergomi model is still a time-consuming task. To overcome this issue, we have designed a novel, hierarchical approach, based on i) adaptive sparse grids quadrature (ASGQ), and ii) quasi-Monte Carlo (QMC). Both techniques are coupled with a Brownian bridge construction and a Richardson extrapolation on the weak error. By uncovering the available regularity, our hierarchical methods demonstrate substantial computational gains with respect to the standard MC method, when reaching a sufficiently small relative error tolerance in the price estimates across different parameter constellations, even for very small values of the Hurst parameter. Our work opens a new research direction in this field, i.e., to investigate the performance of methods other than Monte Carlo for pricing and calibrating under the rBergomi model.
  • 25.02.2020, 14:00: Chiheb Ben Hammouda (KAUST)
    • Title: Numerical Smoothing and Hierarchical Approximations for Efficient Option Pricing  and Density Estimation
    • Abstract: When approximating expectations of certain quantity of interest (QoI), the efficiency and performance of deterministic quadrature methods, such as sparse grids, and hierarchical variance reduction methods such as multilevel Monte Carlo (MLMC), may be highly deteriorated, in different ways, by the low regularity of the QoI with respect to the input parameters. To overcome this issue, a smoothing procedure is needed to uncover the available regularity and improve the performance of the aforementioned numerical methods. In this work, we consider cases where we can not perform an analytic smoothing and introduce a novel numerical smoothing technique, based on root finding combined with a one dimensional integration with respect to a single well-chosen variable. We prove that under appropriate conditions the resulting function of the remaining variables is a highly smooth function, so potentially allowing a higher efficiency of adaptive sparse grids quadrature (ASGQ), in particular when it is combined with hierarchical transformations (Brownian bridge and Richardson extrapolation on the weak error) to treat effectively the high dimensionality.  Our study is motivated by option pricing problems and our main focus is on dynamics where a discretization of the asset price is needed.  Through our analysis and numerical experiments, we illustrate the advantage of combining numerical smoothing with ASGQ, over the Monte Carlo (MC) approach. Furthermore, we demonstrate how the numerical smoothing significantly reduces the kurtosis at the deep levels of MLMC, and also improves the strong convergence rate, when using Euler scheme from 1/2 for the standard case (without smoothing), to 1.  Due to the complexity theorem of MLMC and given a pre-selected tolerance, TOL, this results in an improvement of the complexity from O(TOL^{-2.5}) in the standard case to O(TOL^{-2}\log(TOL)^2). Finally, we show how our numerical smoothing combined with MLMC enables us also to estimate density functions, a situation where standard MLMC fails.
  • 18.02.2020, 14:45: Chiheb Ben Hammouda (KAUST)
    • Title: Importance Sampling for a Robust and Efficient Multilevel Monte Carlo  Estimator for Stochastic Reaction Networks
    • Abstract: The multilevel Monte Carlo (MLMC) method for continuous time Markov chains, first introduced by Anderson and Higham (SIAM Multiscal Model. Simul. 10(1), 2012), is a highly efficient simulation technique that can be used to estimate various statistical quantities for stochastic reaction networks (SRNs), and in particular for stochastic biological systems. Unfortunately, the robustness and performance of the multilevel method can be deteriorated due to the phenomenon of high kurtosis, observed at the deep levels of MLMC, which leads to inaccurate estimates for the sample variance. In this work, we address cases where the high kurtosis phenomenon is due to catastrophic coupling (characteristic of pure jump processes where coupled consecutive paths are identical in most of the simulations, while differences only appear in a very small proportion), and introduce a pathwise dependent importance sampling technique that improves the robustness and efficiency of the multilevel method. Our analysis, along with the conducted numerical experiments, demonstrates that our proposed method significantly reduces the kurtosis at the deep levels of MLMC, and also improves the strong convergence rate. Due to the complexity theorem of MLMC and given a pre-selected tolerance, TOL, this results in an improvement of the complexity from  O(TOL^{-2} \log(TOL)^2)  in the standard case to O(TOL^{-2}).
  • 17.02.2020, 14:00: Truong-Vinh Hoang (Technische Universität Braunschweig)
    • Title: Neural network-based filtering technique for high-dimensional and non-linear data assimilation
    • Abstract: The talk aims at an introduction about the author's research, with a particular focus on the neural network-based filtering technique for data assimilation. We propose a novel ensemble filter for high dimensional and non-linear data assimilation problem. The method trains an artificial neural network (ANN) to approximate the conditional expectation using as training data the prior ensemble and their predicted observations. To avoid over-fitting when training the ANN on ensembles of relatively small sizes, different techniques are employed such as L2 regularisation, dataset augmentation and multilevel method. The trained ANN is then used in computing the assimilated ensemble. Our approach can be understood as a natural generalisation of the ensemble Kalman filter (EnKF) to non-linear updates. Using ANN to approximate the conditional expectation can reduce the intrinsic bias of the EnKF and improve its predictions.  To illustrate the approach, we implement our framework for tracking the states in the Lorenz-64 and Lorenz-93 systems.
    • 14.02.2020, 14:00: Chiheb Ben Hammouda (KAUST)
      • Title: Multilevel Hybrid Split Step Implicit Tau-Leap for Stochastic Reaction Networks
      • Abstract: In biochemically reactive systems with small copy numbers of one or more reactant molecules, the dynamics are dominated by stochastic effects. To approximate those systems, discrete state-space and stochastic simulation approaches have been shown to be more relevant than continuous state-space and deterministic ones.  These stochastic models constitute the theory of Stochastic Reaction Networks (SRNs). In systems characterized by having simultaneously fast and slow timescales, existing discrete space-state stochastic path simulation methods, such as the stochastic simulation algorithm (SSA) and the explicit tau-leap (explicit-TL) method, can be very slow. In this talk, we propose a novel implicit scheme, split-step implicit tau-leap (SSI-TL),  to improve numerical stability and provide efficient simulation algorithms for those systems.  Furthermore,  to estimate statistical quantities related to SRNs,  we propose a novel hybrid Multilevel Monte Carlo (MLMC) estimator in the spirit of the work by Anderson and Higham (SIAM Multiscal Model. Simul. 10(1), 2012). This estimator uses the SSI-TL scheme at levels where the explicit-TL method is not applicable due to numerical stability issues, and then, starting from a certain interface level, it switches to the explicit scheme. We present numerical examples that illustrate the achieved gains of our proposed approach in this context.


    External Links