Uncertainty Quantification Group

Available Links

• https://www.cambridge.org/core/journals/acta-numerica/article/optimal-experimental-design-formulations-and-computations/38BBD0DC1A0386FDF306B6C0167DF7D9

Abstract

Questions of ‘how best to acquire data’ are essential to modelling and prediction in the natural and social sciences, engineering applications, and beyond. Optimal experimental design (OED) formalizes these questions and creates computational methods to answer them. This article presents a systematic survey of modern OED, from its foundations in classical design theory to current research involving OED for complex models. We begin by reviewing criteria used to formulate an OED problem and thus to encode the goal of performing an experiment. We emphasize the flexibility of the Bayesian and decision-theoretic approach, which encompasses information-based criteria that are well-suited to nonlinear and non-Gaussian statistical models. We then discuss methods for estimating or bounding the values of these design criteria; this endeavour can be quite challenging due to strong nonlinearities, high parameter dimension, large per-sample costs, or settings where the model is implicit. A complementary set of computational issues involves optimization methods used to find a design; we discuss such methods in the discrete (combinatorial) setting of observation selection and in settings where an exact design can be continuously parametrized. Finally we present emerging methods for sequential OED that build non-myopic design policies, rather than explicit designs; these methods naturally adapt to the outcomes of past experiments in proposing new experiments, while seeking coordination among all experiments to be performed. Throughout, we highlight important open questions and challenges.

BibTeX Citation

@article{ huan-oedreview-2024,
  keywords = { optimal experimental design },
  doi = { 10.1017/S0962492924000023 },
  title = { Optimal experimental design: Formulations and computations },
  author = { X. Huan and J. Jagalur and Y. M. Marzouk },
  journal = { Acta Numerica },
  volume = { 33 },
  pages = { 715-840 },
  month = { 7 },
  year = { 2024 },
  editor = {  },
}

Available Links

• https://arxiv.org/abs/1807.03712
• https://doi.org/10.1090/mcom/3737

Abstract

We propose a dimension reduction technique for Bayesian inverse problems with nonlinear forward operators, non-Gaussian priors, and non-Gaussian observation noise. The likelihood function is approximated by a ridge function, i.e., a map which depends non-trivially only on a few linear combinations of the parameters. We build this ridge approximation by minimizing an upper bound on the Kullback--Leibler divergence between the posterior distribution and its approximation. This bound, obtained via logarithmic Sobolev inequalities, allows one to certify the error of the posterior approximation. Computing the bound requires computing the second moment matrix of the gradient of the log likelihood function. In practice, a sample-based approximation of the upper bound is then required. We provide an analysis that enables control of the posterior approximation error due to this sampling. Numerical and theoretical comparisons with existing methods illustrate the benefits of the proposed methodology.

BibTeX Citation

@article{ art_73,
  keywords = { dimension reduction, nonlinear Bayesian inverse problem, logarithmic Sobolev inequality, certified error bound, non-asymptotic analysis },
  doi = { 10.1090/mcom/3737 },
  title = { Certified dimension reduction in nonlinear Bayesian inverse problems },
  author = { O. Zahm and T. Cui and K. J. H. Law and A. Spantini and Y. M. Marzouk },
  journal = { Mathematics of Computation },
  volume = { 91 },
  pages = { 1789–1835 },
  month = { 0 },
  year = { 2022 },
  editor = {  },
}

Available Links

• https://arxiv.org/abs/2006.04554
• https://jmlr.csail.mit.edu/papers/v22/20-1023.html

Abstract

We propose and analyze batch greedy heuristics for cardinality constrained maximization of non-submodular non-decreasing set functions. We consider the standard greedy paradigm, along with its distributed greedy and stochastic greedy variants. Our theoretical guarantees are characterized by the combination of submodularity and supermodularity ratios. We argue how these parameters define tight modular bounds based on incremental gains, and provide a novel reinterpretation of the classical greedy algorithm using the minorize-maximize (MM) principle. Based on that analogy, we propose a new class of methods exploiting any plausible modular bound. In the context of optimal experimental design for linear Bayesian inverse problems, we bound the submodularity and supermodularity ratios when the underlying objective is based on mutual information. We also develop novel modular bounds for the mutual information in this setting, and describe certain connections to polyhedral combinatorics. We discuss how algorithms using these modular bounds relate to established statistical notions such as leverage scores and to more recent efforts such as volume sampling. We demonstrate our theoretical findings on synthetic problems and on a real-world climate monitoring example.

BibTeX Citation

@article{ jagalur-batchgreedy-2020,
  keywords = { greedy methods, submodularity, non-submodular functions, optimal experi- mental design, inverse problems, mutual information, uncertainty quantification, Bayesian statistics },
  title = { Batch greedy maximization of non-submodular functions: guarantees and applications to experimental design },
  author = { J. Jagalur-Mohan and Y. M. Marzouk },
  journal = { The Journal of Machine Learning Research },
  volume = { 22 },
  number = { 252 },
  pages = { 1–62 },
  month = { 0 },
  year = { 2021 },
  editor = {  },
}