MIT Uncertainty Quantification Group

We present the fundamentals of a measure transport approach to sampling. The idea is to construct a deterministic coupling – i.e., a transport map – between a complex ‘‘target’’ probability measure of interest and a simpler reference measure. Given a transport map, one can generate arbitrarily many independent and unweighted samples from the target simply by pushing forward reference samples through the map. If the map is endowed with a triangular structure, one can also easily generate samples from conditionals of the target measure. We consider two different and complementary scenarios: first, when only evaluations of the unnormalized target density are available and, second, when the target distribution is known only through a finite collection of samples. We show that in both settings, the desired transports can be characterized as the solutions of variational problems. We then address practical issues associated with the optimization-based construction of transports: choosing finite-dimensional parameterizations of the map, enforcing monotonicity, quantifying the error of approximate transports, and refining approximate transports by enriching the corresponding approximation spaces. Approximate transports can also be used to ‘‘Gaussianize’’ complex distributions and thus precondition conventional asymptotically exact sampling schemes. We place the measure transport approach in broader context, describing connections with other optimization-based samplers, with inference and density estimation schemes using optimal transport, and with alternative transformation-based approaches to simulation. We also sketch current work aimed at the construction of transport maps in high dimensions, exploiting essential features of the target distribution (e.g., conditional independence, low-rank structure). The approaches and algorithms presented here have direct applications to Bayesian computation and to broader problems of stochastic simulation.

@inbook {marzouk-transport-2016,
  address    = "Cham",
  author     = "Y. M. Marzouk and T. Moselhy and M. Parno and A. Spantini",
  booktitle  = "Handbook of Uncertainty Quantification",
  editor     = "R. Ghanem and D. Higdon and H. Owhadi",
  pages      = "1--41",
  publisher  = "Springer International Publishing",
  title      = "Sampling via Measure Transport: An Introduction",
  year       = "2016",
  doi        = "10.1007/978-3-319-11259-6_23-1",
  abstract   = "We present the fundamentals of a measure transport approach to sampling. The idea is to construct a deterministic coupling -- i.e., a transport map -- between a complex ``target'' probability measure of interest and a simpler reference measure. Given a transport map, one can generate arbitrarily many independent and unweighted samples from the target simply by pushing forward reference samples through the map. If the map is endowed with a triangular structure, one can also easily generate samples from conditionals of the target measure. We consider two different and complementary scenarios: first, when only evaluations of the unnormalized target density are available and, second, when the target distribution is known only through a finite collection of samples. We show that in both settings, the desired transports can be characterized as the solutions of variational problems. We then address practical issues associated with the optimization-based construction of transports: choosing finite-dimensional parameterizations of the map, enforcing monotonicity, quantifying the error of approximate transports, and refining approximate transports by enriching the corresponding approximation spaces. Approximate transports can also be used to ``Gaussianize'' complex distributions and thus precondition conventional asymptotically exact sampling schemes. We place the measure transport approach in broader context, describing connections with other optimization-based samplers, with inference and density estimation schemes using optimal transport, and with alternative transformation-based approaches to simulation. We also sketch current work aimed at the construction of transport maps in high dimensions, exploiting essential features of the target distribution (e.g., conditional independence, low-rank structure). The approaches and algorithms presented here have direct applications to Bayesian computation and to broader problems of stochastic simulation."
}

Optimal experimental design (OED) aims to maximize the value of experiments and the data they produce. OED ensures efficient allocation of limited resources, especially when numerous repeated experiments cannot be performed. This chapter presents a fully Bayesian and decision theoretic approach to OED–-accounting for uncertainties in models, model parameters, and experimental outcomes, and allowing optimality to be defined according to a range of possible experimental goals. We demonstrate this approach on two illustrative problems in materials research. The first example is a parameter inference problem. Its goal is to determine a substrate property from the behavior of a film deposited thereon. We design experiments to yield maximal information about the substrate property using only two measurements. The second example is a model selection problem. We design an experiment that optimally distinguishes between two models for helium trapping at interfaces. In both instances, we provide model-based justifications for why the selected experiments are optimal. Moreover, both examples illustrate the utility of reduced-order or surrogate models in optimal experimental design.

@inbook {doi:10.1007/978-3-319-23871-5_2,
  address    = "Cham",
  author     = "R. Aggarwal and M. J. Demkowicz and Y. M. Marzouk",
  booktitle  = "Information Science for Materials Discovery and Design",
  editor     = "T. Lookman and F. Alexander and K. Rajan",
  pages      = "13--44",
  publisher  = "Springer International Publishing",
  title      = "Information-driven experimental design in materials science",
  year       = "2016",
  doi        = "10.1007/978-3-319-23871-5_2",
  abstract   = "Optimal experimental design (OED) aims to maximize the value of experiments and the data they produce. OED ensures efficient allocation of limited resources, especially when numerous repeated experiments cannot be performed. This chapter presents a fully Bayesian and decision theoretic approach to OED---accounting for uncertainties in models, model parameters, and experimental outcomes, and allowing optimality to be defined according to a range of possible experimental goals. We demonstrate this approach on two illustrative problems in materials research. The first example is a parameter inference problem. Its goal is to determine a substrate property from the behavior of a film deposited thereon. We design experiments to yield maximal information about the substrate property using only two measurements. The second example is a model selection problem. We design an experiment that optimally distinguishes between two models for helium trapping at interfaces. In both instances, we provide model-based justifications for why the selected experiments are optimal. Moreover, both examples illustrate the utility of reduced-order or surrogate models in optimal experimental design."
}

@inbook {isbn:9780691150390,
  author     = "Y. M. Marzouk and K. E. Willcox",
  booktitle  = "The Princeton Companion to Applied Mathematics",
  chapter    = "34",
  editor     = "N. Higham and M. Dennis and P. Glendinning and P. Martin and F. Santosa and J. Tanner",
  pages      = "131--134",
  publisher  = "Princeton University Press",
  title      = "Uncertainty Quantification",
  volume     = "II",
  year       = "2015"
}

@incollection {doi:10.1002/9780470685853.ch7,
  author     = "M. Frangos and Y. M. Marzouk and K. Willcox and B. van Bloemen Waanders",
  booktitle  = "Computational Methods for Large Scale Inverse Problems and Uncertainty Quantification",
  chapter    = "7",
  editor     = "L. Biegler and G. Biros and O. Ghattas and M. Heinkenschloss and D. Keyes and B. Mallick and Y. Marzouk and L. Tenorio and B. van Bloemen Waanders and K. Willcox",
  publisher  = "Wiley",
  title      = "Surrogate and reduced-order modeling: a comparison of approaches for large-scale statistical inverse problems",
  year       = "2010",
  doi        = "10.1002/9780470685853.ch7"
}

@book {doi:10.1002/9780470685853,
  author     = "L. Biegler and G. Biros and O. Ghattas and M. Heinkenschloss and D. Keyes and B. Mallick and Y. Marzouk and L. Tenorio and B. van Bloemen Waanders and K. Willcox",
  publisher  = "John Wiley & Sons, Ltd",
  title      = "Large-Scale Inverse Problems and Quantification of Uncertainty",
  year       = "2010",
  doi        = "10.1002/9780470685853",
  keywords   = "statistical methods and approximation methods, inverse problem solving - estimating unknown objects from indirect noisy observations, earth's subsurface mapping and seismic waves, ill-posed inverse problems - in science and engineering, precursors to quantification of uncertainties - prediction and decision-making, inverse problems, determining input - or given noisy data, crucial unmet need for scalable numerical algorithm development - and large-scale inverse problem solution, computational methods - for large-scale inverse problems, inverse problems and statistical characterizations - uncertainties modeled randomly, approximation techniques - replacing forward model with inexpensive surrogates"
}

@misc {button2018,
  author     = "K. Button",
  howpublished = "Aerospace America, March 2018",
  title      = "Engineering Notebook: Wringing Out The Risks"
}

@misc {aamag12,
  author     = "Y. M. Marzouk",
  howpublished = "AeroAstro Magazine No. 12, 2014--2015.",
  title      = "Quantifying uncertainty in the physical world"
}

@misc {aamag7,
  author     = "Y. M. Marzouk and K. E. Willcox",
  howpublished = "AeroAstro Magazine No. 7, 2009--2010",
  title      = "Confronting energy and environment's toughest challenges with computational engineering"
}