TY - BOOK A1 - Sullivan, T. J. T1 - Introduction to Uncertainty Quantification Y1 - 2015 SN - 978-3-319-23394-9 U6 - https://doi.org/10.1007/978-3-319-23395-6 VL - 63 PB - Springer ER - TY - JOUR A1 - Owhadi, Houman A1 - Scovel, Clint A1 - Sullivan, T. J. T1 - On the Brittleness of Bayesian Inference JF - SIAM Review N2 - With the advent of high-performance computing, Bayesian methods are becoming increasingly popular tools for the quantification of uncertainty throughout science and industry. Since these methods can impact the making of sometimes critical decisions in increasingly complicated contexts, the sensitivity of their posterior conclusions with respect to the underlying models and prior beliefs is a pressing question to which there currently exist positive and negative answers. We report new results suggesting that, although Bayesian methods are robust when the number of possible outcomes is finite or when only a finite number of marginals of the data-generating distribution are unknown, they could be generically brittle when applied to continuous systems (and their discretizations) with finite information on the data-generating distribution. If closeness is defined in terms of the total variation (TV) metric or the matching of a finite system of generalized moments, then (1) two practitioners who use arbitrarily close models and observe the same (possibly arbitrarily large amount of) data may reach opposite conclusions; and (2) any given prior and model can be slightly perturbed to achieve any desired posterior conclusion. The mechanism causing brittleness/robustness suggests that learning and robustness are antagonistic requirements, which raises the possibility of a missing stability condition when using Bayesian inference in a continuous world under finite information. Y1 - 2015 U6 - https://doi.org/10.1137/130938633 VL - 57 IS - 4 SP - 566 EP - 582 ER - TY - JOUR A1 - Lie, Han Cheng A1 - Sullivan, T. J. T1 - Cameron--Martin theorems for sequences of Cauchy-distributed random variables JF - arXiv N2 - Given a sequence of Cauchy-distributed random variables defined by a sequence of location parameters and a sequence of scale parameters, we consider another sequence of random variables that is obtained by perturbing the location or scale parameter sequences. Using a result of Kakutani on equivalence of infinite product measures, we provide sufficient conditions for the equivalence of laws of the two sequences. Y1 - 2016 SP - 1608.03784 ER - TY - GEN A1 - Lie, Han Cheng A1 - Sullivan, T. J. T1 - Cameron--Martin theorems for sequences of Cauchy-distributed random variables N2 - Given a sequence of Cauchy-distributed random variables defined by a sequence of location parameters and a sequence of scale parameters, we consider another sequence of random variables that is obtained by perturbing the location or scale parameter sequences. Using a result of Kakutani on equivalence of infinite product measures, we provide sufficient conditions for the equivalence of laws of the two sequences. T3 - ZIB-Report - 16-40 Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-60230 SN - 1438-0064 ER - TY - GEN A1 - Lie, Han Cheng A1 - Sullivan, T. J. A1 - Teckentrup, Aretha T1 - Random forward models and log-likelihoods in Bayesian inverse problems T2 - SIAM/ASA Journal on Uncertainty Quantification N2 - We consider the use of randomised forward models and log-likelihoods within the Bayesian approach to inverse problems. Such random approximations to the exact forward model or log-likelihood arise naturally when a computationally expensive model is approximated using a cheaper stochastic surrogate, as in Gaussian process emulation (kriging), or in the field of probabilistic numerical methods. We show that the Hellinger distance between the exact and approximate Bayesian posteriors is bounded by moments of the difference between the true and approximate log-likelihoods. Example applications of these stability results are given for randomised misfit models in large data applications and the probabilistic solution of ordinary differential equations. T3 - ZIB-Report - 18-03 KW - Bayesian inverse problem KW - random likelihood KW - surrogate model KW - posterior consistency KW - probabilistic numerics KW - uncertainty quantification KW - randomised misfit Y1 - 2018 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-66324 SN - 1438-0064 VL - 6 IS - 4 SP - 1600 EP - 1629 ER - TY - JOUR A1 - Cockayne, Jon A1 - Oates, Chris A1 - Sullivan, T. J. A1 - Girolami, Mark T1 - Bayesian Probabilistic Numerical Methods JF - SIAM Review N2 - Over forty years ago average-case error was proposed in the applied mathematics literature as an alternative criterion with which to assess numerical methods. In contrast to worst-case error, this criterion relies on the construction of a probability measure over candidate numerical tasks, and numerical methods are assessed based on their average performance over those tasks with respect to the measure. This paper goes further and establishes Bayesian probabilistic numerical methods as solutions to certain inverse problems based upon the numerical task within the Bayesian framework. This allows us to establish general conditions under which Bayesian probabilistic numerical methods are well defined, encompassing both the non-linear and non-Gaussian context. For general computation, a numerical approximation scheme is proposed and its asymptotic convergence established. The theoretical development is extended to pipelines of computation, wherein probabilistic numerical methods are composed to solve more challenging numerical tasks. The contribution highlights an important research frontier at the interface of numerical analysis and uncertainty quantification, and a challenging industrial application is presented. Y1 - 2019 U6 - https://doi.org/10.1137/17M1139357 VL - 61 IS - 4 SP - 756 EP - 789 ER - TY - JOUR A1 - Lie, Han Cheng A1 - Sullivan, T. J. A1 - Stuart, Andrew T1 - Strong convergence rates of probabilistic integrators for ordinary differential equations JF - Statistics and Computing N2 - Probabilistic integration of a continuous dynamical system is a way of systematically introducing model error, at scales no larger than errors inroduced by standard numerical discretisation, in order to enable thorough exploration of possible responses of the system to inputs. It is thus a potentially useful approach in a number of applications such as forward uncertainty quantification, inverse problems, and data assimilation. We extend the convergence analysis of probabilistic integrators for deterministic ordinary differential equations, as proposed by Conrad et al.\ (\textit{Stat.\ Comput.}, 2016), to establish mean-square convergence in the uniform norm on discrete- or continuous-time solutions under relaxed regularity assumptions on the driving vector fields and their induced flows. Specifically, we show that randomised high-order integrators for globally Lipschitz flows and randomised Euler integrators for dissipative vector fields with polynomially-bounded local Lipschitz constants all have the same mean-square convergence rate as their deterministic counterparts, provided that the variance of the integration noise is not of higher order than the corresponding deterministic integrator. Y1 - 2019 U6 - https://doi.org/10.1007/s11222-019-09898-6 VL - 29 IS - 6 SP - 1265 EP - 1283 ER - TY - CHAP A1 - Cockayne, Jon A1 - Oates, Chris A1 - Sullivan, T. J. A1 - Girolami, Mark T1 - Probabilistic Numerical Methods for PDE-constrained Bayesian Inverse Problems T2 - Proceedings of the 36th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering N2 - This paper develops meshless methods for probabilistically describing discretisation error in the numerical solution of partial differential equations. This construction enables the solution of Bayesian inverse problems while accounting for the impact of the discretisation of the forward problem. In particular, this drives statistical inferences to be more conservative in the presence of significant solver error. Theoretical results are presented describing rates of convergence for the posteriors in both the forward and inverse problems. This method is tested on a challenging inverse problem with a nonlinear forward model Y1 - 2017 U6 - https://doi.org/10.1063/1.4985359 VL - 1853 SP - 060001-1 EP - 060001-8 ER - TY - JOUR A1 - Schäfer, Florian A1 - Sullivan, T. J. A1 - Owhadi, Houman T1 - Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity JF - Multiscale Modeling and Simulation Y1 - 2021 U6 - https://doi.org/10.1137/19M129526X VL - 19 IS - 2 SP - 688 EP - 730 ER - TY - JOUR A1 - Lie, Han Cheng A1 - Sullivan, T. J. T1 - Equivalence of weak and strong modes of measures on topological vector spaces JF - Inverse Problems N2 - Modes of a probability measure on an infinite-dimensional Banach space X are often defined by maximising the small-radius limit of the ratio of measures of norm balls. Helin and Burger weakened the definition of such modes by considering only balls with centres in proper subspaces of X, and posed the question of when this restricted notion coincides with the unrestricted one. We generalise these definitions to modes of arbitrary measures on topological vector spaces, defined by arbitrary bounded, convex, neighbourhoods of the origin. We show that a coincident limiting ratios condition is a necessary and sufficient condition for the equivalence of these two types of modes, and show that the coincident limiting ratios condition is satisfied in a wide range of topological vector spaces. Y1 - 2018 U6 - https://doi.org/10.1088/1361-6420/aadef2 VL - 34 IS - 11 SP - 115013 ER - TY - JOUR A1 - Schuster, Ingmar A1 - Constantine, Paul A1 - Sullivan, T. J. T1 - Exact active subspace Metropolis–Hastings, with applications to the Lorenz-96 system N2 - We consider the application of active subspaces to inform a Metropolis-Hastings algorithm, thereby aggressively reducing the computational dimension of the sampling problem. We show that the original formulation, as proposed by Constantine, Kent, and Bui-Thanh (SIAM J. Sci. Comput., 38(5):A2779-A2805, 2016), possesses asymptotic bias. Using pseudo-marginal arguments, we develop an asymptotically unbiased variant. Our algorithm is applied to a synthetic multimodal target distribution as well as a Bayesian formulation of a parameter inference problem for a Lorenz-96 system. Y1 - 2017 ER - TY - JOUR A1 - Lie, Han Cheng A1 - Sullivan, T. J. T1 - Quasi-invariance of countable products of Cauchy measures under non-unitary dilations JF - Electronic Communications in Probability Y1 - 2018 U6 - https://doi.org/10.1214/18-ECP113 VL - 23 IS - 8 SP - 1 EP - 6 ER - TY - GEN A1 - Sullivan, T. J. T1 - Well-posed Bayesian inverse problems and heavy-tailed stable Banach space priors N2 - This article extends the framework of Bayesian inverse problems in infinite-dimensional parameter spaces, as advocated by Stuart (Acta Numer. 19:451–559, 2010) and others, to the case of a heavy-tailed prior measure in the family of stable distributions, such as an infinite-dimensional Cauchy distribution, for which polynomial moments are infinite or undefined. It is shown that analogues of the Karhunen–Loève expansion for square-integrable random variables can be used to sample such measures. Furthermore, under weaker regularity assumptions than those used to date, the Bayesian posterior measure is shown to depend Lipschitz continuously in the Hellinger metric upon perturbations of the misfit function and observed data. T3 - ZIB-Report - 16-30 KW - Bayesian inverse problems KW - heavy-tailed distribution KW - Karhunen–Loève expansion KW - stable distribution KW - uncertainty quantification KW - well-posedness Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-59422 SN - 1438-0064 ER - TY - GEN A1 - Cockayne, Jon A1 - Oates, Chris A1 - Sullivan, T. J. A1 - Girolami, Mark T1 - Probabilistic Meshless Methods for Partial Differential Equations and Bayesian Inverse Problems N2 - This paper develops a class of meshless methods that are well-suited to statistical inverse problems involving partial differential equations (PDEs). The methods discussed in this paper view the forcing term in the PDE as a random field that induces a probability distribution over the residual error of a symmetric collocation method. This construction enables the solution of challenging inverse problems while accounting, in a rigorous way, for the impact of the discretisation of the forward problem. In particular, this confers robustness to failure of meshless methods, with statistical inferences driven to be more conservative in the presence of significant solver error. In addition, (i) a principled learning-theoretic approach to minimise the impact of solver error is developed, and (ii) the challenging setting of inverse problems with a non-linear forward model is considered. The method is applied to parameter inference problems in which non-negligible solver error must be accounted for in order to draw valid statistical conclusions. T3 - ZIB-Report - 16-31 KW - Probabilistic Numerics KW - Partial Differential Equations KW - Inverse Problems KW - Meshless Methods KW - Gaussian Processes KW - Pseudo-Marginal MCMC Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-59513 SN - 1438-0064 ER - TY - JOUR A1 - Girolami, Mark A. A1 - Ipsen, Ilse C. F. A1 - Oates, Chris A1 - Owen, Art B. A1 - Sullivan, T. J. T1 - Editorial: Special edition on probabilistic numerics JF - Statistics and Computing Y1 - 2019 U6 - https://doi.org/doi:10.1007/s11222-019-09892-y VL - 29 IS - 6 SP - 1181 EP - 1183 ER - TY - JOUR A1 - Klebanov, Ilja A1 - Schuster, Ingmar A1 - Sullivan, T. J. T1 - A rigorous theory of conditional mean embeddings JF - SIAM Journal on Mathematics of Data Science Y1 - 2020 U6 - https://doi.org/10.1137/19M1305069 VL - 2 IS - 3 SP - 583 EP - 606 ER - TY - JOUR A1 - Nava-Yazdani, Esfandiar A1 - Hege, Hans-Christian A1 - Sullivan, T. J. A1 - von Tycowicz, Christoph T1 - Geodesic Analysis in Kendall's Shape Space with Epidemiological Applications JF - Journal of Mathematical Imaging and Vision N2 - We analytically determine Jacobi fields and parallel transports and compute geodesic regression in Kendall’s shape space. Using the derived expressions, we can fully leverage the geometry via Riemannian optimization and thereby reduce the computational expense by several orders of magnitude over common, nonlinear constrained approaches. The methodology is demonstrated by performing a longitudinal statistical analysis of epidemiological shape data. As an example application we have chosen 3D shapes of knee bones, reconstructed from image data of the Osteoarthritis Initiative (OAI). Comparing subject groups with incident and developing osteoarthritis versus normal controls, we find clear differences in the temporal development of femur shapes. This paves the way for early prediction of incident knee osteoarthritis, using geometry data alone. Y1 - 2020 U6 - https://doi.org/10.1007/s10851-020-00945-w VL - 62 IS - 4 SP - 549 EP - 559 ER - TY - JOUR A1 - Klebanov, Ilja A1 - Sprungk, Björn A1 - Sullivan, T. J. T1 - The linear conditional expectation in Hilbert space JF - Bernoulli Y1 - 2021 U6 - https://doi.org/10.3150/20-BEJ1308 VL - 27 IS - 4 SP - 2299 EP - 2299 ER - TY - JOUR A1 - Bonnet, Luc A1 - Akian, Jean-Luc A1 - Savin, Éric A1 - Sullivan, T. J. T1 - Adaptive reconstruction of imperfectly-observed monotone functions, with applications to uncertainty quantification JF - Algorithms N2 - Motivated by the desire to numerically calculate rigorous upper and lower bounds on deviation probabilities over large classes of probability distributions, we present an adaptive algorithm for the reconstruction of increasing real-valued functions. While this problem is similar to the classical statistical problem of isotonic regression, the optimisation setting alters several characteristics of the problem and opens natural algorithmic possibilities. We present our algorithm, establish sufficient conditions for convergence of the reconstruction to the ground truth, and apply the method to synthetic test cases and a real-world example of uncertainty quantification for aerodynamic design. Y1 - 2020 U6 - https://doi.org/10.3390/a13080196 VL - 13 IS - 8 SP - 196 ER - TY - JOUR A1 - Sullivan, T. J. T1 - Contributed discussion on the article "A Bayesian conjugate gradient method" JF - Bayesian Analysis N2 - The recent article "A Bayesian conjugate gradient method" by Cockayne, Oates, Ipsen, and Girolami proposes an approximately Bayesian iterative procedure for the solution of a system of linear equations, based on the conjugate gradient method, that gives a sequence of Gaussian/normal estimates for the exact solution. The purpose of the probabilistic enrichment is that the covariance structure is intended to provide a posterior measure of uncertainty or confidence in the solution mean. This note gives some comments on the article, poses some questions, and suggests directions for further research. Y1 - 2019 U6 - https://doi.org/10.1214/19-BA1145 VL - 14 IS - 3 SP - 985 EP - 989 ER - TY - CHAP A1 - Teymur, Onur A1 - Lie, Han Cheng A1 - Sullivan, T. J. A1 - Calderhead, Ben T1 - Implicit probabilistic integrators for ODEs T2 - Advances in Neural Information Processing Systems 31 (NIPS 2018) Y1 - 2018 UR - http://papers.nips.cc/paper/7955-implicit-probabilistic-integrators-for-odes ER - TY - JOUR A1 - Lie, Han Cheng A1 - Sullivan, T. J. T1 - Erratum: Equivalence of weak and strong modes of measures on topological vector spaces (2018 Inverse Problems 34 115013) JF - Inverse Problems Y1 - 2018 U6 - https://doi.org/10.1088/1361-6420/aae55b VL - 34 IS - 12 SP - 129601 ER - TY - JOUR A1 - Oates, Chris A1 - Sullivan, T. J. T1 - A modern retrospective on probabilistic numerics JF - Statistics and Computing N2 - This article attempts to place the emergence of probabilistic numerics as a mathematical-statistical research field within its historical context and to explore how its gradual development can be related to modern formal treatments and applications. We highlight in particular the parallel contributions of Sul'din and Larkin in the 1960s and how their pioneering early ideas have reached a degree of maturity in the intervening period, mediated by paradigms such as average-case analysis and information-based complexity. We provide a subjective assessment of the state of research in probabilistic numerics and highlight some difficulties to be addressed by future works. Y1 - 2019 U6 - https://doi.org/10.1007/s11222-019-09902-z VL - 29 IS - 6 SP - 1335 EP - 1351 ER - TY - GEN A1 - Oates, Chris A1 - Cockayne, Jon A1 - Prangle, Dennis A1 - Sullivan, T. J. A1 - Girolami, Mark ED - Hickernell, F. J. ED - Kritzer, P. T1 - Optimality criteria for probabilistic numerical methods T2 - Multivariate Algorithms and Information-Based Complexity N2 - It is well understood that Bayesian decision theory and average case analysis are essentially identical. However, if one is interested in performing uncertainty quantification for a numerical task, it can be argued that the decision-theoretic framework is neither appropriate nor sufficient. To this end, we consider an alternative optimality criterion from Bayesian experimental design and study its implied optimal information in the numerical context. This information is demonstrated to differ, in general, from the information that would be used in an average-case-optimal numerical method. The explicit connection to Bayesian experimental design suggests several distinct regimes in which optimal probabilistic numerical methods can be developed. Y1 - 2020 U6 - https://doi.org/10.1515/9783110635461-005 VL - 27 SP - 65 EP - 88 PB - De Gruyter ER - TY - JOUR A1 - Kersting, Hans A1 - Sullivan, T. J. A1 - Hennig, Philipp T1 - Convergence rates of Gaussian ODE filters JF - Statistics and Computing Y1 - 2020 U6 - https://doi.org/10.1007/s11222-020-09972-4 VL - 30 SP - 1791 EP - 1816 PB - Springer CY - Statistics and Computing ER - TY - JOUR A1 - Lie, Han Cheng A1 - Sullivan, T. J. A1 - Teckentrup, Aretha T1 - Error bounds for some approximate posterior measures in Bayesian inference JF - Numerical Mathematics and Advanced Applications ENUMATH 2019 Y1 - 2020 U6 - https://doi.org/10.1007/978-3-030-55874-1_26 SP - 275 EP - 283 PB - Springer ER - TY - CHAP A1 - Sullivan, T. J. T1 - Well-posedness of Bayesian inverse problems in quasi-Banach spaces with stable priors T2 - 88th Annual Meeting of the International Association of Applied Mathematics and Mechanics (GAMM), Weimar 2017 Y1 - 2017 U6 - https://doi.org/10.1002/pamm.201710402 VL - 17 IS - 1 SP - 871 EP - 874 ER - TY - JOUR A1 - Sullivan, T. J. T1 - Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors JF - Inverse Problems and Imaging Y1 - 2017 U6 - https://doi.org/10.3934/ipi.2017040 VL - 11 IS - 5 SP - 857 EP - 874 ER - TY - JOUR A1 - Ayanbayev, Birzhan A1 - Klebanov, Ilja A1 - Lie, Han Cheng A1 - Sullivan, T. J. T1 - Γ-convergence of Onsager–Machlup functionals: I. With applications to maximum a posteriori estimation in Bayesian inverse problems JF - Inverse Problems Y1 - 2022 U6 - https://doi.org/10.1088/1361-6420/ac3f81 VL - 38 IS - 2 ER - TY - JOUR A1 - Ayanbayev, Birzhan A1 - Klebanov, Ilja A1 - Lie, Han Cheng A1 - Sullivan, T. J. T1 - Γ-convergence of Onsager–Machlup functionals: II. Infinite product measures on Banach spaces JF - Inverse Problems Y1 - 2022 U6 - https://doi.org/10.1088/1361-6420/ac3f82 VL - 38 IS - 2 ER - TY - JOUR A1 - Wang, Junyang A1 - Cockayne, Jon A1 - Chkrebtii, Oksana A1 - Sullivan, T. J. A1 - Oates, Chris T1 - Bayesian numerical methods for nonlinear partial differential equations JF - Statistics and Computing Y1 - 2021 U6 - https://doi.org/10.1007/s11222-021-10030-w VL - 31 IS - 5 ER - TY - GEN A1 - Nava-Yazdani, Esfandiar A1 - Hege, Hans-Christian A1 - von Tycowicz, Christoph A1 - Sullivan, T. J. T1 - A Shape Trajectories Approach to Longitudinal Statistical Analysis N2 - For Kendall’s shape space we determine analytically Jacobi fields and parallel transport, and compute geodesic regression. Using the derived expressions, we can fully leverage the geometry via Riemannian optimization and reduce the computational expense by several orders of magnitude. The methodology is demonstrated by performing a longitudinal statistical analysis of epidemiological shape data. As application example we have chosen 3D shapes of knee bones, reconstructed from image data of the Osteoarthritis Initiative. Comparing subject groups with incident and developing osteoarthritis versus normal controls, we find clear differences in the temporal development of femur shapes. This paves the way for early prediction of incident knee osteoarthritis, using geometry data only. T3 - ZIB-Report - 18-42 KW - shape space, shape trajectories, geodesic regression, longitudinal analysis, osteoarthritis Y1 - 2018 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-69759 SN - 1438-0064 ER -