7145
2019
eng
article
0
--
--
--
Optimality criteria for probabilistic numerical methods
It is well understood that Bayesian decision theory and average case analysis are essentially identical. However, if one is interested in performing uncertainty quantification for a numerical task, it can be argued that the decision-theoretic framework is neither appropriate nor sufficient. To this end, we consider an alternative optimality criterion from Bayesian experimental design and study its implied optimal information in the numerical context. This information is demonstrated to differ, in general, from the information that would be used in an average-case-optimal numerical method. The explicit connection to Bayesian experimental design suggests several distinct regimes in which optimal probabilistic numerical methods can be developed.
1901.04326
under review
Chris. J. Oates
Tim Sullivan
Jon Cockayne
Dennis Prangle
T. J. Sullivan
Mark Girolami
Numerical Mathematics
Sullivan, Tim
no-project
7144
2019
eng
article
0
--
--
--
A modern retrospective on probabilistic numerics
This article attempts to place the emergence of probabilistic numerics as a mathematical-statistical research field within its historical context and to explore how its gradual development can be related to modern formal treatments and applications. We highlight in particular the parallel contributions of Sul'din and Larkin in the 1960s and how their pioneering early ideas have reached a degree of maturity in the intervening period, mediated by paradigms such as average-case analysis and information-based complexity. We provide a subjective assessment of the state of research in probabilistic numerics and highlight some difficulties to be addressed by future works.
Statistics and Computing
1901.04457
To appear
2019
Yes
Chris. J. Oates
Tim Sullivan
T.J. Sullivan
Numerical Mathematics
Sullivan, Tim
no-project
7105
2018
eng
129601
12
34
article
0
--
--
--
Erratum: Equivalence of weak and strong modes of measures on topological vector spaces (2018 Inverse Problems 34 115013)
Inverse Problems
10.1088/1361-6420/aae55b
yes
2018
Han Cheng Lie
Tim Sullivan
T. J. Sullivan
Numerical Mathematics
Sullivan, Tim
no-project
Lie, Han
6023
eng
reportzib
0
--
2016-08-23
--
Cameron--Martin theorems for sequences of Cauchy-distributed random variables
Given a sequence of Cauchy-distributed random variables defined by a sequence of location parameters and a sequence of scale parameters, we consider another sequence of random variables that is obtained by perturbing the location or scale parameter sequences. Using a result of Kakutani on equivalence of infinite product measures, we provide sufficient conditions for the equivalence of laws of the two sequences.
1438-0064
urn:nbn:de:0297-zib-60230
Han Cheng Lie
Han Cheng Lie
T. J. Sullivan
ZIB-Report
16-40
PROBABILITY THEORY AND STOCHASTIC PROCESSES (For additional applications, see 11Kxx, 62-XX, 90-XX, 91-XX, 92-XX, 93-XX, 94-XX)
Numerical Mathematics
Sullivan, Tim
no-project
Lie, Han
https://opus4.kobv.de/opus4-zib/files/6023/ZIB_report.pdf
6014
2016
eng
1608.03784
article
0
--
--
--
Cameron--Martin theorems for sequences of Cauchy-distributed random variables
Given a sequence of Cauchy-distributed random variables defined by a sequence of location parameters and a sequence of scale parameters, we consider another sequence of random variables that is obtained by perturbing the location or scale parameter sequences. Using a result of Kakutani on equivalence of infinite product measures, we provide sufficient conditions for the equivalence of laws of the two sequences.
arXiv
1608.03784
no
Han Cheng Lie
Han Cheng Lie
T. J. Sullivan
Numerical Mathematics
Sullivan, Tim
no-project
Lie, Han
6659
2017
eng
article
0
--
--
--
Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity
1706.02205
no
under review
Florian Schäfer
Tim Sullivan
T. J. Sullivan
Houman Owhadi
Numerical Mathematics
Sullivan, Tim
no-project
6241
2018
2018
eng
1
6
6
8
23
article
0
2018-02-21
--
--
Quasi-invariance of countable products of Cauchy measures under non-unitary dilations
Electronic Communications in Probability
1611.10289
10.1214/18-ECP113
Yes
Han Cheng Lie
Regine Kossick
T. J. Sullivan
Numerical Mathematics
Sullivan, Tim
no-project
6810
2017
eng
857
874
5
11
article
0
--
--
--
Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors
Inverse Problems and Imaging
10.3934/ipi.2017040
1605.05898
yes
urn:nbn:de:0297-zib-59422
T. J. Sullivan
Tim Sullivan
Numerical Mathematics
Sullivan, Tim
no-project
5808
2015
eng
63
book
Springer
0
--
--
--
Introduction to Uncertainty Quantification
978-3-319-23394-9
10.1007/978-3-319-23395-6
Texts in Applied Mathematics
yes
T. J. Sullivan
Tim Sullivan
Numerical Mathematics
Sullivan, Tim
no-project
Uncertainty Quantification
5807
2015
eng
566
582
4
57
article
0
--
--
--
On the Brittleness of Bayesian Inference
With the advent of high-performance computing, Bayesian methods are becoming increasingly popular tools for the quantification of uncertainty throughout science and industry. Since these methods can impact the making of sometimes critical decisions in increasingly complicated contexts, the sensitivity of their posterior conclusions with respect to the underlying models and prior beliefs is a pressing question to which there currently exist positive and negative answers. We report new results suggesting that, although Bayesian methods are robust when the number of possible outcomes is finite or when only a finite number of marginals of the data-generating distribution are unknown, they could be generically brittle when applied to continuous systems (and their discretizations) with finite information on the data-generating distribution. If closeness is defined in terms of the total variation (TV) metric or the matching of a finite system of generalized moments, then (1) two practitioners who use arbitrarily close models and observe the same (possibly arbitrarily large amount of) data may reach opposite conclusions; and (2) any given prior and model can be slightly perturbed to achieve any desired posterior conclusion. The mechanism causing brittleness/robustness suggests that learning and robustness are antagonistic requirements, which raises the possibility of a missing stability condition when using Bayesian inference in a continuous world under finite information.
SIAM Review
10.1137/130938633
yes
Houman Owhadi
Tim Sullivan
Clint Scovel
T. J. Sullivan
Numerical Mathematics
Sullivan, Tim
no-project
Uncertainty Quantification
6661
2017
eng
article
0
--
--
--
Exact active subspace Metropolis–Hastings, with applications to the Lorenz-96 system
We consider the application of active subspaces to inform a Metropolis-Hastings algorithm, thereby aggressively reducing the computational dimension of the sampling problem. We show that the original formulation, as proposed by Constantine, Kent, and Bui-Thanh (SIAM J. Sci. Comput., 38(5):A2779-A2805, 2016), possesses asymptotic bias. Using pseudo-marginal arguments, we develop an asymptotically unbiased variant. Our algorithm is applied to a synthetic multimodal target distribution as well as a Bayesian formulation of a parameter inference problem for a Lorenz-96 system.
1712.02749
no
Ingmar Schuster
Tim Sullivan
Paul Constantine
T. J. Sullivan
Numerical Mathematics
Sullivan, Tim
Schuster, Ingmar
SFB1114 A06
6658
2017
eng
060001-1
060001-8
1853
conferenceobject
0
2017-06-30
--
--
Probabilistic Numerical Methods for PDE-constrained Bayesian Inverse Problems
This paper develops meshless methods for probabilistically describing discretisation error in the numerical solution of partial differential equations. This construction enables the solution of Bayesian inverse problems while accounting for the impact of the discretisation of the forward problem. In particular, this drives statistical inferences to be more conservative in the presence of significant solver error. Theoretical results are presented describing rates of convergence for the posteriors in both the forward and inverse problems. This method is tested on a challenging inverse problem with a nonlinear forward model
Proceedings of the 36th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
10.1063/1.4985359
1701.04006
AIP Conference Proceedings
yes
Jon Cockayne
Tim Sullivan
Chris Oates
T. J. Sullivan
Mark Girolami
Numerical Mathematics
Sullivan, Tim
no-project
5951
eng
reportzib
0
--
2016-05-25
--
Probabilistic Meshless Methods for Partial Differential Equations and Bayesian Inverse Problems
This paper develops a class of meshless methods that are well-suited to statistical inverse problems involving partial differential equations (PDEs). The methods discussed in this paper view the forcing term in the PDE as a random field that induces a probability distribution over the residual error of a symmetric collocation method. This construction enables the solution of challenging inverse problems while accounting, in a rigorous way, for the impact of the discretisation of the forward problem. In particular, this confers robustness to failure of meshless methods, with statistical inferences driven to be more conservative in the presence of significant solver error. In addition, (i) a principled learning-theoretic approach to minimise the impact of solver error is developed, and (ii) the challenging setting of inverse problems with a non-linear forward model is considered. The method is applied to parameter inference problems in which non-negligible solver error must be accounted for in order to draw valid statistical conclusions.
1438-0064
urn:nbn:de:0297-zib-59513
urn:nbn:de:0297-zib-59513
Tim Sullivan
Jon Cockayne
Chris Oates
T. J. Sullivan
Mark Girolami
ZIB-Report
16-31
eng
uncontrolled
Probabilistic Numerics
eng
uncontrolled
Partial Differential Equations
eng
uncontrolled
Inverse Problems
eng
uncontrolled
Meshless Methods
eng
uncontrolled
Gaussian Processes
eng
uncontrolled
Pseudo-Marginal MCMC
Numerical Mathematics
Sullivan, Tim
no-project
Uncertainty Quantification
https://opus4.kobv.de/opus4-zib/files/5951/pmm_pde_bip.pdf
5942
2016
2016
eng
reportzib
0
--
--
--
Well-posed Bayesian inverse problems and heavy-tailed stable Banach space priors
This article extends the framework of Bayesian inverse problems in infinite-dimensional parameter spaces, as advocated by Stuart (Acta Numer. 19:451–559, 2010) and others, to the case of a heavy-tailed prior measure in the family of stable distributions, such as an infinite-dimensional Cauchy distribution, for which polynomial moments are infinite or undefined. It is shown that analogues of the Karhunen–Loève expansion for square-integrable random variables can be used to sample such measures. Furthermore, under weaker regularity assumptions than those used to date, the Bayesian posterior measure is shown to depend Lipschitz continuously in the Hellinger metric upon perturbations of the misfit function and observed data.
urn:nbn:de:0297-zib-59422
10.3934/ipi.2017040
1438-0064
Appeared in: Inverse Problems and Imaging
no
T. J. Sullivan
Tim Sullivan
ZIB-Report
16-30
eng
uncontrolled
Bayesian inverse problems
eng
uncontrolled
heavy-tailed distribution
eng
uncontrolled
Karhunen–Loève expansion
eng
uncontrolled
stable distribution
eng
uncontrolled
uncertainty quantification
eng
uncontrolled
well-posedness
MEASURE AND INTEGRATION (For analysis on manifolds, see 58-XX)
PARTIAL DIFFERENTIAL EQUATIONS
PROBABILITY THEORY AND STOCHASTIC PROCESSES (For additional applications, see 11Kxx, 62-XX, 90-XX, 91-XX, 92-XX, 93-XX, 94-XX)
STATISTICS
NUMERICAL ANALYSIS
Numerical Mathematics
Sullivan, Tim
no-project
Uncertainty Quantification
https://opus4.kobv.de/opus4-zib/files/5942/stable_bip.pdf
6660
2018
eng
115013
11
34
article
0
--
--
--
Equivalence of weak and strong modes of measures on topological vector spaces
Modes of a probability measure on an infinite-dimensional Banach space X are often defined by maximising the small-radius limit of the ratio of measures of norm balls. Helin and Burger weakened the definition of such modes by considering only balls with centres in proper subspaces of X, and posed the question of when this restricted notion coincides with the unrestricted one. We generalise these definitions to modes of arbitrary measures on topological vector spaces, defined by arbitrary bounded, convex, neighbourhoods of the origin. We show that a coincident limiting ratios condition is a necessary and sufficient condition for the equivalence of these two types of modes, and show that the coincident limiting ratios condition is satisfied in a wide range of topological vector spaces.
Inverse Problems
1708.02516
10.1088/1361-6420/aadef2
yes
Han Cheng Lie
Tim Sullivan
T. J. Sullivan
Numerical Mathematics
Sullivan, Tim
no-project
Lie, Han
6809
2017
eng
871
874
1
17
conferenceobject
0
--
--
--
Well-posedness of Bayesian inverse problems in quasi-Banach spaces with stable priors
88th Annual Meeting of the International Association of Applied Mathematics and Mechanics (GAMM), Weimar 2017
10.1002/pamm.201710402
1710.05610
Proceedings in Applied Mathematics and Mechanics
yes
T. J. Sullivan
Tim Sullivan
Numerical Mathematics
Sullivan, Tim
no-project
6632
eng
1600
1629
4
6
article
0
--
2018-01-04
--
Random forward models and log-likelihoods in Bayesian inverse problems
We consider the use of randomised forward models and log-likelihoods within the Bayesian approach to inverse problems. Such random approximations to the exact forward model or log-likelihood arise naturally when a computationally expensive model is approximated using a cheaper stochastic surrogate, as in Gaussian process emulation (kriging), or in the field of probabilistic numerical methods. We show that the Hellinger distance between the exact and approximate Bayesian posteriors is bounded by moments of the difference between the true and approximate log-likelihoods. Example applications of these stability results are given for randomised misfit models in large data applications and the probabilistic solution of ordinary differential equations.
SIAM/ASA Journal on Uncertainty Quantification
1438-0064
urn:nbn:de:0297-zib-66324
1712.05717
10.1137/18M1166523
2018
yes
Han Cheng Lie
Tim Sullivan
T. J. Sullivan
Aretha Teckentrup
ZIB-Report
18-03
eng
uncontrolled
Bayesian inverse problem
eng
uncontrolled
random likelihood
eng
uncontrolled
surrogate model
eng
uncontrolled
posterior consistency
eng
uncontrolled
probabilistic numerics
eng
uncontrolled
uncertainty quantification
eng
uncontrolled
randomised misfit
STATISTICS
NUMERICAL ANALYSIS
COMPUTER SCIENCE (For papers involving machine computations and programs in a specific mathematical area, see Section -04 in that area)
Numerical Mathematics
Sullivan, Tim
SFB1114 A06
https://opus4.kobv.de/opus4-zib/files/6632/random-bip.pdf
6934
2018
eng
conferenceobject
0
--
--
--
Implicit probabilistic integrators for ODEs
Advances in Neural Information Processing Systems 31 (NIPS 2018)
1805.07970
http://papers.nips.cc/paper/7955-implicit-probabilistic-integrators-for-odes
2018
yes
Onur Teymur
Tim Sullivan
Han Cheng Lie
T. J. Sullivan
Ben Calderhead
Numerical Mathematics
Sullivan, Tim
no-project
7146
2018
eng
article
0
--
--
--
Convergence rates of Gaussian ODE filters
1807.09737
under review
no
Hans Kersting
Tim Sullivan
T. J. Sullivan
Philipp Hennig
Numerical Mathematics
Sullivan, Tim
no-project
6975
eng
reportzib
0
--
2018-08-01
--
A Shape Trajectories Approach to Longitudinal Statistical Analysis
For Kendall’s shape space we determine analytically Jacobi fields and parallel transport, and compute geodesic regression. Using the derived expressions, we can fully leverage the geometry via Riemannian optimization and reduce the computational expense by several orders of magnitude. The methodology is demonstrated by performing a longitudinal statistical analysis of epidemiological shape data.
As application example we have chosen 3D shapes of knee bones, reconstructed from image data of the Osteoarthritis Initiative. Comparing subject groups with incident and developing osteoarthritis versus normal controls, we find clear differences in the temporal development of femur shapes. This paves the way for early prediction of incident knee osteoarthritis, using geometry data only.
1438-0064
urn:nbn:de:0297-zib-69759
Esfandiar Nava-Yazdani
Hans-Christian Hege
Hans-Christian Hege
Christoph von Tycowicz
T. J. Sullivan
ZIB-Report
18-42
eng
uncontrolled
shape space, shape trajectories, geodesic regression, longitudinal analysis, osteoarthritis
PROBABILITY AND STATISTICS
DIFFERENTIAL GEOMETRY (For differential topology, see 57Rxx. For foundational questions of differentiable manifolds, see 58Axx)
STATISTICS
Visual Data Analysis
Visual Data Analysis in Science and Engineering
Therapy Planning
Hege, Hans-Christian
Tycowicz, Christoph von
Navayazdani, Esfandiar
ECMath-CH15
https://opus4.kobv.de/opus4-zib/files/6975/shape_trj.pdf
6657
2019
eng
article
0
--
--
--
Strong convergence rates of probabilistic integrators for ordinary differential equations
Probabilistic integration of a continuous dynamical system is a way of systematically introducing model error, at scales no larger than errors inroduced by standard numerical discretisation, in order to enable thorough exploration of possible responses of the system to inputs. It is thus a potentially useful approach in a number of applications such as forward uncertainty quantification, inverse problems, and data assimilation. We extend the convergence analysis of probabilistic integrators for deterministic ordinary differential equations, as proposed by Conrad et al.\ (\textit{Stat.\ Comput.}, 2016), to establish mean-square convergence in the uniform norm on discrete- or continuous-time solutions under relaxed regularity assumptions on the driving vector fields and their induced flows. Specifically, we show that randomised high-order integrators for globally Lipschitz flows and randomised Euler integrators for dissipative vector fields with polynomially-bounded local Lipschitz constants all have the same mean-square convergence rate as their deterministic counterparts, provided that the variance of the integration noise is not of higher order than the corresponding deterministic integrator.
Statistics and Computing
1703.03680
yes
To appear
2018
Han Cheng Lie
Tim Sullivan
T. J. Sullivan
Andrew Stuart
Numerical Mathematics
Sullivan, Tim
no-project
Lie, Han
6616
2019
eng
article
0
--
--
--
Bayesian Probabilistic Numerical Methods
Over forty years ago average-case error was proposed in the applied mathematics literature as an alternative criterion with which to assess numerical methods. In contrast to worst-case error, this criterion relies on the construction of a probability measure over candidate numerical tasks, and numerical methods are assessed based on their average performance over those tasks with respect to the measure. This paper goes further and establishes Bayesian probabilistic numerical methods as solutions to certain inverse problems based upon the numerical task within the Bayesian framework. This allows us to establish general conditions under which Bayesian probabilistic numerical methods are well defined, encompassing both the non-linear and non-Gaussian context. For general computation, a numerical approximation scheme is proposed and its asymptotic convergence established. The theoretical development is extended to pipelines of computation, wherein probabilistic numerical methods are composed to solve more challenging numerical tasks. The contribution highlights an important research frontier at the interface of numerical analysis and uncertainty quantification, and a challenging industrial application is presented.
SIAM Review
1702.03673
Yes
To appear
2019
Jon Cockayne
Tim Sullivan
Chris Oates
T. J. Sullivan
Mark Girolami
Numerical Mathematics
Sullivan, Tim
no-project