TY - GEN A1 - Sagnol, Guillaume T1 - Network-related problems in Optimal Experimental Design and Second Order Cone Programming N2 - In the past few years several applications of optimal experimental designs have emerged to optimize the measurements in communication networks. The optimal design problems arising from this kind of applications share three interesting properties: (i) measurements are only available at a small number of locations of the network; (ii) each monitor can simultaneously measure several quantities, which can be modeled by ``multiresponse experiments"; (iii) the observation matrices depend on the topology of the network. In this paper, we give an overview of these experimental design problems and recall recent results for the computation of optimal designs by Second Order Cone Programming (SOCP). New results for the network-monitoring of a discrete time process are presented. In particular, we show that the optimal design problem for the monitoring of an AR1 process can be reduced to the standard form and we give experimental results. T3 - ZIB-Report - 11-52 KW - Optimal Experimental Design KW - Second Order Cone Programming KW - Network measurements Y1 - 2012 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-14942 SN - 1438-0064 VL - 51 IS - 51 SP - 161 EP - 171 ER - TY - GEN A1 - Klebanov, Ilja A1 - Sikorski, Alexander A1 - Schütte, Christof A1 - Röblitz, Susanna T1 - Prior estimation and Bayesian inference from large cohort data sets N2 - One of the main goals of mathematical modelling in systems biology related to medical applications is to obtain patient-specific parameterisations and model predictions. In clinical practice, however, the number of available measurements for single patients is usually limited due to time and cost restrictions. This hampers the process of making patient-specific predictions about the outcome of a treatment. On the other hand, data are often available for many patients, in particular if extensive clinical studies have been performed. Using these population data, we propose an iterative algorithm for contructing an informative prior distribution, which then serves as the basis for computing patient-specific posteriors and obtaining individual predictions. We demonsrate the performance of our method by applying it to a low-dimensional parameter estimation problem in a toy model as well as to a high-dimensional ODE model of the human menstrual cycle, which represents a typical example from systems biology modelling. T3 - ZIB-Report - 16-09 Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-57475 SN - 1438-0064 ER - TY - GEN A1 - Klebanov, Ilja A1 - Sikorski, Alexander A1 - Schütte, Christof A1 - Röblitz, Susanna T1 - Empirical Bayes Methods for Prior Estimation in Systems Medicine N2 - One of the main goals of mathematical modelling in systems medicine related to medical applications is to obtain patient-specific parameterizations and model predictions. In clinical practice, however, the number of available measurements for single patients is usually limited due to time and cost restrictions. This hampers the process of making patient-specific predictions about the outcome of a treatment. On the other hand, data are often available for many patients, in particular if extensive clinical studies have been performed. Therefore, before applying Bayes’ rule separately to the data of each patient (which is typically performed using a non-informative prior), it is meaningful to use empirical Bayes methods in order to construct an informative prior from all available data. We compare the performance of four priors - a non-informative prior and priors chosen by nonparametric maximum likelihood estimation (NPMLE), by maximum penalized lilelihood estimation (MPLE) and by doubly-smoothed maximum likelihood estimation (DS-MLE) - by applying them to a low-dimensional parameter estimation problem in a toy model as well as to a high-dimensional ODE model of the human menstrual cycle, which represents a typical example from systems biology modelling. T3 - ZIB-Report - 16-57 KW - Parameter estimation KW - Bayesian inference KW - Bayesian hierarchical modelling KW - NPMLE KW - MPLE KW - DS-MLE KW - EM algorithm KW - Jeffreys prior KW - reference prior KW - hyperparameter KW - hyperprior KW - principle of maximum entropy Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-61307 SN - 1438-0064 ER - TY - GEN A1 - Lie, Han Cheng A1 - Sullivan, T. J. A1 - Teckentrup, Aretha T1 - Random forward models and log-likelihoods in Bayesian inverse problems T2 - SIAM/ASA Journal on Uncertainty Quantification N2 - We consider the use of randomised forward models and log-likelihoods within the Bayesian approach to inverse problems. Such random approximations to the exact forward model or log-likelihood arise naturally when a computationally expensive model is approximated using a cheaper stochastic surrogate, as in Gaussian process emulation (kriging), or in the field of probabilistic numerical methods. We show that the Hellinger distance between the exact and approximate Bayesian posteriors is bounded by moments of the difference between the true and approximate log-likelihoods. Example applications of these stability results are given for randomised misfit models in large data applications and the probabilistic solution of ordinary differential equations. T3 - ZIB-Report - 18-03 KW - Bayesian inverse problem KW - random likelihood KW - surrogate model KW - posterior consistency KW - probabilistic numerics KW - uncertainty quantification KW - randomised misfit Y1 - 2018 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-66324 SN - 1438-0064 VL - 6 IS - 4 SP - 1600 EP - 1629 ER - TY - GEN A1 - Sagnol, Guillaume A1 - Harman, Radoslav T1 - Computing exact D-optimal designs by mixed integer second order cone programming N2 - Let the design of an experiment be represented by an $s$-dimensional vector $\vec{w}$ of weights with non-negative components. Let the quality of $\vec{w}$ for the estimation of the parameters of the statistical model be measured by the criterion of $D$-optimality defined as the $m$-th root of the determinant of the information matrix $M(\vec{w})=\sum_{i=1}^s w_iA_iA_i^T$, where $A_i$, $i=1,...,s$, are known matrices with $m$ rows. In the paper, we show that the criterion of $D$-optimality is second-order cone representable. As a result, the method of second order cone programming can be used to compute an approximate $D$-optimal design with any system of linear constraints on the vector of weights. More importantly, the proposed characterization allows us to compute an \emph{exact} $D$-optimal design, which is possible thanks to high-quality branch-and-cut solvers specialized to solve mixed integer second order cone problems. We prove that some other widely used criteria are also second order cone representable, for instance the criteria of $A$-, and $G$-optimality, as well as the criteria of $D_K$- and $A_K$-optimality, which are extensions of $D$-, and $A$-optimality used in the case when only a specific system of linear combinations of parameters is of interest. We present several numerical examples demonstrating the efficiency and universality of the proposed method. We show that in many cases the mixed integer second order cone programming approach allows us to find a provably optimal exact design, while the standard heuristics systematically miss the optimum. T3 - ZIB-Report - 13-39 KW - Optimal Design of Experiments KW - D-optimality KW - SOCP Representability KW - Exact Designs Y1 - 2013 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-41932 SN - 1438-0064 ER - TY - GEN A1 - Sullivan, T. J. T1 - Well-posed Bayesian inverse problems and heavy-tailed stable Banach space priors N2 - This article extends the framework of Bayesian inverse problems in infinite-dimensional parameter spaces, as advocated by Stuart (Acta Numer. 19:451–559, 2010) and others, to the case of a heavy-tailed prior measure in the family of stable distributions, such as an infinite-dimensional Cauchy distribution, for which polynomial moments are infinite or undefined. It is shown that analogues of the Karhunen–Loève expansion for square-integrable random variables can be used to sample such measures. Furthermore, under weaker regularity assumptions than those used to date, the Bayesian posterior measure is shown to depend Lipschitz continuously in the Hellinger metric upon perturbations of the misfit function and observed data. T3 - ZIB-Report - 16-30 KW - Bayesian inverse problems KW - heavy-tailed distribution KW - Karhunen–Loève expansion KW - stable distribution KW - uncertainty quantification KW - well-posedness Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-59422 SN - 1438-0064 ER - TY - GEN A1 - Harman, Radoslav A1 - Sagnol, Guillaume T1 - Computing D-optimal experimental designs for estimating treatment contrasts under the presence of a nuisance time trend N2 - We prove a mathematical programming characterisation of approximate partial D-optimality under general linear constraints. We use this characterisation with a branch-and-bound method to compute a list of all exact D-optimal designs for estimating a pair of treatment contrasts in the presence of a nuisance time trend up to the size of 24 consecutive trials. T3 - ZIB-Report - 15-02 KW - optimal design of experiments KW - nuisance trend KW - branch and bound Y1 - 2015 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-53640 SN - 1438-0064 ER - TY - GEN A1 - Nava-Yazdani, Esfandiar A1 - Hege, Hans-Christian A1 - von Tycowicz, Christoph A1 - Sullivan, T. J. T1 - A Shape Trajectories Approach to Longitudinal Statistical Analysis N2 - For Kendall’s shape space we determine analytically Jacobi fields and parallel transport, and compute geodesic regression. Using the derived expressions, we can fully leverage the geometry via Riemannian optimization and reduce the computational expense by several orders of magnitude. The methodology is demonstrated by performing a longitudinal statistical analysis of epidemiological shape data. As application example we have chosen 3D shapes of knee bones, reconstructed from image data of the Osteoarthritis Initiative. Comparing subject groups with incident and developing osteoarthritis versus normal controls, we find clear differences in the temporal development of femur shapes. This paves the way for early prediction of incident knee osteoarthritis, using geometry data only. T3 - ZIB-Report - 18-42 KW - shape space, shape trajectories, geodesic regression, longitudinal analysis, osteoarthritis Y1 - 2018 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-69759 SN - 1438-0064 ER - TY - GEN A1 - Sagnol, Guillaume T1 - A Class of Semidefinite Programs with rank-one solutions N2 - We show that a class of semidefinite programs (SDP) admits a solution that is a positive semidefinite matrix of rank at most $r$, where $r$ is the rank of the matrix involved in the objective function of the SDP. The optimization problems of this class are semidefinite packing problems, which are the SDP analogs to vector packing problems. Of particular interest is the case in which our result guarantees the existence of a solution of rank one: we show that the computation of this solution actually reduces to a Second Order Cone Program (SOCP). We point out an application in statistics, in the optimal design of experiments. T3 - ZIB-Report - 11-51 KW - SDP KW - Semidefinite Packing Problem KW - rank 1-solution KW - Low-rank solutions KW - SOCP KW - Optimal Experimental Design KW - Multiresponse experiments Y1 - 2012 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-14933 SN - 1438-0064 ER - TY - GEN A1 - Klebanov, Ilja A1 - Sikorski, Alexander A1 - Schütte, Christof A1 - Röblitz, Susanna T1 - Empirical Bayes Methods, Reference Priors, Cross Entropy and the EM Algorithm N2 - When estimating a probability density within the empirical Bayes framework, the non-parametric maximum likelihood estimate (NPMLE) usually tends to overfit the data. This issue is usually taken care of by regularization - a penalization term is subtracted from the marginal log-likelihood before the maximization step, so that the estimate favors smooth solutions, resulting in the so-called maximum penalized likelihood estimation (MPLE). The majority of penalizations currently in use are rather arbitrary brute-force solutions, which lack invariance under transformation of the parameters(reparametrization) and measurements. This contradicts the principle that, if the underlying model has several equivalent formulations, the methods of inductive inference should lead to consistent results. Motivated by this principle and using an information-theoretic point of view, we suggest an entropy-based penalization term that guarantees this kind of invariance. The resulting density estimate can be seen as a generalization of reference priors. Using the reference prior as a hyperprior, on the other hand, is argued to be a poor choice for regularization. We also present an insightful connection between the NPMLE, the cross entropy and the principle of minimum discrimination information suggesting another method of inference that contains the doubly-smoothed maximum likelihood estimation as a special case. T3 - ZIB-Report - 16-56 KW - parameter estimation KW - Bayesian inference KW - Bayesian hierarchical modeling KW - hyperparameter KW - hyperprior KW - EM algorithm KW - NPMLE KW - MPLE KW - DS-MLE KW - principle of maximum entropy KW - cross entropy KW - minimum discrimination information KW - reference prior KW - Jeffreys prior Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-61230 SN - 1438-0064 ER - TY - GEN A1 - Witzig, Jakob A1 - Beckenbach, Isabel A1 - Eifler, Leon A1 - Fackeldey, Konstantin A1 - Gleixner, Ambros A1 - Grever, Andreas A1 - Weber, Marcus T1 - Mixed-Integer Programming for Cycle Detection in Non-reversible Markov Processes N2 - In this paper, we present a new, optimization-based method to exhibit cyclic behavior in non-reversible stochastic processes. While our method is general, it is strongly motivated by discrete simulations of ordinary differential equations representing non-reversible biological processes, in particular molecular simulations. Here, the discrete time steps of the simulation are often very small compared to the time scale of interest, i.e., of the whole process. In this setting, the detection of a global cyclic behavior of the process becomes difficult because transitions between individual states may appear almost reversible on the small time scale of the simulation. We address this difficulty using a mixed-integer programming model that allows us to compute a cycle of clusters with maximum net flow, i.e., large forward and small backward probability. For a synthetic genetic regulatory network consisting of a ring-oscillator with three genes, we show that this approach can detect the most productive overall cycle, outperforming classical spectral analysis methods. Our method applies to general non-equilibrium steady state systems such as catalytic reactions, for which the objective value computes the effectiveness of the catalyst. T3 - ZIB-Report - 16-39 KW - Non-reversible Markov Processes KW - NESS KW - Mixed-Integer Programming KW - Markov State Models Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-60353 SN - 1438-0064 ER - TY - THES A1 - Hanik, Martin T1 - Geometric Data Analysis: Advancements of the Statistical Methodology and Applications N2 - Data analysis has become fundamental to our society and comes in multiple facets and approaches. Nevertheless, in research and applications, the focus was primarily on data from Euclidean vector spaces. Consequently, the majority of methods that are applied today are not suited for more general data types. Driven by needs from fields like image processing, (medical) shape analysis, and network analysis, more and more attention has recently been given to data from non-Euclidean spaces---particularly (curved) manifolds. It has led to the field of geometric data analysis whose methods explicitly take the structure (for example, the topology and geometry) of the underlying space into account. This thesis contributes to the methodology of geometric data analysis by generalizing several fundamental notions from multivariate statistics to manifolds. We thereby focus on two different viewpoints. First, we use Riemannian structures to derive a novel regression scheme for general manifolds that relies on splines of generalized Bézier curves. It can accurately model non-geodesic relationships, for example, time-dependent trends with saturation effects or cyclic trends. Since Bézier curves can be evaluated with the constructive de Casteljau algorithm, working with data from manifolds of high dimensions (for example, a hundred thousand or more) is feasible. Relying on the regression, we further develop a hierarchical statistical model for an adequate analysis of longitudinal data in manifolds, and a method to control for confounding variables. We secondly focus on data that is not only manifold- but even Lie group-valued, which is frequently the case in applications. We can only achieve this by endowing the group with an affine connection structure that is generally not Riemannian. Utilizing it, we derive generalizations of several well-known dissimilarity measures between data distributions that can be used for various tasks, including hypothesis testing. Invariance under data translations is proven, and a connection to continuous distributions is given for one measure. A further central contribution of this thesis is that it shows use cases for all notions in real-world applications, particularly in problems from shape analysis in medical imaging and archaeology. We can replicate or further quantify several known findings for shape changes of the femur and the right hippocampus under osteoarthritis and Alzheimer's, respectively. Furthermore, in an archaeological application, we obtain new insights into the construction principles of ancient sundials. Last but not least, we use the geometric structure underlying human brain connectomes to predict cognitive scores. Utilizing a sample selection procedure, we obtain state-of-the-art results. KW - Regression in Riemannian manifolds KW - Bi-invariant statistics KW - Bézier splines KW - Shape analysis KW - Brain connectomes KW - Ancient sundials Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:188-refubium-40087-8 PB - Refubium ER -