TY - JOUR A1 - Sun, Jingtong A1 - Berner, Julius A1 - Richter, Lorenz A1 - Zeinhofer, Marius A1 - Müller, Johannes A1 - Azizzadenesheli, Kamyar A1 - Anandkumar, Anima T1 - Dynamical Measure Transport and Neural PDE Solvers for Sampling N2 - The task of sampling from a probability density can be approached as transporting a tractable density function to the target, known as dynamical measure transport. In this work, we tackle it through a principled unified framework using deterministic or stochastic evolutions described by partial differential equations (PDEs). This framework incorporates prior trajectory-based sampling methods, such as diffusion models or Schrödinger bridges, without relying on the concept of time-reversals. Moreover, it allows us to propose novel numerical methods for solving the transport task and thus sampling from complicated targets without the need for the normalization constant or data samples. We employ physics-informed neural networks (PINNs) to approximate the respective PDE solutions, implying both conceptional and computational advantages. In particular, PINNs allow for simulation- and discretization-free optimization and can be trained very efficiently, leading to significantly better mode coverage in the sampling task compared to alternative methods. Moreover, they can readily be fine-tuned with Gauss-Newton methods to achieve high accuracy in sampling. Y1 - 2024 ER - TY - CHAP A1 - Richter, Lorenz A1 - Berner, Julius T1 - Improved sampling via learned diffusions T2 - International Conference on Learning Representations 2024 N2 - Recently, a series of papers proposed deep learning-based approaches to sample from unnormalized target densities using controlled diffusion processes. In this work, we identify these approaches as special cases of the Schrödinger bridge problem, seeking the most likely stochastic evolution between a given prior distribution and the specified target. We further generalize this framework by introducing a variational formulation based on divergences between path space measures of time-reversed diffusion processes. This abstract perspective leads to practical losses that can be optimized by gradient-based algorithms and includes previous objectives as special cases. At the same time, it allows us to consider divergences other than the reverse Kullback-Leibler divergence that is known to suffer from mode collapse. In particular, we propose the so-called log-variance loss, which exhibits favorable numerical properties and leads to significantly improved performance across all considered approaches. Y1 - 2024 UR - https://openreview.net/pdf?id=h4pNROsO06 ER - TY - JOUR A1 - Blessing, Denis A1 - Berner, Julius A1 - Richter, Lorenz A1 - Domingo-Enrich, Carles A1 - Du, Yuanqi A1 - Vahdat, Arash A1 - Neumann, Gerhard T1 - Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference N2 - Solving stochastic optimal control problems with quadratic control costs can be viewed as approximating a target path space measure, e.g. via gradient-based optimization. In practice, however, this optimization is challenging in particular if the target measure differs substantially from the prior. In this work, we therefore approach the problem by iteratively solving constrained problems incorporating trust regions that aim for approaching the target measure gradually in a systematic way. It turns out that this trust region based strategy can be understood as a geometric annealing from the prior to the target measure, where, however, the incorporated trust regions lead to a principled and educated way of choosing the time steps in the annealing path. We demonstrate in multiple optimal control applications that our novel method can improve performance significantly, including tasks in diffusion-based sampling, transition path sampling, and fine-tuning of diffusion models. Y1 - 2025 ER - TY - CHAP A1 - Blessing, Denis A1 - Berner, Julius A1 - Richter, Lorenz A1 - Neumann, Gerhard T1 - Underdamped Diffusion Bridges with Applications to Sampling T2 - 13th International Conference on Learning Representations (ICLR 2025) N2 - We provide a general framework for learning diffusion bridges that transport prior to target distributions. It includes existing diffusion models for generative modeling, but also underdamped versions with degenerate diffusion matrices, where the noise only acts in certain dimensions. Extending previous findings, our framework allows to rigorously show that score matching in the underdamped case is indeed equivalent to maximizing a lower bound on the likelihood. Motivated by superior convergence properties and compatibility with sophisticated numerical integration schemes of underdamped stochastic processes, we propose \emph{underdamped diffusion bridges}, where a general density evolution is learned rather than prescribed by a fixed noising process. We apply our method to the challenging task of sampling from unnormalized densities without access to samples from the target distribution. Across a diverse range of sampling problems, our approach demonstrates state-of-the-art performance, notably outperforming alternative methods, while requiring significantly fewer discretization steps and no hyperparameter tuning. Y1 - 2025 UR - https://openreview.net/attachment?id=Q1QTxFm0Is&name=pdf ER - TY - JOUR A1 - Berner, Julius A1 - Richter, Lorenz A1 - Sendera, Marcin A1 - Rector-Brooks, Jarrid A1 - Malkin, Nikolay T1 - From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster training N2 - We study the problem of training neural stochastic differential equations, or diffusion models, to sample from a Boltzmann distribution without access to target samples. Existing methods for training such models enforce time-reversal of the generative and noising processes, using either differentiable simulation or off-policy reinforcement learning (RL). We prove equivalences between families of objectives in the limit of infinitesimal discretization steps, linking entropic RL methods (GFlowNets) with continuous-time objects (partial differential equations and path space measures). We further show that an appropriate choice of coarse time discretization during training allows greatly improved sample efficiency and the use of time-local objectives, achieving competitive performance on standard sampling benchmarks with reduced computational cost. Y1 - 2025 ER - TY - CHAP A1 - Chen, Junhua A1 - Richter, Lorenz A1 - Berner, Julius A1 - Blessing, Denis A1 - Neumann, Gerhard A1 - Anandkumar, Anima T1 - Sequential Controlled Langevin Diffusions T2 - 13th International Conference on Learning Representations (ICLR 2025) N2 - An effective approach for sampling from unnormalized densities is based on the idea of gradually transporting samples from an easy prior to the complicated target distribution. Two popular methods are (1) Sequential Monte Carlo (SMC), where the transport is performed through successive annealed densities via prescribed Markov chains and resampling steps, and (2) recently developed diffusion-based sampling methods, where a learned dynamical transport is used. Despite the common goal, both approaches have different, often complementary, advantages and drawbacks. The resampling steps in SMC allow focusing on promising regions of the space, often leading to robust performance. While the algorithm enjoys asymptotic guarantees, the lack of flexible, learnable transitions can lead to slow convergence. On the other hand, diffusion-based samplers are learned and can potentially better adapt themselves to the target at hand, yet often suffer from training instabilities. In this work, we present a principled framework for combining SMC with diffusion-based samplers by viewing both methods in continuous time and considering measures on path space. This culminates in the new Sequential Controlled Langevin Diffusion (SCLD) sampling method, which is able to utilize the benefits of both methods and reaches improved performance on multiple benchmark problems, in many cases using only 10% of the training budget of previous diffusion-based samplers. Y1 - 2025 UR - https://openreview.net/pdf?id=dImD2sgy86 ER - TY - CHAP A1 - Berner, Julius A1 - Richter, Lorenz A1 - Ullrich, Karen T1 - An optimal control perspective on diffusion-based generative modeling T2 - Transactions on Machine Learning Research N2 - We establish a connection between stochastic optimal control and generative models based on stochastic differential equations (SDEs) such as recently developed diffusion probabilistic models. In particular, we derive a Hamilton-Jacobi-Bellman equation that governs the evolution of the log-densities of the underlying SDE marginals. This perspective allows to transfer methods from optimal control theory to generative modeling. First, we show that the evidence lower bound is a direct consequence of the well-known verification theorem from control theory. Further, we develop a novel diffusion-based method for sampling from unnormalized densities -- a problem frequently occurring in statistics and computational sciences. Y1 - 2024 UR - https://openreview.net/forum?id=oYIjw37pTP ER - TY - CHAP A1 - Richter, Lorenz A1 - Berner, Julius T1 - Robust SDE-Based Variational Formulations for Solving Linear PDEs via Deep Learning T2 - Proceedings of the 39th International Conference on Machine Learning, PMLR N2 - The combination of Monte Carlo methods and deep learning has recently led to efficient algorithms for solving partial differential equations (PDEs) in high dimensions. Related learning problems are often stated as variational formulations based on associated stochastic differential equations (SDEs), which allow the minimization of corresponding losses using gradient-based optimization methods. In respective numerical implementations it is therefore crucial to rely on adequate gradient estimators that exhibit low variance in order to reach convergence accurately and swiftly. In this article, we rigorously investigate corresponding numerical aspects that appear in the context of linear Kolmogorov PDEs. In particular, we systematically compare existing deep learning approaches and provide theoretical explanations for their performances. Subsequently, we suggest novel methods that can be shown to be more robust both theoretically and numerically, leading to substantial performance improvements. Y1 - 2022 VL - 162 SP - 18649 EP - 18666 ER -