TY - GEN A1 - Schütte, Christof A1 - Klus, Stefan A1 - Hartmann, Carsten T1 - Overcoming the Timescale Barrier in Molecular Dynamics: Transfer Operators, Variational Principles, and Machine Learning N2 - One of the main challenges in molecular dynamics is overcoming the “timescale barrier”, a phrase used to describe that in many realistic molecular systems, biologically important rare transitions occur on timescales that are not accessible to direct numerical simulation, not even on the largest or specifically dedicated supercomputers. This article discusses how to circumvent the timescale barrier by a collection of transfer operator-based techniques that have emerged from dynamical systems theory, numerical mathematics, and machine learning over the last two decades. We will focus on how transfer operators can be used to approximate the dynamical behavior on long timescales, review the introduction of this approach into molecular dynamics, and outline the respective theory as well as the algorithmic development from the early numerics-based methods, via variational reformulations, to modern data-based techniques utilizing and improving concepts from machine learning. Furthermore, its relation to rare event simulation techniques will be explained, revealing a broad equivalence of variational principles for long-time quantities in MD. The article will mainly take a mathematical perspective and will leave the application to real-world molecular systems to the more than 1000 research articles already written on this subject. T3 - ZIB-Report - 22-25 Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-88637 SN - 1438-0064 ER - TY - JOUR A1 - Quer, Jannes A1 - Ribera Borrell, Enric T1 - Connecting Stochastic Optimal Control and Reinforcement Learning JF - Journal of Mathematical Physics N2 - In this paper the connection between stochastic optimal control and reinforcement learning is investigated. Our main motivation is to apply importance sampling to sampling rare events which can be reformulated as an optimal control problem. By using a parameterised approach the optimal control problem becomes a stochastic optimization problem which still raises some open questions regarding how to tackle the scalability to high-dimensional problems and how to deal with the intrinsic metastability of the system. To explore new methods we link the optimal control problem to reinforcement learning since both share the same underlying framework, namely a Markov Decision Process (MDP). For the optimal control problem we show how the MDP can be formulated. In addition we discuss how the stochastic optimal control problem can be interpreted in the framework of reinforcement learning. At the end of the article we present the application of two different reinforcement learning algorithms to the optimal control problem and a comparison of the advantages and disadvantages of the two algorithms. Y1 - 2024 U6 - https://doi.org/10.1063/5.0140665 VL - 65 IS - 8 ER - TY - JOUR A1 - Frank, Anna-Simone A1 - Sikorski, Alexander A1 - Röblitz, Susanna T1 - Spectral clustering of Markov chain transition matrices with complex eigenvalues JF - Journal of Computational and Applied Mathematics N2 - The Robust Perron Cluster Analysis (PCCA+) has become a popular spectral clustering algorithm for coarse-graining transition matrices of nearly decomposable Markov chains with transition states. Originally developed for reversible Markov chains, the algorithm only worked for transition matrices with real eigenvalues. In this paper, we therefore extend the theoretical framework of PCCA+ to Markov chains with a complex eigen-decomposition. We show that by replacing a complex conjugate pair of eigenvectors by their real and imaginary components, a real representation of the same subspace is obtained, which is suitable for the cluster analysis. We show that our approach leads to the same results as the generalized PCCA+ (GPCCA), which replaces the complex eigen-decomposition by a conceptually more difficult real Schur decomposition. We apply the method on non-reversible Markov chains, including circular chains, and demonstrate its efficiency compared to GPCCA. The experiments are performed in the Matlab programming language and codes are provided. Y1 - 2024 U6 - https://doi.org/10.1016/j.cam.2024.115791 VL - 444 SP - 115791 ER - TY - JOUR A1 - Lelièvre, Tony A1 - Zhang, Wei T1 - Pathwise estimates for effective dynamics: the case of nonlinear vectorial reaction coordinates JF - Multiscale Modeling and Simulation N2 - Effective dynamics using conditional expectation was proposed in [F. Legoll and T. Lelièvre, Nonlinearity, 2010] to approximate the essential dynamics of high-dimensional diffusion processes along a given reaction coordinate. The approximation error of the effective dynamics when it is used to approximate the behavior of the original dynamics has been considered in recent years. As a continuation of the previous work [F. Legoll, T. Lelièvre, and S. Olla, Stoch. Process. Appl, 2017], in this paper we obtain pathwise estimates for effective dynamics when the reaction coordinate function is either nonlinear or vector-valued. Y1 - 2018 U6 - https://doi.org/10.1137/18M1186034 IS - 17 SP - 1019 EP - 1051 ER - TY - JOUR A1 - Hartmann, Carsten A1 - Schütte, Christof A1 - Zhang, Wei T1 - Jarzynski's equality, fluctuation theorems, and variance reduction: Mathematical analysis and numerical algorithms JF - Journal of Statistical Physics N2 - In this paper, we study Jarzynski's equality and fluctuation theorems for diffusion processes. While some of the results considered in the current work are known in the (mainly physics) literature, we review and generalize these nonequilibrium theorems using mathematical arguments, therefore enabling further investigations in the mathematical community. On the numerical side, variance reduction approaches such as importance sampling method are studied in order to compute free energy differences based on Jarzynski's equality. Y1 - 2018 U6 - https://doi.org/10.1007/s10955-019-02286-4 VL - 175 IS - 6 SP - 1214 EP - 1261 ER - TY - JOUR A1 - Akhyar, Fatima-Zahrae A1 - Zhang, Wei A1 - Stoltz, Gabriel A1 - Schütte, Christof T1 - Generative modeling of conditional probability distributions on the level-sets of collective variables N2 - Given a probability distribution $\mu$ in $\mathbb{R}^d$ represented by data, we study in this paper the generative modeling of its conditional probability distributions on the level-sets of a collective variable $\xi: \mathbb{R}^d \rightarrow \mathbb{R}^k$, where $1 \le k