TY - GEN A1 - Quer, Jannes A1 - Lie, Han Cheng T1 - Some connections between importance sampling and enhanced sampling methods in molecular dynamics N2 - Enhanced sampling methods play an important role in molecular dynamics, because they enable the collection of better statistics of rare events that are important in many physical phenomena. We show that many enhanced sampling methods can be viewed as methods for performing importance sampling, by identifying important correspondences between the language of molecular dynamics and the language of probability theory. We illustrate these connections by highlighting the similarities between the rare event simulation method of Hartmann and Schütte (J. Stat. Mech. Theor. Exp., 2012), and the enhanced sampling method of Valsson and Parrinello (Phys. Rev. Lett. 113, 090601). We show that the idea of changing a probability measure is fundamental to both enhanced sampling and importance sampling. T3 - ZIB-Report - 17-30 Y1 - 2017 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-64289 SN - 1438-0064 ER - TY - THES A1 - Quer, Jannes T1 - Importance Sampling for metastable dynamical systems in molecular dynamics Y1 - 2018 ER - TY - GEN A1 - Quer, Jannes A1 - Donati, Luca A1 - Keller, Bettina A1 - Weber, Marcus T1 - An automatic adaptive importance sampling algorithm for molecular dynamics in reaction coordinates N2 - In this article we propose an adaptive importance sampling scheme for dynamical quantities of high dimensional complex systems which are metastable. The main idea of this article is to combine a method coming from Molecular Dynamics Simulation, Metadynamics, with a theorem from stochastic analysis, Girsanov's theorem. The proposed algorithm has two advantages compared to a standard estimator of dynamic quantities: firstly, it is possible to produce estimators with a lower variance and, secondly, we can speed up the sampling. One of the main problems for building importance sampling schemes for metastable systems is to find the metastable region in order to manipulate the potential accordingly. Our method circumvents this problem by using an assimilated version of the Metadynamics algorithm and thus creates a non-equilibrium dynamics which is used to sample the equilibrium quantities. T3 - ZIB-Report - 17-09 KW - Adaptive Importance Sampling KW - Molecular Dynamics KW - Metastability KW - Variance Reduction KW - Non Equilibrium Sampling KW - Metadynamics KW - Girsanov Y1 - 2017 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-62075 SN - 1438-0064 ER - TY - GEN A1 - Weber, Marcus A1 - Quer, Jannes T1 - Estimating exit rates in rare event dynamical systems via extrapolation N2 - In this article we present a new idea for approximating exit rates for diffusion processes living in a craggy landscape. We are especially interested in the exit rates of a process living in a metastable regions. Due to the fact that Monte Carlo simulations perform quite poor and are very computational expensive in this setting we create several similar situations with a smoothed potential. For this we introduce a new parameter $\lambda \in [0,1]$ ($\lambda = 1$ very smoothed potential, $\lambda=0$ original potential) into the potential which controls the influence the smoothing. We then sample the exit rate for different parameters $\lambda$ the exit rate from a given region. Due to the fact that $\lambda$ is connected to the exit rate we can use this dependency to approximate the real exit rate. The method can be seen as something between hyperdynamics and temperature accelerated MC. T3 - ZIB-Report - 15-54 KW - rare event sampling, smoothing, membership functions, perturbed potential Y1 - 2015 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-56622 SN - 1438-0064 ER - TY - JOUR A1 - Quer, Jannes A1 - Donati, Luca A1 - Keller, Bettina A1 - Weber, Marcus T1 - An automatic adaptive importance sampling algorithm for molecular dynamics in reaction coordinates JF - SIAM Journal on Scientific Computing N2 - In this article we propose an adaptive importance sampling scheme for dynamical quantities of high dimensional complex systems which are metastable. The main idea of this article is to combine a method coming from Molecular Dynamics Simulation, Metadynamics, with a theorem from stochastic analysis, Girsanov's theorem. The proposed algorithm has two advantages compared to a standard estimator of dynamic quantities: firstly, it is possible to produce estimators with a lower variance and, secondly, we can speed up the sampling. One of the main problems for building importance sampling schemes for metastable systems is to find the metastable region in order to manipulate the potential accordingly. Our method circumvents this problem by using an assimilated version of the Metadynamics algorithm and thus creates a non-equilibrium dynamics which is used to sample the equilibrium quantities. Y1 - 2018 U6 - https://doi.org/10.1137/17m1124772 VL - 40 IS - 2 SP - A653 EP - A670 ER - TY - JOUR A1 - Quer, Jannes A1 - Ribera Borrell, Enric T1 - Connecting Stochastic Optimal Control and Reinforcement Learning JF - Journal of Mathematical Physics N2 - In this article we study the connection of stochastic optimal control and reinforcement learning. Our main motivation is an importance sampling application to rare events sampling which can be reformulated as an optimal control problem. By using a parameterized approach the optimal control problem turns into a stochastic optimization problem which still presents some open questions regarding how to tackle the scalability to high-dimensional problems and how to deal with the intrinsic metastability of the system. With the aim to explore new methods we connect the optimal control problem to reinforcement learning since both share the same underlying framework namely a Markov decision process (MDP). We show how the MDP can be formulated for the optimal control problem. Furthermore, we discuss how the stochastic optimal control problem can be interpreted in a reinforcement learning framework. At the end of the article we present the application of two different reinforcement learning algorithms to the optimal control problem and compare the advantages and disadvantages of the two algorithms. Y1 - 2023 ER - TY - JOUR A1 - Ribera Borrell, Enric A1 - Quer, Jannes A1 - Richter, Lorenz A1 - Schütte, Christof T1 - Improving control based importance sampling strategies for metastable diffusions via adapted metadynamics JF - SIAM Journal on Scientific Computing (SISC) N2 - Sampling rare events in metastable dynamical systems is often a computationally expensive task and one needs to resort to enhanced sampling methods such as importance sampling. Since we can formulate the problem of finding optimal importance sampling controls as a stochastic optimization problem, this then brings additional numerical challenges and the convergence of corresponding algorithms might as well suffer from metastabilty. In this article we address this issue by combining systematic control approaches with the heuristic adaptive metadynamics method. Crucially, we approximate the importance sampling control by a neural network, which makes the algorithm in principle feasible for high dimensional applications. We can numerically demonstrate in relevant metastable problems that our algorithm is more effective than previous attempts and that only the combination of the two approaches leads to a satisfying convergence and therefore to an efficient sampling in certain metastable settings. KW - importance sampling KW - stochastic optimal control KW - rare event simulation KW - metastability KW - neural networks KW - metadynamics Y1 - 2023 U6 - https://doi.org/10.1137/22M1503464 SP - S298 EP - S323 ER - TY - GEN A1 - Ribera Borrell, Enric A1 - Quer, Jannes A1 - Richter, Lorenz A1 - Schütte, Christof T1 - Improving control based importance sampling strategies for metastable diffusions via adapted metadynamics N2 - Sampling rare events in metastable dynamical systems is often a computationally expensive task and one needs to resort to enhanced sampling methods such as importance sampling. Since we can formulate the problem of finding optimal importance sampling controls as a stochastic optimization problem, this then brings additional numerical challenges and the convergence of corresponding algorithms might as well suffer from metastabilty. In this article we address this issue by combining systematic control approaches with the heuristic adaptive metadynamics method. Crucially, we approximate the importance sampling control by a neural network, which makes the algorithm in principle feasible for high dimensional applications. We can numerically demonstrate in relevant metastable problems that our algorithm is more effective than previous attempts and that only the combination of the two approaches leads to a satisfying convergence and therefore to an efficient sampling in certain metastable settings. T3 - ZIB-Report - 21-40 KW - importance sampling KW - stochastic optimal control KW - rare event simulation KW - metastability KW - neural networks KW - metadynamics Y1 - 2021 SN - 1438-0064 ER - TY - JOUR A1 - Quer, Jannes A1 - Lie, Han Cheng T1 - Some connections between importance sampling and enhanced sampling methods in molecular dynamics JF - Journal of Chemical Physics N2 - Enhanced sampling methods play an important role in molecular dynamics, because they enable the collection of better statistics of rare events that are important in many physical phenomena. We show that many enhanced sampling methods can be viewed as methods for performing importance sampling, by identifying important correspondences between the language of molecular dynamics and the language of probability theory. We illustrate these connections by highlighting the similarities between the rare event simulation method of Hartmann and Schütte (J. Stat. Mech. Theor. Exp., 2012), and the enhanced sampling method of Valsson and Parrinello (Phys. Rev. Lett. 113, 090601). We show that the idea of changing a probability measure is fundamental to both enhanced sampling and importance sampling. Y1 - 2017 ER -