TY - JOUR A1 - Zhang, Wei A1 - Hartmann, Carsten A1 - von Kleist, Max T1 - Optimal control of Markov jump processes: Asymptotic analysis, algorithms and applications to the modeling of chemical reaction systems JF - Communications in Mathematical Sciences N2 - Markov jump processes are widely used to model natural and engineered processes. In the context of biological or chemical applications one typically refers to the chemical master equation (CME), which models the evolution of the probability mass of any copy-number combination of the interacting particles. When many interacting particles (“species”) are considered, the complexity of the CME quickly increases, making direct numerical simulations impossible. This is even more problematic when one aims at controlling the Markov jump processes defined by the CME. In this work, we study both open loop and feedback optimal control problems of the Markov jump processes in the case that the controls can only be switched at fixed control stages. Based on Kurtz’s limit theorems, we prove the convergence of the respective control value functions of the underlying Markov decision problem as the copy numbers of the species go to infinity. In the case of the optimal control problem on a finite time-horizon, we propose a hybrid control policy algorithm to overcome the difficulties due to the curse of dimensionality when the copy number of the involved species is large. Two numerical examples demonstrate the suitability of both the analysis and the proposed algorithms. Y1 - 2018 U6 - https://doi.org/10.4310/CMS.2018.v16.n2.a1 SP - 293 EP - 331 ER - TY - JOUR A1 - Sharma, Upanshu A1 - Zhang, Wei T1 - Non-reversible sampling schemes on submanifolds JF - SIAM Journal on Numerical Analysis N2 - Calculating averages with respect to probability measures on submanifolds is often necessary in various application areas such as molecular dynamics, computational statistical mechanics and Bayesian statistics. In recent years, various numerical schemes have been proposed in the literature to study this problem based on appropriate reversible constrained stochastic dynamics. In this paper we present and analyse a non-reversible generalisation of the projection-based scheme developed by one of the authors [ESAIM: M2AN, 54 (2020), pp. 391-430]. This scheme consists of two steps - starting from a state on the submanifold, we first update the state using a non-reversible stochastic differential equation which takes the state away from the submanifold, and in the second step we project the state back onto the manifold using the long-time limit of a ordinary differential equation. We prove the consistency of this numerical scheme and provide quantitative error estimates for estimators based on finite-time running averages. Furthermore, we present theoretical analysis which shows that this scheme outperforms its reversible counterpart in terms of asymptotic variance. We demonstrate our findings on an illustrative test example. Y1 - 2020 U6 - https://doi.org/10.1137/20M1378752 VL - 59 IS - 6 SP - 2989 EP - 3031 ER - TY - JOUR A1 - Zhang, Wei T1 - Ergodic SDEs on submanifolds and related numerical sampling schemes JF - ESAIM: Mathematical Modelling and Numerical Analysis N2 - In many applications, it is often necessary to sample the mean value of certain quantity with respect to a probability measure $\mu$ on the level set of a smooth function ξ:R^d→R^k, 1≤k