## 65C40 Computational Markov chains

### Refine

#### Year of publication

#### Document Type

- ZIB-Report (9)
- Article (1)

#### Keywords

- $n$-pentane molecule (2)
- almost invariant sets (2)
- bridge sampling (2)
- cluster analysis (2)
- hierarchical annealing (2)
- metastability (2)
- metastable sets (2)
- Alanine dipeptide (1)
- Chemical Master Equations (1)
- Conformation Dynamics (1)

#### Institute

Gene Regulatory Networks are powerful models for describing the mechanisms and dynamics inside a cell. These networks are generally large in dimension and seldom yield analytical formulations. It was shown that studying the conditional expectations between dimensions (vertices or species) of a network could lead to drastic dimension reduction. These conditional expectations were classically given by solving equations of motions derived from the Chemical Master Equation. In this paper we deviate from this convention and take an Algebraic approach instead. That is, we explore the consequences of conditional expectations being described by a polynomial function. There are two main results in this work. Firstly: if the conditional expectation can be described by a polynomial function, then coefficients of this polynomial function can be reconstructed using the classical moments. And secondly: there are dimensions in Gene Regulatory Networks which inherently have conditional expectations with algebraic forms. We demonstrate through examples, that the theory derived in this work can be used to develop new and effective numerical schemes for forward simulation and parameter inference. The algebraic line of investigation of conditional expectations has considerable scope to be applied to many different aspects of Gene Regulatory Networks; this paper serves as a preliminary commentary in this direction.

Finding metastable sets as dominant structures of Markov processes has been shown to be especially useful in modeling interesting slow dynamics of various real world complex processes. Furthermore, coarse graining of such processes based on their dominant structures leads to better understanding and dimension reduction of observed systems. However, in many cases, e.g. for nonreversible Markov processes, dominant structures are often not formed by metastable sets but by important cycles or mixture of both. This paper aims at understanding and identifying these different types of dominant structures for reversible as well as nonreversible ergodic Markov processes. Our algorithmic approach generalizes spectral based methods for reversible process by using Schur decomposition techniques which can tackle also nonreversible cases. We illustrate the mathematical construction of our new approach by numerical experiments.

Recent years have seen an increased interest in non-equilibrium molecular dynamics (NEMD) simulations, especially for molecular systems with periodic forcing by external fields, e.g., in the context of studying effects of electromagnetic radiation on the human body tissue. Lately, an NEMD methods with local thermostating has been proposed that allows for studying non-equilibrium processes in a statistically reliable and thermodynamically consistent way. In this article, we demonstrate how to construct Markov State Models (MSMs) for such NEMD simulations. MSM building has been well-established for systems in equilibrium where MSMs with just a few (macro-)states allow for accurate reproduction of the essential kinetics of the molecular system under consideration. Non-equilibrium MSMs have been lacking so far. The article presents how to construct such MSMs and illustrates their validity and usefulness for the case of conformation dynamics of alanine dipeptide in an external electric field.

The rebinding effect is a phenomenon which occurs when observing a ligand-receptor binding process.
On the macro scale this process comprises the Markov property.
This Makovian view is spoiled when switching to the atomistic scale of a binding process.
We therefore suggest a model which accurately describes the rebinding effect on the atomistic scale by allowing ''intermediate'' bound states.
This allows us to define an indicator for the magnitude of rebinding and to formulate an optimization problem.
The results form our examples show good agreement with data form laboratory.

The enormous time lag between fast atomic motion and complex pro- tein folding events makes it almost impossible to compute molecular dy- namics on a high resolution. A common way to tackle this problem is to model the system dynamics as a Markov process. Yet for large molec- ular systems the resulting Markov chains can hardly be handled due to the curse of dimensionality. Coarse graining methods can be used to re- duce the dimension of a Markov chain, but it is still unclear how far the coarse grained Markov chain resembles the original system. In order to answer this question, two different coarse-graining methods were analysed and compared: a classical set-based reduction method and an alternative subspace-based approach, which is based on membership vectors instead of sets. On the basis of a small toy system, it could be shown, that in con- trast to the subset-based approach, the subspace-based reduction method preserves the Markov property as well as the essential dynamics of the original system.

In this article we aim at an efficient sampling of the stationary distribution of dynamical systems in the presence of metastabilities. In the past decade many sophisticated algorithms have been inven ted in this field. We do not want to simply add a further one. We address the problem that one has applied a sampling algorithm for a dynamical system many times. This leads to different samplings which more or less represent the stationary distribution partially very well, but which are still far away from ergodicity or from the global stationary distribution. We will show how these samplings can be joined together in order to get one global sampling of the stationary distribution.

The dynamic behavior of molecules can often be described by Markov processes. From computational molecular simulations one can derive transition rates or transition probabilities between subsets of the discretized conformational space. On the basis of this dynamic information, the spatial subsets are combined into a small number of so-called metastable molecular conformations. This is done by clustering methods like the Robust Perron Cluster Analysis (PCCA+). Up to now it is an open question how this coarse graining in space can be transformed to a coarse graining of the Markov chain while preserving the essential dynamic information. In the following article we aim at a consistent coarse graining of transition probabilities or rates on the basis of metastable conformations such that important physical and mathematical relations are preserved. This approach is new because PCCA+ computes molecular conformations as linear combinations of the dominant eigenvectors of the transition matrix which does not hold for other clustering methods.

The paper surveys recent progress in the mathematical modelling and simulation of essential molecular dynamics. Particular emphasis is put on computational drug design wherein time scales of $msec$ up to $min$ play the dominant role. Classical long-term molecular dynamics computations, however, would run into ill-conditioned initial value problems already after time spans of only $psec=10^{-12} sec$. Therefore, in order to obtain results for times of pharmaceutical interest, a combined deterministic-stochastic model is needed. The concept advocated in this paper is the direct identification of metastable conformations together with their life times and their transition patterns. It can be interpreted as a {\em transfer operator} approach corresponding to some underlying hybrid Monte Carlo process, wherein short-term trajectories enter. Once this operator has been discretized, which is a hard problem of its own, a stochastic matrix arises. This matrix is then treated by {\em Perron cluster analysis}, a recently developed cluster analysis method involving the numerical solution of an eigenproblem for a Perron cluster of eigenvalues. In order to avoid the 'curse of dimension', the construction of appropriate boxes for the spatial discretization of the Markov operator requires careful consideration. As a biomolecular example we present a rather recent SARS protease inhibitor.

Uncoupling-coupling Monte Carlo (UCMC) combines uncoupling techniques for finite Markov chains with Markov chain Monte Carlo methodology. UCMC aims at avoiding the typical metastable or trapping behavior of Monte Carlo techniques. From the viewpoint of Monte Carlo, a slowly converging long-time Markov chain is replaced by a limited number of rapidly mixing short-time ones. Therefore, the state space of the chain has to be hierarchically decomposed into its metastable conformations. This is done by means of combining the technique of conformation analysis as recently introduced by the authors, and appropriate annealing strategies. We present a detailed examination of the uncoupling-coupling procedure which uncovers its theoretical background, and illustrates the hierarchical algorithmic approach. Furthermore, application of the UCMC algorithm to the $n$-pentane molecule allows us to discuss the effect of its crucial steps in a typical molecular scenario.

Uncoupling-coupling Monte Carlo (UCMC) combines uncoupling techniques for finite Markov chains with Markov chain Monte Carlo methodology. By determining almost invariant sets of the associated Markov operator, the Monte Carlo sampling splits by a hierarchical annealing process into the essential regions of the state space; therefore UCMC aims at avoiding the typical metastable behavior of Monte Carlo techniques. From the viewpoint of Monte Carlo, a slowly converging long-time Markov chain is replaced by a limited number of rapidly mixing short-time ones. The correct weighting factors for the various Markov chains are obtained via a coupling matrix, that connects the samplings from the different almost invariant sets. The underlying mathematical structure of this approach is given by a general examination of the uncoupling-coupling procedure. Furthermore, the overall algorithmic scheme of UCMC is applied to the $n$-pentane molecule, a well-known example from molecular dynamics.