## 60J20 Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) [See also 90B30, 91D10, 91D35, 91E40]

### Refine

#### Document Type

- ZIB-Report (11)

#### Language

- English (11)

#### Has Fulltext

- yes (11)

#### Is part of the Bibliography

- no (11)

#### Keywords

#### Institute

- Numerical Mathematics (7)
- ZIB Allgemein (4)

We investigate metastable dynamical systems subject to non-stationary forcing as they appear in molecular dynamics for systems driven by external fields. We show, that if the strength of the forcing is inversely proportional to the length of the slow metastable time scales of the unforced system, then the effective behavior of the forced system on slow time scales can be described by a low-dimensional reduced master equation. Our construction is explicit and uses the multiscale perturbation expansion method called two-timing, or method of multiple scales. The reduced master equation—a Markov state model—can be assembled by constructing two equilibrium Markov state models; one for the unforced system, and one for a slightly perturbed one.

We utilize the theory of coherent sets to build Markov state models for non- equilibrium molecular dynamical systems. Unlike for systems in equilibrium, “meta- stable” sets in the non-equilibrium case may move as time evolves. We formalize this concept by relying on the theory of coherent sets, based on this we derive finite-time non-stationary Markov state models, and illustrate the concept and its main differences to equilibrium Markov state modeling on simple, one-dimensional examples.

Finding metastable sets as dominant structures of Markov processes has been shown to be especially useful in modeling interesting slow dynamics of various real world complex processes. Furthermore, coarse graining of such processes based on their dominant structures leads to better understanding and dimension reduction of observed systems. However, in many cases, e.g. for nonreversible Markov processes, dominant structures are often not formed by metastable sets but by important cycles or mixture of both. This paper aims at understanding and identifying these different types of dominant structures for reversible as well as nonreversible ergodic Markov processes. Our algorithmic approach generalizes spectral based methods for reversible process by using Schur decomposition techniques which can tackle also nonreversible cases. We illustrate the mathematical construction of our new approach by numerical experiments.

Markov State Modelling as a concept for a coarse grained description of the essential kinetics of a molecular system in equilibrium has gained a lot of atten- tion recently. The last 10 years have seen an ever increasing publication activity on how to construct Markov State Models (MSMs) for very different molecular systems ranging from peptides to proteins, from RNA to DNA, and via molecu- lar sensors to molecular aggregation. Simultaneously the accompanying theory behind MSM building and approximation quality has been developed well be- yond the concepts and ideas used in practical applications. This article reviews the main theoretical results, provides links to crucial new developments, outlines the full power of MSM building today, and discusses the essential limitations still to overcome.

Techniques for ﬁnding metastable or almost invariant sets have been investigated, e.g., for deterministic dynamical systems in set-oriented numerics, for stochastic processes in molecular dynamics, and for random walks on complex networks. Most prominent algorithms are based on spectral apporaches and identify metastable sets via the doimant eigenvalues of the transfer operator associated with the dynamical system under consideration. These algorithms require the dominant eigenvalues to be real-valued. However, for many types of dynamics, e.g. for non-reversible Markov chains, this condition is not met. In this paper we utilize the hitting time apporach to metastable sets and demonstrate how the wellknown statements about optimal metastable decompositions of reversible chains can be reformulated for non-reversible chains if one switches from a spectral approach to an exit time approach. The performance of the resulting algorithm is illustrated by numerical experiments on random walks on complex networks.

The problem of decomposing networks into modules (or clusters) has gained much attention in recent years, as it can account for a coarsegrained description of complex systems, often revealing functional subunits of these systems. A variety of module detection algorithms have been proposed, mostly oriented towards finding hard partitionings of undirected networks. Despite the increasing number of fuzzy clustering methods for directed networks, many of these approaches tend to neglect important directional information. In this paper, we present a novel random walk based approach for finding fuzzy partitions of directed, weighted networks, where edge directions play a crucial role in defining how well nodes in a module are interconnected. We will show that cycle decomposition of a random walk process connects the notion of network modules and information transport in a network, leading to a new, symmetric measure of node communication. Finally, we will use this measure to introduce a communication graph, for which we will show that although being undirected it inherits all necessary information about modular structures from the original network.

We investigate the problem of finding modules (or clusters, communities) in directed networks. Until now, most articles on this topic have been oriented towards finding complete network partitions despite the fact that this often is unwanted. We present a novel random walk based approach for non-complete partitions of the directed network into modules in which some nodes do not belong to only one of the modules but to several or to none at all. The new random walk process is reversible even for directed networks but inherits all necessary information about directions and structure of the original network. We demonstrate the performance of the new method in application to a real-world earthquake network.

As has been shown recently, the identification of metastable chemical conformations leads to a Perron cluster eigenvalue problem for a reversible Markov operator. Naive discretization of this operator would suffer from combinatorial explosion. As a first remedy, a pre-identification of essential degrees of freedom out of the set of torsion angles had been applied up to now. The present paper suggests a different approach based on neural networks: its idea is to discretize the Markov operator via self-organizing (box) maps. The thus obtained box discretization then serves as a prerequisite for the subsequent Perron cluster analysis. Moreover, this approach also permits exploitation of additional structure within embedded simulations. As it turns out, the new method is fully automatic and efficient also in the treatment of biomolecules. This is exemplified by numerical results.

The statistical behavior of deterministic and stochastic dynamical systems may be described using transfer operators, which generalize the notion of Frobenius Perron and Koopman operators. Since numerical techniques to analyze dynamical systems based on eigenvalues problems for the corresponding transfer operator have emerged, bounds on its essential spectral radius became of interest. This article shows that they are also of great theoretical interest. We give an analytical representation of the essential spectral radius in $L^{1}\!(\mu)$, which then is exploited to analyze the asymptotical properties of transfer operators by combining results from functional analysis, Markov operators and Markov chain theory. In particular, it is shown, that an essential spectral radius less than $1$, constrictiveness and some weak form'' of the so--called Doeblin condition are equivalent. Finally, we apply the results to study three main problem classes: deterministic systems, stochastically perturbed deterministic systems and stochastic systems.

Recently, a novel concept for the computation of essential features of the dynamics of Hamiltonian systems (such as molecular dynamics) has been proposed. The realization of this concept had been based on subdivision techniques applied to the Frobenius--Perron operator for the dynamical system. The present paper suggests an alternative but related concept that merges the conceptual advantages of the dynamical systems approach with the appropriate statistical physics framework. This approach allows to define the phrase ``conformation'' in terms of the dynamical behavior of the molecular system and to characterize the dynamical stability of conformations. In a first step, the frequency of conformational changes is characterized in statistical terms leading to the definition of some Markov operator $T$ that describes the corresponding transition probabilities within the canonical ensemble. In a second step, a discretization of $T$ via specific hybrid Monte Carlo techniques is shown to lead to a stochastic matrix $P$. With these theoretical preparations, an identification algorithm for conformations is applicable. It is demonstrated that the discretization of $T$ can be restricted to few essential degrees of freedom so that the combinatorial explosion of discretization boxes is prevented and biomolecular systems can be attacked. Numerical results for the n-pentane molecule and the triribonucleotide adenylyl\emph{(3'-5')}cytidylyl\emph{(3'-5')}cytidin are given and interpreted.