TY - JOUR A1 - Winkelmann, Stefanie A1 - Schütte, Christof A1 - von Kleist, Max T1 - Markov Control with Rare State Observation JF - International Journal of Biomathematics and Biostatistics Y1 - 2012 UR - http://publications.mi.fu-berlin.de/1177/ ER - TY - JOUR A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - Hybrid models for chemical reaction networks: Multiscale theory and application to gene regulatory systems JF - The Journal of Chemical Physics N2 - Well-mixed stochastic chemical kinetics are properly modeled by the chemical master equation (CME) and associated Markov jump processes in molecule number space. If the reactants are present in large amounts, however, corresponding simulations of the stochastic dynamics become computationally expensive and model reductions are demanded. The classical model reduction approach uniformly rescales the overall dynamics to obtain deterministic systems characterized by ordinary differential equations, the well-known mass action reaction rate equations. For systems with multiple scales, there exist hybrid approaches that keep parts of the system discrete while another part is approximated either using Langevin dynamics or deterministically. This paper aims at giving a coherent overview of the different hybrid approaches, focusing on their basic concepts and the relation between them. We derive a novel general description of such hybrid models that allows expressing various forms by one type of equation. We also check in how far the approaches apply to model extensions of the CME for dynamics which do not comply with the central well-mixed condition and require some spatial resolution. A simple but meaningful gene expression system with negative self-regulation is analysed to illustrate the different approximation qualities of some of the hybrid approaches discussed. Especially, we reveal the cause of error in the case of small volume approximations. Y1 - 2017 U6 - https://doi.org/10.1063/1.4986560 VL - 147 IS - 11 SP - 114115-1 EP - 114115-18 ER - TY - JOUR A1 - Winkelmann, Stefanie T1 - Markov Control with Rare State Observation: Average Optimality JF - Markov Processes and Related Fields N2 - This paper investigates the criterion of long-term average costs for a Markov decision process (MDP) which is not permanently observable. Each observation of the process produces a fixed amount of information costs which enter the considered performance criterion and preclude from arbitrarily frequent state testing. Choosing the rare observation times is part of the control procedure. In contrast to the theory of partially observable Markov decision processes, we consider an arbitrary continuous-time Markov process on a finite state space without further restrictions on the dynamics or the type of interaction. Based on the original Markov control theory, we redefine the control model and the average cost criterion for the setting of information costs. We analyze the constant of average costs for the case of ergodic dynamics and present an optimality equation which characterizes the optimal choice of control actions and observation times. For this purpose, we construct an equivalent freely observable MDP and translate the well-known results from the original theory to the new setting. Y1 - 2017 VL - 23 SP - 1 EP - 34 ER - TY - JOUR A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - The Spatiotemporal Master Equation: Approximation of Reaction-Diffusion Dynamics via Markov State Modeling JF - Journal of Chemical Physics N2 - Accurate modeling and numerical simulation of reaction kinetics is a topic of steady interest.We consider the spatiotemporal chemical master equation (ST-CME) as a model for stochastic reaction-diffusion systems that exhibit properties of metastability. The space of motion is decomposed into metastable compartments and diffusive motion is approximated by jumps between these compartments. Treating these jumps as first-order reactions, simulation of the resulting stochastic system is possible by the Gillespie method. We present the theory of Markov state models (MSM) as a theoretical foundation of this intuitive approach. By means of Markov state modeling, both the number and shape of compartments and the transition rates between them can be determined. We consider the ST-CME for two reaction-diffusion systems and compare it to more detailed models. Moreover, a rigorous formal justification of the ST-CME by Galerkin projection methods is presented. Y1 - 2016 U6 - https://doi.org/10.1063/1.4971163 VL - 145 IS - 21 ER - TY - JOUR A1 - Duwal, Sulav A1 - Winkelmann, Stefanie A1 - Schütte, Christof A1 - von Kleist, Max T1 - Optimal Treatment Strategies in the Context of 'Treatment for Prevention' against HIV/1 in Resource-Poor Settings JF - PloS Computational Biology N2 - An estimated 2.7 million new HIV-1 infections occurred in 2010. `Treatment-for-prevention’ may strongly prevent HIV-1 transmission. The basic idea is that immediate treatment initiation rapidly decreases virus burden, which reduces the number of transmittable viruses and thereby the probability of infection. However, HIV inevitably develops drug resistance, which leads to virus rebound and nullifies the effect of `treatment-for-prevention’ for the time it remains unrecognized. While timely conducted treatment changes may avert periods of viral rebound, necessary treatment options and diagnostics may be lacking in resource-constrained settings. Within this work, we provide a mathematical platform for comparing different treatment paradigms that can be applied to many medical phenomena. We use this platform to optimize two distinct approaches for the treatment of HIV-1: (i) a diagnostic-guided treatment strategy, based on infrequent and patient-specific diagnostic schedules and (ii) a pro-active strategy that allows treatment adaptation prior to diagnostic ascertainment. Both strategies are compared to current clinical protocols (standard of care and the HPTN052 protocol) in terms of patient health, economic means and reduction in HIV-1 onward transmission exemplarily for South Africa. All therapeutic strategies are assessed using a coarse-grained stochastic model of within-host HIV dynamics and pseudo-codes for solving the respective optimal control problems are provided. Our mathematical model suggests that both optimal strategies (i)-(ii) perform better than the current clinical protocols and no treatment in terms of economic means, life prolongation and reduction of HIV-transmission. The optimal diagnostic-guided strategy suggests rare diagnostics and performs similar to the optimal pro-active strategy. Our results suggest that ‘treatment-for-prevention’ may be further improved using either of the two analyzed treatment paradigms. Y1 - 2015 U6 - https://doi.org/10.1371/journal.pcbi.1004200 VL - 11 IS - 4 ER - TY - GEN A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - Hybrid Models for Chemical Reaction Networks: Multiscale Theory and Application to Gene Regulatory Systems N2 - Well-mixed stochastic chemical kinetics are properly modelled by the chemical master equation (CME) and associated Markov jump processes in molecule number space. If the reactants are present in large amounts, however, corresponding simulations of the stochastic dynamics become computationally expensive and model reductions are demanded. The classical model reduction approach uniformly rescales the overall dynamics to obtain deterministic systems characterized by ordinary differential equations, the well-known mass action reaction rate equations. For systems with multiple scales there exist hybrid approaches that keep parts of the system discrete while another part is approximated either using Langevin dynamics or deterministically. This paper aims at giving a coherent overview of the different hybrid approaches, focusing on their basic concepts and the relation between them. We derive a novel general description of such hybrid models that allows to express various forms by one type of equation. We also check in how far the approaches apply to model extensions of the CME for dynamics which do not comply with the central well-mixed condition and require some spatial resolution. A simple but meaningful gene expression system with negative self-regulation is analysed to illustrate the different approximation qualities of some of the hybrid approaches discussed. T3 - ZIB-Report - 17-29 Y1 - 2017 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-64264 ER - TY - THES A1 - Winkelmann, Stefanie T1 - Markov Decision Processes with Information Costs Y1 - 2013 UR - http://www.diss.fu-berlin.de/diss/receive/FUDISS_thesis_000000094689 ER - TY - GEN A1 - Winkelmann, Stefanie A1 - Schütte, Christof A1 - von Kleist, Max T1 - Markov Control Processes with Rare State Observation: Theory and Application to Treatment Scheduling in HIV-1 N2 - Markov Decision Processes (MDP) or Partially Observable MDPs (POMDP) are used for modelling situations in which the evolution of a process is partly random and partly controllable. These MDP theories allow for computing the optimal control policy for processes that can continuously or frequently be observed, even if only partially. However, they cannot be applied if state observation is very costly and therefore rare (in time). We present a novel MDP theory for rare, costly observations and derive the corresponding Bellman equation. In the new theory, state information can be derived for a particular cost after certain, rather long time intervals. The resulting information costs enter into the total cost and thus into the optimization criterion. This approach applies to many real world problems, particularly in the medical context, where the medical condition is examined rather rarely because examination costs are high. At the same time, the approach allows for efficient numerical realization. We demonstrate the usefulness of the novel theory by determining, from the national economic perspective, optimal therapeutic policies for the treatment of the human immunodefficiency virus (HIV) in resource-rich and resource-poor settings. Based on the developed theory and models, we discover that available drugs may not be utilized efficiently in resource-poor settings due to exorbitant diagnostic costs. T3 - ZIB-Report - 13-34 KW - information costs KW - hidden state KW - bellmann equation KW - optimal therapeutic policies KW - diagnostic frequency KW - resource-poor KW - resource-rich Y1 - 2013 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-41955 SN - 1438-0064 ER - TY - JOUR A1 - Winkelmann, Stefanie A1 - Schütte, Christof A1 - von Kleist, Max T1 - Markov Control Processes with Rare State Observation: Sensitivity Analysis with Respect to Optimal Treatment Strategies against HIV-1 JF - International Journal of Biomathematics and Biostatistics N2 - We present the theory of “Markov decision processes (MDP) with rare state observation” and apply it to optimal treatment scheduling and diagnostic testing to mitigate HIV-1 drug resistance development in resource-poor countries. The developed theory assumes that the state of the process is hidden and can only be determined by making an examination. Each examination produces costs which enter into the considered cost functional so that the resulting optimization problem includes finding optimal examination times. This is a realistic ansatz: In many real world applications, like HIV-1 treatment scheduling, the information about the disease evolution involves substantial costs, such that examination and control are intimately connected. However, a perfect compliance with the optimal strategy can rarely be achieved. This may be particularly true for HIV-1 resistance testing in resource-constrained countries. In the present work, we therefore analyze the sensitivity of the costs with respect to deviations from the optimal examination times both analytically and for the considered application. We discover continuity in the cost-functional with respect to the examination times. For the HIV-application, moreover, sensitivity towards small deviations from the optimal examination rule depends on the disease state. Furthermore, we compare the optimal rare-control strategy to (i) constant control strategies (one action for the remaining time) and to (ii) the permanent control of the original, fully observed MDP. This comparison is done in terms of expected costs and in terms of life-prolongation. The proposed rare-control strategy offers a clear benefit over a constant control, stressing the usefulness of medical testing and informed decision making. This indicates that lower-priced medical tests could improve HIV treatment in resource-constrained settings and warrants further investigation. Y1 - 2013 VL - 2 IS - 1 ER - TY - JOUR A1 - Winkelmann, Stefanie A1 - Schütte, Christof A1 - von Kleist, Max T1 - Markov Control Processes with Rare State Observation: Theory and Application to Treatment Scheduling in HIV-1 JF - Communications in Mathematical Sciences N2 - Markov Decision Processes (MDP) or Partially Observable MDPs (POMDP) are used for modelling situations in which the evolution of a process is partly random and partly controllable. These MDP theories allow for computing the optimal control policy for processes that can continuously or frequently be observed, even if only partially. However, they cannot be applied if state observation is very costly and therefore rare (in time). We present a novel MDP theory for rare, costly observations and derive the corresponding Bellman equation. In the new theory, state information can be derived for a particular cost after certain, rather long time intervals. The resulting information costs enter into the total cost and thus into the optimization criterion. This approach applies to many real world problems, particularly in the medical context, where the medical condition is examined rather rarely because examination costs are high. At the same time, the approach allows for efficient numerical realization. We demonstrate the usefulness of the novel theory by determining, from the national economic perspective, optimal therapeutic policies for the treatment of the human immunodeficiency virus (HIV) in resource-rich and resource-poor settings. Based on the developed theory and models, we discover that available drugs may not be utilized efficiently in resource-poor settings due to exorbitant diagnostic costs. Y1 - 2014 U6 - https://doi.org/10.4310/CMS.2014.v12.n5.a4 VL - 12 IS - 5 SP - 859 EP - 877 ER - TY - JOUR A1 - Djurdjevac Conrad, Natasa A1 - Helfmann, Luzie A1 - Zonker, Johannes A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - Human mobility and innovation spreading in ancient times: a stochastic agent-based simulation approach JF - EPJ Data Science N2 - Human mobility always had a great influence on the spreading of cultural, social and technological ideas. Developing realistic models that allow for a better understanding, prediction and control of such coupled processes has gained a lot of attention in recent years. However, the modeling of spreading processes that happened in ancient times faces the additional challenge that available knowledge and data is often limited and sparse. In this paper, we present a new agent-based model for the spreading of innovations in the ancient world that is governed by human movements. Our model considers the diffusion of innovations on a spatial network that is changing in time, as the agents are changing their positions. Additionally, we propose a novel stochastic simulation approach to produce spatio-temporal realizations of the spreading process that are instructive for studying its dynamical properties and exploring how different influences affect its speed and spatial evolution. Y1 - 2018 U6 - https://doi.org/10.1140/epjds/s13688-018-0153-9 VL - 7 IS - 1 PB - EPJ Data Science ET - EPJ Data Science ER - TY - JOUR A1 - Helfmann, Luzie A1 - Djurdjevac Conrad, Natasa A1 - Djurdjevac, Ana A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - From interacting agents to density-based modeling with stochastic PDEs JF - Communications in Applied Mathematics and Computational Science N2 - Many real-world processes can naturally be modeled as systems of interacting agents. However, the long-term simulation of such agent-based models is often intractable when the system becomes too large. In this paper, starting from a stochastic spatio-temporal agent-based model (ABM), we present a reduced model in terms of stochastic PDEs that describes the evolution of agent number densities for large populations. We discuss the algorithmic details of both approaches; regarding the SPDE model, we apply Finite Element discretization in space which not only ensures efficient simulation but also serves as a regularization of the SPDE. Illustrative examples for the spreading of an innovation among agents are given and used for comparing ABM and SPDE models. Y1 - 2021 U6 - https://doi.org/10.2140/camcos.2021.16.1 VL - 16 IS - 1 SP - 1 EP - 32 ER - TY - GEN A1 - Helfmann, Luzie A1 - Djurdjevac Conrad, Natasa A1 - Djurdjevac, Ana A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - From interacting agents to density-based modeling with stochastic PDEs N2 - Many real-world processes can naturally be modeled as systems of interacting agents. However, the long-term simulation of such agent-based models is often intractable when the system becomes too large. In this paper, starting from a stochastic spatio-temporal agent-based model (ABM), we present a reduced model in terms of stochastic PDEs that describes the evolution of agent number densities for large populations. We discuss the algorithmic details of both approaches; regarding the SPDE model, we apply Finite Element discretization in space which not only ensures efficient simulation but also serves as a regularization of the SPDE. Illustrative examples for the spreading of an innovation among agents are given and used for comparing ABM and SPDE models. T3 - ZIB-Report - 19-21 Y1 - 2019 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-73456 SN - 1438-0064 ER - TY - BOOK A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - Stochastic Dynamics in Computational Biology T3 - Frontiers in Applied Dynamical Systems: Reviews and Tutorials Y1 - 2020 SN - 978-3-030-62386-9 U6 - https://doi.org/10.1007/978-3-030-62387-6 VL - 8 PB - Springer International Publishing ER - TY - GEN A1 - Winkelmann, Stefanie T1 - Markov Control with Rare State Observation: Average Optimality N2 - This paper investigates the criterion of long-term average costs for a Markov decision process (MDP) which is not permanently observable. Each observation of the process produces a fixed amount of \textit{information costs} which enter the considered performance criterion and preclude from arbitrarily frequent state testing. Choosing the \textit{rare} observation times is part of the control procedure. In contrast to the theory of partially observable Markov decision processes, we consider an arbitrary continuous-time Markov process on a finite state space without further restrictions on the dynamics or the type of interaction. Based on the original Markov control theory, we redefine the control model and the average cost criterion for the setting of information costs. We analyze the constant of average costs for the case of ergodic dynamics and present an optimality equation which characterizes the optimal choice of control actions and observation times. For this purpose, we construct an equivalent freely observable MDP and translate the well-known results from the original theory to the new setting. T3 - ZIB-Report - 16-59 KW - Markov decision process KW - partial observability KW - average optimality KW - information costs Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-60981 SN - 1438-0064 ER - TY - GEN A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - The spatiotemporal master equation: approximation of reaction-diffusion dynamics via Markov state modeling N2 - Accurate modeling and numerical simulation of reaction kinetics is a topic of steady interest. We consider the spatiotemporal chemical master equation (ST-CME) as a model for stochastic reaction-diffusion systems that exhibit properties of metastability. The space of motion is decomposed into metastable compartments and diffusive motion is approximated by jumps between these compartments. Treating these jumps as first-order reactions, simulation of the resulting stochastic system is possible by the Gillespie method. We present the theory of Markov state models (MSM) as a theoretical foundation of this intuitive approach. By means of Markov state modeling, both the number and shape of compartments and the transition rates between them can be determined. We consider the ST-CME for two reaction-diffusion systems and compare it to more detailed models. Moreover, a rigorous formal justification of the ST-CME by Galerkin projection methods is presented. T3 - ZIB-Report - 16-60 KW - reaction-diffusion KW - stochastic chemical kinetics KW - chemical master equation Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-60999 SN - 1438-0064 ER - TY - JOUR A1 - Winkelmann, Stefanie A1 - Zonker, Johannes A1 - Schütte, Christof A1 - Djurdjevac Conrad, Natasa T1 - Mathematical modeling of spatio-temporal population dynamics and application to epidemic spreading JF - Mathematical Biosciences N2 - Agent based models (ABMs) are a useful tool for modeling spatio-temporal population dynamics, where many details can be included in the model description. Their computational cost though is very high and for stochastic ABMs a lot of individual simulations are required to sample quantities of interest. Especially, large numbers of agents render the sampling infeasible. Model reduction to a metapopulation model leads to a significant gain in computational efficiency, while preserving important dynamical properties. Based on a precise mathematical description of spatio-temporal ABMs, we present two different metapopulation approaches (stochastic and piecewise deterministic) and discuss the approximation steps between the different models within this framework. Especially, we show how the stochastic metapopulation model results from a Galerkin projection of the underlying ABM onto a finite-dimensional ansatz space. Finally, we utilize our modeling framework to provide a conceptual model for the spreading of COVID-19 that can be scaled to real-world scenarios. Y1 - 2021 U6 - https://doi.org/10.1016/j.mbs.2021.108619 VL - 336 PB - Elsevier ER - TY - JOUR A1 - Straube, Arthur A1 - Winkelmann, Stefanie A1 - Schütte, Christof A1 - Höfling, Felix T1 - Stochastic pH oscillations in a model of the urea–urease reaction confined to lipid vesicles JF - J. Phys. Chem. Lett. N2 - The urea-urease clock reaction is a pH switch from acid to basic that can turn into a pH oscillator if it occurs inside a suitable open reactor. We numerically study the confinement of the reaction to lipid vesicles, which permit the exchange with an external reservoir by differential transport, enabling the recovery of the pH level and yielding a constant supply of urea molecules. For microscopically small vesicles, the discreteness of the number of molecules requires a stochastic treatment of the reaction dynamics. Our analysis shows that intrinsic noise induces a significant statistical variation of the oscillation period, which increases as the vesicles become smaller. The mean period, however, is found to be remarkably robust for vesicle sizes down to approximately 200 nm, but the periodicity of the rhythm is gradually destroyed for smaller vesicles. The observed oscillations are explained as a canard-like limit cycle that differs from the wide class of conventional feedback oscillators. Y1 - 2021 U6 - https://doi.org/10.1021/acs.jpclett.1c03016 VL - 12 SP - 9888 EP - 9893 ER - TY - JOUR A1 - Boltz, Horst-Holger A1 - Sirbu, Alexei A1 - Stelzer, Nina A1 - de Lanerolle, Primal A1 - Winkelmann, Stefanie A1 - Annibale, Paolo T1 - The Impact of Membrane Protein Diffusion on GPCR Signaling JF - Cells N2 - Spatiotemporal signal shaping in G protein-coupled receptor (GPCR) signaling is now a well-established and accepted notion to explain how signaling specificity can be achieved by a superfamily sharing only a handful of downstream second messengers. Dozens of Gs-coupled GPCR signals ultimately converge on the production of cAMP, a ubiquitous second messenger. This idea is almost always framed in terms of local concentrations, the differences in which are maintained by means of spatial separation. However, given the dynamic nature of the reaction-diffusion processes at hand, the dynamics, in particular the local diffusional properties of the receptors and their cognate G proteins, are also important. By combining some first principle considerations, simulated data, and experimental data of the receptors diffusing on the membranes of living cells, we offer a short perspective on the modulatory role of local membrane diffusion in regulating GPCR-mediated cell signaling. Our analysis points to a diffusion-limited regime where the effective production rate of activated G protein scales linearly with the receptor–G protein complex’s relative diffusion rate and to an interesting role played by the membrane geometry in modulating the efficiency of coupling Y1 - 2022 U6 - https://doi.org/10.3390/cells11101660 VL - 11 IS - 10 SP - 1660 ER - TY - JOUR A1 - Ernst, Ariane A1 - Schütte, Christof A1 - Sigrist, Stephan A1 - Winkelmann, Stefanie T1 - Variance of filtered signals: Characterization for linear reaction networks and application to neurotransmission dynamics JF - Mathematical Biosciences N2 - Neurotransmission at chemical synapses relies on the calcium-induced fusion of synaptic vesicles with the presynaptic membrane. The distance to the calcium channels determines the release probability and thereby the postsynaptic signal. Suitable models of the process need to capture both the mean and the variance observed in electrophysiological measurements of the postsynaptic current. In this work, we propose a method to directly compute the exact first- and second-order moments for signals generated by a linear reaction network under convolution with an impulse response function, rendering computationally expensive numerical simulations of the underlying stochastic counting process obsolete. We show that the autocorrelation of the process is central for the calculation of the filtered signal’s second-order moments, and derive a system of PDEs for the cross-correlation functions (including the autocorrelations) of linear reaction networks with time-dependent rates. Finally, we employ our method to efficiently compare different spatial coarse graining approaches for a specific model of synaptic vesicle fusion. Beyond the application to neurotransmission processes, the developed theory can be applied to any linear reaction system that produces a filtered stochastic signal. Y1 - 2022 U6 - https://doi.org/10.1016/j.mbs.2021.108760 VL - 343 ER - TY - GEN A1 - Ernst, Ariane A1 - Schütte, Christof A1 - Sigrist, Stephan A1 - Winkelmann, Stefanie T1 - Variance of filtered signals: Characterization for linear reaction networks and application to neurotransmission dynamics N2 - Neurotransmission at chemical synapses relies on the calcium-induced fusion of synaptic vesicles with the presynaptic membrane. The distance to the calcium channels determines the release probability and thereby the postsynaptic signal. Suitable models of the process need to capture both the mean and the variance observed in electrophysiological measurements of the postsynaptic current. In this work, we propose a method to directly compute the exact first- and second-order moments for signals generated by a linear reaction network under convolution with an impulse response function, rendering computationally expensive numerical simulations of the underlying stochastic counting process obsolete. We show that the autocorrelation of the process is central for the calculation of the filtered signal’s second-order moments, and derive a system of PDEs for the cross-correlation functions (including the autocorrelations) of linear reaction networks with time-dependent rates. Finally, we employ our method to efficiently compare different spatial coarse graining approaches for a specific model of synaptic vesicle fusion. Beyond the application to neurotransmission processes, the developed theory can be applied to any linear reaction system that produces a filtered stochastic signal. T3 - ZIB-Report - 21-15 KW - linear reaction networks KW - cross-correlation KW - neurotransmission Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-82674 SN - 1438-0064 ER - TY - JOUR A1 - del Razo, Mauricio A1 - Frömberg, Daniela A1 - Straube, Arthur A1 - Schütte, Christof A1 - Höfling, Felix A1 - Winkelmann, Stefanie T1 - A probabilistic framework for particle-based reaction–diffusion dynamics using classical Fock space representations JF - Letters in Mathematical Physics Y1 - 2022 U6 - https://doi.org/10.1007/s11005-022-01539-w VL - 112 IS - 49 ER - TY - JOUR A1 - Winkelmann, Stefanie A1 - Schütte, Christof A1 - von Kleist, Max T1 - Markov Control Processes with Rare State Observation JF - Communications in Mathematical Sciences Y1 - 2012 UR - http://publications.imp.fu-berlin.de/1176/ VL - 12 IS - 859 ER - TY - JOUR A1 - Schütte, Christof A1 - Winkelmann, Stefanie A1 - Hartmann, Carsten T1 - Optimal control of molecular dynamics using Markov state models JF - Math. Program. (Series B) Y1 - 2012 UR - http://publications.imp.fu-berlin.de/1107/ U6 - https://doi.org/10.1007/s10107-012-0547-6 VL - 134 IS - 1 SP - 259 EP - 282 ER - TY - JOUR A1 - Ernst, Ariane A1 - Falkenhagen, Undine A1 - Winkelmann, Stefanie T1 - Model reduction for calcium-induced vesicle fusion dynamics JF - Proceedings in Applied Mathematics & Mechanics N2 - In this work, we adapt an established model for the Ca2+-induced fusion dynamics of synaptic vesicles and employ a lumping method to reduce its complexity. In the reduced system, sequential Ca2+-binding steps are merged to a single releasable state, while keeping the important dependence of the reaction rates on the local Ca2+ concentration. We examine the feasibility of this model reduction for a representative stimulus train over the physiologically relevant site-channel distances. Our findings show that the approximation error is generally small and exhibits an interesting nonlinear and non-monotonic behavior where it vanishes for very low distances and is insignificant at intermediary distances. Furthermore, we give expressions for the reduced model’s reaction rates and suggest that our approach may be used to directly compute effective fusion rates for assessing the validity of a fusion model, thereby circumventing expensive simulations. Y1 - 2023 U6 - https://doi.org/10.1002/pamm.202300184 VL - 23 IS - 4 ER - TY - JOUR A1 - del Razo, Mauricio A1 - Winkelmann, Stefanie A1 - Klein, Rupert A1 - Höfling, Felix T1 - Chemical diffusion master equation: formulations of reaction-diffusion processes on the molecular level JF - Journal of Mathematical Physics N2 - The chemical diffusion master equation (CDME) describes the probabilistic dynamics of reaction--diffusion systems at the molecular level [del Razo et al., Lett. Math. Phys. 112:49, 2022]; it can be considered the master equation for reaction--diffusion processes. The CDME consists of an infinite ordered family of Fokker--Planck equations, where each level of the ordered family corresponds to a certain number of particles and each particle represents a molecule. The equations at each level describe the spatial diffusion of the corresponding set of particles, and they are coupled to each other via reaction operators --linear operators representing chemical reactions. These operators change the number of particles in the system, and thus transport probability between different levels in the family. In this work, we present three approaches to formulate the CDME and show the relations between them. We further deduce the non-trivial combinatorial factors contained in the reaction operators, and we elucidate the relation to the original formulation of the CDME, which is based on creation and annihilation operators acting on many-particle probability density functions. Finally we discuss applications to multiscale simulations of biochemical systems among other future prospects. Y1 - 2023 U6 - https://doi.org/10.1063/5.0129620 VL - 64 IS - 1 ER - TY - JOUR A1 - Lücke, Marvin A1 - Heitzig, Jobst A1 - Koltai, Péter A1 - Molkethin, Nora A1 - Winkelmann, Stefanie T1 - Large population limits of Markov processes on random networks JF - Stochastic Processes and their Applications N2 - We consider time-continuous Markovian discrete-state dynamics on random networks of interacting agents and study the large population limit. The dynamics are projected onto low-dimensional collective variables given by the shares of each discrete state in the system, or in certain subsystems, and general conditions for the convergence of the collective variable dynamics to a mean-field ordinary differential equation are proved. We discuss the convergence to this mean-field limit for a continuous-time noisy version of the so-called "voter model" on Erdős-Rényi random graphs, on the stochastic block model, as well as on random regular graphs. Moreover, a heterogeneous population of agents is studied. For each of these types of interaction networks, we specify the convergence conditions in dependency on the corresponding model parameters. Y1 - 2023 U6 - https://doi.org/10.1016/j.spa.2023.09.007 VL - 166 ER - TY - JOUR A1 - Ernst, Ariane A1 - Unger, Nathalie A1 - Schütte, Christof A1 - Walter, Alexander A1 - Winkelmann, Stefanie T1 - Rate-limiting recovery processes in neurotransmission under sustained stimulation JF - Mathematical Biosciences N2 - At chemical synapses, an arriving electric signal induces the fusion of vesicles with the presynaptic membrane, thereby releasing neurotransmitters into the synaptic cleft. After a fusion event, both the release site and the vesicle undergo a recovery process before becoming available for reuse again. Of central interest is the question which of the two restoration steps acts as the limiting factor during neurotrans-mission under high-frequency sustained stimulation. In order to investigate this question, we introduce a novel non-linear reaction network which involves explicit recovery steps for both the vesicles and the release sites, and includes the induced time-dependent output current. The associated reaction dynamics are formulated by means of ordinary differential equations (ODEs), as well as via the associated stochastic jump process. While the stochastic jump model describes a single release site, the average over many release sites is close to the ODE solution and shares its periodic structure. The reason for this can be traced back to the insight that recovery dynamics of vesicles and release sites are statistically almost independent. A sensitivity analysis on the recovery rates based on the ODE formulation reveals that neither the vesicle nor the release site recovery step can be identified as the essential rate-limiting step but that the rate- limiting feature changes over the course of stimulation. Under sustained stimulation the dynamics given by the ODEs exhibit transient dynamics leading from an initial depression of the postsynaptic response to an asymptotic periodic orbit, while the individual trajectories of the stochastic jump model lack the oscillatory behavior an asymptotic periodicity of the ODE-solution. Y1 - 2023 U6 - https://doi.org/10.1016/j.mbs.2023.109023 VL - 362 ER - TY - JOUR A1 - Straube, Arthur A1 - Winkelmann, Stefanie A1 - Höfling, Felix T1 - Accurate reduced models for the pH oscillations in the urea-urease reaction confined to giant lipid vesicles JF - The Journal of Physical Chemistry B N2 - This theoretical study concerns a pH oscillator based on the urea-urease reaction confined to giant lipid vesicles. Under suitable conditions, differential transport of urea and hydrogen ion across the unilamellar vesicle membrane periodically resets the pH clock that switches the system from acid to basic, resulting in self-sustained oscillations. We analyse the structure of the phase flow and of the limit cycle, which controls the dynamics for giant vesicles and dominates the pronouncedly stochastic oscillations in small vesicles of submicrometer size. To this end, we derive reduced models, which are amenable to analytic treatments that are complemented by numerical solutions, and obtain the period and amplitude of the oscillations as well as the parameter domain, where oscillatory behavior persists. We show that the accuracy of these predictions is highly sensitive to the employed reduction scheme. In particular, we suggest an accurate two-variable model and show its equivalence to a three-variable model that admits an interpretation in terms of a chemical reaction network. The faithful modeling of a single pH oscillator appears crucial for rationalizing experiments and understanding communication of vesicles and synchronization of rhythms. Y1 - 2023 U6 - https://doi.org/10.1021/acs.jpcb.2c09092 VL - 127 IS - 13 SP - 2955 EP - 2967 ER - TY - JOUR A1 - Montefusco, Alberto A1 - Schütte, Christof A1 - Winkelmann, Stefanie T1 - A route to the hydrodynamic limit of a reaction-diffusion master equation using gradient structures JF - SIAM Journal on Applied Mathematics N2 - The reaction-diffusion master equation (RDME) is a lattice-based stochastic model for spatially resolved cellular processes. It is often interpreted as an approximation to spatially continuous reaction-diffusion models, which, in the limit of an infinitely large population, may be described by means of reaction-diffusion partial differential equations. Analyzing and understanding the relation between different mathematical models for reaction-diffusion dynamics is a research topic of steady interest. In this work, we explore a route to the hydrodynamic limit of the RDME which uses gradient structures. Specifically, we elaborate on a method introduced in [J. Maas and A. Mielke, J. Stat. Phys., 181 (2020), pp. 2257–2303] in the context of well-mixed reaction networks by showing that, once it is complemented with an appropriate limit procedure, it can be applied to spatially extended systems with diffusion. Under the assumption of detailed balance, we write down a gradient structure for the RDME and use the method in order to produce a gradient structure for its hydrodynamic limit, namely, for the corresponding RDPDE. Y1 - 2023 U6 - https://doi.org/10.1137/22M1488831 VL - 83 IS - 2 SP - 837 EP - 861 ER - TY - JOUR A1 - Steudle, Gesine A1 - Winkelmann, Stefanie A1 - Fürst, Steffen A1 - Wolf, Sarah T1 - Understanding Memory Mechanisms in in Socio-Technical Systems: the Case of an Agent-based Mobility Model N2 - This paper explores memory mechanisms in complex socio-technical systems, using a mobility demand model as an example case. We simplified a large-scale agent-based mobility model into a Markov process and discover that the mobility decision process is non-Markovian. This is due to its dependence on the system’s history, including social structure and local infrastructure, which evolve based on prior mobility decisions. To make the process Markovian, we extend the state space by incorporating two history-dependent components. Although our model is a very much reduced version of the original one, it remains too complex for the application of usual analytic methods. Instead, we employ simulations to examine the functionalities of the two history-dependent components. We think that the structure of the analyzed stochastic process is exemplary for many socio-technical, -economic, -ecological systems. Additionally, it exhibits analogies with the framework of extended evolution, which has previously been used to study cultural evolution. Y1 - 2024 U6 - https://doi.org/10.17617/2.3562016 ER - TY - JOUR A1 - Lücke, Marvin A1 - Winkelmann, Stefanie A1 - Heitzig, Jobst A1 - Molkenthin, Nora A1 - Koltai, Péter T1 - Learning interpretable collective variables for spreading processes on networks JF - Physical Review E N2 - Collective variables (CVs) are low-dimensional projections of high-dimensional system states. They are used to gain insights into complex emergent dynamical behaviors of processes on networks. The relation between CVs and network measures is not well understood and its derivation typically requires detailed knowledge of both the dynamical system and the network topology. In this Letter, we present a data-driven method for algorithmically learning and understanding CVs for binary-state spreading processes on networks of arbitrary topology. We demonstrate our method using four example networks: the stochastic block model, a ring-shaped graph, a random regular graph, and a scale-free network generated by the Albert-Barabási model. Our results deliver evidence for the existence of low-dimensional CVs even in cases that are not yet understood theoretically. Y1 - 2024 U6 - https://doi.org/10.1103/PhysRevE.109.L022301 VL - 2 SP - L022301 ET - 109 ER - TY - JOUR A1 - Thies, Arne A1 - Sunkara, Vikram A1 - Ray, Sourav A1 - Wulkow, Hanna A1 - Celik, M. Özgür A1 - Yergöz, Fatih A1 - Schütte, Christof A1 - Stein, Christoph A1 - Weber, Marcus A1 - Winkelmann, Stefanie T1 - Modelling altered signalling of G-protein coupled receptors in inflamed environment to advance drug design JF - Scientific Reports N2 - We previously reported the successful design, synthesis and testing of the prototype opioid painkiller NFEPP that does not elicit adverse side effects. The design process of NFEPP was based on mathematical modelling of extracellular interactions between G-protein coupled receptors (GPCRs) and ligands, recognizing that GPCRs function differently under pathological versus healthy conditions. We now present an additional and novel stochastic model of GPCR function that includes intracellular dissociation of G-protein subunits and modulation of plasma membrane calcium channels and their dependence on parameters of inflamed and healthy tissue (pH, radicals). The model is validated against in vitro experimental data for the ligands NFEPP and fentanyl at different pH values and radical concentrations. We observe markedly reduced binding affinity and calcium channel inhibition for NFEPP at normal pH compared to lower pH, in contrast to the effect of fentanyl. For increasing radical concentrations, we find enhanced constitutive G-protein activation but reduced ligand binding affinity. Assessing the different effects, the results suggest that, compared to radicals, low pH is a more important determinant of overall GPCR function in an inflamed environment. Future drug design efforts should take this into account. Y1 - 2023 U6 - https://doi.org/10.1038/s41598-023-27699-w VL - 13 IS - 607 ER - TY - GEN A1 - Ray, Sourav A1 - Thies, Arne A1 - Sunkara, Vikram A1 - Wulkow, Hanna A1 - Celik, Özgür A1 - Yergöz, Fatih A1 - Schütte, Christof A1 - Stein, Christoph A1 - Weber, Marcus A1 - Winkelmann, Stefanie T1 - Modelling altered signalling of G-protein coupled receptors in inflamed environment to advance drug design N2 - Initiated by mathematical modelling of extracellular interactions between G-protein coupled receptors (GPCRs) and ligands in normal versus diseased (inflamed) environments, we previously reported the successful design, synthesis and testing of the prototype opioid painkiller NFEPP that does not elicit adverse side effects. Uniquely, this design recognised that GPCRs function differently under pathological versus healthy conditions. We now present a novel stochastic model of GPCR function that includes intracellular dissociation of G-protein subunits and modulation of plasma membrane calcium channels associated with parameters of inflamed tissue (pH, radicals). By means of molecular dynamics simulations, we also assessed qualitative changes of the reaction rates due to additional disulfide bridges inside the GPCR binding pocket and used these rates for stochastic simulations of the corresponding reaction jump process. The modelling results were validated with in vitro experiments measuring calcium currents and G-protein activation. We found markedly reduced G-protein dissociation and calcium channel inhibition induced by NFEPP at normal pH, and enhanced constitutive G-protein activation but lower probability of ligand binding with increasing radical concentrations. These results suggest that, compared to radicals, low pH is a more important determinant of overall GPCR function in an inflamed environment. Future drug design efforts should take this into account. T3 - ZIB-Report - 21-19 Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-82797 SN - 1438-0064 ER - TY - JOUR A1 - Niemann, Jan-Hendrik A1 - Winkelmann, Stefanie A1 - Wolf, Sarah A1 - Schütte, Christof T1 - Agent-based modeling: Population limits and large timescales JF - Chaos: An Interdisciplinary Journal of Nonlinear Science N2 - Modeling, simulation and analysis of interacting agent systems is a broad field of research, with existing approaches reaching from informal descriptions of interaction dynamics to more formal, mathematical models. In this paper, we study agent-based models (ABMs) given as continuous-time stochastic processes and their pathwise approximation by ordinary and stochastic differential equations (ODEs and SDEs, respectively) for medium to large populations. By means of an appropriately adapted transfer operator approach we study the behavior of the ABM process on long time scales. We show that, under certain conditions, the transfer operator approach allows to bridge the gap between the pathwise results for large populations on finite timescales, i.e., the SDE limit model, and approaches built to study dynamical behavior on long time scales like large deviation theory. The latter provides a rigorous analysis of rare events including the associated asymptotic rates on timescales that scale exponentially with the population size. We demonstrate that it is possible to reveal metastable structures and timescales of rare events of the ABM process by finite-length trajectories of the SDE process for large enough populations. This approach has the potential to drastically reduce computational effort for the analysis of ABMs. Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-77309 SN - 1438-0064 VL - 31 IS - 3 ER - TY - JOUR A1 - Engel, Maximilian A1 - Olicón-Méndez, Guillermo A1 - Wehlitz, Nathalie A1 - Winkelmann, Stefanie T1 - Synchronization and random attractors in reaction jump processes JF - Journal of Dynamics and Differential Equations N2 - This work explores a synchronization-like phenomenon induced by common noise for continuous-time Markov jump processes given by chemical reaction networks. Based on Gillespie’s stochastic simulation algorithm, a corresponding random dynamical system is formulated in a two-step procedure, at first for the states of the embedded discrete-time Markov chain and then for the augmented Markov chain including random jump times. We uncover a time-shifted synchronization in the sense that—after some initial waiting time—one trajectory exactly replicates another one with a certain time delay. Whether or not such a synchronization behavior occurs depends on the combination of the initial states. We prove this partial time-shifted synchronization for the special setting of a birth-death process by analyzing the corresponding two-point motion of the embedded Markov chain and determine the structure of the associated random attractor. In this context, we also provide general results on existence and form of random attractors for discrete-time, discrete-space random dynamical systems. Y1 - 2024 U6 - https://doi.org/10.1007/s10884-023-10345-4 ER - TY - GEN A1 - Straube, Arthur A1 - Winkelmann, Stefanie A1 - Höfling, Felix T1 - Accurate reduced models for the pH oscillations in the urea-urease reaction confined to giant lipid vesicles N2 - Our theoretical study concerns an urea-urease-based pH oscillator confined to giant lipid vesicles. Under suitable conditions, differential transport of urea and hydrogen ion across the unilamellar vesicle membrane periodically resets the pH clock that switches the system from acid to basic, resulting in self-sustained oscillations. We analyse the structure of the limit cycle, which controls the dynamics for giant vesicles and dominates the strongly stochastic oscillations in small vesicles of submicrometer size. To this end, we derive reduced models, amenable to analytic treatments, and show that the accuracy of predictions, including the period of oscillations, is highly sensitive to the choice of the reduction scheme. In particular, we suggest an accurate two-variable model and show its equivalence to a three-variable model that admits an interpretation in terms of a chemical reaction network. The accurate description of a single pH oscillator appears crucial for rationalizing experiments and understanding communication of vesicles and synchronization of rhythms. T3 - ZIB-Report - 22-21 Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-88179 SN - 1438-0064 ER - TY - JOUR A1 - Montefusco, Alberto A1 - Helfmann, Luzie A1 - Okunola, Toluwani A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - Partial mean-field model for neurotransmission dynamics JF - Mathematical Biosciences N2 - This article addresses reaction networks in which spatial and stochastic effects are of crucial importance. For such systems, particle-based models allow us to describe all microscopic details with high accuracy. However, they suffer from computational inefficiency if particle numbers and density get too large. Alternative coarse-grained-resolution models reduce computational effort tremendously, e.g., by replacing the particle distribution by a continuous concentration field governed by reaction-diffusion PDEs. We demonstrate how models on the different resolution levels can be combined into hybrid models that seamlessly combine the best of both worlds, describing molecular species with large copy numbers by macroscopic equations with spatial resolution while keeping the stochastic-spatial particle-based resolution level for the species with low copy numbers. To this end, we introduce a simple particle-based model for the binding dynamics of ions and vesicles at the heart of the neurotransmission process. Within this framework, we derive a novel hybrid model and present results from numerical experiments which demonstrate that the hybrid model allows for an accurate approximation of the full particle-based model in realistic scenarios. Y1 - 2024 U6 - https://doi.org/10.1016/j.mbs.2024.109143 VL - 369 ER -