TY - GEN A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - Hybrid Models for Chemical Reaction Networks: Multiscale Theory and Application to Gene Regulatory Systems N2 - Well-mixed stochastic chemical kinetics are properly modelled by the chemical master equation (CME) and associated Markov jump processes in molecule number space. If the reactants are present in large amounts, however, corresponding simulations of the stochastic dynamics become computationally expensive and model reductions are demanded. The classical model reduction approach uniformly rescales the overall dynamics to obtain deterministic systems characterized by ordinary differential equations, the well-known mass action reaction rate equations. For systems with multiple scales there exist hybrid approaches that keep parts of the system discrete while another part is approximated either using Langevin dynamics or deterministically. This paper aims at giving a coherent overview of the different hybrid approaches, focusing on their basic concepts and the relation between them. We derive a novel general description of such hybrid models that allows to express various forms by one type of equation. We also check in how far the approaches apply to model extensions of the CME for dynamics which do not comply with the central well-mixed condition and require some spatial resolution. A simple but meaningful gene expression system with negative self-regulation is analysed to illustrate the different approximation qualities of some of the hybrid approaches discussed. T3 - ZIB-Report - 17-29 Y1 - 2017 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-64264 ER - TY - GEN A1 - Winkelmann, Stefanie A1 - Schütte, Christof A1 - von Kleist, Max T1 - Markov Control Processes with Rare State Observation: Theory and Application to Treatment Scheduling in HIV-1 N2 - Markov Decision Processes (MDP) or Partially Observable MDPs (POMDP) are used for modelling situations in which the evolution of a process is partly random and partly controllable. These MDP theories allow for computing the optimal control policy for processes that can continuously or frequently be observed, even if only partially. However, they cannot be applied if state observation is very costly and therefore rare (in time). We present a novel MDP theory for rare, costly observations and derive the corresponding Bellman equation. In the new theory, state information can be derived for a particular cost after certain, rather long time intervals. The resulting information costs enter into the total cost and thus into the optimization criterion. This approach applies to many real world problems, particularly in the medical context, where the medical condition is examined rather rarely because examination costs are high. At the same time, the approach allows for efficient numerical realization. We demonstrate the usefulness of the novel theory by determining, from the national economic perspective, optimal therapeutic policies for the treatment of the human immunodefficiency virus (HIV) in resource-rich and resource-poor settings. Based on the developed theory and models, we discover that available drugs may not be utilized efficiently in resource-poor settings due to exorbitant diagnostic costs. T3 - ZIB-Report - 13-34 KW - information costs KW - hidden state KW - bellmann equation KW - optimal therapeutic policies KW - diagnostic frequency KW - resource-poor KW - resource-rich Y1 - 2013 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-41955 SN - 1438-0064 ER - TY - GEN A1 - Helfmann, Luzie A1 - Djurdjevac Conrad, Natasa A1 - Djurdjevac, Ana A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - From interacting agents to density-based modeling with stochastic PDEs N2 - Many real-world processes can naturally be modeled as systems of interacting agents. However, the long-term simulation of such agent-based models is often intractable when the system becomes too large. In this paper, starting from a stochastic spatio-temporal agent-based model (ABM), we present a reduced model in terms of stochastic PDEs that describes the evolution of agent number densities for large populations. We discuss the algorithmic details of both approaches; regarding the SPDE model, we apply Finite Element discretization in space which not only ensures efficient simulation but also serves as a regularization of the SPDE. Illustrative examples for the spreading of an innovation among agents are given and used for comparing ABM and SPDE models. T3 - ZIB-Report - 19-21 Y1 - 2019 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-73456 SN - 1438-0064 ER - TY - GEN A1 - Winkelmann, Stefanie T1 - Markov Control with Rare State Observation: Average Optimality N2 - This paper investigates the criterion of long-term average costs for a Markov decision process (MDP) which is not permanently observable. Each observation of the process produces a fixed amount of \textit{information costs} which enter the considered performance criterion and preclude from arbitrarily frequent state testing. Choosing the \textit{rare} observation times is part of the control procedure. In contrast to the theory of partially observable Markov decision processes, we consider an arbitrary continuous-time Markov process on a finite state space without further restrictions on the dynamics or the type of interaction. Based on the original Markov control theory, we redefine the control model and the average cost criterion for the setting of information costs. We analyze the constant of average costs for the case of ergodic dynamics and present an optimality equation which characterizes the optimal choice of control actions and observation times. For this purpose, we construct an equivalent freely observable MDP and translate the well-known results from the original theory to the new setting. T3 - ZIB-Report - 16-59 KW - Markov decision process KW - partial observability KW - average optimality KW - information costs Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-60981 SN - 1438-0064 ER - TY - GEN A1 - Winkelmann, Stefanie A1 - Schütte, Christof T1 - The spatiotemporal master equation: approximation of reaction-diffusion dynamics via Markov state modeling N2 - Accurate modeling and numerical simulation of reaction kinetics is a topic of steady interest. We consider the spatiotemporal chemical master equation (ST-CME) as a model for stochastic reaction-diffusion systems that exhibit properties of metastability. The space of motion is decomposed into metastable compartments and diffusive motion is approximated by jumps between these compartments. Treating these jumps as first-order reactions, simulation of the resulting stochastic system is possible by the Gillespie method. We present the theory of Markov state models (MSM) as a theoretical foundation of this intuitive approach. By means of Markov state modeling, both the number and shape of compartments and the transition rates between them can be determined. We consider the ST-CME for two reaction-diffusion systems and compare it to more detailed models. Moreover, a rigorous formal justification of the ST-CME by Galerkin projection methods is presented. T3 - ZIB-Report - 16-60 KW - reaction-diffusion KW - stochastic chemical kinetics KW - chemical master equation Y1 - 2016 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-60999 SN - 1438-0064 ER - TY - GEN A1 - Ernst, Ariane A1 - Schütte, Christof A1 - Sigrist, Stephan A1 - Winkelmann, Stefanie T1 - Variance of filtered signals: Characterization for linear reaction networks and application to neurotransmission dynamics N2 - Neurotransmission at chemical synapses relies on the calcium-induced fusion of synaptic vesicles with the presynaptic membrane. The distance to the calcium channels determines the release probability and thereby the postsynaptic signal. Suitable models of the process need to capture both the mean and the variance observed in electrophysiological measurements of the postsynaptic current. In this work, we propose a method to directly compute the exact first- and second-order moments for signals generated by a linear reaction network under convolution with an impulse response function, rendering computationally expensive numerical simulations of the underlying stochastic counting process obsolete. We show that the autocorrelation of the process is central for the calculation of the filtered signal’s second-order moments, and derive a system of PDEs for the cross-correlation functions (including the autocorrelations) of linear reaction networks with time-dependent rates. Finally, we employ our method to efficiently compare different spatial coarse graining approaches for a specific model of synaptic vesicle fusion. Beyond the application to neurotransmission processes, the developed theory can be applied to any linear reaction system that produces a filtered stochastic signal. T3 - ZIB-Report - 21-15 KW - linear reaction networks KW - cross-correlation KW - neurotransmission Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-82674 SN - 1438-0064 ER - TY - GEN A1 - Ray, Sourav A1 - Thies, Arne A1 - Sunkara, Vikram A1 - Wulkow, Hanna A1 - Celik, Özgür A1 - Yergöz, Fatih A1 - Schütte, Christof A1 - Stein, Christoph A1 - Weber, Marcus A1 - Winkelmann, Stefanie T1 - Modelling altered signalling of G-protein coupled receptors in inflamed environment to advance drug design N2 - Initiated by mathematical modelling of extracellular interactions between G-protein coupled receptors (GPCRs) and ligands in normal versus diseased (inflamed) environments, we previously reported the successful design, synthesis and testing of the prototype opioid painkiller NFEPP that does not elicit adverse side effects. Uniquely, this design recognised that GPCRs function differently under pathological versus healthy conditions. We now present a novel stochastic model of GPCR function that includes intracellular dissociation of G-protein subunits and modulation of plasma membrane calcium channels associated with parameters of inflamed tissue (pH, radicals). By means of molecular dynamics simulations, we also assessed qualitative changes of the reaction rates due to additional disulfide bridges inside the GPCR binding pocket and used these rates for stochastic simulations of the corresponding reaction jump process. The modelling results were validated with in vitro experiments measuring calcium currents and G-protein activation. We found markedly reduced G-protein dissociation and calcium channel inhibition induced by NFEPP at normal pH, and enhanced constitutive G-protein activation but lower probability of ligand binding with increasing radical concentrations. These results suggest that, compared to radicals, low pH is a more important determinant of overall GPCR function in an inflamed environment. Future drug design efforts should take this into account. T3 - ZIB-Report - 21-19 Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-82797 SN - 1438-0064 ER - TY - JOUR A1 - Niemann, Jan-Hendrik A1 - Winkelmann, Stefanie A1 - Wolf, Sarah A1 - Schütte, Christof T1 - Agent-based modeling: Population limits and large timescales JF - Chaos: An Interdisciplinary Journal of Nonlinear Science N2 - Modeling, simulation and analysis of interacting agent systems is a broad field of research, with existing approaches reaching from informal descriptions of interaction dynamics to more formal, mathematical models. In this paper, we study agent-based models (ABMs) given as continuous-time stochastic processes and their pathwise approximation by ordinary and stochastic differential equations (ODEs and SDEs, respectively) for medium to large populations. By means of an appropriately adapted transfer operator approach we study the behavior of the ABM process on long time scales. We show that, under certain conditions, the transfer operator approach allows to bridge the gap between the pathwise results for large populations on finite timescales, i.e., the SDE limit model, and approaches built to study dynamical behavior on long time scales like large deviation theory. The latter provides a rigorous analysis of rare events including the associated asymptotic rates on timescales that scale exponentially with the population size. We demonstrate that it is possible to reveal metastable structures and timescales of rare events of the ABM process by finite-length trajectories of the SDE process for large enough populations. This approach has the potential to drastically reduce computational effort for the analysis of ABMs. Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-77309 SN - 1438-0064 VL - 31 IS - 3 ER - TY - GEN A1 - Straube, Arthur A1 - Winkelmann, Stefanie A1 - Höfling, Felix T1 - Accurate reduced models for the pH oscillations in the urea-urease reaction confined to giant lipid vesicles N2 - Our theoretical study concerns an urea-urease-based pH oscillator confined to giant lipid vesicles. Under suitable conditions, differential transport of urea and hydrogen ion across the unilamellar vesicle membrane periodically resets the pH clock that switches the system from acid to basic, resulting in self-sustained oscillations. We analyse the structure of the limit cycle, which controls the dynamics for giant vesicles and dominates the strongly stochastic oscillations in small vesicles of submicrometer size. To this end, we derive reduced models, amenable to analytic treatments, and show that the accuracy of predictions, including the period of oscillations, is highly sensitive to the choice of the reduction scheme. In particular, we suggest an accurate two-variable model and show its equivalence to a three-variable model that admits an interpretation in terms of a chemical reaction network. The accurate description of a single pH oscillator appears crucial for rationalizing experiments and understanding communication of vesicles and synchronization of rhythms. T3 - ZIB-Report - 22-21 Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-88179 SN - 1438-0064 ER -