## Markov Control with Rare State Observation: Average Optimality

Please always quote using this URN: urn:nbn:de:0297-zib-60981

- This paper investigates the criterion of long-term average costs for a Markov decision process (MDP) which is not permanently observable. Each observation of the process produces a fixed amount of \textit{information costs} which enter the considered performance criterion and preclude from arbitrarily frequent state testing. Choosing the \textit{rare} observation times is part of the control procedure. In contrast to the theory of partially observable Markov decision processes, we consider an arbitrary continuous-time Markov process on a finite state space without further restrictions on the dynamics or the type of interaction. Based on the original Markov control theory, we redefine the control model and the average cost criterion for the setting of information costs. We analyze the constant of average costs for the case of ergodic dynamics and present an optimality equation which characterizes the optimal choice of control actions and observation times. For this purpose, we construct an equivalent freely observable MDP and translate the well-known results from the original theory to the new setting.

Author: | Stefanie Winkelmann |
---|---|

Document Type: | ZIB-Report |

Tag: | Markov decision process; average optimality; information costs; partial observability |

MSC-Classification: | 49-XX CALCULUS OF VARIATIONS AND OPTIMAL CONTROL; OPTIMIZATION [See also 34H05, 34K35, 65Kxx, 90Cxx, 93-XX] / 49Nxx Miscellaneous topics / 49N30 Problems with incomplete information [See also 93C41] |

90-XX OPERATIONS RESEARCH, MATHEMATICAL PROGRAMMING / 90Cxx Mathematical programming [See also 49Mxx, 65Kxx] / 90C39 Dynamic programming [See also 49L20] | |

90-XX OPERATIONS RESEARCH, MATHEMATICAL PROGRAMMING / 90Cxx Mathematical programming [See also 49Mxx, 65Kxx] / 90C40 Markov and semi-Markov decision processes | |

CCS-Classification: | G. Mathematics of Computing |

Date of first Publication: | 2016/11/18 |

Series (Serial Number): | ZIB-Report (16-59) |

ISSN: | 1438-0064 |

Published in: | Markov Processes and Related Fields 23 (2017) pp. 1-34 |