TY - JOUR A1 - Gabor, Thomas A1 - Sedlmeier, Andreas A1 - Phan, Thomy A1 - Ritz, Fabian A1 - Kiermeier, Marie A1 - Belzner, Lenz A1 - Kempter, Bernhard A1 - Klein, Cornel A1 - Sauer, Horst A1 - Schmid, Reiner A1 - Wieghardt, Jan A1 - Zeller, Marc A1 - Linnhoff-Popien, Claudia T1 - The scenario coevolution paradigm BT - adaptive quality assurance for adaptive systems JF - International Journal on Software Tools for Technology Transfer N2 - Systems are becoming increasingly more adaptive, using techniques like machine learning to enhance their behavior on their own rather than only through human developers programming them. We analyze the impact the advent of these new techniques has on the discipline of rigorous software engineering, especially on the issue of quality assurance. To this end, we provide a general description of the processes related to machine learning and embed them into a formal framework for the analysis of adaptivity, recognizing that to test an adaptive system a new approach to adaptive testing is necessary. We introduce scenario coevolution as a design pattern describing how system and test can work as antagonists in the process of software evolution. While the general pattern applies to large-scale processes (including human developers further augmenting the system), we show all techniques on a smaller-scale example of an agent navigating a simple smart factory. We point out new aspects in software engineering for adaptive systems that may be tackled naturally using scenario coevolution. This work is a substantially extended take on Gabor et al. (International symposium on leveraging applications of formal methods, Springer, pp 137–154, 2018). UR - https://doi.org/10.1007/s10009-020-00560-5 Y1 - 2020 UR - https://doi.org/10.1007/s10009-020-00560-5 UR - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:573-13929 SN - 1433-2787 VL - 22 IS - 4 SP - 457 EP - 476 PB - Springer CY - Berlin ER - TY - CHAP A1 - Sedlmeier, Andreas A1 - Gabor, Thomas A1 - Phan, Thomy A1 - Belzner, Lenz A1 - Linnhoff-Popien, Claudia ED - Rocha, Ana ED - Steels, Luc ED - Herik, Jaap van den T1 - Uncertainty-based out-of-distribution classification in deep reinforcement learning T2 - Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART N2 - Robustness to out-of-distribution (OOD) data is an important goal in building reliable machine learning systems. As a first step towards a solution, we consider the problem of detecting such data in a value-based deep reinforcement learning (RL) setting. Modelling this problem as a one-class classification problem, we propose a framework for uncertainty-based OOD classification: UBOOD. It is based on the effect that an agent’s epistemic uncertainty is reduced for situations encountered during training (in-distribution), and thus lower than for unencountered (OOD) situations. Being agnostic towards the approach used for estimating epistemic uncertainty, combinations with different uncertainty estimation methods, e.g. approximate Bayesian inference methods or ensembling techniques are possible. Evaluation shows that the framework produces reliable classification results when combined with ensemble-based estimators, while the combination with concrete dropout-based estimators fails to r eliably detect OOD situations. UR - https://doi.org/10.5220/0008949905220529 KW - Uncertainty in AI KW - Out-of-Distribution Classification KW - Deep Reinforcement Learning Y1 - 2020 UR - https://doi.org/10.5220/0008949905220529 SN - 978-989-758-395-7 SN - 2184-433X SP - 522 EP - 529 PB - SciTePress CY - Setúbal ER - TY - CHAP A1 - Phan, Thomy A1 - Belzner, Lenz A1 - Gabor, Thomas A1 - Sedlmeier, Andreas A1 - Ritz, Fabian A1 - Linnhoff-Popien, Claudia T1 - Resilient multi-agent reinforcement learning with adversarial value decomposition T2 - AAAI-21 / IAAI-21 / EAAI-21 Proceedings UR - https://ojs.aaai.org/index.php/AAAI/article/view/17348 KW - Multiagent Learning KW - Adversarial Learning & Robustness KW - Adversarial Agents KW - Reinforcement Learning Y1 - 2021 UR - https://ojs.aaai.org/index.php/AAAI/article/view/17348 SN - 978-1-57735-866-4 SN - 2374-3468 VL - 35 IS - 13 SP - 11308 EP - 11316 PB - AAAI Press CY - Palo Alto (CA) ER - TY - CHAP A1 - Feld, Sebastian A1 - Sedlmeier, Andreas A1 - Friedrich, Markus A1 - Franz, Jan A1 - Belzner, Lenz T1 - Bayesian Surprise in Indoor Environments T2 - SIGSPATIAL '19 : Proceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems UR - https://doi.org/10.1145/3347146.3359358 KW - Bayesian Surprise KW - Novelty KW - Salience KW - Isovist Analysis KW - Indoor Location- Based Service KW - Indoor LBS KW - Indoor Navigation KW - Trajectory Characterization Y1 - 2019 UR - https://doi.org/10.1145/3347146.3359358 SN - 978-1-4503-6909-1 PB - ACM CY - New York ER - TY - CHAP A1 - Feld, Sebastian A1 - Sedlmeier, Andreas A1 - Illium, Steffen A1 - Belzner, Lenz T1 - Trajectory annotation using sequences of spatial perception T2 - SIGSPATIAL '18 : Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems UR - https://doi.org/10.1145/3274895.3274968 KW - Spatial Syntax KW - Isovist Analysis KW - Geospatial Trajectories KW - Indoor Navigation KW - Auto-Encoder KW - Artificial Neural Networks Y1 - 2018 UR - https://doi.org/10.1145/3274895.3274968 SN - 978-1-4503-5889-7 SP - 329 EP - 338 PB - ACM CY - New York ER - TY - JOUR A1 - Sedlmeier, Andreas A1 - Gabor, Thomas A1 - Phan, Thomy A1 - Belzner, Lenz T1 - Uncertainty-Based Out-of-Distribution Detection in Deep Reinforcement Learning JF - Digitale Welt UR - https://doi.org/10.1007/s42354-019-0238-z Y1 - 2019 UR - https://doi.org/10.1007/s42354-019-0238-z SN - 2569-1996 VL - 4 IS - 1 SP - 74 EP - 78 PB - Digitale Welt Academy CY - München ER -