• Treffer 2 von 3
Zurück zur Trefferliste

Uncovering Instabilities in Variational-Quantum Deep Q-Networks

  • Deep Reinforcement Learning (RL) has considerably advanced over the past decade. At the same time, state-of-the-art RL algorithms require a large computational budget in terms of training time to converge. Recent work has started to approach this problem through the lens of quantum computing, which promises theoretical speed-ups for several traditionally hard tasks. In this work, we examine a class of hybrid quantumclassical RL algorithms that we collectively refer to as variational quantum deep Q-networks (VQ-DQN). We show that VQ-DQN approaches are subject to instabilities that cause the learned policy to diverge, study the extent to which this afflicts reproduciblity of established results based on classical simulation, and perform systematic experiments to identify potential explanations for the observed instabilities. Additionally, and in contrast to most existing work on quantum reinforcement learning, we execute RL algorithms on an actual quantum processing unit (an IBM Quantum Device) and investigate differences in behaviour between simulated and physical quantum systems that suffer from implementation deficiencies. Our experiments show that, contrary to opposite claims in the literature, it cannot be conclusively decided if known quantum approaches, even if simulated without physical imperfections, can provide an advantage as compared to classical approaches. Finally, we provide a robust, universal and well-tested implementation of VQ-DQN as a reproducible testbed for future experiments.

Metadaten exportieren

Weitere Dienste

Teilen auf Twitter Suche bei Google Scholar Anzahl der Zugriffe auf dieses Dokument
Metadaten
Verfasserangaben:Maja Franz, Lucas Wolf, Maniraman Periyasamy, Christian Ufrecht, Daniel D. Scherer, Axel Plinge, Christopher Mutschler, Wolfgang MauererORCiDGND
DOI:https://doi.org/10.1016/j.jfranklin.2022.08.021
ISSN:0016-0032
Titel des übergeordneten Werkes (Englisch):Journal of the Franklin Institute
Verlag:Elsevier
Dokumentart:Artikel aus einer Zeitschrift/Zeitung
Sprache der Veröffentlichung:Englisch
Jahr der Veröffentlichung:2022
Datum der Freischaltung:11.04.2022
Freies Schlagwort / Tag:Artificial Intelligence; Computer Science; Quantum Physics
Auflage:In Press, Corrected Proof
Bemerkung:
Corresponding author: Maja Franz
Fakultäten / Institute / Einrichtungen:Fakultät Informatik und Mathematik
Fakultät Informatik und Mathematik / Labor für Digitalisierung (LFD)
Begutachtungsstatus:peer-reviewed
Forschungsschwerpunkt:Digitalisierung
OpenAccess Publikationsweg:Hybrid Open Access - OA-Veröffentlichung in einer Subskriptionszeitschrift/-medium
Corresponding author der OTH Regensburg
Lizenz (Deutsch):Creative Commons - CC BY - Namensnennung 4.0 International