TY - CHAP A1 - Bock, Sebastian A1 - Weiß, Martin Georg T1 - Non-Convergence and Limit Cycles in the Adam Optimizer T2 - Proceedings of the 28th International Conference on Artificial Neural Networks, 2019, Munich, Germany, September 17-19 N2 - One of the most popular training algorithms for deep neural networks is the Adaptive Moment Estimation (Adam) introduced by Kingma and Ba. Despite its success in many applications there is no satisfactory convergence analysis: only local convergence can be shown for batch mode under some restrictions on the hyperparameters, counterexamples exist for incremental mode. Recent results show that for simple quadratic objective functions limit cycles of period 2 exist in batch mode, but only for atypical hyperparameters, and only for the algorithm without bias correction. We extend the convergence analysis to all choices of the hyperparameters for quadratic functions. This finally answers the question of convergence for Adam in batch mode to the negative. We analyze the stability of these limit cycles and relate our analysis to other results where approximate convergence was shown, but under the additional assumption of bounded gradients which does not apply to quadratic functions. The investigation heavily relies on the use of computer algebra due to the complexity of the equations. KW - Adam optimizer KW - Convergence KW - Computer algebra KW - Dynamical system KW - Limit cycle KW - Neuronales Netz KW - Maschinelles Lernen KW - Optimierungsalgorithmus KW - Konvergenz 〈Informationstechnik〉 KW - Computeralgebra Y1 - 2019 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-490 UR - https://doi.org/10.1007/978-3-030-30484-3_20 SP - 232 EP - 243 ER - TY - CHAP A1 - Bock, Sebastian A1 - Weiß, Martin Georg T1 - A Proof of Local Convergence for the Adam Optimizer T2 - Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), 2019, Budapest, Hungary, July 14-19 N2 - Adaptive Moment Estimation (Adam) is a very popular training algorithm for deep neural networks, implemented in many machine learning frameworks. To the best of the authors knowledge no complete convergence analysis exists for Adam. The contribution of this paper is a method for the local convergence analysis in batch mode for a deterministic fixed training set, which gives necessary conditions for the hyperparameters of the Adam algorithm. Due to the local nature of the arguments the objective function can be non-convex but must be at least twice continuously differentiable. KW - Non-convex optimization KW - Adam optimizer KW - Convergence KW - Momentum method KW - Dynamical system KW - Fixed point KW - Neuronales Netz KW - Maschinelles Lernen KW - Optimierungsalgorithmus KW - Konvergenz 〈Informationstechnik〉 Y1 - 2019 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-501 UR - https://doi.org/10.1109/IJCNN.2019.8852239 VL - 2019 SP - 1 EP - 8 ER - TY - CHAP A1 - Weiß, Martin Georg ED - Holderbaum, William ED - Selig, J. M. T1 - Optimization of Cartesian Tasks with Configuration Selection T2 - 2nd IMA Conference on Mathematics of Robotics: online September 8–10, 2021 N2 - A basic task in the design of an industrial robot application is the relative placement of robot and workpiece. Process points are defined in Cartesian coordinates relative to the workpiece coordinate system, and the workpiece has to be located such that the robot can reach all points. Finding such a location is still an iterative procedure based on the developers’ intuition. One difficulty is the choice of one of the several solutions of the backward transform of a typical 6R robot. We present a novel algorithm that simultaneously optimizes the workpiece location and the robot configuration at all process points using higher order optimization algorithms. A key ingredient is the extension of the robot with a virtual prismatic axis. The practical feasibility of the approach is shown with an example using a commercial industrial robot. KW - Configuration KW - Differentiable optimization KW - Virtual axis Y1 - 2022 SN - 978-3-030-91351-9 U6 - https://doi.org/10.1007/978-3-030-91352-6_16 SP - 153 EP - 160 PB - Springer International Publishing CY - Cham ER - TY - CHAP A1 - Weiß, Martin Georg A1 - Volbert, Klaus ED - Falter, Thomas T1 - Intelligente Steuerung von Industrierobotern T2 - Zweite OTH-Clusterkonferenz 18. Januar 2017 Techbase, Regensburg Y1 - 2017 CY - Regensburg ER - TY - CHAP A1 - Weiß, Martin Georg ED - Lenarcic, Jadran ED - Parenti-Castelli, Vincenzo T1 - Optimal Object Placement Using a Virtual Axis T2 - Advances in Robot Kinematics, ARK 2018 N2 - A basic task in the design of a robotic production cell is the relative placement of robot and workpiece. The fundamental requirement is that the robot can reach all process positions; only then one can think further optimization. Therefore an algorithm that automatically places an object into the workspace is very desirable. However many iterative optimization algorithms cannot guarantee that all intermediate steps are reachable, resulting in complicated procedures. We present a novel approach which extends a robot by a virtual prismatic joint - which measures the distance to the workspace - such that any TCP frames are reachable. This allows higher order nonlinear programming algorithms to be used for placement of an object alone as well as the optimal placement under some differentiable criterion. KW - Inverse kinematics KW - Nonsmooth optimization KW - Optimization KW - Virtual joint KW - Workspace Cartesian tasks Y1 - 2019 SN - 978-3-319-93187-6 U6 - https://doi.org/10.1007/978-3-319-93188-3_14 VL - 8 SP - 116 EP - 123 PB - Springer International Publishing CY - Cham ER -