Überblick Statistik: Seitenaufrufe (grau) und PDF-Downloads (blau)

A Novel Exponential Continuous Learning Rate Adaption Gradient Descent Optimization Method

  • We present two novel, fast gradient based optimizer algorithms with dynamic learning rate. The main idea is to adapt the learning rate α by situational awareness, mainly striving for orthogonal neighboring gradients. The method has a high success and fast convergence rate and relies much less on hand-tuned hyper-parameters, providing greater universality. It scales linearly (of order O(n)) with dimension and is rotation invariant, thereby overcoming known limitations. The method is presented in two variants C2Min and P2Min, with slightly different control. Their impressive performance is demonstrated by experiments on several benchmark data-sets (ranging from MNIST to Tiny ImageNet) against the state-of-the-art optimizers Adam and Lion.

Download full text files

Export metadata

Additional Services

Search Google Scholar
Metadaten
URN:urn:nbn:de:kobv:526-opus4-20785
Publisher DOI:https://doi.org/10.52825/th-wildau-ensp.v2i.2939
Author:Alexander KleinsorgeORCiD, Alexander FauckORCiDGND, Stefan KupperORCiD
Parent Title (German):Wildauer Konferenz für Künstliche Intelligenz 2025 (WiKKI25)
Series (Serial Number):TH Wildau Engineering and Natural Sciences Proceedings (2)
Document Type:Conference Proceeding
Language:English
Year of Publication:2025
Publisher:TIB Open Publishing
Place of publication:Hannover
Faculties an central facilities:Fachbereich Ingenieur- und Naturwissenschaften
Editor:Jörg Reiff-StephanORCiDGND, Anja Beuster
Publishing Institution:Technische Hochschule Wildau
Tag:neural network; optimizer; training
Source:Kleinsorge, A., Fauck, A., & Kupper, S. (2025). A Novel Exponential Continuous Learning Rate Adaption Gradient Descent Optimization Method. TH Wildau Engineering and Natural Sciences Proceedings , 2. https://doi.org/10.52825/th-wildau-ensp.v2i.2939
Dewey Decimal Classification:0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 006 Spezielle Computerverfahren
5 Naturwissenschaften und Mathematik / 51 Mathematik / 519 Wahrscheinlichkeiten, angewandte Mathematik
Funding:Publikationsfonds für Open-Access-Monografien des Landes Brandenburg
Licence (German):Creative Commons - CC BY - Namensnennung 4.0 International
Release Date:2025/09/26
Verstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.