Continual learning for neural regression networks to cope with concept drift in industrial processes using convex optimisation
- Process models in industrial applications, e.g. predictive maintenance or automation, are subject to both divergence from the underlying system due to their time-variant nature and to high complexity resulting from a wide operational range being covered. Hence, regression models require high accuracy for the present system state and at the same time need to be valid across the whole system operating space. While accuracy for the current system state can be gained by updating the model on the current data, the overall validity must often be retrieved from historical or design data. We propose a method to find an appropriate compromise for these two demands. A pre-trained artificial neural network (ANN) is continually updated on the current sensor data stream using convex optimisation. Thus, a unique and optimal solution is generated in each update step, while robust regression accuracy on the domain that is not covered by the arriving data subset is maintained. This is achieved byProcess models in industrial applications, e.g. predictive maintenance or automation, are subject to both divergence from the underlying system due to their time-variant nature and to high complexity resulting from a wide operational range being covered. Hence, regression models require high accuracy for the present system state and at the same time need to be valid across the whole system operating space. While accuracy for the current system state can be gained by updating the model on the current data, the overall validity must often be retrieved from historical or design data. We propose a method to find an appropriate compromise for these two demands. A pre-trained artificial neural network (ANN) is continually updated on the current sensor data stream using convex optimisation. Thus, a unique and optimal solution is generated in each update step, while robust regression accuracy on the domain that is not covered by the arriving data subset is maintained. This is achieved by introducing a data management system to provide some historical data, constraining the optimisation problem and manipulating the architecture of the ANN. Models updated with this method show reasonable stability but display plastic behaviour at the current operating point.…
Author: | Wolfgang Grote-Ramm, David Lanuschny, Finn Lorenzen, Marcel Oliveira Brito, Felix Schönig |
---|---|
Fachbereich/Einrichtung: | Hochschule Düsseldorf / Fachbereich - Maschinenbau und Verfahrenstechnik |
Document Type: | Article |
Year of Completion: | 2023 |
Language of Publication: | English |
Publisher: | Elsevier |
Place of publication: | New York |
Parent Title (English): | Engineering Applications of Artificial Intelligence |
Volume: | 120 |
Article Number: | 105927 |
Page Number: | 10 |
DOI: | https://doi.org/10.1016/j.engappai.2023.105927 |
ISSN: | 1873-6769 |
Tag: | Catastrophic forgetting; Concept drift; Continual learning; Convex optimization |
Dewey Decimal Classification: | 6 Technik, Medizin, angewandte Wissenschaften / 62 Ingenieurwissenschaften / 620 Ingenieurwissenschaften und zugeordnete Tätigkeiten |
Licence (German): | keine Lizenz - nur Metadaten |
Release Date: | 2024/01/24 |