Applying Time Series Extrinsic Regression to Parameter Estimation Problems for Dynamic Models - an Alternative for Gradient-Free Approaches?
- Time series analysis is a well-established field within the machine learning community, with two prominent applications being time-series forecasting, i.e., surrogate models, predicting the next time step for the systems outputs, and time-series classification, where complete timeseries are mapped to discrete labels, e.g. a sensor is either working or defective. Time-Series Extrinsic Regression (TSER), however, is a method for predicting continuous, time-invariant variables from a time series by learning the relation between these underlying parameters and the complete dynamic time series of the outputs without focusing on the recent states. E.g., it can be used to predict the heart rate based on an ECG signal. TSER as a research field was only established in 2021, but it is gaining traction ever since and it is used e.g. in the field of manufacturing technology to predict steel surface roughness from laser reflection measurements. It is applied, when there are no modelsTime series analysis is a well-established field within the machine learning community, with two prominent applications being time-series forecasting, i.e., surrogate models, predicting the next time step for the systems outputs, and time-series classification, where complete timeseries are mapped to discrete labels, e.g. a sensor is either working or defective. Time-Series Extrinsic Regression (TSER), however, is a method for predicting continuous, time-invariant variables from a time series by learning the relation between these underlying parameters and the complete dynamic time series of the outputs without focusing on the recent states. E.g., it can be used to predict the heart rate based on an ECG signal. TSER as a research field was only established in 2021, but it is gaining traction ever since and it is used e.g. in the field of manufacturing technology to predict steel surface roughness from laser reflection measurements. It is applied, when there are no models available.
Parameter Estimation (PE) is a common task in chemical engineering. It is used to adjust model parameters to better fit existing dynamic models to experimental time series data. This becomes more challenging in higher dimensions and for dynamic systems, where sensitivity and identifiability may change over time. There already exists a multitude of algorithms to solve the problem, including second-order methods that leverage information from Jacobian and Hessian matrices, as well as gradient-free optimization techniques, such as particle swarm optimization (PSO) or simulated annealing. However, with the growing establishment of machine learning (ML) in an increasing number of domains, the question arises as to whether, and if so, how, ML in general and TSER in particular can be employed to solve PE problems.
This study marks the first application of TSER to PE problems. A comparative analysis is conducted between TSER and PSO, in terms of prediction accuracy, computational cost and data efficiency. We investigate, whether it is viable to use TSER, when there is a model available.
Our methodology to regress model parameters via ML builds on the typical assumption, that a structurally correct and rigorous model, which can be simulated at low cost, is available. At the beginning, the boundaries of the parameter space are defined. This space is then sampled using Sobol sequences and the model is simulated. The resulting trajectories, along with their corresponding parameters, constitute the training data set. These trajectories are transformed through application of the “RandOm Convolutional Kernel Transform” method resulting in novel features, which are subsequently used to train the regressor model. This regressor returns predictions for the parameters.
In a case study, the method is applied to predict the heat transfer and kinetic parameters of a batch reactor based on simulated data. However, real measurements are often not continuously available, but are taken only at rare, discrete points in time, and different variables are measured at different, asynchronous intervals. This is also mimicked in the synthetic training data, so the influence of heterogeneity on the results can be shown and over- or undersampling strategies are applied to counteract the effect.…


| Autor*innen: | Torben TalisORCiD |
|---|---|
| Koautor*innen: | John Paul GerakisORCiD, Christian HoffmannORCiD, Jens-Uwe repkeORCiD |
| Dokumenttyp: | Vortrag |
| Veröffentlichungsform: | Präsentation |
| Sprache: | Englisch |
| Jahr der Erstveröffentlichung: | 2025 |
| Organisationseinheit der BAM: | 2 Prozess- und Anlagensicherheit |
| 2 Prozess- und Anlagensicherheit / 2.2 Prozesssimulation | |
| DDC-Klassifikation: | Technik, Medizin, angewandte Wissenschaften / Chemische Verfahrenstechnik / Chemische Verfahrenstechnik |
| Freie Schlagwörter: | Machine Learning; Parameter Estimation; Time Series Extrinsic Regression |
| Themenfelder/Aktivitätsfelder der BAM: | Chemie und Prozesstechnik |
| Chemie und Prozesstechnik / Anlagensicherheit und Prozesssimulation | |
| Veranstaltung: | European Symposium on Computer Aided Process Engineering 35 |
| Veranstaltungsort: | Gent, Belgium |
| Beginndatum der Veranstaltung: | 06.07.2025 |
| Enddatum der Veranstaltung: | 09.07.2025 |
| Verfügbarkeit des Dokuments: | Datei im Netzwerk der BAM verfügbar ("Closed Access") |
| Datum der Freischaltung: | 17.12.2025 |
| Referierte Publikation: | Nein |
| Eingeladener Vortrag (wissenschaftliche Konferenzen): | Nein |

