Open Access

An ML approach for parameter estimation of dynamic models is proposed, in which Time Series Extrinsic Regression (TSER) is used to learn the inverse mapping of the time series output to the underlying time-invariant parameters. To our knowledge, this is the first deliberate application of TSER and its methods to parameter estimation of dynamic models.

Time series analysis is a well-established field within the machine learning community, with two prominent applications being time-series forecasting, i.e., surrogate models, predicting the next time step for the systems outputs, and time-series classification, where complete timeseries are mapped to discrete labels, e.g. a sensor is either working or defective. Time-Series Extrinsic Regression (TSER), however, is a method for predicting continuous, time-invariant variables from a time series by learning the relation between these underlying parameters and the complete dynamic time series of the outputs without focusing on the recent states. E.g., it can be used to predict the heart rate based on an ECG signal. TSER as a research field was only established in 2021, but it is gaining traction ever since and it is used e.g. in the field of manufacturing technology to predict steel surface roughness from laser reflection measurements. It is applied, when there are no models available. Parameter Estimation (PE) is a common task in chemical engineering. It is used to adjust model parameters to better fit existing dynamic models to experimental time series data. This becomes more challenging in higher dimensions and for dynamic systems, where sensitivity and identifiability may change over time. There already exists a multitude of algorithms to solve the problem, including second-order methods that leverage information from Jacobian and Hessian matrices, as well as gradient-free optimization techniques, such as particle swarm optimization (PSO) or simulated annealing. However, with the growing establishment of machine learning (ML) in an increasing number of domains, the question arises as to whether, and if so, how, ML in general and TSER in particular can be employed to solve PE problems. This study marks the first application of TSER to PE problems. A comparative analysis is conducted between TSER and PSO, in terms of prediction accuracy, computational cost and data efficiency. We investigate, whether it is viable to use TSER, when there is a model available. Our methodology to regress model parameters via ML builds on the typical assumption, that a structurally correct and rigorous model, which can be simulated at low cost, is available. At the beginning, the boundaries of the parameter space are defined. This space is then sampled using Sobol sequences and the model is simulated. The resulting trajectories, along with their corresponding parameters, constitute the training data set. These trajectories are transformed through application of the “RandOm Convolutional Kernel Transform” method resulting in novel features, which are subsequently used to train the regressor model. This regressor returns predictions for the parameters. In a case study, the method is applied to predict the heat transfer and kinetic parameters of a batch reactor based on simulated data. However, real measurements are often not continuously available, but are taken only at rare, discrete points in time, and different variables are measured at different, asynchronous intervals. This is also mimicked in the synthetic training data, so the influence of heterogeneity on the results can be shown and over- or undersampling strategies are applied to counteract the effect.