Refine
Year of publication
- 2020 (3)
Language
- English (3)
Has Fulltext
- yes (3)
Keywords
- Hamilton-Jacobi-Bellman equations (1)
- Neural ODEs (1)
- Optimal control problems (1)
- Residual Neural Networks (1)
- Supervised Learning (1)
- deep learning (1)
- linear-quadratic (1)
- long-time behavior (1)
- optimal control (1)
- the turnpike property (1)
In this work, we analyze the consequences that the so-called turnpike property has on the long-time behavior of the value function corresponding to an optimal control problem. As a by-product, we obtain the long-time behavior of the solution to the associated Hamilton-Jacobi-Bellman equation.
In order to carry out our study, we use the setting of a finite-dimensional linear-quadratic optimal control problem, for which the turnpike property is well understood. We prove that, when the time horizon T tends to infinity, the value function converges to a travelling-front like solution of the form W(x) + c T + λ. In addition, we provide a control interpretation of each of these three terms in the spirit of the turnpike theory. Finally, we compare this asymptotic decomposition with the existing results on long-time behavior for Hamilton-Jacobi equations. We stress that in our case, the Hamiltonian is not coercive in the momentum variable, a case rarely considered in the classical literature about Hamilton-Jacobi equations.
We present a new proof of the turnpike property for nonlinear optimal control problems, when the running target is a steady control-state pair of the underlying dynamics. Our strategy combines the construction of suboptimal quasi-turnpike trajectories via controllability, and a bootstrap argument, and does not rely on analyzing the optimality system or linearization techniques. This in turn allows us to address several optimal control problems for finite-dimensional, control-affine systems with globally Lipschitz (possibly nonsmooth) nonlinearities, without any smallness conditions on the initial data or the running target. These results are motivated by the large-layer regime of residual neural networks, commonly used in deep learning applications. We show that our methodology is applicable to controlled PDEs as well, such as the semilinear wave and heat equation with a globally Lipschitz nonlinearity, once again without any smallness assumptions.
It is by now well-known that practical deep supervised learning may roughly be cast as an optimal control problem for a specific discrete-time, nonlinear dynamical system called an artificial neural network. In this work, we consider the continuous-time formulation of the deep supervised learning problem, and study the latter’s behavior when the final time horizon increases, a fact that can be interpreted as increasing the number of layers in the neural network setting.
When considering the classical regularized empirical risk minimization problem, we show that, in long time, the optimal states converge to zero training error, namely approach the zero training error regime, whilst the optimal control parameters approach, on an appropriate scale, minimal norm parameters with corresponding states precisely in the zero training error regime. This result provides an alternative theoretical underpinning to the notion that neural networks learn best in the overparametrized regime, when seen from the large layer perspective.
We also propose a learning problem consisting of minimizing a cost with a state tracking term, and establish the well-known turnpike property, which indicates that the solutions of the learning problem in long time intervals consist of three pieces, the first and the last of which being transient short-time arcs, and the middle piece being a long-time arc staying exponentially close to the optimal solution of an associated static learning problem. This property in fact stipulates a quantitative estimate for the number of layers required to reach the zero training error regime.
Both of the aforementioned asymptotic regimes are addressed in the context of continuous-time and continuous space-time neural networks, the latter taking the form of nonlinear, integro-differential equations, hence covering residual neural networks with both fixed and possibly variable depths.