TY - JOUR A1 - Ruiz-Balet, Domenec A1 - Zuazua, Enrique T1 - Neural ODE Control for Classification, Approximation and Transport N2 - We analyze Neural Ordinary Differential Equations (NODEs) from a control theoretical perspective to address some of the main properties and paradigms of Deep Learning (DL), in particular, data classification and universal approximation. These objectives are tackled and achieved from the perspective of the simultaneous control of systems of NODEs. For instance, in the context of classification, each item to be classified corresponds to a different initial datum for the control problem of the NODE, to be classified, all of them by the same common control, to the location (a subdomain of the euclidean space) associated to each label. Our proofs are genuinely nonlinear and constructive, allowing us to estimate the complexity of the control strategies we develop. The nonlinear nature of the activation functions governing the dynamics of NODEs under consideration plays a key role in our proofs, since it allows deforming half of the phase space while the other half remains invariant, a property that classical models in mechanics do not fulfill. This very property allows to build elementary controls inducing specific dynamics and transformations whose concatenation, along with properly chosen hyperplanes, allows achieving our goals in finitely many steps. The nonlinearity of the dynamics is assumed to be Lipschitz. Therefore, our results apply also in the particular case of the ReLU activation function. We also present the counterparts in the context of the control of neural transport equations, establishing a link between optimal transport and deep neural networks. KW - data classification KW - Neural ODEs KW - Optimal Transport KW - simultaneous control KW - deep learning Y1 - 2021 ER - TY - JOUR A1 - Zuazua, Enrique T1 - Asymptotic behavior of scalar convection-diffusion equations N2 - In these lecture notes, we address the problem of large-time asymptotic behaviour of the solutions to scalar convection-diffusion equations set in [katex]\mathbb{R}^N[/katex]. The large-time asymptotic behaviour of the solutions to many convection-diffusion equations is strongly linked with the behavior of the initial data at infinity. In fact, when the initial datum is integrable and of mass [katex]M[/katex], the solutions to the equations under consideration oftentimes behave like the associated self-similar profile of mass [katex]M[/katex], thus emphasising the role of scaling variables in these scenarios. However, these equations can also manifest other asymptotic behaviors, including weakly non-linear, linear or strongly non-linear behavior depending on the form of the convective term. We give an exhaustive presentation of several results and techniques, where we clearly distinguish the role of the spatial dimension and the form of the nonlinear convective term. Y1 - 2020 ER - TY - JOUR A1 - Ruiz-Balet, Domenec A1 - Zuazua, Enrique T1 - Control under constraints for multi-dimensional reaction-diffusion monostable and bistable equations N2 - Dynamic phenomena in social and biological sciences can often be modeled employing reaction diffusion equations. Frequently in applications, their control plays an important role when avoiding population extinction or propagation of infectious diseases, enhancing multicultural features, etc. When addressing these issues from a mathematical viewpoint one of the main challenges is that, because of the intrinsic nature of the models under consideration, the solution, typically a proportion or a density function, needs to preserve given lower and upper bounds (taking values in [0; 1])). Controlling the system to the desired final configuration then becomes complex, and sometimes even impossible. In the present work, we analyze the controllability to constant steady states of spatially homogeneous semilinear heat equations, with constraints in the state, and using boundary controls, which is indeed a natural way of acting on the system in the present context. The nonlinearities considered are among the most frequent: monostable and bistable ones. We prove that controlling the system to a constant steadystate may become impossible when the diffusivity is too small (or when the domain is large), due to the existence of barrier functions. When such an obstruction does not arise, we build sophisticated control strategies combining the dissipativity of the system, the existence of traveling waves, some connectivity of the set of steady states. This connectivity allows building paths that the controlled trajectories can follow, in a long time, with small oscillations, preserving the natural constraints of the system. This kind of strategy was successfully implemented in one space dimension, where phase plane analysis techniques allowed to decode the nature of the set of steady states. These techniques fail in the present multidimensional setting. We employ a fictitious domain technique, extending the system to a larger ball, and building paths of radially symmetric solution that can then be restricted to the original domain. The results are illustrated by numerical simulations of these models that find several applications, such as the extinction of minority languages or the survival of rare species in sufficiently large reserved areas. KW - Constraints KW - Controllability KW - Mathematical biology KW - Reaction-diffusion Y1 - 2020 U6 - https://doi.org/https://doi.org/10.1016/j.matpur.2020.08.006 VL - 143 SP - 345 EP - 375 ER - TY - JOUR A1 - Ko, Dongnam A1 - Zuazua, Enrique T1 - Model predictive control with random batch methods for a guiding problem N2 - We model, simulate and control the guiding problem for a herd of evaders under the action of repulsive drivers. The problem is formulated in an optimal control framework, where the drivers (controls) aim to guide the evaders (states) to a desired region of the Euclidean space. The numerical simulation of such models quickly becomes unfeasible for a large number of interacting agents. To reduce the computational cost, we use the Random Batch Method (RBM), which provides a computationally feasible approximation of the dynamics. At each time step, the RBM randomly divides the set of particles into small subsets (batches), considering only the interactions inside each batch. Due to the averaging effect, the RBM approximation converges to the exact dynamics as the time discretization gets finer. We propose an algorithm that leads to the optimal control of a fixed RBM approximated trajectory using a classical gradient descent. The resulting control is not optimal for the original complete system, but rather for the reduced RBM model. We then adopt a Model Predictive Control (MPC) strategy to handle the error in the dynamics. While the system evolves in time, the MPC strategy consists in periodically updating the state and computing the optimal control over a long-time horizon, which is implemented recursively in a shorter time-horizon. This leads to a semi-feedback control strategy. Through numerical experiments we show that the combination of RBM and MPC leads to a significant reduction of the computational cost, preserving the capacity of controlling the overall dynamics. KW - Agent-based models KW - Guiding problem KW - large scale complex systems KW - Random Batch Method KW - Model Predictive Control Y1 - 2020 U6 - https://doi.org/https://doi.org/10.1142/S0218202521500329 N1 - Ko Dongnam, Zuazua Enrique. Model predictive control with random batch methods for a guiding problem (2021). Math. Models Methods Appl. Sci., Vol. 31, No. 8, pp. 1569-1592. (2021) DOI: https://doi.org/10.1142/S0218202521500329 VL - 31 IS - 8 SP - 1569 EP - 1592 PB - Math. Models Methods Appl. Sci. ER - TY - JOUR A1 - Biccari, Umberto A1 - Navarro-Quiles, Ana A1 - Zuazua, Enrique T1 - Stochastic optimization methods for the simultaneous control of parameter-dependent systems N2 - We address the application of stochastic optimization methods for the simultaneous control of parameter-dependent systems. In particular, we focus on the classical Stochastic Gradient Descent (SGD) approach of Robbins and Monro, and on the recently developed Continuous Stochastic Gradient (CSG) algorithm. We consider the problem of computing simultaneous controls through the minimization of a cost functional defined as the superposition of individual costs for each realization of the system. We compare the performances of these stochastic approaches, in terms of their computational complexity, with those of the more classical Gradient Descent (GD) and Conjugate Gradient (CG) algorithms, and we discuss the advantages and disadvantages of each methodology. In agreement with well-established results in the machine learning context, we show how the SGD and CSG algorithms can significantly reduce the computational burden when treating control problems depending on a large amount of parameters. This is corroborated by numerical experiments. KW - Parameter-dependent systems KW - simultaneous controllability KW - stochastic optimization KW - computational cost Y1 - 2020 ER - TY - JOUR A1 - Esteve, Carlos A1 - Kouhkouh, H A1 - Pighin, Dario A1 - Zuazua, Enrique T1 - The Turnpike property and the long-time behavior of the Hamilton-Jacobi equation N2 - In this work, we analyze the consequences that the so-called turnpike property has on the long-time behavior of the value function corresponding to an optimal control problem. As a by-product, we obtain the long-time behavior of the solution to the associated Hamilton-Jacobi-Bellman equation. In order to carry out our study, we use the setting of a finite-dimensional linear-quadratic optimal control problem, for which the turnpike property is well understood. We prove that, when the time horizon T tends to infinity, the value function converges to a travelling-front like solution of the form W(x) + c T + λ. In addition, we provide a control interpretation of each of these three terms in the spirit of the turnpike theory. Finally, we compare this asymptotic decomposition with the existing results on long-time behavior for Hamilton-Jacobi equations. We stress that in our case, the Hamiltonian is not coercive in the momentum variable, a case rarely considered in the classical literature about Hamilton-Jacobi equations. KW - Optimal control problems KW - long-time behavior KW - the turnpike property KW - Hamilton-Jacobi-Bellman equations KW - linear-quadratic Y1 - 2020 ER - TY - JOUR A1 - Gontran, Lance A1 - Trélat, Emmanuel A1 - Zuazua, Enrique T1 - Shape turnpike for linear parabolic PDE models N2 - We introduce and study the turnpike property for time-varying shapes, within the viewpoint of optimal control. We focus here on second-order linear parabolic equations where the shape acts as a source term and we seek the optimal time-varying shape that minimizes a quadratic criterion. We first establish existence of optimal solutions under some appropriate sufficient conditions. We then provide necessary conditions for optimality in terms of adjoint equations and, using the concept of strict dissipativity, we prove that state and adjoint satisfy the measure-turnpike property, meaning that the extremal time-varying solution remains essentially close to the optimal solution of an associated static problem. We show that the optimal shape enjoys the exponential turnpike property in term of Hausdorff distance for a Mayer quadratic cost. We illustrate the turnpike phenomenon in optimal shape design with several numerical simulations. Y1 - 2020 U6 - https://doi.org/https://doi.org/10.1016/j.sysconle.2020.104733 VL - 142 PB - Syst. Control. Lett. ER - TY - JOUR A1 - Esteve, Carlos A1 - Geshkovski, Borjan A1 - Pighin, Dario A1 - Zuazua, Enrique T1 - Turnpike in Lipschitz-nonlinear optimal control N2 - We present a new proof of the turnpike property for nonlinear optimal control problems, when the running target is a steady control-state pair of the underlying dynamics. Our strategy combines the construction of suboptimal quasi-turnpike trajectories via controllability, and a bootstrap argument, and does not rely on analyzing the optimality system or linearization techniques. This in turn allows us to address several optimal control problems for finite-dimensional, control-affine systems with globally Lipschitz (possibly nonsmooth) nonlinearities, without any smallness conditions on the initial data or the running target. These results are motivated by the large-layer regime of residual neural networks, commonly used in deep learning applications. We show that our methodology is applicable to controlled PDEs as well, such as the semilinear wave and heat equation with a globally Lipschitz nonlinearity, once again without any smallness assumptions. Y1 - ER - TY - JOUR A1 - Bárcena, J.A. A1 - Zuazua, Enrique T1 - Averaged dynamics and control for heat equations with random diffusion N2 - Abstract. This paper deals with the averaged dynamics for heat equations in the degenerate case where the diffusivity coefficient, assumed to be constant, is allowed to take the null value. First we prove that the averaged dynamics is analytic. This allows to show that, most often, the averaged dynamics enjoys the property of unique continuation and is approximately controllable. We then determine if the averaged dynamics is actually null controllable or not depending on how the density of averaging behaves when the diffusivity vanishes. In the critical density threshold the dynamics of the average is similar to the $\frac{1}{2}$-fractional Laplacian, which is wellknown to be critical in the context of the controllability of fractional diffusion processes. Null controllability then fails (resp. holds) when the density weights more (resp. less) in the null diffusivity regime than in this critical regime. KW - Averaged controllability, averaged observability, observability, random heat equation Y1 - 2020 ER - TY - JOUR A1 - Esteve, C A1 - Zuazua, Enrique T1 - The Inverse Problem for Hamilton-Jacobi equations and Semiconcave Envelopes N2 - We study the inverse problem, or inverse design problem, for a time-evolution Hamilton-Jacobi equation. More precisely, given a target function [katex]u_T[/katex] and a time horizon [katex]T > 0[/katex], we aim to construct all the initial conditions for which the viscosity solution coincides with [katex]u_T[/katex] at time [katex]T[/katex]. As it is common in this kind of nonlinear equations, the target might not be reachable. We first study the existence of at least one initial condition leading the system to the given target. The natural candidate, which indeed allows determining the reachability of [katex]u_T[/katex] , is the one obtained by reversing the direction of time in the equation, considering [katex]u_T[/katex] as terminal condition. In this case, we use the notion of backward viscosity solution, that provides existence and uniqueness for the terminal-value problem. We also give an equivalent reachability condition based on a differential inequality, that relates the reachability of the target with its semiconcavity properties. Then, for the case when [katex]u_T[/katex] is reachable, we construct the set of all initial conditions for which the solution coincides with [katex]u_T[/katex] at time [katex]T[/katex]. Note that in general, such initial conditions are not unique. Finally, for the case when the target [katex]u_T[/katex] is not necessarily reachable, we study the projection of [katex]u_T[/katex] on the set of reachable targets, obtained by solving the problem backward and then forward in time. This projection is then identified with the solution of a fully nonlinear obstacle problem, and can be interpreted as the semiconcave envelope of [katex]u_T[/katex] , i.e. the smallest reachable target bounded from below by [katex]u_T[/katex] . KW - Remove term: Hamilton-Jacobi equation Hamilton-Jacobi equation KW - inverse design problem KW - obstacle problems KW - semiconcave envelopes Y1 - 2020 ER - TY - JOUR A1 - Esteve, Carlos A1 - Geshkovski, Borjan A1 - Pighin, Dario A1 - Zuazua, Enrique T1 - Large-time asymptotics in deep learning N2 - It is by now well-known that practical deep supervised learning may roughly be cast as an optimal control problem for a specific discrete-time, nonlinear dynamical system called an artificial neural network. In this work, we consider the continuous-time formulation of the deep supervised learning problem, and study the latter’s behavior when the final time horizon increases, a fact that can be interpreted as increasing the number of layers in the neural network setting. When considering the classical regularized empirical risk minimization problem, we show that, in long time, the optimal states converge to zero training error, namely approach the zero training error regime, whilst the optimal control parameters approach, on an appropriate scale, minimal norm parameters with corresponding states precisely in the zero training error regime. This result provides an alternative theoretical underpinning to the notion that neural networks learn best in the overparametrized regime, when seen from the large layer perspective. We also propose a learning problem consisting of minimizing a cost with a state tracking term, and establish the well-known turnpike property, which indicates that the solutions of the learning problem in long time intervals consist of three pieces, the first and the last of which being transient short-time arcs, and the middle piece being a long-time arc staying exponentially close to the optimal solution of an associated static learning problem. This property in fact stipulates a quantitative estimate for the number of layers required to reach the zero training error regime. Both of the aforementioned asymptotic regimes are addressed in the context of continuous-time and continuous space-time neural networks, the latter taking the form of nonlinear, integro-differential equations, hence covering residual neural networks with both fixed and possibly variable depths. KW - deep learning KW - Neural ODEs KW - optimal control KW - Residual Neural Networks KW - Supervised Learning Y1 - 2020 ER - TY - JOUR A1 - Heiland, Jan A1 - Zuazua, Enrique T1 - Classical system theory revisited for Turnpike in standard state space systems and impulse controllable descriptor systems N2 - The concept of turnpike connects the solution of long but finite time horizon optimal control problems with steady state optimal controls. A key ingredient of the analysis of the turnpike is the linear quadratic regulator problem and the convergence of the solution of the associated differential Riccati equation as the terminal time approaches infinity. This convergence has been investigated in linear systems theory in the 1980s. We extend classical system theoretic results for the investigation of turnpike properties of standard state space systems and descriptor systems. We present conditions for turnpike in the nondetectable case and for impulse controllable descriptor systems. For the latter, in line with the theory for standard linear systems, we establish existence and convergence of solutions to a generalized differential Riccati equation. KW - descriptor systems KW - linear systems KW - long time behavior KW - optimal control KW - Riccati equations Y1 - 2020 ER - TY - JOUR A1 - Biccari, Umberto A1 - Zuazua, Enrique T1 - A Stochastic Approach to the Synchronization of Coupled Oscillators N2 - This paper deals with an optimal control problem associated with the Kuramoto model describing the dynamical behavior of a network of coupled oscillators. Our aim is to design a suitable control function allowing us to steer the system to a synchronized configuration in which all the oscillators are aligned on the same phase. This control is computed via the minimization of a given cost functional associated with the dynamics considered. For this minimization, we propose a novel approach based on the combination of a standard Gradient Descent (GD) methodology with the recently-developed Random Batch Method (RBM) for the efficient numerical approximation of collective dynamics. Our simulations show that the employment of RBM improves the performances of the GD algorithm, reducing the computational complexity of the minimization process and allowing for a more efficient control calculation. KW - coupled oscillators KW - gradient descent KW - Kuramoto model KW - optimal control KW - random batch method Y1 - 2020 U6 - https://doi.org/10.3389/fenrg.2020.00115 VL - 8 ER - TY - JOUR A1 - Gugat, Martin A1 - Schuster, Michael A1 - Zuazua, Enrique T1 - M. Gugat, M. Schuster, E. Zuazua. The Finite-Time Turnpike Phenomenon for Optimal Control Problems: Stabilization by Non-Smooth Tracking Terms, in “Stabilization of Distributed Parameter Systems: Design Methods and Applications”. Grigory Sklyar Alexander Zuyev Eds., ICIAM 2019 SEMA SIMAI Springer Series 2, p. 17-42. ISSN 2199-3041 N2 - In this paper, problems of optimal control are considered where in the objective function, in addition to the control cost, there is a tracking term that measures the distance to a desired stationary state. The tracking term is given by some norm, and therefore it is in general not differentiable. In the optimal control problem, the initial state is prescribed. We assume that the system is either exactly controllable in the classical sense or nodal profile controllable. We show that both for systems that are governed by ordinary differential equations and for infinite-dimensional systems, for example, for boundary control systems governed by the wave equation, under certain assumptions, the optimal system state is steered exactly to the desired state after finite time. Y1 - 2020 VL - SEMA SIMAI Springer Series 2 SP - 17 EP - 42 PB - Springer International Publishing ET - Grigory Sklyar Alexander Zuyev Eds., ICIAM 2019 ER - TY - JOUR A1 - Sarac, Yesim A1 - Zuazua, Enrique T1 - Sidewise control of 1-d waves N2 - We analyze the sidewise controllability for the variable coefficients one-dimensional wave equation. The control is acting on one extreme of the string with the aim that the solution tracks a given path at the otherfree end. This sidewise control problem is also often referred to as nodal profile or tracking control. First, the problem is reformulated as a dual observability property for the corresponding adjoint system. Using sidewiseenergy propagation arguments the sidewise observability is shown to hold, ina sufficiently large time, in the class of BV-coefficients. We also present a number of open problems and perspectives for further research. KW - 1-d wave equations KW - BV-coefficients KW - nodal profile con-trol Y1 - 2021 ER - TY - INPR A1 - Ruiz-Balet, Domènec A1 - Affili, Elisa A1 - Zuazua, Enrique T1 - Interpolation and approximation via Momentum ResNets and Neural ODEs N2 - In this article, we explore the effects of memory terms in continuous-layer Deep Residual Networks by studying Neural ODEs (NODEs). We investigate two types of models. On one side, we consider the case of Residual Neural Networks with dependence on multiple layers, more precisely Momentum ResNets. On the other side, we analyse a Neural ODE with auxiliary states playing the role of memory states. We examine the interpolation and universal approximation properties for both architectures through a simultaneous control perspective. We also prove the ability of the second model to represent sophisticated maps, such as parametrizations of time-dependent functions. Numerical simulations complement our study. Y1 - 2021 ER -