TY - JOUR A1 - Ruiz-Balet, Domenec A1 - Zuazua, Enrique T1 - Neural ODE Control for Classification, Approximation and Transport N2 - We analyze Neural Ordinary Differential Equations (NODEs) from a control theoretical perspective to address some of the main properties and paradigms of Deep Learning (DL), in particular, data classification and universal approximation. These objectives are tackled and achieved from the perspective of the simultaneous control of systems of NODEs. For instance, in the context of classification, each item to be classified corresponds to a different initial datum for the control problem of the NODE, to be classified, all of them by the same common control, to the location (a subdomain of the euclidean space) associated to each label. Our proofs are genuinely nonlinear and constructive, allowing us to estimate the complexity of the control strategies we develop. The nonlinear nature of the activation functions governing the dynamics of NODEs under consideration plays a key role in our proofs, since it allows deforming half of the phase space while the other half remains invariant, a property that classical models in mechanics do not fulfill. This very property allows to build elementary controls inducing specific dynamics and transformations whose concatenation, along with properly chosen hyperplanes, allows achieving our goals in finitely many steps. The nonlinearity of the dynamics is assumed to be Lipschitz. Therefore, our results apply also in the particular case of the ReLU activation function. We also present the counterparts in the context of the control of neural transport equations, establishing a link between optimal transport and deep neural networks. KW - data classification KW - Neural ODEs KW - Optimal Transport KW - simultaneous control KW - deep learning Y1 - 2021 ER - TY - INPR A1 - Egerer, Jonas A1 - Grimm, Veronika A1 - Grübel, Julia A1 - Zöttl, Gregor T1 - Long-run market equilibria in coupled energy sectors: A study of uniqueness N2 - We propose an equilibrium model for coupled markets of multiple energy sectors. The agents in our model are operators of sector-specific production and sector-coupling technologies, as well as price-sensitive consumers with varying demand. We analyze long-run investment in production capacity in each sector and investment in coupling capacity between sectors, as well as production decisions determined at repeated spot markets. We show that in our multi-sector model, multiplicity of equilibria may occur, even if all assumptions hold that would be sufficient for uniqueness in a single-sector model. We then contribute to the literature by deriving sufficient conditions for the uniqueness of short- and long-run market equilibrium in coupled markets of multiple energy sectors. We illustrate via simple examples that these conditions are indeed required to guarantee uniqueness in general. The uniqueness result is an important step to be able to incorporate the proposed market equilibrium problem in more complex computational multilevel equilibrium models, in which uniqueness of lower levels is a prerequisite for obtaining meaningful solutions. Our analysis also paves the way to understand and analyze more complex sector coupling models in the future. KW - Energy Markets KW - Sector Coupling KW - Regional Pricing KW - Uniqueness KW - Short- and Long-Run Market Equilibrium Y1 - 2021 ER - TY - JOUR A1 - Kleinert, Thomas A1 - Manns, Julian A1 - Schmidt, Martin A1 - Weninger, Dieter T1 - Presolving Linear Bilevel Optimization Problems JF - EURO Journal on Computational Optimization N2 - Linear bilevel optimization problems are known to be strongly NP-hard and the computational techniques to solve these problems are often motivated by techniques from single-level mixed-integer optimization. Thus, during the last years and decades many branch-and-bound methods, cutting planes, or heuristics have been proposed. On the other hand, there is almost no literature on presolving linear bilevel problems although presolve is a very important ingredient in state-of-the-art mixed-integer optimization solvers. In this paper, we carry over standard presolve techniques from single-level optimization to bilevel problems and show that this needs to be done with great caution since a naive application of well-known techniques does often not lead to correctly presolved bilevel models. Our numerical study shows that presolve can also be very beneficial for bilevel problems but also highlights that these methods have a more heterogeneous effect on the solution process compared to what is known from single-level optimization. As a side result, our numerical experiments reveal that there is an urgent need for better and more heterogeneous test instance libraries to further propel the field of computational bilevel optimization. KW - Linear Bilevel Optimization KW - Presolve KW - Computational Analysis Y1 - 2021 U6 - https://doi.org/10.1016/j.ejco.2021.100020 IS - 9 ER - TY - JOUR A1 - Domschke, Pia A1 - Kolb, Oliver A1 - Lang, Jens T1 - Fast and Reliable Transient Simulation and Continuous Optimization of Large-Scale Gas Networks N2 - We are concerned with the simulation and optimization of large-scale gas pipeline systems in an error-controlled environment. The gas flow dynamics is locally approximated by sufficiently accurate physical models taken from a hierarchy of decreasing complexity and varying over time. Feasible work regions of compressor stations consisting of several turbo compressors are included by semiconvex approximations of aggregated characteristic fields. A discrete adjoint approach within a first-discretize-then-optimize strategy is proposed and a sequential quadratic programming with an active set strategy is applied to solve the nonlinear constrained optimization problems resulting from a validation of nominations. The method proposed here accelerates the computation of near-term forecasts of sudden changes in the gas management and allows for an economic control of intra-day gas flow schedules in large networks. Case studies for real gas pipeline systems show the remarkable performance of the new method. Y1 - 2021 U6 - https://doi.org/https://doi.org/10.1007/s00186-021-00765-7 PB - Mathematical Methods of Operations Research ER - TY - JOUR A1 - Frenzel, David A1 - Lang, Jens T1 - A Third-Order Weighted Essentially Non-Oscillatory Scheme in Optimal Control Problems Governed by Nonlinear Hyperbolic Conservation Laws N2 - The weighted essentially non-oscillatory (WENO) methods are popular and effective spatial discretization methods for nonlinear hyperbolic partial differential equations. Although these methods are formally first-order accurate when a shock is present, they still have uniform high-order accuracy right up to the shock location. In this paper, we propose a novel third-order numerical method for solving optimal control problems subject to scalar nonlinear hyperbolic conservation laws. It is based on the first-disretize-then-optimize approach and combines a discrete adjoint WENO scheme of third order with the classical strong stability preserving three-stage third-order Runge-Kutta method SSPRK3. We analyze its approximation properties and apply it to optimal control problems of tracking-type with non-smooth target states. Comparisons to common first-order methods such as the Lax-Friedrichs and Engquist-Osher method show its great potential to achieve a higher accuracy along with good resolution around discontinuities. Y1 - 2021 U6 - https://doi.org/https://doi.org/10.1007/s10589-021-00295-2 VL - Computational Optimization and Applications IS - 80 SP - 301 EP - 320 ER - TY - JOUR A1 - Lang, Jens A1 - Domschke, Pia A1 - Strauch, Elisa T1 - Adaptive Single- and Multilevel Stochastic Collocation Methods for Uncertain Gas Transport in Large-Scale Networks N2 - In this paper, we are concerned with the quantification of uncertainties that arise from intra-day oscillations in the demand for natural gas transported through large-scale networks. The short-term transient dynamics of the gas flow is modelled by a hierarchy of hyperbolic systems of balance laws based on the isentropic Euler equations. We extend a novel adaptive strategy for solving elliptic PDEs with random data, recently proposed and analysed by Lang, Scheichl, and Silvester [J. Comput. Phys., 419:109692, 2020], to uncertain gas transport problems. Sample-dependent adaptive meshes and a model refinement in the physical space is combined with adaptive anisotropic sparse Smolyak grids in the stochastic space. A single-level approach which balances the discretization errors of the physical and stochastic approximations and a multilevel approach which additionally minimizes the computational costs are considered. Two examples taken from a public gas library demonstrate the reliability of the error control of expectations calculated from random quantities of interest, and the further use of stochastic interpolants to, e.g., approximate probability density functions of minimum and maximum pressure values at the exits of the network. Y1 - 2021 VL - In: Mesh Generation and Adaptation, Cutting-Edge Techniques. R. Sevilla, S. Perotto, K. Morgan (eds.), SEMA-SIMAI Springer Series IS - Vol. 30 SP - 113 EP - 135 ER - TY - JOUR A1 - Gräßle, Carmen A1 - Hinze, Michael A1 - Lang, Jens A1 - Ullmann, Sebastian T1 - POD model order reduction with space-adapted snapshots for incompressible flows N2 - We consider model order reduction based on proper orthogonal decomposition (POD) for unsteady incompressible Navier-Stokes problems, assuming that the snapshots are given by spatially adapted finite element solutions. We propose two approaches of deriving stable POD-Galerkin reduced-order models for this context. In the first approach, the pressure term and the continuity equation are eliminated by imposing a weak incompressibility constraint with respect to a pressure reference space. In the second approach, we derive an inf-sup stable velocity-pressure reduced-order model by enriching the velocity reduced space with supremizers computed on a velocity reference space. For problems with inhomogeneous Dirichlet conditions, we show how suitable lifting functions can be obtained from standard adaptive finite element computations. We provide a numerical comparison of the considered methods for a regularized lid-driven cavity problem. Y1 - 2019 U6 - https://doi.org/doi:10.1007/s10444-019-09716-7 VL - Advances in Computational Mathematics IS - 45 SP - 2401 EP - 2428 ER - TY - JOUR A1 - Lang, Jens A1 - Schmitt, Bernhard A. T1 - Discrete Adjoint Implicit Peer Methods in Optimal Control N2 - It is well known that in the first-discretize-then-optimize approach in the control of ordinary differential equations the adjoint method may converge under additional order conditions only. For Peer two-step methods we derive such adjoint order conditions and pay special attention to the boundary steps. For $s$-stage methods, we prove convergence of order s for the state variables if the adjoint method satisfies the conditions for order s-1, at least. We remove some bottlenecks at the boundaries encountered in an earlier paper of the first author et al. [J. Comput. Appl. Math., 262:73--86, 2014] and discuss the construction of 3-stage methods for the order pair (3,2) in detail including some matrix background for the combined forward and adjoint order conditions. The impact of nodes having equal differences is highlighted. It turns out that the most attractive methods are related to BDF. Three 3-stage methods are constructed which show the expected orders in numerical tests. Y1 - 2020 U6 - https://doi.org/https://doi.org/10.1016/j.cam.2022.114596 VL - Journal of Computational and Applied Mathematics IS - 416:114596 ER - TY - JOUR A1 - Lang, Jens A1 - Scheichl, Robert A1 - Silvester, David T1 - A Fully Adaptive Multilevel Stochastic Collocation Strategy for Solving Elliptic PDEs with Random Data N2 - We propose and analyse a fully adaptive strategy for solving elliptic PDEs with random data in this work. A hierarchical sequence of adaptive mesh refinements for the spatial approximation is combined with adaptive anisotropic sparse Smolyak grids in the stochastic space in such a way as to minimize the computational cost. The novel aspect of our strategy is that the hierarchy of spatial approximations is sample dependent so that the computational effort at each collocation point can be optimised individually. We outline a rigorous analysis for the convergence and computational complexity of the adaptive multilevel algorithm and we provide optimal choices for error tolerances at each level. Two numerical examples demonstrate the reliability of the error control and the significant decrease in the complexity that arises when compared to single level algorithms and multilevel algorithms that employ adaptivity solely in the spatial discretisation or in the collocation procedure. Y1 - 2021 U6 - https://doi.org/doi:10.1016/j.jcp.2020.109692 VL - Journal of Computational Physics IS - 419 ER - TY - INPR A1 - Heitsch, Holger A1 - Henrion, René A1 - Kleinert, Thomas A1 - Schmidt, Martin T1 - On Convex Lower-Level Black-Box Constraints in Bilevel Optimization with an Application to Gas Market Models with Chance Constraints N2 - Bilevel optimization is an increasingly important tool to model hierarchical decision making. However, the ability of modeling such settings makes bilevel problems hard to solve in theory and practice. In this paper, we add on the general difficulty of this class of problems by further incorporating convex black-box constraints in the lower level. For this setup, we develop a cutting-plane algorithm that computes approximate bilevel-feasible points. We apply this method to a bilevel model of the European gas market in which we use a joint chance constraint to model uncertain loads. Since the chance constraint is not available in closed form, this fits into the black-box setting studied before. For the applied model, we use further problem-specific insights to derive bounds on the objective value of the bilevel problem. By doing so, we are able to show that we solve the application problem to approximate global optimality. In our numerical case study we are thus able to evaluate the welfare sensitivity in dependence of the achieved safety level of uncertain load coverage. KW - Bilevel optimization KW - Black-box constraints KW - Chance constraints KW - Cutting planes KW - European gas market Y1 - 2021 ER -