TY - JOUR A1 - Petkovic, Milena A1 - Chen, Ying A1 - Gamrath, Inken A1 - Gotzes, Uwe A1 - Hadjidimitrou, Natalia Selini A1 - Zittel, Janina A1 - Xu, Xiaofei A1 - Koch, Thorsten T1 - A hybrid approach for high precision prediction of gas flows JF - Energy Systems N2 - About 23% of the German energy demand is supplied by natural gas. Additionally, for about the same amount Germany serves as a transit country. Thereby, the German network represents a central hub in the European natural gas transport network. The transport infrastructure is operated by transmissions system operators (TSOs). The number one priority of the TSOs is to ensure the security of supply. However, the TSOs have only very limited knowledge about the intentions and planned actions of the shippers (traders). Open Grid Europe (OGE), one of Germany’s largest TSO, operates a high-pressure transport network of about 12,000 km length. With the introduction of peak-load gas power stations, it is of great importance to predict in- and out-flow of the network to ensure the necessary flexibility and security of supply for the German Energy Transition (“Energiewende”). In this paper, we introduce a novel hybrid forecast method applied to gas flows at the boundary nodes of a transport network. This method employs an optimized feature selection and minimization. We use a combination of a FAR, LSTM and mathematical programming to achieve robust high-quality forecasts on real-world data for different types of network nodes. Y1 - 2022 U6 - https://doi.org/10.1007/s12667-021-00466-4 VL - 13 SP - 383 EP - 408 ER - TY - GEN A1 - Petkovic, Milena A1 - Chen, Ying A1 - Gamrath, Inken A1 - Gotzes, Uwe A1 - Hadjidimitriou, Natalia Selini A1 - Zittel, Janina A1 - Xu, Xiaofei A1 - Koch, Thorsten T1 - A Hybrid Approach for High Precision Prediction of Gas Flows N2 - About 20% of the German energy demand is supplied by natural gas. Ad- ditionally, for about twice the amount Germany serves as a transit country. Thereby, the German network represents a central hub in the European natural gas transport network. The transport infrastructure is operated by so-called transmissions system operators or TSOs. The number one priority of the TSOs is to ensure security of supply. However, the TSOs have no knowledge of the intentions and planned actions of the shippers (traders). Open Grid Europe (OGE), one of Germany’s largest TSO, operates a high- pressure transport network of about 12.000 km length. Since flexibility and security of supply is of utmost importance to the German Energy Transition (“Energiewende”) especially with the introduction of peak-load gas power stations, being able to predict in- and out-flow of the network is of great importance. In this paper we introduce a new hybrid forecast method applied to gas flows at the boundary nodes of a transport network. The new method employs optimized feature minimization and selection. We use a combination of an FAR, LSTM DNN and mathematical programming to achieve robust high quality forecasts on real world data for different types of network nodes. Keywords: Gas Forecast, Time series, Hybrid Method, FAR, LSTM, Mathematical Optimisation T3 - ZIB-Report - 19-26 Y1 - 2019 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-73525 SN - 1438-0064 ER - TY - GEN A1 - Cook, William A1 - Koch, Thorsten A1 - Steffy, Daniel A1 - Wolter, Kati T1 - A Hybrid Branch-and-Bound Approach for Exact Rational Mixed-Integer Programming N2 - We present an exact rational solver for mixed-integer linear programming that avoids the numerical inaccuracies inherent in the floating-point computations used by existing software. This allows the solver to be used for establishing theoretical results and in applications where correct solutions are critical due to legal and financial consequences. Our solver is a hybrid symbolic/numeric implementation of LP-based branch-and-bound, using numerically-safe methods for all binding computations in the search tree. Computing provably accurate solutions by dynamically choosing the fastest of several safe dual bounding methods depending on the structure of the instance, our exact solver is only moderately slower than an inexact floating-point branch-and-bound solver. The software is incorporated into the SCIP optimization framework, using the exact LP solver QSopt_ex and the GMP arithmetic library. Computational results are presented for a suite of test instances taken from the MIPLIB and Mittelmann libraries and for a new collection of numerically difficult instances. T3 - ZIB-Report - 12-49 Y1 - 2012 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-17171 SN - 1438-0064 ER - TY - JOUR A1 - Cook, William A1 - Koch, Thorsten A1 - Steffy, Daniel A1 - Wolter, Kati T1 - A hybrid branch-and-bound approach for exact rational mixed-integer programming JF - Mathematical Programming Computation N2 - We present an exact rational solver for mixed-integer linear programming that avoids the numerical inaccuracies inherent in the floating-point computations used by existing software. This allows the solver to be used for establishing theoretical results and in applications where correct solutions are critical due to legal and financial consequences. Our solver is a hybrid symbolic/numeric implementation of LP-based branch-and-bound, using numerically-safe methods for all binding computations in the search tree. Computing provably accurate solutions by dynamically choosing the fastest of several safe dual bounding methods depending on the structure of the instance, our exact solver is only moderately slower than an inexact floating-point branch-and-bound solver. The software is incorporated into the SCIP optimization framework, using the exact LP solver QSopt_ex and the GMP arithmetic library. Computational results are presented for a suite of test instances taken from the MIPLIB and Mittelmann libraries and for a new collection of numerically difficult instances. Y1 - 2013 U6 - https://doi.org/10.1007/s12532-013-0055-6 VL - 5 IS - 3 SP - 305 EP - 344 ER - TY - GEN A1 - Arnold, Thomas A1 - Berthold, Timo A1 - Heinz, Stefan A1 - Vigerske, Stefan A1 - Henrion, René A1 - Grötschel, Martin A1 - Koch, Thorsten A1 - Tischendorf, Caren A1 - Römisch, Werner ED - Deuflhard, Peter ED - Grötschel, Martin ED - Hömberg, Dietmar ED - Horst, Ulrich ED - Kramer, Jürg ED - Mehrmann, Volker ED - Polthier, Konrad ED - Schmidt, Frank ED - Schütte, Christof ED - Skutella, Martin ED - Sprekels, Jürgen T1 - A Jack of all Trades? Solving stochastic mixed-integer nonlinear constraint programs T2 - MATHEON - Mathematics for Key Technologies N2 - Natural gas is one of the most important energy sources in Germany and Europe. In recent years, political regulations have led to a strict separation of gas trading and gas transport, thereby assigning a central role in energy politics to the transportation and distribution of gas. These newly imposed political requirements influenced the technical processes of gas transport in such a way that the complex task of planning and operating gas networks has become even more intricate. Mathematically, the combination of discrete decisions on the configuration of a gas transport network, the nonlinear equations describing the physics of gas, and the uncertainty in demand and supply yield large-scale and highly complex stochastic mixed-integer nonlinear optimization problems. The Matheon project "Optimization of Gas Transport" takes the key role of making available the necessary core technology to solve the mathematical optimization problems which model the topology planning and the operation of gas networks. An important aspect of the academic impact is the free availability of our framework. As a result of several years of research and development, it is now possible to download a complete state-of-the-art framework for mixed-integer linear and nonlinear programming in source code at http://scip.zib.de Y1 - 2014 U6 - https://doi.org/10.4171/137 VL - 1 SP - 135 EP - 146 PB - European Mathematical Society ER - TY - GEN A1 - Arnold, Thomas A1 - Berthold, Timo A1 - Heinz, Stefan A1 - Vigerske, Stefan A1 - Henrion, René A1 - Grötschel, Martin A1 - Koch, Thorsten A1 - Tischendorf, Caren A1 - Römisch, Werner T1 - A Jack of all Trades? Solving stochastic mixed-integer nonlinear constraint programs N2 - Natural gas is one of the most important energy sources in Germany and Europe. In recent years, political regulations have led to a strict separation of gas trading and gas transport, thereby assigning a central role in energy politics to the transportation and distribution of gas. These newly imposed political requirements influenced the technical processes of gas transport in such a way that the complex task of planning and operating gas networks has become even more intricate. Mathematically, the combination of discrete decisions on the configuration of a gas transport network, the nonlinear equations describing the physics of gas, and the uncertainty in demand and supply yield large-scale and highly complex stochastic mixed-integer nonlinear optimization problems. The Matheon project "Optimization of Gas Transport" takes the key role of making available the necessary core technology to solve the mathematical optimization problems which model the topology planning and the operation of gas networks. An important aspect of the academic impact is the free availability of our framework. As a result of several years of research and development, it is now possible to download a complete state-of-the-art framework for mixed-integer linear and nonlinear programming in source code at http://scip.zib.de T3 - ZIB-Report - 14-14 KW - gas transport optimization KW - mixed integer nonlinear programming Y1 - 2014 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-49947 SN - 1438-0064 ER - TY - GEN A1 - Rehfeldt, Daniel A1 - Hobbie, Hannes A1 - Schönheit, David A1 - Gleixner, Ambros A1 - Koch, Thorsten A1 - Möst, Dominik T1 - A massively parallel interior-point solver for linear energy system models with block structure N2 - Linear energy system models are often a crucial component of system design and operations, as well as energy policy consulting. Such models can lead to large-scale linear programs, which can be intractable even for state-of-the-art commercial solvers|already the available memory on a desktop machine might not be sufficient. Against this backdrop, this article introduces an interior-point solver that exploits common structures of linear energy system models to efficiently run in parallel on distributed memory systems. The solver is designed for linear programs with doubly bordered block-diagonal constraint matrix and makes use of a Schur complement based decomposition. Special effort has been put into handling large numbers of linking constraints and variables as commonly observed in energy system models. In order to handle this strong linkage, a distributed preconditioning of the Schur complement is used. In addition, the solver features a number of more generic techniques such as parallel matrix scaling and structure-preserving presolving. The implementation is based on the existing parallel interior-point solver PIPS-IPM. We evaluate the computational performance on energy system models with up to 700 million non-zero entries in the constraint matrix, and with more than 200 million columns and 250 million rows. This article mainly concentrates on the energy system model ELMOD, which is a linear optimization model representing the European electricity markets by the use of a nodal pricing market clearing. It has been widely applied in the literature on energy system analyses during the recent years. However, it will be demonstrated that the new solver is also applicable to other energy system models. T3 - ZIB-Report - 19-41 Y1 - 2019 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-74321 SN - 1438-0064 N1 - In the meantime, this report got published as a journal article: https://opus4.kobv.de/opus4-zib/frontdoor/index/index/searchtype/authorsearch/author/Hannes+Hobbie/docId/8191/start/1/rows/10 Please use this journal reference when citing this work. ER - TY - JOUR A1 - Rehfeldt, Daniel A1 - Hobbie, Hannes A1 - Schönheit, David A1 - Koch, Thorsten A1 - Möst, Dominik A1 - Gleixner, Ambros T1 - A massively parallel interior-point solver for LPs with generalized arrowhead structure, and applications to energy system models JF - European Journal of Operational Research N2 - Linear energy system models are a crucial component of energy system design and operations, as well as energy policy consulting. If detailed enough, such models lead to large-scale linear programs, which can be intractable even for the best state-of-the-art solvers. This article introduces an interior-point solver that exploits common structures of energy system models to efficiently run in parallel on distributed-memory systems. The solver is designed for linear programs with doubly-bordered block-diagonal constraint matrix and makes use of a Schur complement based decomposition. In order to handle the large number of linking constraints and variables commonly observed in energy system models, a distributed Schur complement preconditioner is used. In addition, the solver features a number of more generic techniques such as parallel matrix scaling and structure-preserving presolving. The implementation is based on the solver PIPS-IPM. We evaluate the computational performance on energy system models with up to four billion nonzero entries in the constraint matrix—and up to one billion columns and one billion rows. This article mainly concentrates on the energy system model ELMOD, which is a linear optimization model representing the European electricity markets by the use of a nodal pricing market-clearing. It has been widely applied in the literature on energy system analyses in recent years. However, it will be demonstrated that the new solver is also applicable to other energy system models. Y1 - 2022 U6 - https://doi.org/10.1016/j.ejor.2021.06.063 VL - 296 IS - 1 SP - 60 EP - 71 ER - TY - JOUR A1 - Johnston, Jennifer A1 - Kelley, Richard A1 - Crawford, Thomas A1 - Morton, D. A1 - Agarwala, Richa A1 - Koch, Thorsten A1 - Schäffer, Alejandro A1 - Francomano, Clair A1 - Biesecker, Leslie T1 - A novel nemaline myopathy in the Amish caused by a mutation in troponin T1 JF - American Journal of Human Genetics Y1 - 2000 UR - http://www.journals.uchicago.edu/AJHG/journal/issues/v67n4/002057/002057.html VL - 67 SP - 814 EP - 821 ER - TY - JOUR A1 - Koch, Thorsten A1 - Chen, Ying A1 - Lim, Kian Guan A1 - Xu, Xiaofei A1 - Zakiyeva, Nazgul T1 - A review study of functional autoregressive models with application to energy forecasting JF - WIREs Computational Statistics Y1 - 2020 U6 - https://doi.org/10.1002/wics.1525 N1 - https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=7693&context=lkcsb_research ER - TY - GEN A1 - Hiller, Benjamin A1 - Koch, Thorsten A1 - Schewe, Lars A1 - Schwarz, Robert A1 - Schweiger, Jonas T1 - A System to Evaluate Gas Network Capacities: Concepts and Implementation N2 - In 2005 the European Union liberalized the gas market with a disruptive change and decoupled trading of natural gas from its transport. The gas is now transported by independent so-called transmissions system operators or TSOs. The market model established by the European Union views the gas transmission network as a black box, providing shippers (gas traders and consumers) the opportunity to transport gas from any entry to any exit. TSOs are required to offer the maximum possible capacities at each entry and exit such that any resulting gas flow can be realized by the network. The revenue from selling these capacities more than one billion Euro in Germany alone, but overestimating the capacity might compromise the security of supply. Therefore, evaluating the available transport capacities is extremely important to the TSOs. This is a report on a large project in mathematical optimization, set out to develop a new toolset for evaluating gas network capacities. The goals and the challenges as they occurred in the project are described, as well as the developments and design decisions taken to meet the requirements. T3 - ZIB-Report - 18-11 Y1 - 2018 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-67438 SN - 1438-0064 N1 - An earlier version of this report is available as ZR 17-03 at https://opus4.kobv.de/opus4-zib/frontdoor/index/index/docId/6193. ER - TY - JOUR A1 - Hiller, Benjamin A1 - Koch, Thorsten A1 - Schewe, Lars A1 - Schwarz, Robert A1 - Schweiger, Jonas T1 - A System to Evaluate Gas Network Capacities: Concepts and Implementation JF - European Journal of Operational Research N2 - In 2005 the European Union liberalized the gas market with a disruptive change and decoupled trading of natural gas from its transport. The gas is now trans- ported by independent so-called transmissions system operators or TSOs. The market model established by the European Union views the gas transmission network as a black box, providing shippers (gas traders and consumers) the opportunity to transport gas from any entry to any exit. TSOs are required to offer the maximum possible capacities at each entry and exit such that any resulting gas flow can be realized by the network. The revenue from selling these capacities more than one billion Euro in Germany alone, but overestimating the capacity might compromise the security of supply. Therefore, evaluating the available transport capacities is extremely important to the TSOs. This is a report on a large project in mathematical optimization, set out to develop a new toolset for evaluating gas network capacities. The goals and the challenges as they occurred in the project are described, as well as the developments and design decisions taken to meet the requirements. Y1 - 2018 VL - 270 IS - 3 SP - 797 EP - 808 ER - TY - GEN A1 - Hiller, Benjamin A1 - Koch, Thorsten A1 - Schewe, Lars A1 - Schwarz, Robert A1 - Schweiger, Jonas T1 - A System to Evaluate Gas Network Capacities: Concepts and Implementation N2 - Since 2005, the gas market in the European Union is liberalized and the trading of natural gas is decoupled from its transport. The transport is done by so-called transmissions system operators or TSOs. The market model established by the European Union views the gas transmission network as a black box, providing shippers (gas traders and consumers) the opportunity to transport gas from any entry to any exit. TSOs are required to offer maximum independent capacities at each entry and exit such that the resulting gas flows can be realized by the network without compromising security of supply. Therefore, evaluating the available transport capacities is extremely important to the TSOs. This paper gives an overview of the toolset for evaluating gas network capacities that has been developed within the ForNe project, a joint research project of seven research partners initiated by Open Grid Europe, Germany's biggest TSO. While most of the relevant mathematics is described in the book "Evaluating Gas Network Capacities", this article sketches the system as a whole, describes some developments that have taken place recently, and gives some details about the current implementation. T3 - ZIB-Report - 17-03 KW - operations research in energy KW - gas network optimization KW - entry-exit model KW - freely allocable capacity KW - large-scale mixed-integer nonlinear programming Y1 - 2017 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-61931 SN - 1438-0064 N1 - A revised and extended version is available as ZIB-Report 18-11. ER - TY - GEN A1 - Turner, Mark A1 - Koch, Thorsten A1 - Serrano, Felipe A1 - Winkler, Michael T1 - Adaptive Cut Selection in Mixed-Integer Linear Programming N2 - Cut selection is a subroutine used in all modern mixed-integer linear programming solvers with the goal of selecting a subset of generated cuts that induce optimal solver performance. These solvers have millions of parameter combinations, and so are excellent candidates for parameter tuning. Cut selection scoring rules are usually weighted sums of different measurements, where the weights are parameters. We present a parametric family of mixed-integer linear programs together with infinitely many family-wide valid cuts. Some of these cuts can induce integer optimal solutions directly after being applied, while others fail to do so even if an infinite amount are applied. We show for a specific cut selection rule, that any finite grid search of the parameter space will always miss all parameter values, which select integer optimal inducing cuts in an infinite amount of our problems. We propose a variation on the design of existing graph convolutional neural networks, adapting them to learn cut selection rule parameters. We present a reinforcement learning framework for selecting cuts, and train our design using said framework over MIPLIB 2017. Our framework and design show that adaptive cut selection does substantially improve performance over a diverse set of instances, but that finding a single function describing such a rule is difficult. Code for reproducing all experiments is available at https://github.com/Opt-Mucca/Adaptive-Cutsel-MILP. T3 - ZIB-Report - 22-04 KW - cut selection KW - mixed-integer linear programming KW - reinforcement learning Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-86055 SN - 1438-0064 ER - TY - JOUR A1 - Turner, Mark A1 - Koch, Thorsten A1 - Serrano, Felipe A1 - Winkler, Michael T1 - Adaptive Cut Selection in Mixed-Integer Linear Programming JF - Open Journal of Mathematical Optimization N2 - Cutting plane selection is a subroutine used in all modern mixed-integer linear programming solvers with the goal of selecting a subset of generated cuts that induce optimal solver performance. These solvers have millions of parameter combinations, and so are excellent candidates for parameter tuning. Cut selection scoring rules are usually weighted sums of different measurements, where the weights are parameters. We present a parametric family of mixed-integer linear programs together with infinitely many family-wide valid cuts. Some of these cuts can induce integer optimal solutions directly after being applied, while others fail to do so even if an infinite amount are applied. We show for a specific cut selection rule, that any finite grid search of the parameter space will always miss all parameter values, which select integer optimal inducing cuts in an infinite amount of our problems. We propose a variation on the design of existing graph convolutional neural networks, adapting them to learn cut selection rule parameters. We present a reinforcement learning framework for selecting cuts, and train our design using said framework over MIPLIB 2017 and a neural network verification data set. Our framework and design show that adaptive cut selection does substantially improve performance over a diverse set of instances, but that finding a single function describing such a rule is difficult. Code for reproducing all experiments is available at https://github.com/Opt-Mucca/Adaptive-Cutsel-MILP. Y1 - 2023 U6 - https://doi.org/10.5802/ojmo.25 VL - 4 SP - 5 ER - TY - CHAP A1 - Koch, Thorsten A1 - Rehfeldt, Daniel A1 - Shinano, Yuji T1 - An exact high performance solver for Steiner tree problems in graphs and related problems T2 - Modeling, Simulation and Optimization of Complex Processes HPSC 2018 Y1 - 2020 PB - Springer ER - TY - CHAP A1 - Cook, William A1 - Koch, Thorsten A1 - Steffy, Daniel A1 - Wolter, Kati ED - Günlük, Oktay ED - Woeginger, Gerhard T1 - An Exact Rational Mixed-Integer Programming Solver T2 - IPCO 2011 Y1 - 2011 U6 - https://doi.org/10.1007/978-3-642-20807-2_9 VL - 6655 SP - 104 EP - 116 ER - TY - GEN A1 - Cook, William A1 - Koch, Thorsten A1 - Steffy, Daniel A1 - Wolter, Kati T1 - An Exact Rational Mixed-Integer Programming Solver N2 - We present an exact rational solver for mixed-integer linear programming that avoids the numerical inaccuracies inherent in the floating-point computations used by existing software. This allows the solver to be used for establishing theoretical results and in applications where correct solutions are critical due to legal and financial consequences. Our solver is a hybrid symbolic/numeric implementation of LP-based branch-and-bound, using numerically-safe methods for all binding computations in the search tree. Computing provably accurate solutions by dynamically choosing the fastest of several safe dual bounding methods depending on the structure of the instance, our exact solver is only moderately slower than an inexact floating-point branch-and-bound solver. The software is incorporated into the SCIP optimization framework, using the exact LP solver QSopt_ex and the GMP arithmetic library. Computational results are presented for a suite of test instances taken from the MIPLIB and Mittelmann collections. T3 - ZIB-Report - 11-07 KW - mixed integer programming KW - branch-and-bound KW - exact computation Y1 - 2011 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-12329 ER - TY - JOUR A1 - Hoppmann-Baum, Kai A1 - Hennings, Felix A1 - Zittel, Janina A1 - Gotzes, Uwe A1 - Spreckelsen, Eva-Maria A1 - Spreckelsen, Klaus A1 - Koch, Thorsten T1 - An Optimization Approach for the Transient Control of Hydrogen Transport Networks JF - Mathematical Methods of Operations Research Y1 - 2021 IS - Special Issue on Energy Networks ER - TY - JOUR A1 - Chen, Ying A1 - Koch, Thorsten A1 - Zakiyeva, Nazgul A1 - Liu, Kailiang A1 - Xu, Zhitong A1 - Chen, Chun-houh A1 - Nakano, Junji A1 - Honda, Keisuke T1 - Article's Scientific Prestige: Measuring the Impact of Individual Articles in the Web of Science N2 - We performed a citation analysis on the Web of Science publications consisting of more than 63 million articles and 1.45 billion citations on 254 subjects from 1981 to 2020. We proposed the Article’s Scientific Prestige (ASP) metric and compared this metric to number of citations (#Cit) and journal grade in measuring the scientific impact of individual articles in the large-scale hierarchical and multi-disciplined citation network. In contrast to #Cit, ASP, that is computed based on the eigenvector centrality, considers both direct and indirect citations, and provides steady-state evaluation cross different disciplines. We found that ASP and #Cit are not aligned for most articles, with a growing mismatch amongst the less cited articles. While both metrics are reliable for evaluating the prestige of articles such as Nobel Prize winning articles, ASP tends to provide more persuasive rankings than #Cit when the articles are not highly cited. The journal grade, that is eventually determined by a few highly cited articles, is unable to properly reflect the scientific impact of individual articles. The number of references and coauthors are less relevant to scientific impact, but subjects do make a difference. T3 - ZIB-Report - 22-07 Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0297-zib-86380 VL - 17 IS - 1 SP - 101379 ER -