TY - CHAP A1 - Gholami, Masoud A1 - Schintke, Florian A1 - Schütt, Thorsten T1 - Checkpoint Scheduling for Shared Usage of Burst-Buffers in Supercomputers T2 - Proceedings of the 47th International Conference on Parallel Processing Companion; SRMPDS 2018: The 14th International Workshop on Scheduling and Resource Management for Parallel and Distributed Systems N2 - User-defined and system-level checkpointing have contrary properties. While user-defined checkpoints are smaller and simpler to recover, system-level checkpointing better knows the global system's state and parameters like the expected mean time to failure (MTTF) per node. Both approaches lead to non-optimal checkpoint time, intervals, sizes, or I/O bandwidth when concurrent checkpoints conflict and compete for it. We combine user-defined and system-level checkpointing to exploit the benefits and avoid the drawbacks of each other. Thus, applications frequently offer to create checkpoints. The system accepts such offers according to the current status and implied costs to recalculate from the last checkpoint or denies them, i.e., immediately lets continue the application without checkpoint creation. To support this approach, we develop economic models for multi-application checkpointing on shared I/O resources that are dedicated for checkpointing (e.g. burst-buffers) by defining an appropriate goal function and solving a global optimization problem. Using our models, the checkpoints of applications on a supercomputer are scheduled to effectively use the available I/O bandwidth and minimize the failure overhead (checkpoint creations plus recalculations). Our simulations show an overall reduction in failure overhead of all nodes of up to 30% for a typical supercomputer workload (HLRN). We can also derive the most cost effective burst-buffer bandwidth for a given node's MTTF and application workload. Y1 - 2018 U6 - https://doi.org/10.1145/3229710.3229755 SP - 44:1 EP - 44:10 ER - TY - CHAP A1 - Weinhold, Carsten A1 - Lackorzynski, Adam A1 - Bierbaum, Jan A1 - Küttler, Martin A1 - Planeta, Maksym A1 - Weisbach, Hannes A1 - Hille, Matthias A1 - Härtig, Hermann A1 - Margolin, Alexander A1 - Sharf, Dror A1 - Levy, Ely A1 - Gak, Pavel A1 - Barak, Amnon A1 - Gholami, Masoud A1 - Schintke, Florian A1 - Schütt, Thorsten A1 - Reinefeld, Alexander A1 - Lieber, Matthias A1 - Nagel, Wolfgang T1 - FFMK: A Fast and Fault-Tolerant Microkernel-Based System for Exascale Computing T2 - Software for Exascale Computing - SPPEXA 2016-2019 Y1 - 2019 U6 - https://doi.org/10.1007/978-3-030-47956-5_16 SP - 483 EP - 516 PB - Springer ER - TY - CHAP A1 - Gholami, Masoud A1 - Schintke, Florian T1 - Multilevel Checkpoint/Restart for Large Computational Jobs on Distributed Computing Resources T2 - 2019 IEEE 38th Symposium on Reliable Distributed Systems (SRDS) Y1 - 2019 U6 - https://doi.org/10.1109/SRDS47363.2019.00025 SP - 143 EP - 152 ER - TY - CHAP A1 - Kleineweber, Christoph A1 - Reinefeld, Alexander A1 - Schütt, Thorsten T1 - QoS-aware Storage Virtualization for Cloud File Systems T2 - Proceedings of the 1st ACM International Workshop on Programmable File Systems Y1 - 2014 U6 - https://doi.org/10.1145/2603941.2603944 SP - 19 EP - 26 ER - TY - CHAP A1 - Wende, Florian A1 - Steinke, Thomas A1 - Reinefeld, Alexander ED - Gray, A. ED - Smith, L. ED - Weiland, M. T1 - The Impact of Process Placement and Oversubscription on Application Performance: A Case Study for Exascale Computing T2 - Proceedings of the 3rd International Conference on Exascale Applications and Software, EASC 2015 Y1 - 2015 SN - 978 -0-9 926615 -1-9 SP - 13 EP - 18 PB - The University of Edinburgh ER - TY - CHAP A1 - Weinhold, Carsten A1 - Lackorzynski, Adam A1 - Bierbaum, Jan A1 - Küttler, Martin A1 - Planeta, Maksym A1 - Härtig, Hermann A1 - Shiloh, Amnon A1 - Levy, Ely A1 - Ben-Nun, Tal A1 - Barak, Amnon A1 - Steinke, Thomas A1 - Schütt, Thorsten A1 - Fajerski, Jan A1 - Reinefeld, Alexander A1 - Lieber, Matthias A1 - Nagel, Wolfgang T1 - FFMK: A Fast and Fault-tolerant Microkernel-based System for Exascale Computing T2 - SPPEXA Symposium 2016 Y1 - 2016 U6 - https://doi.org/10.1007/978-3-319-40528-5_18 ER - TY - CHAP A1 - Fajerski, J. A1 - Noack, Matthias A1 - Reinefeld, Alexander A1 - Schintke, Florian A1 - Schütt, Thorsten A1 - Steinke, Thomas T1 - Fast In-Memory Checkpointing with POSIX API for Legacy Exascale-Applications T2 - SPPEXA Symposium 2016 Y1 - 2016 U6 - https://doi.org/10.1007/978-3-319-40528-5_19 ER - TY - CHAP A1 - Gholami, Masoud A1 - Schintke, Florian T1 - Combining XOR and Partner Checkpointing for Resilient Multilevel Checkpoint/Restart T2 - 2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS) Y1 - 2021 U6 - https://doi.org/10.1109/IPDPS49936.2021.00036 SP - 277 EP - 288 ER -