TY - CHAP A1 - Schmidtke, Robert A1 - Schintke, Florian A1 - Schütt, Thorsten T1 - From Application to Disk: Tracing I/O Through the Big Data Stack T2 - High Performance Computing ISC High Performance 2018 International Workshops, Frankfurt/Main, Germany, June 24 - 28, 2018, Revised Selected Papers, Workshop on Performance and Scalability of Storage Systems (WOPSSS) N2 - Typical applications in data science consume, process and produce large amounts of data, making disk I/O one of the dominating — and thus worthwhile optimizing — factors of their overall performance. Distributed processing frameworks, such as Hadoop, Flink and Spark, hide a lot of complexity from the programmer when they parallelize these applications across a compute cluster. This exacerbates reasoning about I/O of both the application and the framework, through the distributed file system, such as HDFS, down to the local file systems. We present SFS (Statistics File System), a modular framework to trace each I/O request issued by the application and any JVM-based big data framework involved, mapping these requests to actual disk I/O. This allows detection of inefficient I/O patterns, both by the applications and the underlying frameworks, and builds the basis for improving I/O scheduling in the big data software stack. Y1 - 2018 U6 - https://doi.org/10.1007/978-3-030-02465-9_6 SP - 89 EP - 102 ER - TY - CHAP A1 - Schmidtke, Robert A1 - Laubender, Guido A1 - Steinke, Thomas T1 - Big Data Analytics on Cray XC Series DataWarp using Hadoop, Spark and Flink T2 - CUG Proceedings Y1 - 2016 ER - TY - JOUR A1 - Schmidtke, Robert A1 - Erleben, Kenny T1 - Chunked Bounding Volume Hierarchies for Fast Digital Prototyping using Volumetric Meshes JF - IEEE Transactions on Visualization and Computer Graphics N2 - We present a novel approach to using Bounding Volume Hierarchies (BVHs) for collision detection of volumetric meshes for digital prototyping based on accurate simulation. In general, volumetric meshes contain more primitives than surface meshes, which in turn means larger BVHs. To manage these larger BVHs, we propose an algorithm for splitting meshes into smaller chunks with a limited-size BVH each. Limited-height BVHs make guided, all-pairs testing of two chunked meshes well-suited for GPU implementation. This is because the dynamically generated work during BVH traversal becomes bounded. Chunking is simple to implement compared to dynamic load balancing methods and can result in an overall two orders of magnitude speedup on GPUs. This indicates that dynamic load balancing may not be a well suited scheme for the GPU. The overall application timings showed that data transfers were not the bottleneck. Instead, the conversion to and from OpenCL friendly data structures was causing serious performance impediments. Still, a simple OpenMP acceleration of the conversion allowed the GPU solution to beat the CPU solution by 20%. We demonstrate our results using rigid and deformable body scenes of varying complexities on a variety of GPUs. Y1 - 2018 U6 - https://doi.org/10.1109/TVCG.2017.2784441 SN - 1941-0506 VL - 24 IS - 12 SP - 3044 EP - 3057 ER -