• search hit 2 of 96
Back to Result List

Don’t Tune Twice: Reusing Tuning Setups for SQL-on-Hadoop Queries

  • SQL-on-Hadoop processing engines have become state-of-the-art in data lake analysis. However, the skills required to tune such systems are rare. This has inspired automated tuning advisors which profile the query workload and produce tuning setups for the low-level MapReduce jobs. Yet with highly dynamic query workloads, repeated re-tuning costs time and money in IaaS environments. In this paper, we focus on reducing the costs for up-front tuning. At the heart of our approach is the observation that a SQL query is compiled into a query plan of MapReduce jobs. While the plans differ from query to query, single jobs tend to be similar between queries. We introduce the notion of the code signature of a MapReduce job and, based on this, our concept of job similarity. We show that we can effectively recycle tuning setups from similar MapReduce jobs already profiled. In doing so, we can leverage any third-party tuning adviser for MapReduce engines. We are able to show that by recycling tuning setups, we can reduce the time spent on profiling by 50% in the TPC-H benchmark.

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics
Metadaten
Author:Edson Ramiro Lucas Filho, Eduardo Cunha de Almeida, Stefanie ScherzingerORCiD
DOI:https://doi.org/10.1007/978-3-030-33223-5_9
Parent Title (English):Conceptual Modeling : 38th International Conference, ER 2019, Salvador, Brazil, November 4-7, 2019, Proceedings
Publisher:Springer
Place of publication:Cham
Document Type:conference proceeding (article)
Language:English
Year of first Publication:2019
Release Date:2022/07/11
First Page:93
Last Page:107
Andere Schriftenreihe:Lecture Notes in Computer Science ; 11788
Institutes:Fakultät Informatik und Mathematik
research focus:Digitalisierung
Licence (German):Keine Lizenz - Es gilt das deutsche Urheberrecht: § 53 UrhG