The search result changed since you submitted your search request. Documents might be displayed in a different sort order.
  • search hit 2 of 16
Back to Result List

Uncovering the evolution history of data lakes

  • Data accumulating in data lakes can become inaccessible in the long run when its semantics are not available. The heterogeneity of data formats and the sheer volumes of data collections prohibit cleaning and unifying the data manually. Thus, tools for automated data lake analysis are of great interest. In this paper, we target the particular problem of reconstructing the schema evolution history from data lakes. Knowing how the data is structured, and how this structure has evolved over time, enables programmatic access to the lake. By deriving a sequence of schema versions, rather than a single schema, we take into account structural changes over time. Moreover, we address the challenge of detecting inclusion dependencies. This is a prerequisite for mapping between succeeding schema versions, and in particular, detecting nontrivial changes such as a property having been moved or copied. We evaluate our approach for detecting inclusion dependencies using the MovieLens dataset, as well an adaption of a dataset containing botanical descriptions, to cover specific edge cases.

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics
Metadaten
Author:Meike Klettke, Hannes Awolin, Uta StörlORCiDGND, Daniel Müller, Stefanie ScherzingerORCiD, Uta StorlORCiDGND, Daniel Muller
DOI:https://doi.org/10.1109/BigData.2017.8258204
Parent Title (English):2017 IEEE International Conference on Big Data (Big Data),,11-14 Dec. 2017, Boston, MA, USA
Publisher:IEEE
Document Type:conference proceeding (article)
Language:English
Year of first Publication:2017
Release Date:2022/05/05
Tag:Data mining; Grippers; Lakes; NoSQL databases; Protocols; evolution operations; history; inclusion dependencies; integrity constraints; schema version extraction
First Page:2462
Last Page:2471
Institutes:Fakultät Informatik und Mathematik
Begutachtungsstatus:peer-reviewed
research focus:Digitalisierung
Licence (German):Keine Lizenz - Es gilt das deutsche Urheberrecht: § 53 UrhG