Curating Variational Data in Application Development
- Building applications for processing data lakes is a software engineering challenge. We present Darwin, a middleware for applications that operate on variational data. This concerns data with heterogeneous structure, usually stored within a schema-flexible NoSQL database. Darwin assists application developers in essential data and schema curation tasks: Upon request, Darwin extracts a schema description, discovers the history of schema versions, and proposes mappings between these versions. Users of Darwin may interactively choose which mappings are most realistic. Darwin is further capable of rewriting queries at runtime, to ensure that queries also comply with legacy data. Alternatively, Darwin can migrate legacy data to reduce the structural heterogeneity. Using Darwin, developers may thus evolve their data in sync with their code. In our hands-on demo, we curate synthetic as well as real-life datasets.
Author: | Uta StörlORCiDGND, Daniel Müller, Alexander Tekleab, Stephane Tolale, Julian Stenzel, Meike Klettke, Stefanie ScherzingerORCiD, Uta StorlORCiDGND, Daniel Muller |
---|---|
DOI: | https://doi.org/10.1109/ICDE.2018.00187 |
Parent Title (English): | 2018 IEEE 34th International Conference on Data Engineering, 16-19 April 2018, Paris, France |
Publisher: | IEEE |
Document Type: | conference proceeding (article) |
Language: | English |
Year of first Publication: | 2018 |
Release Date: | 2022/05/05 |
Tag: | Data mining; Evolution (biology); NoSQL databases; Software; Task analysis; data migration; history; query rewriting; schema evolution; schema management; variational data |
First Page: | 1605 |
Last Page: | 1608 |
Institutes: | Fakultät Informatik und Mathematik |
Begutachtungsstatus: | peer-reviewed |
research focus: | Digitalisierung |
Licence (German): | Keine Lizenz - Es gilt das deutsche Urheberrecht: § 53 UrhG |