NoSQL schema evolution and big data migration at scale

  • This paper explores scalable implementation strategies for carrying out lazy schema evolution in NoSQL data stores. For decades, schema evolution has been an evergreen in database research. Yet new challenges arise in the context of cloud-hosted data backends: With all database reads and writes charged by the provider, migrating the entire data instance eagerly into a new schema can be prohibitively expensive. Thus, lazy migration may be more cost-efficient, as legacy entities are only migrated in case they are actually accessed by the application. Related work has shown that the overhead of migrating data lazily is affordable when a single evolutionary change is carried out, such as adding a new property. In this paper, we focus on long-term schema evolution, where chains of pending schema evolution operations may have to be applied. Chains occur when legacy entities written several application releases back are finally accessed by the application. We discuss strategies for dealing with chains of evolution operations, in particular, the composition into a single, equivalent composite migration that performs the required version jump. Our experiments with MongoDB focus on scalable implementation strategies. Our lineup further compares the number of write operations, and thus, the operational costs of different data migration strategies.

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics
Metadaten
Author:Meike Klettke, Uta StörlORCiDGND, Manuel Shenavai, Stefanie ScherzingerORCiD, Uta StorlORCiDGND
DOI:https://doi.org/10.1109/BigData.2016.7840924
Parent Title (English):2016 IEEE International Conference on Big Data (Big Data), 5-8 Dec. 2016, Washington, DC
Publisher:IEEE
Document Type:conference proceeding (article)
Language:English
Year of first Publication:2016
Release Date:2022/05/05
Tag:Big data; Context; Data Migration Strategies; Data models; Databases; Incremental Migration; Lazy Composite Migration; Lazy Migration; NoSQL databases; Predictive Migration; Production; Runtime; Software; schema evolution
First Page:2764
Last Page:2774
Institutes:Fakultät Informatik und Mathematik
Begutachtungsstatus:peer-reviewed
research focus:Information und Kommunikation
Licence (German):Keine Lizenz - Es gilt das deutsche Urheberrecht: § 53 UrhG