Why It Is Time for Yet Another Schema Evolution Benchmark - Visionary Paper

Database schema evolution is one of the grand challenges in data management research and practice. In this paper, we survey available schema evolution benchmarks. We argue that existing benchmarks do not reflect the more recent demands on database applications, such as online data migration for high availability applications, agile release strategies, and schema changes that affect more than one data model. We conclude that a new generation of schema evolution benchmarks is called for, which reflect these new demands, especially in the context of schema evolution in NoSQL data stores.

[1]  Adam Silberstein,et al.  Benchmarking cloud serving systems with YCSB , 2010, SoCC '10.

[2]  Meike Klettke,et al.  NoSQL schema evolution and big data migration at scale , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[3]  Jeyhun Karimov,et al.  PolyBench: The First Benchmark for Polystores , 2018, TPCTC.

[4]  Isil Dillig,et al.  Synthesizing database programs for schema refactoring , 2019, PLDI.

[5]  VassiliadisPanos,et al.  Growing up with stability , 2015 .

[6]  John F. Roddick,et al.  A survey of schema versioning issues for database systems , 1995, Inf. Softw. Technol..

[7]  Wolfgang Lehner,et al.  Living in Parallel Realities: Co-Existing Schema Versions with a Bidirectional Database Evolution Language , 2017, SIGMOD Conference.

[8]  Apostolos V. Zarras,et al.  Growing up with stability: How open-source relational databases evolve , 2015, Inf. Syst..

[9]  Meike Klettke,et al.  Managing Schema Evolution in NoSQL Data Stores , 2013, DBPL.

[10]  Jiaheng Lu,et al.  UniBench: A Benchmark for Multi-model Database Management Systems , 2018, TPCTC.

[11]  Mark Lukas Möller,et al.  Query Rewriting for Continuously Evolving NoSQL Databases , 2019, ER.

[12]  Dong Qiu,et al.  An empirical analysis of the co-evolution of schema and code in database applications , 2013, ESEC/FSE 2013.

[13]  Wang Chiew Tan,et al.  STBenchmark: towards a benchmark for mapping systems , 2008, Proc. VLDB Endow..

[14]  Edgar H. Sibley,et al.  Evolution of Data-Base Management Systems , 1976, CSUR.

[15]  Marieke Huisman,et al.  A Benchmark for Online Non-blocking Schema Transformations , 2015, DATA.

[16]  Meike Klettke,et al.  MigCast: Putting a Price Tag on Data Model Evolution in NoSQL Data Stores , 2019, SIGMOD Conference.

[17]  Tudor Dumitras,et al.  Evolving NoSQL Databases without Downtime , 2015, 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[18]  Iulian Neamtiu,et al.  Collateral evolution of applications and databases , 2009, IWPSE-Evol '09.

[19]  Xiaoyi Lu,et al.  On Big Data Benchmarking , 2014, BPOE@ASPLOS/VLDB.

[20]  Scott J. Ambler,et al.  Refactoring Databases: Evolutionary Database Design , 2006 .

[21]  Iulian Neamtiu,et al.  Schema evolution analysis for embedded databases , 2011, 2011 IEEE 27th International Conference on Data Engineering Workshops.

[22]  Lizy Kurian John,et al.  Benchmarking Big Data Systems: A Review , 2018, IEEE Transactions on Services Computing.

[23]  Carlo Curino,et al.  Schema Evolution in Wikipedia - Toward a Web Information System Benchmark , 2008, ICEIS.