Online, Asynchronous Schema Change in F1

We introduce a protocol for schema evolution in a globally distributed database management system with shared data, stateless servers, and no global membership. Our protocol is asynchronous--it allows different servers in the database system to transition to a new schema at different times--and online--all servers can access and update all data during a schema change. We provide a formal model for determining the correctness of schema changes under these conditions, and we demonstrate that many common schema changes can cause anomalies and database corruption. We avoid these problems by replacing corruption-causing schema changes with a sequence of schema changes that is guaranteed to avoid corrupting the database so long as all servers are no more than one schema version behind at any time. Finally, we discuss a practical implementation of our protocol in F1, the database management system that stores data for Google AdWords.

[1]  Gottfried Vossen,et al.  Transactional Information Systems: Theory, Algorithms, and the Practice of Concurrency Control and Recovery , 2002 .

[2]  Carlo Curino,et al.  The PRISM Workwench: Database Schema Evolution without Tears , 2009, 2009 IEEE 25th International Conference on Data Engineering.

[3]  Nina Edelweiss,et al.  Schema versioning: queries to the generalized temporal database system , 1999, Proceedings. Tenth International Workshop on Database and Expert Systems Applications. DEXA 99.

[4]  Zohra Bellahsene.  Schema Evolution in Data Warehouses , 2002, Knowledge and Information Systems.

[5]  Frank Dabek,et al.  Large-scale Incremental Processing Using Distributed Transactions and Notifications , 2010, OSDI.

[6]  H SchollMarc,et al.  Transactional information systems , 2001 .

[7]  John F. Roddick,et al.  Schema evolution in database systems: an annotated bibliography , 1992, SGMD.

[8]  Angela Bonifati,et al.  Schema mapping verification: the spicy way , 2008, EDBT '08.

[9]  Philip A. Bernstein,et al.  Automated Co-evolution of Conceptual Models, Physical Databases, and Mappings , 2010, ER.

[10]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[11]  Christopher Frost,et al.  Spanner: Google's Globally-Distributed Database , 2012, OSDI.

[12]  Eladio Domínguez,et al.  Model-Driven, View-Based Evolution of Relational Databases , 2008, DEXA.

[13]  Robert H. Thomas,et al.  A Majority consensus approach to concurrency control for multiple copy databases , 1979, ACM Trans. Database Syst..

[14]  Carsten Sapia,et al.  On Schema Evolution in Multidimensional Databases , 1999, DaWaK.

[15]  B Praveen Kumar,et al.  Mariposa a Wide-Area Distributed Database System , 2010, ICCA 2010.

[16]  Matteo Golfarelli,et al.  X-Time: Schema Versioning and Cross-Version Querying in Data Warehouses , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[17]  John F. Roddick,et al.  SQL/SE: a query language extension for databases supporting schema evolution , 1992, SGMD.

[18]  Elke A. Rundensteiner,et al.  Maintaining data warehouses over changing information sources , 2000, CACM.

[19]  Mikael Ronström.  On-Line Schema Update for a Telecom Database , 2000, ICDE.

[20]  Jørgen Løland,et al.  A Non-Blocking Relational Schema Change , 2022 .

[21]  Ian Rae,et al.  F1: A Distributed SQL Database That Scales , 2013, Proc. VLDB Endow..

[22]  Svein-Olaf Hvasshovd,et al.  Online, Non-blocking Relational Schema Changes , 2006, EDBT.

[23]  Carlo Curino,et al.  Update Rewriting and Integrity Constraint Maintenance in a Schema Evolution Support System: PRISM++ , 2010, Proc. VLDB Endow..

[24]  Michael Stonebraker,et al.  Mariposa: a wide-area distributed database system , 1996, The VLDB Journal.

[25]  Philip A. Bernstein,et al.  Model Management and Schema Mappings: Theory and Practice , 2007, VLDB.

[26]  George Papastefanatos,et al.  Language Extensions for the Automation of Database Schema Evolution , 2008, ICEIS.

[27]  Alexandra Poulovassilis,et al.  Schema Evolution in Heterogeneous Database Architectures, A Schema Transformation Approach , 2002, CAiSE.