Enterprise Application-Specific Data Management

Enterprise applications are presently built on a 20-year old data management infrastructure that was designed to meet a specific set of requirements for OLTP systems. In the meantime, enterprise applications have become more sophisticated, data set sizes have increased, requirements on the freshness of input data have been strengthened, and the time allotted for completing business processes has been reduced. To meet these challenges, enterprise applications have become increasingly complicated to make up for short-comings in the data management infrastructure. To address this issue we investigate recent trends in data management such as main memory databases, column stores and compression techniques with regards to the workload requirements and data characteristics derived from actual customer systems. We show that a main memory column store is better suited for to days enterprise systems, which we validate by using SAP’s Net Weaver Business Warehouse Accelerator and a realistic set of data from an inventory management application.

[1]  Antoni Wolski,et al.  Lazy Aggregates for Real-Time OLAP , 1999, DaWaK.

[2]  Martin L. Kersten,et al.  Database Architecture Optimized for the New Bottleneck: Memory Access , 1999, VLDB.

[3]  Evaggelia Pitoura,et al.  ETL queues for active data warehousing , 2005, IQIS '05.

[4]  David J. DeWitt,et al.  A case for fractured mirrors , 2003, The VLDB Journal.

[5]  Daniel J. Abadi,et al.  Integrating compression and execution in column-oriented database systems , 2006, SIGMOD Conference.

[6]  Alexander Zeier,et al.  Optimizing Write Performance for Read Optimized Databases , 2010, DASFAA.

[7]  Werner Vogels,et al.  Dynamo: amazon's highly available key-value store , 2007, SOSP.

[8]  Setrag Khoshafian,et al.  A decomposition storage model , 1985, SIGMOD Conference.

[9]  Hasso Plattner,et al.  A common database approach for OLTP and OLAP using an in-memory column database , 2009, SIGMOD Conference.

[10]  Michael Stonebraker,et al.  The End of an Architectural Era (It's Time for a Complete Rewrite) , 2007, VLDB.

[11]  Clark D. French,et al.  “One size fits all” database architectures do not work for DSS , 1995, SIGMOD '95.

[12]  Alexander Zeier,et al.  A Hybrid Row-Column OLTP Database Architecture for Operational Reporting , 2008, BIRTE.

[13]  W. H. Inmon,et al.  Building the Operational Data Store , 1995 .

[14]  Holger Pirk,et al.  Cache Conscious Data Layouting for In-Memory Databases , 2010 .

[15]  Peter Sanders,et al.  Compressed Inverted Indexes for In-Memory Search Engines , 2008, ALENEX.

[16]  Per-Åke Larson,et al.  Eager Aggregation and Lazy Aggregation , 1995, VLDB.

[17]  Sang Kyun Cha,et al.  P*TIME: Highly Scalable OLTP DBMS for Managing Update-Intensive Stream Workload , 2004, VLDB.

[18]  Sven Helmer,et al.  The implementation and performance of compressed databases , 2000, SGMD.

[19]  F. Tödtling,et al.  One size fits all?: Towards a differentiated regional innovation policy approach , 2005 .

[20]  Gordon V. Cormack,et al.  Data compression on a database system , 1985, CACM.

[21]  Hector Garcia-Molina,et al.  Main Memory Database Systems: An Overview , 1992, IEEE Trans. Knowl. Data Eng..

[22]  Marcin Zukowski,et al.  MonetDB/X100: Hyper-Pipelining Query Execution , 2005, CIDR.

[23]  Henrik Loeser,et al.  "One Size Fits All": An Idea Whose Time Has Come and Gone? , 2011, BTW.

[24]  Alexander Zeier,et al.  Vertical partioning in insert-only scenarios for enterprise applications , 2009, 2009 16th International Conference on Industrial Engineering and Engineering Management.

[25]  Martin L. Kersten,et al.  Generic Database Cost Models for Hierarchical Memory Systems , 2002, VLDB.

[26]  Michael Stonebraker,et al.  C-Store: A Column-oriented DBMS , 2005, VLDB.

[27]  Michael Stonebraker,et al.  Implementation techniques for main memory database systems , 1984, SIGMOD '84.

[28]  Nihar R. Mahapatra,et al.  The processor-memory bottleneck: problems and solutions , 1999, CROS.

[29]  David J. DeWitt,et al.  DBMSs on a Modern Processor: Where Does Time Go? , 1999, VLDB.

[30]  David J. DeWitt,et al.  Materialization Strategies in a Column-Oriented DBMS , 2007, 2007 IEEE 23rd International Conference on Data Engineering.