A review of the rationale and architectures of PJama - a durable, flexible, evolvable and scalable orthogonally persistent programming platform

A primary goal of research into orthogonal persistence is to simplify significantly the construction, maintenance and operation of applications in order to save software costs, extend the range of applications and improve users' experiences. To test such claims we need relevant experiments. To mount such experiments requires an industrial-strength persistent programming platform. The PJama project is an attempt to build such a platform and initiate those experiments. We report our design decisions and their consequences evaluated by four years of experience. We have reached a range of platforms, demonstrated orthogonality and provided durability, schema evolution with instance reformatting, platform migration and recovery. The application programming interface is now close to minimal, while we support open systems through a resumable-programming model. Our architecture is flexible and supports a range of optimisations. Performance measurements and current applications attest to our progress, but it is still possible to identify major research questions, and the experiments to test the utility of orthogonal persistence are still in their early stages.

[1]  Ben Shneiderman,et al.  Visual information seeking: tight coupling of dynamic query filters with starfield displays , 1994, CHI '94.

[2]  Ronald Morrison,et al.  An Approach to Persistent Programming , 1989, Comput. J..

[3]  Liwu Li,et al.  Java Virtual Machine , 1998 .

[4]  Mahadev Satyanarayanan,et al.  Lightweight recoverable virtual memory , 1993, SOSP '93.

[5]  W. Paul Cockshott,et al.  High-Performance Operations Using a Compressed Database Architecture , 1998, Comput. J..

[6]  Joachim W. Schmidt,et al.  Next Generation Information System Technology , 1991, Lecture Notes in Computer Science.

[7]  Huw Evans,et al.  Porting a Distributed System to PJama: Orthogonal Persistence for Java? , 1998, POS/PJW.

[8]  François Bancilhon,et al.  Building an Object-Oriented Database System, The Story of O2 , 1992 .

[9]  Patrick Valduriez,et al.  Customizable Concurrency Control for Persistent Java , 1997, Advanced Transaction Models and Architectures.

[10]  Alex Garthwaite,et al.  The GC Interface in the EVM 1 , 1998 .

[11]  Malcolm P. Atkinson,et al.  Evolutionary Data Conversion in the PJama Persistent Language , 1999, ECOOP Workshops.

[12]  Michael Stonebraker,et al.  Readings in Database Systems , 1988 .

[13]  Duane Szafron,et al.  Type system for an object-oriented database programming language , 1999 .

[14]  Florian Matthes,et al.  An open system architecture for a persistent object store , 1992 .

[15]  David Maier,et al.  Persistent Object Systems , 1995, Workshops in Computing.

[16]  Malcolm P. Atkinson,et al.  Orthogonal Persistence for Java? - A Mid-term Report , 1998, POS/PJW.

[17]  M. F. Challis The JACKDAW database package , 1974 .

[18]  P. R. Wilson,et al.  Pointer swizzling at page fault time: efficiently and compatibly supporting huge address spaces on standard hardware , 1992, [1992] Proceedings of the Second International Workshop on Object Orientation in Operating Systems.

[19]  C. Mohan,et al.  An efficient and flexible method for archiving a data base , 1993, SIGMOD Conference.

[20]  Ronald Morrison,et al.  Variadic Genericity Through Linguistic Reflection: A Performance Evaluation , 1998, POS/PJW.

[21]  Graham Hamilton,et al.  Jdbc Database Access with Java: A Tutorial and Annotated Reference , 1997 .

[22]  Alfred L. Brown,et al.  Persistent object stores , 1988 .

[24]  Frank Yellin,et al.  The Java Virtual Machine Specification , 1996 .

[25]  Rajiv Arora,et al.  Java server performance: A case study of building efficient, scalable Jvms , 2000, IBM Syst. J..

[26]  Tony Printezis Analysing a Simple Disk Garbage Collector , 1996 .

[27]  Ronald Morrison,et al.  Hyper-Programming in Java , 1999, POS/PJW.

[28]  Melanie E. Goward,et al.  The DNA sequence of human chromosome 22 , 1999, Nature.

[29]  David Maier,et al.  Proceedings of the Sixth International Workshop on Persistent Object Systems , 1994 .

[30]  Daniel Schneider,et al.  Automatic Persistent Memory Management for the Spotless Java Virtual Machine on the Palm Connected Organizer , 2001, Java Virtual Machine Research and Technology Symposium.

[31]  David Maier,et al.  Persistent object systems : proceedings of the Sixth International Workshop on Persistent Object Systems, Tarascon, Provence, France, 5-9 September 1994 , 1995 .

[32]  Barbara Liskov,et al.  Distributed programming in Argus , 1988, CACM.

[33]  Malcolm P. Atkinson,et al.  Bulk Types for Large Scale Programming , 1990, East/West Database Workshop.

[34]  Laurent Daynès,et al.  High-performance, space-efficient, automated object locking , 2001, Proceedings 17th International Conference on Data Engineering.

[35]  Malcolm P. Atkinson,et al.  Evaluating Usability Aspects of PJama Based on Source Code Measurements , 1998, POS/PJW.

[36]  Craig Chambers,et al.  Debugging optimized code with dynamic deoptimization , 1992, PLDI '92.

[37]  Sophie Cluet,et al.  Database Programming Languages , 1997, Lecture Notes in Computer Science.

[38]  Guy L. Steele,et al.  The Java Language Specification , 1996 .

[39]  WilkesJohn,et al.  The HP AutoRAID hierarchical storage system , 1996 .

[40]  Alan Jay Smith,et al.  Analysis of the Characteristics of Production Database Workloads and Comparison with the TPC Benchmarks , 1999 .

[41]  David J. DeWitt,et al.  The oo7 Benchmark , 1993, SIGMOD Conference.

[42]  Roberto V. Zicari,et al.  Overview and Progress Report of the ESSE Project: Supporting Object-Oriented Database Schema Analysis and Evolution , 1993, TOOLS.

[43]  Miguel Mira da Silva,et al.  Automating Type-safe RPC , 1995, RIDE-DOM.

[44]  Michael J. Carey,et al.  The design of the E programming language , 1993, TOPL.

[45]  John Rosenberg,et al.  Grasshopper: An Orthogonally Persistent Operating System , 1994, Comput. Syst..

[46]  Jochen Liedtke,et al.  The mungi single‐address‐space operating system , 1998, Softw. Pract. Exp..

[47]  Narain H. Gehani,et al.  ODE (Object Database and Environment): the language and the data model , 1989, SIGMOD '89.

[48]  Edward D. Lazowska,et al.  Techniques for file system simulation , 1994, Softw. Pract. Exp..

[49]  Quintin I. Cutts,et al.  Optimizing the Read and Write Barriers for Orthogonal Persistence , 1998, POS/PJW.

[50]  Satoshi Matsuoka,et al.  OpenJIT: An Open-Ended, Reflective JIT Compiler Framework for Java , 2000, ECOOP.

[51]  Malcolm P. Atkinson,et al.  Issues Raised by Three Years of Developing PJama: An Orthogonally Persistent Platform for Java , 1999, ICDT.

[52]  Toshiaki Yasue,et al.  Overview of the IBM Java Just-in-Time Compiler , 2000, IBM Syst. J..

[53]  Malcolm P. Atkinson,et al.  Persistent Foudations for Scalable Multi-Paradigmal Systems , 1992, IWDOM.

[54]  Wolfgang Klas,et al.  Persistence in the Object-Oriented Database Programming Language VML , 1992 .

[55]  Ela Hunt,et al.  Case study: Use of computer tools in locating a human disease gene , 1999 .

[56]  David S. Munro,et al.  Concurrent Shadow Paging in the Flask Architecture , 1994, POS.

[57]  Roger S. Barga,et al.  Phoenix: making applications robust , 1999, SIGMOD '99.

[58]  Laurent Daynès,et al.  Implementation of automated fine‐granularity locking in a persistent programming language , 2000, Softw. Pract. Exp..

[59]  Tobias Murer,et al.  Global Names: Support for Managing Software in a World of Virtual Organizations , 1999, SCM.

[60]  Peter Buneman,et al.  Database programming in Machiavelli—a polymorphic language with static type inference , 1989, SIGMOD '89.

[61]  Carl Staelin,et al.  The HP AutoRAID hierarchical storage system , 1995, SOSP.

[62]  Grzegorz Czajkowski Application isolation in the JavaTM Virtual Machine , 2000, OOPSLA.

[63]  Grzegorz Czajkowski,et al.  Application isolation in the Java Virtual Machine , 2000, OOPSLA '00.

[64]  Malcolm P. Atkinson,et al.  Fully Integrated Data Environments , 2000, Esprit Basic Research Series.

[65]  Hamid Pirahesh,et al.  ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging , 1998 .

[66]  Stanley B. Zdonik,et al.  Inheritance as an Incremental Modification Mechanism or What Like Is and Isn't Like , 1988, ECOOP.

[67]  Vivek Singhal,et al.  Texas: An Efficient, Portable Persistent Store , 1992, POS.

[68]  William A. Woods,et al.  Natural Language Technology in Precision Content Retrieval , 1998 .

[69]  Seth J. White Pointer Swizzling Techniques for Object-Oriented Database Systems , 1994, Technical Report / University of Wisconsin, Madison / Computer Sciences Department.

[70]  Ole J. Anfindsen,et al.  Conditional Conflict Serializability: An Application Oriented Correctness Criterion , 1998, J. Database Manag..

[71]  John Rosenberg,et al.  The MONADS Architecture - A Layered View , 1990, Workshop on Persistent Objects.

[72]  Florian Matthes Higher-Order Persistent Polymorphic Programming in Tycoon , 2000 .

[73]  Jochen Liedtke,et al.  A persistent system in real use-experiences of the first 13 years , 1993, Proceedings Third International Workshop on Object Orientation in Operating Systems.

[74]  Olivier Gruber,et al.  Nested Actions in Eos , 1992, POS.

[75]  Bruce G. Lindsay,et al.  Transaction management in the R* distributed database management system , 1986, TODS.

[76]  Michael Stonebraker,et al.  Object-Relational DBMSs: The Next Great Wave , 1995 .

[77]  Huw Evans,et al.  Zones, contracts and absorbing changes: an approach to software evolution , 1999, OOPSLA '99.

[78]  Florian Matthes,et al.  Persistent Threads , 1994, VLDB.

[79]  Antonios Printezis,et al.  Management of long-running high-performance persistent object stores , 2000 .

[80]  Miguel Leitão Bignolas Mira da Silva,et al.  Models of higher-order, type-safe, distributed computation over autonomous persistent object stores , 1996 .

[81]  Michael L. Van de Vanter Coordinated Editing of Versioned Packages in the JP Programming Environment , 1998, SCM.

[82]  Matthias Nussbaum Database Transaction Models for Advanced Applications , 1992 .

[83]  Laurent Daynès,et al.  Implementation of automated fine‐granularity locking in a persistent programming language , 2000 .

[84]  Patrick Valduriez,et al.  On the Cost of Lock Inheritance in Lock Managers Supporting Nested Transactions , 1994, BDA.

[85]  Miguel Castro,et al.  Providing Persistent Objects in Distributed Systems , 1999, ECOOP.

[86]  Peter Wegner,et al.  Interaction as a Framework for Modeling , 1997, Conceptual Modeling.

[87]  Antony L. Hosking,et al.  PM3: An Orthogonal Persistent Systems Programming Language - Design, Implementation, Performance , 1999, VLDB.

[88]  Marvin H. Solomon,et al.  A trace-based simulation of pointer swizzling techniques , 1995, Proceedings of the Eleventh International Conference on Data Engineering.

[89]  Martin Odersky,et al.  Making the future safe for the past: adding genericity to the Java programming language , 1998, OOPSLA '98.

[90]  Francis Wai,et al.  Distributed PS-algol , 1989, POS.

[91]  Andrew Hume,et al.  Gecko: Tracking a Very Large Billing System , 2000, USENIX Annual Technical Conference, General Track.

[92]  Sandra Heiler,et al.  Distributed Object Management , 1992, Int. J. Cooperative Inf. Syst..

[93]  Partha Dasgupta,et al.  The Clouds distributed operating system: functional description, implementation details and related work , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[94]  Mark R. Tuttle,et al.  Redo Recovery after System Crashes , 1995, VLDB.

[95]  Joachim W. Schmidt,et al.  Database Programming Languages: Bulk Types and Persistent Data. 3rd International Workshop, August 27-30, 1991, Nafplion, Greece, Proceedings , 1991 .

[96]  Brian T. Lewis,et al.  Efficient Barriers for Persistent Object Caching in a High-Performance Java TM Virtual Machine , 1999 .

[97]  Narain H. Gehani,et al.  Rationale for the Design of Persistence and Query Processing Facilities in the Database Programming Language O++ , 1989, DBPL.

[98]  Patrick Valduriez,et al.  Locking in OODBMS client supporting nested transactions , 1995, Proceedings of the Eleventh International Conference on Data Engineering.

[99]  Arif Merchant,et al.  Capacity planning with phased workloads , 1998, WOSP '98.

[100]  David S. Munro,et al.  The Napier88 Persistent Programming Language and Environment , 2000 .

[101]  Jack A. Orenstein,et al.  The ObjectStore database system , 1991, CACM.

[102]  David S. Munro On the integration of concurrency, distribution and persistence , 1994 .

[103]  Philippe Collet,et al.  Towards a Consistent Viewpoint on Consistency for Persistent Applications , 2000, Objects and Databases.

[104]  Malcolm P. Atkinson,et al.  EFDM: Extended Functional Data Model , 1986, Comput. J..

[105]  Malcolm P. Atkinson,et al.  Design Issues for Persistent Java: A Type-Safe, Object-Oriented, Orthogonally Persistent System , 1996, POS.

[106]  Amer Diwan,et al.  Partial redundancy elimination for access path expressions , 1999, Softw. Pract. Exp..

[107]  J. Gregory Morrisett,et al.  Composing first-class transactions , 1994, TOPL.

[108]  Malcolm P. Atkinson,et al.  Scalable and Recoverable Implementation of Object Evolution for the PJama1 Platform , 2000, POS.

[109]  Malcolm P. Atkinson,et al.  An efficient object promotion algorithm for persistent object systems , 2001, Softw. Pract. Exp..

[110]  Peter Burden,et al.  A Comparison of Two Persistent Storage Tools for Implementing a Search Engine , 2000, POS.

[111]  Peter Buneman,et al.  Polymorphism and type inference in database programming , 1996, TODS.

[112]  Ronald Morrison,et al.  Linguistic reflection in Java : a quantitative assessment , 1998 .

[113]  Quintin I. Cutts,et al.  Swizzle Barrier Optimizations for Orthogonal Persistence in Java , 1998, POS/PJW.

[114]  Jack C. Wileden,et al.  Toward Assessing Approaches to Persistence for Java , 1997 .

[115]  Misha Dmitriev Class and Data Evolution Support in the PJama Persistent Platform , 2000 .

[116]  Patrick Valduriez,et al.  Efficient support for customizing concurrency control in Persistent Java , 1996 .

[117]  Alan Dearle,et al.  Persistent Operating System Support for Java , 1996 .

[118]  Computer Staff,et al.  Transaction processing , 1994 .

[119]  Erik Odberg,et al.  Category Classes: Flexible Classification and Evolution in Object-Oriented Databases , 1994, CAiSE.

[120]  Misha Dmitriev The First Experience of Class Evolution Support in PJama , 1998, POS/PJW.

[121]  C. Mohan,et al.  Repeating History Beyond ARIES , 1999, VLDB.

[122]  Stephan Korsholm Transparent, Scalable, Efficient OO-Persistence , 1999, ECOOP Workshops.

[123]  Michael L. Van De Vanter,et al.  Software configuration management in an object oriented database , 1995 .

[124]  Irving L. Traiger,et al.  System R: relational approach to database management , 1976, TODS.

[125]  Garth A. Gibson,et al.  Self-managing network-attached storage , 1996, CSUR.

[126]  Gernot Heiser,et al.  The Mungi Single-Address-Space Operating System , 1994, Softw. Pract. Exp..

[127]  Walter F. Tichy,et al.  Should Computer Scientists Experiment More? , 1998, Computer.

[128]  R. Connor,et al.  Can Java Persist ? , 1996 .

[129]  Stanley B. Zdonik,et al.  Version Management in an Object-Oriented Database , 1986, Advanced Programming Environments.

[130]  Ronald Morrison,et al.  OCB : An object/class browser for Java , 1997 .

[131]  R. Venkatesh,et al.  Persistent Java Objects in 3 Tier Architectures , 1998, POS/PJW.

[132]  Huw Evans Why Object Serialization is Inappropriate for Providing Persistence in Java , 2000 .

[133]  Ronald Morrison,et al.  Linguistic reflection in Java , 1998 .

[134]  R. Connor,et al.  Design Issues for Persistent Java: a type-safe, object-oriented, orthogonally persistent system , 1996 .

[135]  Malcolm P. Atkinson,et al.  A scalable model of distribution promoting autonomy of and cooperation between PJava object stores , 1997, Proceedings of the Thirtieth Hawaii International Conference on System Sciences.

[136]  Malcolm P. Atkinson,et al.  An orthogonally persistent Java , 1996, SGMD.

[137]  Pascal Molli,et al.  TuaMotu: Support for Telecooperative Engineering Applications using Replicated Versions , 1998 .

[138]  Susan Spence,et al.  PJRMI: remote method invocation for a persistent system , 1999, Proceedings of the International Symposium on Distributed Objects and Applications.

[139]  Malcolm P. Atkinson,et al.  Algorithms for a persistent heap , 1983, Softw. Pract. Exp..

[140]  Malcolm P. Atkinson,et al.  Main-Memory Management to support Orthogonal Persistence for Java , 1997 .

[141]  Irving L. Traiger,et al.  A history and evaluation of System R , 1981, CACM.

[142]  Ole Lehrmann Madsen,et al.  Object-oriented programming in the BETA programming language , 1993 .

[143]  Bowen Alpern,et al.  Implementing jalapeño in Java , 1999, OOPSLA '99.

[144]  Richard Cooper,et al.  Constructing Database Systems in a Persistent Environment , 1987, VLDB.

[145]  Roberto Zicari,et al.  A framework for schema updates in an object-oriented database system , 1991, [1991] Proceedings. Seventh International Conference on Data Engineering.

[146]  Roy Goldman,et al.  DataGuides: Enabling Query Formulation and Optimization in Semistructured Databases , 1997, VLDB.

[147]  Michael F. Challis Database Consistency and Integrity in a Multi-User Environment , 1978, JCDKB.

[148]  Antero Taivalsaari,et al.  The Spotless System: Implementing a Java System for the Palm Connected Organizer , 1999 .

[149]  John Wilkes,et al.  An introduction to disk drive modeling , 1994, Computer.

[150]  Philip Wadler,et al.  Featherweight Java: a minimal core calculus for Java and GJ , 2001, TOPL.

[151]  Malcolm P. Atkinson,et al.  Design Issues in a Map Language , 1992, DBPL.

[152]  Satish M. Thatte,et al.  Persistent Memory: A Storage Architecture for Object-Oriented Database Systems , 1986, OODBS.

[153]  C. R. Landau,et al.  Object-Oriented Transaction Processing in the KeyKOS Microkernel , 1993, USENIX Microkernels and Other Kernel Architectures Symposium.

[154]  David J. DeWitt,et al.  A Performance Study of Alternative Object Faulting and Pointer Swizzling Strategies , 1992, VLDB.

[155]  Mark R. Tuttle,et al.  Logical logging to extend recovery to new domains , 1999, SIGMOD '99.

[156]  Brian T. Lewis,et al.  Architecture of the PEVM: A High-Performance Orthogonally Persistent Java Virtual Machine , 2000, POS.

[157]  Scott Nettles,et al.  Persistent object systems : principles and practice : the 7th International Workshop on Persistent Object Systems , 1997 .

[158]  William A. Woods,et al.  Conceptual Indexing: A Better Way to Organize Knowledge , 1997 .

[159]  Jonathan M. Smith,et al.  EROS: a fast capability system , 1999, SOSP.

[160]  Stephen J. Green,et al.  Linguistic Knowledge can Improve Information Retrieval , 2000, ANLP.

[161]  Dag I. K. Sjoberg Thesaurus-based methodologies and tools for maintaining persistent application systems , 1993 .