Improving Data Consistency for Mobile File Access Using Isolation-Only Transactions

Disconnected operation based on optimistic replication has been demonstrated as an effective technique enabling mobile computers to access shared data in distributed file systems. To guard against inconsistencies resulted from partitioned data sharing, past research has focused on detecting and resolving write/write conflicts. However, experience shows that undetected read/write conflicts pose a subtle but serious threat to data integrity in mobile file access. Solving this problem is critical for the future success of mobile computing. This dissertation shows that isolation-only transaction (IOT), an upward compatible transaction mechanism for the Unix File System, is a viable solution to this problem. The central idea of the IOT model is imposing serializability-based isolation requirements on partitioned transaction executions. Transactions executed on a disconnected client stay in a tentative state until the client regains connection to relevant servers. They are committed to the servers as soon as they pass consistency validation. Invalidated transactions are automatically or manually resolved to ensure global consistency. Powerful resolution mechanisms such as automatic transaction re-execution and application specific resolver invocation can transparently resolve conflicts for many common Unix applications. In addition, a concise conflict representation scheme enables application semantics to be smoothly integrated for only conflict resolution and consistency validation. The practical usability of IOT is further enhanced by a flexible interactive interface, full compatibility with existing Unix applications, and the ability to retain overall file system scalability, security and transparency. A working IOT implementation in the Coda file system has been developed and used in experiments in software development and document processing applications. Quantitative evaluation based on controlled experiments and trace-driven simulations establish that the IOT model is scalable and incurs modest performance and resource overhead. The main contributions of this thesis research are the following: the design of an isolationonly transaction model specialized for improving mobile file consistency while preserving upward compatibility with existing Unix applications; the development of a working IOT implementation in the Coda file system; experimentation and evaluation demonstrating the feasibility and practicality of the IOT model.

[1]  Abraham Silberschatz,et al.  Distributed file systems: concepts and examples , 1990, CSUR.

[2]  M. Herlihy A quorum-consensus replication method for abstract data types , 1986, TOCS.

[3]  Mahadev Satyanarayanan,et al.  Scale and performance in a distributed file system , 1988, TOCS.

[4]  Marvin Theimer,et al.  Managing update conflicts in Bayou, a weakly connected replicated storage system , 1995, SOSP.

[5]  Gail E. Kaiser,et al.  Concurrency control in advanced database applications , 1991, CSUR.

[6]  Mahadev Satyanarayanan,et al.  Supporting application-specific resolution in an optimistically replicated file system , 1993, Proceedings of IEEE 4th Workshop on Workstation Operating Systems. WWOS-III.

[7]  Steve R. Kleiman,et al.  Vnodes: An Architecture for Multiple File System Types in Sun UNIX , 1986, USENIX Summer.

[8]  Alfred Z. Spector,et al.  Distributed transactions for reliable systems , 1985, SOSP '85.

[9]  Alexander Thomasian,et al.  Analysis of some optimistic concurrency control schemes based on certification , 1985, SIGMETRICS 1985.

[10]  Maria Ebling,et al.  SynRGen: an extensible file reference generator , 1994, SIGMETRICS.

[11]  Ii Richard George Guy,et al.  FICUS: a very large scale reliable distributed file system , 1992 .

[12]  Mahadev Satyanarayanan,et al.  Coda: A Highly Available File System for a Distributed Workstation Environment , 1990, IEEE Trans. Computers.

[13]  Irving L. Traiger,et al.  Transactions and consistency in distributed database systems , 1982, TODS.

[14]  Andreas Reuter,et al.  Transaction Processing: Concepts and Techniques , 1992 .

[15]  Scott Shenker,et al.  Epidemic algorithms for replicated database maintenance , 1988, OPSR.

[16]  Mahadev Satyanarayanan,et al.  Efficient User-Level File Cache Management on the Sun Vnode Interface , 1990, USENIX Summer.

[17]  Qi Lu,et al.  Improving data consistency in mobile computing using isolation-only transactions , 1995, Proceedings 5th Workshop on Hot Topics in Operating Systems (HotOS-V).

[18]  Frank B. Schmuck,et al.  Experience with transactions in QuickSilver , 1991, SOSP '91.

[19]  Mahadev Satyanarayanan,et al.  An Empirical Study of a Highly Available File System , 1994, SIGMETRICS.

[20]  Mahesh K. Marina,et al.  Performance of route caching strategies in Dynamic Source Routing , 2001, Proceedings 21st International Conference on Distributed Computing Systems Workshops.

[21]  Maria Ebling,et al.  Exploiting weak connectivity for mobile file access , 1995, SOSP.

[22]  David A. Goldberg,et al.  Design and Implementation of the Sun Network Filesystem , 1985, USENIX Conference Proceedings.

[23]  Mahadev Satyanarayanan,et al.  Long Term Distributed File Reference Tracing: Implementation and Experience , 1996, Softw. Pract. Exp..

[24]  Mahadev Satyanarayanan,et al.  Disconnected Operation in the Coda File System , 1999, Mobidata.

[25]  Alexander Thomasian,et al.  Analysis of Some Optimistic Concurrency Control Schemes Based on Certification , 1985, SIGMETRICS.

[26]  Erhard Rahm,et al.  A new distributed optimistic concurrency control method and a comparison of its performance with two-phase locking , 1990, Proceedings.,10th International Conference on Distributed Computing Systems.

[27]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[28]  Qi Lu,et al.  Isolation-only transactions for mobile computing , 1994, OPSR.

[29]  Barbara Liskov,et al.  Guardians and Actions: Linguistic Support for Robust, Distributed Programs , 1983, TOPL.

[30]  Bruce Walker,et al.  The LOCUS distributed operating system , 1983, SOSP '83.

[31]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[32]  Erhard Rahm,et al.  Distributed optimistic concurrency control for high performance transaction processing , 1990, Proceedings. PARBASE-90: International Conference on Databases, Parallel Architectures, and Their Applications.

[33]  Mahadev Satyanarayanan,et al.  Scale and performance in a distributed file system , 1987, SOSP '87.

[34]  Toshimi Minoura,et al.  Resilient Extended True-Copy Token Scheme for a Distributed Database System , 1982, IEEE Transactions on Software Engineering.

[35]  Marvin Theimer,et al.  QuickSilver support for access to data in large, geographically dispersed systems , 1989, [1989] Proceedings. The 9th International Conference on Distributed Computing Systems.

[36]  Mahadev Satyanarayanan,et al.  Andrew: a distributed personal computing environment , 1986, CACM.

[37]  Hector Garcia-Molina,et al.  Read-only transactions in a distributed database , 1982, TODS.

[38]  David K. Gifford,et al.  Weighted voting for replicated data , 1979, SOSP '79.

[39]  Divyakant Agrawal,et al.  The Performance of Protocols Based on Locks with Ordered Sharing , 1994, IEEE Trans. Knowl. Data Eng..

[40]  Raymond A. Lorie,et al.  Physical integrity in a large segmented database , 1977, TODS.

[41]  Mahadev Satyanarayanan,et al.  Log-based directory resolution in the Coda file system , 1993, [1993] Proceedings of the Second International Conference on Parallel and Distributed Information Systems.

[42]  Hector Garcia-Molina,et al.  Using semantic knowledge for transaction processing in a distributed database , 1983, TODS.

[43]  Mahadev Satyanarayanan,et al.  Flexible and Safe Resolution of File Conflicts , 1995, USENIX.

[44]  John S. Heidemann,et al.  Resolving File Conflicts in the Ficus File System , 1994, USENIX Summer.

[45]  John S. Heidemann,et al.  Implementation of the Ficus Replicated File System , 1990, USENIX Summer.

[46]  Randy Pausch,et al.  Adding input and output to the transactional model , 1988 .

[47]  Puneet Kumar,et al.  Mitigating the Effects of Optimistic Replication in a Distributed File System , 1994 .

[48]  Alley Stoughton,et al.  Detection of Mutual Inconsistency in Distributed Systems , 1983, IEEE Transactions on Software Engineering.

[49]  Donald F. Towsley,et al.  Modeling the effects of data and resource contention on the performance of optimistic concurrency control protocols , 1988, Proceedings. Fourth International Conference on Data Engineering.

[50]  Susan B. Davidson,et al.  Optimism and consistency in partitioned distributed database systems , 1984, TODS.

[51]  David S. H. Rosenthal,et al.  Evolving the Vnode interface , 1990, USENIX Summer.

[52]  Mahadev Satyanarayanan,et al.  Coda File System User and System Administrators Manual , 1995 .

[53]  James J. Kistler Disconnected Operation in a Distributed File System , 1995, Lecture Notes in Computer Science.

[54]  Irene Greif,et al.  Replicated document management in a group communication system , 1988, CSCW '88.

[55]  Susan B. Davidson,et al.  An optimistic protocol for partitioned distributed database systems , 1982 .

[56]  Mahadev Satyanarayanan,et al.  Scalable, secure, and highly available distributed file access , 1990, Computer.

[57]  John K. Ousterhout,et al.  Tcl and the Tk Toolkit , 1994 .

[58]  Maurice Herlihy,et al.  ARGUS REFERENCE MANUAL , 1987 .

[59]  Mahadev Satyanarayanan,et al.  Lightweight recoverable virtual memory , 1993, SOSP '93.

[60]  John T. Kohl,et al.  The Zephyr Notification Service , 1988, USENIX Winter.

[61]  J. T. Robinson,et al.  On optimistic methods for concurrency control , 1979, TODS.

[62]  J. D. Day,et al.  A principle for resilient sharing of distributed resources , 1976, ICSE '76.

[63]  Alfred Z. Spector,et al.  Distributed transactions for reliable systems , 1985, SOSP 1985.

[64]  Qi Lu,et al.  Experience with Disconnected Operation in a Mobile Environment , 1993, Symposium on Mobile and Location-Independent Computing.