Optimizing Remote File Access for Parallel and Distributed Network Applications

This paper presents a paradigm for remote file access called Smart File Objects (SFOs). The SFO is a realization of the ELFS (Extensible File Systems) concept of files as typed objects, but it is applied to wide-area networks (J. Karpovich et al., in “Proceedings of the 9th OOPLSA,” 1994). The SFO is an object-oriented application- specific file access paradigm designed to address the bottleneck imposed by high latency, low bandwidth, unpredictable, and unreliable networks such as the current Internet. Newly emerging network applications such as multimedia, metacomputing, and collaboratories will have different sensitivities to these network “features.” These applications will require a more flexible file access mechanism than what is provided by conventional distributed file systems. The SFO uses application and network information to adaptively prefetch and cache needed data in parallel with the execution of the application to mitigate the impact of the network. Preliminary results indicate that the SFO can provide substantial performance gains for network applications.

[1]  Mahadev Satyanarayanan,et al.  An empirical study of a wide-area distributed file system , 1996, TOCS.

[2]  Shaogang Gong,et al.  Prediction and Adaptation , 2000 .

[3]  Andrew S. Grimshaw,et al.  The Legion vision of a worldwide virtual computer , 1997, Commun. ACM.

[4]  Jeffrey K. Hollingsworth,et al.  Prediction and adaptation in Active Harmony , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[5]  E. A. West,et al.  No pain and gain! - experiences with Mentat on a biological application , 1993, Concurr. Pract. Exp..

[6]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[7]  R. Wolski,et al.  Predicting the CPU availability of time‐shared Unix systems on the computational grid , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[8]  T.M. Madhyastha,et al.  Intelligent, adaptive file system policy selection , 1996, Proceedings of 6th Symposium on the Frontiers of Massively Parallel Computation (Frontiers '96).

[9]  James C. French,et al.  Extensible File Systems (ELFS): An Object-Oriented Approach to High Performance File I/O , 1994, OOPSLA.

[10]  Mor Harchol-Balter,et al.  Exploiting process lifetime distributions for dynamic load balancing , 1996, SIGMETRICS '96.

[11]  Jeffrey S. Vetter,et al.  Autopilot: adaptive control of distributed applications , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[12]  Anna R. Karlin,et al.  Implementing cooperative prefetching and caching in a globally-managed memory system , 1998, SIGMETRICS '98/PERFORMANCE '98.

[13]  Jon B. Weissman.  Smart file objects: a remote file access paradigm , 1999, IOPADS '99.

[14]  Jon B. Weissman,et al.  Smart multimedia file objects , 1999, Proceedings 1999 IEEE Workshop on Internet Applications (Cat. No.PR00197).

[15]  Andrew S. Grimshaw,et al.  Easy-to-use object-oriented parallel processing with Mentat , 1993, Computer.

[16]  Richard Wolski,et al.  Forecasting network performance to support dynamic scheduling using the network weather service , 1997, Proceedings. The Sixth IEEE International Symposium on High Performance Distributed Computing (Cat. No.97TB100183).

[17]  James C. French,et al.  Extensible file system (ELFS): an object-oriented approach to high performance file I/O , 1994, OOPSLA '94.

[18]  Jeanna Neefe Matthews,et al.  Serverless network file systems , 1996, TOCS.

[19]  David E. Culler,et al.  WebOS: operating system services for wide area applications , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[20]  Ian T. Foster,et al.  Remote I/O: fast access to distant storage , 1997, IOPADS '97.

[21]  Andrew S. Grimshaw,et al.  The LegionVision of a Worldwide Virtual , 1997 .