Supporting application-tailored grid file system sessions with WSRF-based services

This paper presents novel service-based grid data management middleware that leverages standards defined by WSRF specifications to create and manage dynamic grid file system sessions. A unique aspect of the service is that the sessions it creates can be customized to address application data transfer needs. Application-tailored configurations enable selection of both performance-related features (block-based partial file transfers and/or whole-file transfers, cache parameters and consistency models) and reliability features (file system copy-on-write checkpointing to aid recovery of client-side failures; replication, autonomous failure detection and data access redirection for server-side failures). These enhancements, in addition to cross-domain user identity mapping and encrypted communication, are implemented via user level proxies managed by the service, requiring no changes to existing kernels. Sessions established using the service is mounted as distributed file systems and can be used transparently by unmodified binary applications. The paper analyzes the use of the service to support virtual machine based grid systems and workflow execution, and also reports on the performance and reliability of service managed wide-area file system sessions with experiments based on scientific applications (NanoMOS/Matlab, CHID, GAUSS and SPECseis).

[1]  Renato J. O. Figueiredo,et al.  Enhancing the scalability and usability of computational grids via logical user accounts and virtual file systems , 2001, Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001.

[2]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[3]  Renato J. O. Figueiredo,et al.  VP/GFS: an Architecture for Virtual Private Grid File Systems , 2003 .

[4]  Marty Humphrey,et al.  Exploiting WSRF and WSRF.NET for remote job execution in grid environments , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[5]  Xiaomin Zhu,et al.  From virtualized resources to virtual computing grids: the In-VIGO system , 2005, Future Gener. Comput. Syst..

[6]  Andrea C. Arpaci-Dusseau,et al.  Flexibility, manageability, and performance in a Grid storage appliance , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.

[7]  David Mazières,et al.  Fast and secure distributed read-only file system , 2000, TOCS.

[8]  Mahadev Satyanarayanan,et al.  Internet suspend/resume , 2002, Proceedings Fourth IEEE Workshop on Mobile Computing Systems and Applications.

[9]  M. Bozyigit,et al.  User-level process checkpoint and restore for migration , 2001, OPSR.

[10]  Ian T. Foster,et al.  The anatomy of the grid: enabling scalable virtual organizations , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[11]  Ian T. Foster,et al.  Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing , 2001, 2001 Eighteenth IEEE Symposium on Mass Storage Systems and Technologies.

[12]  David Mazières,et al.  A Toolkit for User-Level File Systems , 2001, USENIX Annual Technical Conference, General Track.

[13]  Renato J. O. Figueiredo,et al.  Distributed file system support for virtual machines in grid computing , 2004, Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004..

[14]  Miron Livny,et al.  Checkpoint and Migration of UNIX Processes in the Condor Distributed Processing System , 1997 .

[15]  Renato J. O. Figueiredo,et al.  VMPlants: Providing and Managing Virtual Machine Execution Environments for Grid Computing , 2004, Proceedings of the ACM/IEEE SC2004 Conference.

[16]  Douglas Thain,et al.  The Kangaroo approach to data movement on the Grid , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[17]  Andrea C. Arpaci-Dusseau,et al.  Explicit Control in the Batch-Aware Distributed File System , 2004, NSDI.

[18]  M. Humphrey,et al.  LegionFS: A Secure and Scalable File System Supporting Cross-Domain High-Performance Applications , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[19]  Renato J. O. Figueiredo,et al.  Distributed File System Virtualization Techniques Supporting On-Demand Virtual Machine Environments for Grid Computing , 2006, Cluster Computing.

[20]  Steven Tuecke,et al.  The Physiology of the Grid An Open Grid Services Architecture for Distributed Systems Integration , 2002 .

[21]  Renato J. O. Figueiredo,et al.  The PUNCH virtual file system: seamless access to decentralized storage services in a computational grid , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[22]  David Brumley,et al.  Virtual Appliances for Deploying and Maintaining Software , 2003, LISA.

[23]  Peter A. Dinda,et al.  Towards Virtual Networks for Virtual Machine Grid Computing , 2004, Virtual Machine Research and Technology Symposium.

[24]  José A. B. Fortes,et al.  PUNCH: An architecture for Web-enabled wide-area network-computing , 2004, Cluster Computing.

[25]  Carl Smith,et al.  NFS Version 3: Design and Implementation , 1994, USENIX Summer.

[26]  Renato J. O. Figueiredo,et al.  Seamless Access to Decentralized Storage Services in Computational Grids via a Virtual File System , 2004, Cluster Computing.

[27]  Renato J. O. Figueiredo,et al.  A case for grid computing on virtual machines , 2003, 23rd International Conference on Distributed Computing Systems, 2003. Proceedings..

[28]  Renato J. O. Figueiredo,et al.  Single sign-on in In-VIGO: role-based access via delegation mechanisms using short-lived user identities , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[29]  Ian T. Foster,et al.  GASS: a data movement and access service for wide area computing systems , 1999, IOPADS '99.