Data Reservoir: Utilization of Multi-Gigabit Backbone Network for Data-Intensive Research

We propose data sharing facility for data intensive scientific research, "Data Reservoir"; which is optimized to transfer huge amount of data files between distant places fully Utilizing multi-gigabit backbone network. In addition, "Data Reservoir" can be used as an ordinary UNIX server in local network without any modification of server softwares. We use low-level protocol and hierarchical striping to realize (1) separation of bulk data transfer and local accesses by cashing, (2) file-system transparency, I.e. interoperable whatever in higher layer than disk driver, including file system. (3) scalability for network and storage. This paper shows our design, implementation using iSCSI protocol [1] and their performances for both 1Gbps model in the real network and 10Gbps model in our laboratory.

[1]  Chandramohan A. Thekkath,et al.  Petal: distributed virtual disks , 1996, ASPLOS VII.

[2]  Robert B. Ross,et al.  PVFS: A Parallel File System for Linux Clusters , 2000, Annual Linux Showcase & Conference.

[3]  Steven Levine,et al.  Porting the SGI XFS File System to Linux , 2000, USENIX Annual Technical Conference, FREENIX Track.

[4]  Andrew A. Chien,et al.  PPFS: a high performance portable parallel file system , 1995, ICS '95.

[5]  H. Koga,et al.  Data reservoir: a new approach to data-intensive scientific computation , 2002, Proceedings International Symposium on Parallel Architectures, Algorithms and Networks. I-SPAN'02.