The Hydra Filesystem: A Distrbuted Storage Famework

Hydra File System (HFS) is an experimental framework for constructing parallel and distributed filesystems. While parallel and distributed applications requiring scalable and flexible access to storage and retrieval are becoming more commonplace, parallel and distributed filesystems remain difficult to deploy easily and configure for different needs. HFS aims to be different by being true to the tradition of highperformance computing while employing modern design patterns to allow various policies to be configured on a per instance basis (e.g. storage, communication, security, and indexing schemes). We describe a working prototype (available for public download) that has been implemented in the Python programming language.

[1]  George K. Thiruvathukal,et al.  Scalable Approaches for Supporting MPI-IO Atomicity , 2006, Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06).

[2]  Frank B. Schmuck,et al.  GPFS: A Shared-Disk File System for Large Computing Clusters , 2002, FAST.

[3]  Ian Clarke,et al.  Freenet: A Distributed Anonymous Information Storage and Retrieval System , 2000, Workshop on Design Issues in Anonymity and Unobservability.

[4]  Howard Gobioff,et al.  The Google file system , 2003, SOSP '03.

[5]  Margo I. Seltzer,et al.  Berkeley DB , 1999, USENIX Annual Technical Conference, FREENIX Track.

[6]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[7]  GhemawatSanjay,et al.  The Google file system , 2003 .

[8]  Tao Yang,et al.  A Self-Organizing Storage Cluster for Parallel Data-Intensive Applications , 2004, Proceedings of the ACM/IEEE SC2004 Conference.