EnLIGHTened Computing: An architecture for co-allocating network, compute, and other grid resources for high-end applications

Many emerging high performance applications require distributed infrastructure that is significantly more powerful and flexible than traditional grids. Such applications require the optimization, close integration, and control of all grid resources, including networks. The EnLIGHTened (ENL) computing project has designed an architectural framework that allows grid applications to dynamically request (in-advance or on-demand) any type of grid resource: computers, storage, instruments, and deterministic, high-bandwidth network paths, including lightpaths. Based on application requirements, the ENL middleware communicates with grid resource managers and, when availability is verified, co-allocates all the necessary resources. ENLpsilas domain network manager controls all network resource allocations to dynamically setup and delete dedicated circuits using generalized multiprotocol label switching (GMPLS) control plane signaling. In order to make optimal brokering decisions, the ENL middleware uses near-real-time performance information about grid resources. A prototype of this architectural framework on a national-scale testbed implementation has been used to demonstrate a small number of applications. Based on this, a set of changes for the middleware have been laid out and are being implemented.

[1]  G. Allen,et al.  Supporting Efficient Execution in Heterogeneous Distributed Computing Environments with Cactus and Globus , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[2]  Italo Epicoco,et al.  The Grid Resource Broker, a ubiquitous grid computing framework , 2002, Sci. Program..

[3]  Lars-Olof Burchard.  Networks with Advance Reservations: Applications, Architecture, and Performance , 2005, Journal of Network and Systems Management.

[4]  Petr Holub,et al.  Distributed and collaborative visualization of large data sets using high-speed networks , 2006, Future Gener. Comput. Syst..

[5]  Joel H. Saltz,et al.  Visualization of Large Data Sets with the Active Data Repository , 2001, IEEE Computer Graphics and Applications.

[6]  Robert L. Grossman,et al.  TeraScope: distributed visual data mining of terascale data sets over photonic networks , 2003, Future Gener. Comput. Syst..

[7]  Andrew A. Chien,et al.  The OptIPuter , 2003, CACM.

[8]  Franco Travostino,et al.  Grid networks : enabling grids with advanced communication technology , 2006 .

[9]  Henri E. Bal,et al.  Griz: experience with remote visualization over an optical grid , 2003, Future Gener. Comput. Syst..

[10]  Ian T. Foster,et al.  Supporting Efficient Execution in Heterogeneous Distributed Computing Environments with Cactus and Globus , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[11]  Jon MacLaren.  Co-allocation of compute and network resources using HARC , 2007 .

[12]  Steven Tuecke,et al.  Internet X.509 Public Key Infrastructure (PKI) Proxy Certificate Profile , 2004, RFC.

[13]  Shantenu Jha,et al.  NEKTAR, SPICE and Vortonics: using federated grids for large scale scientific applications , 2006, 2006 IEEE Challenges of Large Applications in Distributed Environments.

[14]  Joel H. Saltz,et al.  DataCutter: Middleware for Filtering Very Large Scientific Datasets on Archival Storage Systems , 2000, IEEE Symposium on Mass Storage Systems.

[15]  H. T. Mouftah,et al.  Routing and wavelength assignment for advance reservation in wavelength-routed WDM optical networks , 2002, 2002 IEEE International Conference on Communications. Conference Proceedings. ICC 2002 (Cat. No.02CH37333).

[16]  Leslie Lamport,et al.  Consensus on transaction commit , 2004, TODS.

[17]  Gigi Karmous-Edwards,et al.  Dynamic scheduling of network resources with advance reservations in optical grids , 2008, Int. J. Netw. Manag..

[18]  Johan Tordsson,et al.  A Grid Resource Broker Supporting Advance Reservations and Benchmark-Based Resource Selection , 2004, PARA.

[19]  Daniel S. Katz,et al.  Montage: a grid portal and software toolkit for science-grade astronomical image mosaicking , 2009, Int. J. Comput. Sci. Eng..

[20]  Klara Nahrstedt,et al.  A distributed resource management architecture that supports advance reservations and co-allocation , 1999, 1999 Seventh International Workshop on Quality of Service. IWQoS'99. (Cat. No.98EX354).

[21]  Hans-Christian Hege,et al.  Interactive exploration of large remote micro-CT scans , 2004, IEEE Visualization 2004.

[22]  John Shalf,et al.  The Cactus Worm: Experiments with Dynamic Resource Discovery and Allocation in a Grid Environment , 2001, Int. J. High Perform. Comput. Appl..

[23]  Rajiv K. Kalia,et al.  Hybrid quantum mechanical/molecular dynamics simulation on parallel computers: density functional theory on real-space multigrids , 2002 .

[24]  Rizos Sakellariou,et al.  A taxonomy of grid monitoring systems , 2005, Future Gener. Comput. Syst..

[25]  Jon MacLaren,et al.  HARC: The Highly-Available Resource Co-allocator , 2007, OTM Conferences.

[26]  Xi Wang,et al.  A Flexible Advance Reservation Model for Multi-Domain WDM Optical Networks , 2006, 2006 3rd International Conference on Broadband Communications, Networks and Systems.

[27]  Louisiana State University,et al.  Numerical Methods for the Simulation of Dynamical Mass Transfer in Binaries , 2002 .

[28]  Alan Simpson,et al.  End-to-End Bandwidth Allocation and Reservation for Grid applications , 2006, 2006 3rd International Conference on Broadband Communications, Networks and Systems.