Programming environments for multidisciplinary Grid communities

As the power of computational Grids increases, there is a corresponding need for better usability for large and diverse communities. The focus in this paper is on supporting multidisciplinary communities of scientists and engineers. We discuss requirements for Grid computing environments (GCEs) in this context, and describe several core support technologies developed to meet these requirements. Our work extends the notion of a programming environment beyond the compile–schedule–execute paradigm, to include functionality such as collaborative application composition, information services, and data and simulation management. Systems designed for five different applications communities are described. These systems illustrate common needs and characteristics arising in multidisciplinary communities and motivate a high‐level design framework for building GCEs that meet those needs. Copyright © 2002 John Wiley & Sons, Ltd.

[1]  Andrew S. Grimshaw,et al.  Object-based approaches , 1998 .

[2]  Reagan Moore,et al.  Data-intensive computing and digital libraries , 1998, CACM.

[3]  John R. Rice,et al.  Recommender Systems for Problem Solving Environments , 1997 .

[4]  Naren Ramakrishnan,et al.  Mining and visualizing recommendation spaces for elliptic PDEs with continuous attributes , 2000, TOMS.

[5]  D. B. Davis,et al.  Sun Microsystems Inc. , 1993 .

[6]  Reagan Moore,et al.  Data-intensive computing , 1998 .

[7]  Dennis Gannon,et al.  Developing component architectures for distributed scientific problem solving , 1998 .

[8]  Daniel A. Reed,et al.  Virtue: Performance Visualization of Parallel and Distributed Applications , 1999, Computer.

[9]  John Shalf,et al.  Cactus Tools for Grid Applications , 2001, Cluster Computing.

[10]  Joseph M. Hellerstein,et al.  Optimization techniques for queries with expensive methods , 1998, TODS.

[11]  Benjamin J. Kaipers,et al.  Qualitative Simulation , 1989, Artif. Intell..

[12]  Scott W. Haney,et al.  Rapid Application Development and Enhanced Code Interoperability using the POOMA Framework , 1998 .

[13]  Hisham K. Hamadeh,et al.  Gene Chips and Functional Genomics , 2000, American Scientist.

[14]  James R. Rice,et al.  From Scientific Software Libraries to Problem Solving Environments John R. Rice , 1996 .

[15]  Brian J. N. Wylie,et al.  Software Tools for Scalable Multilevel Application Engineering , 1997, Int. J. High Perform. Comput. Appl..

[16]  James C. Browne,et al.  Compositional Development of Performance Models in Poems , 2000, Int. J. High Perform. Comput. Appl..

[17]  David H. Laidlaw,et al.  The application visualization system: a computational environment for scientific visualization , 1989, IEEE Computer Graphics and Applications.

[18]  Hector Garcia-Molina,et al.  Meaningful change detection in structured data , 1997, SIGMOD '97.

[19]  Jacques Chassin de Kergommeaux,et al.  Flexible performance visualization of parallel and distributed applications , 2003, Future Gener. Comput. Syst..

[20]  Dennis Gannon,et al.  The Linear System Analyzer , 2000 .

[21]  Charles D. Hansen,et al.  Interactive Simulation and Visualization , 1999, Computer.

[22]  John Shalf,et al.  Solving Einstein's Equations on Supercomputers , 1999, Computer.

[23]  Jr. Allen B. Tucker,et al.  The Computer Science and Engineering Handbook , 1997 .

[24]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[25]  Ian T. Foster,et al.  The Anatomy of the Grid: Enabling Scalable Virtual Organizations , 2001, Int. J. High Perform. Comput. Appl..

[26]  Naren Ramakrishnan,et al.  Networked agents for scientific computing , 1999, CACM.

[27]  Rajkumar Buyya,et al.  The Grid: International Efforts in Global Computing , 2000 .

[28]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[29]  Paul A. Fishwick,et al.  Simulation model design and execution - building digital worlds , 1995 .

[30]  Ian T. Foster,et al.  GASS: a data movement and access service for wide area computing systems , 1999, IOPADS '99.

[31]  P. Pandurang Nayak,et al.  Automated Modeling of Physical Systems , 1995, Lecture Notes in Computer Science.

[32]  Gul Agha,et al.  Research directions in concurrent object-oriented programming , 1993 .

[33]  Susan L. Graham,et al.  The high-performance computing continuum , 1998, CACM.

[34]  James P. Ahrens,et al.  Ligature: Component Architecture for High Performance Applications , 2000, Int. J. High Perform. Comput. Appl..

[35]  Benjamin Kuipers,et al.  Qualitative Simulation , 1986, Artificial Intelligence.

[36]  M Eidson Thomas,et al.  A component-based programming model for composite, distributed applications , 2001 .

[37]  Dennis G. Kafura,et al.  Symphony: a Java-based composition and manipulation framework for distributed legacy resources , 1999, 1999 Proceedings International Symposium on Software Engineering for Parallel and Distributed Systems.

[38]  Rizos Sakellariou,et al.  Application Representations for Multiparadigm Performance Modeling of Large-Scale Parallel Scientific Codes , 2000, Int. J. High Perform. Comput. Appl..

[39]  Jason Novotny,et al.  The Grid Portal Development Kit , 2002, Concurr. Comput. Pract. Exp..

[40]  Kenneth D. Forbus.  Qualitative Reasoning , 1997, The Computer Science and Engineering Handbook.

[41]  Dennis Gannon,et al.  Distributed pC++ Basic Ideas for an Object Parallel Language , 1993, Sci. Program..

[42]  Henri Casanova,et al.  Netsolve: a Network-Enabled Server for Solving Computational Science Problems , 1997, Int. J. High Perform. Comput. Appl..

[43]  Dennis Gannon,et al.  A component based services architecture for building distributed applications , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.

[44]  Mary K. Vernon,et al.  Poems: end-to-end performance design of large parallel adaptive computational systems , 1998, WOSP '98.

[45]  Naren Ramakrishnan,et al.  Expresso-A PSE for Bioinformatics: Finding Answers with Microarray Technology , 2001 .

[46]  M. Stonebraker,et al.  SEQUOIA 2000-a reflection on the first three years , 1994, Seventh International Working Conference on Scientific and Statistical Database Management.

[47]  Mark Young,et al.  An object oriented visual programming language toolkit , 1995, COMG.

[48]  Gregor von Laszewski,et al.  Distance Visualization: Data Exploration on the Grid , 1999, Computer.

[49]  Bruce W. Porter,et al.  Automated Modeling of Complex Systems to Answer Prediction Questions , 1997, Artif. Intell..

[50]  Bernard Grossman,et al.  Response Surface Models Combining Linear and Euler Aerodynamics for Supersonic Transport Design , 1999 .

[51]  Steve Kubica,et al.  Cantata: visual programming environment for the Khoros system , 1995, COMG.

[52]  Scott R. Kohn,et al.  Language Interoperability Mechanisms For High-Performance Scientific Applications , 1998 .

[53]  Ian T. Foster,et al.  Grid information services for distributed resource sharing , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[54]  James Demmel,et al.  Programming tools and environments , 1998, CACM.

[55]  Naren Ramakrishnan,et al.  Note on generalization in experimental algorithmics , 2000, TOMS.

[56]  Naren Ramakrishnan,et al.  PYTHIA-II: a knowledge/database system for managing performance data and recommending scientific software , 2000, TOMS.

[57]  Ian T. Foster,et al.  Compositional parallel programming languages , 1996, TOPL.

[58]  Clifford A. Shaffer,et al.  Lightweight data management for compositional modeling in problem solving environments , 2001 .

[59]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[60]  Clifford A. Shaffer,et al.  From landscapes to waterscapes: A PSE for landuse change analysis , 2003, Engineering with Computers.

[61]  Clifford A. Shaffer,et al.  VizCraft: a problem-solving environment for aircraft configuration design , 2001, Comput. Sci. Eng..

[62]  Miron Livny,et al.  Zoo: a desktop experiment management environment , 1997, SIGMOD '97.

[63]  R. Ritt,et al.  The modeling of physical systems , 1956, IRE Transactions on Antennas and Propagation.

[64]  Clifford A. Shaffer,et al.  WBCSim: A Prototype Problem Solving Environment for Wood-Based Composites Simulations , 1998, Engineering with Computers.

[65]  Anthony G. Cohn,et al.  Qualitative Reasoning , 1987, Advanced Topics in Artificial Intelligence.

[66]  Dan Suciu,et al.  Data on the Web: From Relations to Semistructured Data and XML , 1999 .

[67]  Jack Dongarra,et al.  Application-specific tools , 1998 .

[68]  Lenwood S. Heath,et al.  Studying the Functional Genomics of Stress Responses in Loblolly Pine With the Expresso Microarray Experiment Management System , 2002, Comparative and functional genomics.

[69]  Narendran Ramakrishnan.  Experiences with an Algorithm Recommender System , 1999 .

[70]  Naren Ramakrishnan,et al.  Mining scientific data , 2001, Adv. Comput..

[71]  Brian Falkenhainer,et al.  Compositional Modeling: Finding the Right Model for the Job , 1991, Artif. Intell..

[72]  K. Mani Chandy,et al.  CC++: A Declarative Concurrent Object Oriented Programming Notation , 1993 .

[73]  Geoffrey C. Fox,et al.  High-performance commodity computing , 1998 .

[74]  Francine Berman,et al.  The AppLeS Parameter Sweep Template: User-Level Middleware for the Grid , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[75]  Amin Vahdat,et al.  The CRISIS Wide Area Security Architecture , 1998, USENIX Security Symposium.

[76]  Ken Kennedy,et al.  Telescoping languages: a compiler strategy for implementation of high-level domain-specific programming systems , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[77]  Naren Ramakrishnan,et al.  The Expresso Microarray Experiment Management System: The Functional Genomics of Stress Responses in Loblolly Pine , 2001, ArXiv.