The Networked Telescope: Progress Toward a Grid Architecture for Pipeline Processing

Pipeline processing systems for modern telescopes are widely considered critical for addressing the problem of ever increasing data rates; however, routine use of fully automated processing systems may discourage the typical user from exploring processing parameter space or trying out new techniques. This issue might be particularly important with regard to radio interferometer data in which the post-calibration processing required to create an image for scientific analysis is not well defined. We describe our architecture for the BIMA Image Pipeline which attempts to address this issue. The pipeline by default is automated and uses NCSA supercomputers to carry out the processing; however, as we further develop the system, we will progressively add tools that enable the astronomer to guide the automated processing. This same system can also be used by the astronomer to create new processing projects using data from the archive. We report our progress on this system, highlighting the design features that allow for greater user interaction, including a research-oriented data model, web-based portal interfaces, and use of the AIPS++ toolkit. The ultimate goal is to evolve the system into a flexible, computational grid for processing radio interferometer data. 1. Vision: The Data Life Cycle within a Grid-based Ecosystem The BIMA Data Archive was built to deliver data automatically in real–time from the BIMA interferometer to a repository at NCSA where it can easily be accessed from NCSA supercomputers for high–performance processing or delivered to astronomers via the Web for local processing (Crutcher 1994, Plante & Crutcher 1997). We are now in the process of expanding the archive system to support automated calibration and construction of images from the raw visibility data, a process traditionally done interactively by the investigating astronomer. Pipeline processing of modern astronomical data is a problem well-suited to a Grid environment because of the inherent distributed nature of the hardware, software, data, and people involved. As part of NCSA’s efforts to build Grid infrastructure, we have adopted a Grid model for implementing the BIMA Image Pipeline. Astronomy Department, University of Illinois Urbana–Champaign