Data Provenance and Management in Radio Astronomy: A Stream Computing Approach

New approaches for data provenance and data management (DPDM) are required for mega science projects like the Square Kilometer Array, characterized by extremely large data volume and intense data rates, therefore demanding innovative and highly efficient computational paradigms. In this context, we explore a stream-computing approach with the emphasis on the use of accelerators. In particular, we make use of a new generation of high performance stream-based parallelization middleware known as InfoSphere Streams. Its viability for managing and ensuring interoperability and integrity of signal processing data pipelines is demonstrated in radio astronomy.

[1]  Ramesh Subramonian,et al.  LogP: towards a realistic model of parallel computation , 1993, PPOPP '93.

[2]  Kristen Rohlfs,et al.  Tools of Radio Astronomy , 1986 .

[3]  Deepak S. Turaga,et al.  Resource Management for Networked Classifiers in Distributed Stream Mining Systems , 2006, Sixth International Conference on Data Mining (ICDM'06).

[4]  J. Bunton.  New Generation Correlators , 2005 .

[5]  Chris Broekema,et al.  The Lofar Central Processing Facility Architecture , 2004 .

[6]  E. Keane,et al.  Rotating Radio Transients , 2011, 1109.6896.

[7]  S. Weston.  Development of Very Long Baseline Interferometry (VLBI) techniques in New Zealand: Array simulation, image synthesis and analysis , 2008 .

[8]  Kun-Lung Wu,et al.  A code generation approach to optimizing high-performance distributed data stream processing , 2009, CIKM.

[9]  Philip S. Yu,et al.  SPADE: the system s declarative stream processing engine , 2008, SIGMOD Conference.

[10]  Navendu Jain,et al.  Design, implementation, and evaluation of the linear road bnchmark on the stream processing core , 2006, SIGMOD Conference.

[11]  Chris Broekema,et al.  The Lofar Central Processing Facility Architecture , 2004 .

[12]  Ana Lucia Varbanescu,et al.  Building high-resolution sky images using the Cell/B.E. , 2009, HiPC 2009.

[13]  Alain Biem,et al.  A streaming approach to radio astronomy imaging , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  W. Becker.  Neutron Stars and Pulsars , 2009 .

[15]  Juan Carlos Guzman,et al.  The Australian SKA Pathfinder (ASKAP) software architecture , 2010, Astronomical Telescopes + Instrumentation.

[16]  Chun Zhang,et al.  Storing and querying ordered XML using a relational database system , 2002, SIGMOD '02.

[17]  Jesper Andersson,et al.  An Adaptive High-Performance Service Architecture , 2005, Electron. Notes Theor. Comput. Sci..

[18]  H. Bondi,et al.  The Gravitational Lens Effect , 1964 .

[20]  Ibm Redbooks,et al.  Programming the Cell Broadband Engine Architecture: Examples and Best Practices , 2008 .