Flexible Light Curves Generation System for Astronomical Catalogs

Light curves are fundamental tools for variable star astronomy. They describe the change of celestial object’s light intensity as time goes on. Because of the sharp increase of data in astronomical researches, the general methods are now utilized cannot meet the requirement of time-domain astronomical observation. In this paper, FLCGS, a Flexible Light Curves Generation System is proposed to achieve scalability and parallelism for generating light curves from astronomical catalogs. We design metadata files for light curves generation based on sky partition. Moreover, a new partition strategy is defined to keep workload balance. The function works very well via dynamic programming when the distribution of Big Data is skewed. We focus on the cross-matching between celestial objects from different catalogs and introduce a new method to determine whether they are the same celestial objects for light curves generation. Experimental results show that FLCGS is nearly 11 times faster than using MySQL, especially when the data is massive.

[1]  F. J. Carrera,et al.  Probabilistic multi-catalogue positional cross-match , 2016, 1609.00818.

[2]  Hilo,et al.  THE ELEVENTH AND TWELFTH DATA RELEASES OF THE SLOAN DIGITAL SKY SURVEY: FINAL DATA FROM SDSS-III , 2015, 1501.00963.

[3]  K. Gorski,et al.  HEALPix: A Framework for High-Resolution Discretization and Fast Analysis of Data Distributed on the Sphere , 2004, astro-ph/0409513.

[4]  F. Ochsenbein,et al.  The VizieR database of astronomical catalogues , 2000, astro-ph/0002122.

[5]  J. Anthony Tyson,et al.  Large Synoptic Survey Telescope: Overview , 2002, SPIE Astronomical Telescopes + Instrumentation.

[6]  Pavlos Protopapas,et al.  Computational Intelligence Challenges and Applications on Large-Scale Astronomical Time Series Databases , 2014, IEEE Computational Intelligence Magazine.

[7]  Alexander S. Szalay,et al.  Splitting the Sky - HTM and HEALPix , 2001 .

[8]  Aniruddha R. Thakar,et al.  The Hierarchical Triangular Mesh , 2001 .

[9]  Jizhou Sun,et al.  A Paralleled Large-Scale Astronomical Cross-Matching Function , 2009, ICA3PP.

[10]  Arie Shoshani,et al.  Parallel data analysis directly on scientific file formats , 2014, SIGMOD Conference.

[11]  Tamás Budavári,et al.  Xmatch: GPU Enhanced Astronomic Catalog Cross-Matching , 2013 .

[12]  Anastasia Ailamaki,et al.  NoDB: efficient query execution on raw data files , 2015, Commun. ACM.

[13]  Alexander S. Szalay,et al.  The Zones Algorithm for Finding Points-Near-a-Point or Cross-Matching Spatial Datasets , 2007, ArXiv.

[14]  Rainer Beck,et al.  Square kilometre array , 2010, Scholarpedia.

[15]  Peng Du,et al.  New cross-matching algorithm in large-scale catalogs with ThreadPool technique , 2014 .

[16]  Mark Taylor,et al.  TOPCAT & STIL: Starlink Table/VOTable Processing Software , 2005 .

[17]  Wei Li,et al.  Optimizing the Join Operation on Hive to Accelerate Cross-Matching in Astronomy , 2014, 2014 IEEE International Parallel & Distributed Processing Symposium Workshops.

[18]  Jaroslav Pokorný,et al.  NoSQL databases: a step to database scalability in web environment , 2011, iiWAS '11.

[19]  Qiong Luo,et al.  Multi-Assignment Single Joins for Parallel Cross-Match of Astronomic Catalogs on Heterogeneous Clusters , 2016, SSDBM.

[20]  Jizhou Sun,et al.  AQUAdexIM: highly efficient in-memory indexing and querying of astronomy time series images , 2016 .

[21]  S. Sánchez-Expósito,et al.  AstroTaverna - Building workflows with Virtual Observatory services , 2014, Astron. Comput..