Harnessing the Power of Adversarial Prompting and Large Language Models for Robust Hypothesis Generation in Astronomy

This study investigates the application of Large Language Models (LLMs), specifically GPT-4, within Astronomy. We employ in-context prompting, supplying the model with up to 1000 papers from the NASA Astrophysics Data System, to explore the extent to which performance can be improved by immersing the model in domain-specific literature. Our findings point towards a substantial boost in hypothesis generation when using in-context prompting, a benefit that is further accentuated by adversarial prompting. We illustrate how adversarial prompting empowers GPT-4 to extract essential details from a vast knowledge base to produce meaningful hypotheses, signaling an innovative step towards employing LLMs for scientific research in Astronomy.

[1]  Henrique Pondé de Oliveira Pinto,et al.  GPT-4 Technical Report , 2023, 2303.08774.

[2]  M. Hayden,et al.  Chasing the impact of the Gaia-Sausage-Enceladus merger on the formation of the Milky Way thick disc , 2022, Monthly Notices of the Royal Astronomical Society: Letters.

[3]  Alexander J. Smola,et al.  Automatic Chain of Thought Prompting in Large Language Models , 2022, ICLR.

[4]  Song-Chun Zhu,et al.  Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering , 2022, NeurIPS.

[5]  D. Schuurmans,et al.  Self-Consistency Improves Chain of Thought Reasoning in Language Models , 2022, ICLR.

[6]  Chris Callison-Burch,et al.  A Feasibility Study of Answer-Unaware Question Generation for Education , 2022, FINDINGS.

[7]  Ryan J. Lowe,et al.  Training language models to follow instructions with human feedback , 2022, NeurIPS.

[8]  Dale Schuurmans,et al.  Chain of Thought Prompting Elicits Reasoning in Large Language Models , 2022, NeurIPS.

[9]  Sang Michael Xie,et al.  An Explanation of In-context Learning as Implicit Bayesian Inference , 2021, ICLR.

[10]  G. Karypis,et al.  Meta-learning via Language Model In-context Tuning , 2021, ACL.

[11]  Christine L. Borgman,et al.  From Data Processes to Data Products: Knowledge Infrastructures in Astronomy , 2021, Harvard Data Science Review.

[12]  Yelong Shen,et al.  LoRA: Low-Rank Adaptation of Large Language Models , 2021, ICLR.

[13]  Joe Davison,et al.  Compacter: Efficient Low-Rank Hypercomplex Adapter Layers , 2021, NeurIPS.

[14]  Lidong Bing,et al.  On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation , 2021, ACL.

[15]  C. Conselice,et al.  Galaxy Evolution in All Five CANDELS Fields and IllustrisTNG: Morphological, Structural, and the Major Merger Evolution to z ∼ 3 , 2021, The Astrophysical Journal.

[16]  P. J. Richards,et al.  Gaia Early Data Release 3: Summary of the contents and survey properties , 2020, 2012.01533.

[17]  Katherine Elkins,et al.  Can GPT-3 Pass a Writer’s Turing Test? , 2020, Journal of Cultural Analytics.

[18]  J. Bagla,et al.  Study of galaxies on large-scale filaments in simulations , 2020, 2007.01149.

[19]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[20]  I. Chilingarian,et al.  Coupling local to global star formation in spiral galaxies: the effect of differential rotation , 2020, 2003.02848.

[21]  Jesse Vig,et al.  Visualizing Attention in Transformer-Based Language Representation Models , 2019, ArXiv.

[22]  S. Lesteven,et al.  LISA VII: Open Science: At the Frontiers of Librarianship , 2015 .

[23]  Stephen S. Murray,et al.  ADS: The Next Generation Search Platform , 2014, ArXiv.

[24]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .