When Geoscience Meets Generative AI and Large Language Models: Foundations, Trends, and Future Challenges

Generative Artificial Intelligence (GAI) represents an emerging field that promises the creation of synthetic data and outputs in different modalities. GAI has recently shown impressive results across a large spectrum of applications ranging from biology, medicine, education, legislation, computer science, and finance. As one strives for enhanced safety, efficiency, and sustainability, generative AI indeed emerges as a key differentiator and promises a paradigm shift in the field. This article explores the potential applications of generative AI and large language models in geoscience. The recent developments in the field of machine learning and deep learning have enabled the generative model's utility for tackling diverse prediction problems, simulation, and multi‐criteria decision‐making challenges related to geoscience and Earth system dynamics. This survey discusses several GAI models that have been used in geoscience comprising generative adversarial networks (GANs), physics‐informed neural networks (PINNs), and generative pre‐trained transformer (GPT)‐based structures. These tools have helped the geoscience community in several applications, including (but not limited to) data generation/augmentation, super‐resolution, panchromatic sharpening, haze removal, restoration, and land surface changing. Some challenges still remain, such as ensuring physical interpretation, nefarious use cases, and trustworthiness. Beyond that, GAI models show promises to the geoscience community, especially with the support to climate change, urban science, atmospheric science, marine science, and planetary science through their extraordinary ability to data‐driven modelling and uncertainty quantification.

[1]  Chao Shi,et al.  Multi-scale generative adversarial networks (GAN) for generation of three-dimensional subsurface geological models from limited boreholes and prior geological knowledge , 2024, Computers and Geotechnics.

[2]  Niklas Linde,et al.  Deep generative networks for multivariate fullstack seismic data inversion using inverse autoregressive flows , 2024, Computers & Geosciences.

[3]  Wenchao Tang,et al.  Diffusion models for spatio-temporal-spectral fusion of homogeneous Gaofen-1 satellite platforms , 2024, International Journal of Applied Earth Observation and Geoinformation.

[4]  Anqin Zhang,et al.  Conditional stochastic simulation of fluvial reservoirs using multi-scale concurrent generative adversarial networks , 2024, Computational Geosciences.

[5]  S. Jana,et al.  TrustLLM: Trustworthiness in Large Language Models , 2024, ArXiv.

[6]  Muzammal Naseer,et al.  GeoChat: Grounded Large Vision-Language Model for Remote Sensing , 2023, ArXiv.

[7]  Ender Demir,et al.  ChatGPT: Unlocking the Future of NLP in Finance , 2023, SSRN Electronic Journal.

[8]  Elaine Mulcahy,et al.  Further delays in tackling greenhouse gas emissions at COP28 will be an act of negligence , 2023, The Lancet.

[9]  Azul Garza,et al.  TimeGPT-1 , 2023, 2310.03589.

[10]  Niklas Boers,et al.  When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System , 2023, ArXiv.

[11]  Shraddha M. Naik,et al.  Ten years of generative adversarial nets (GANs): a survey of the state-of-the-art , 2023, Mach. Learn. Sci. Technol..

[12]  D. Gašević,et al.  Can Large Language Models Provide Feedback to Students? A Case Study on ChatGPT , 2023, 2023 IEEE International Conference on Advanced Learning Technologies (ICALT).

[13]  Mala Deep Upadhaya,et al.  Performance of ChatGPT on USMLE: Unlocking the Potential of Large Language Models for AI-Assisted Medical Education , 2023, ArXiv.

[14]  Kevin D. Ashley,et al.  Explaining Legal Concepts with Augmented Large Language Models (GPT-4) , 2023, ArXiv.

[15]  U. Kumar,et al.  Prediction of transportation index for urban patterns in small and medium-sized Indian cities using hybrid RidgeGAN model , 2023, Scientific reports.

[16]  Rose E. Wang,et al.  Is ChatGPT a Good Teacher Coach? Measuring Zero-Shot Performance For Scoring and Providing Actionable Insights on Classroom Instruction , 2023, BEA.

[17]  Nirav Patel Generative Artificial Intelligence and Remote Sensing: A perspective on the past and the future [Perspectives] , 2023, IEEE Geoscience and Remote Sensing Magazine.

[18]  Qing Yang,et al.  XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameters , 2023, CIKM.

[19]  Vivek Natarajan,et al.  Towards Expert-Level Medical Question Answering with Large Language Models , 2023, ArXiv.

[20]  Bo Du,et al.  Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model , 2023, NeurIPS.

[21]  Lingming Zhang,et al.  Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation , 2023, NeurIPS.

[22]  Guijin Son,et al.  Beyond Classification: Financial Reasoning in State-of-the-Art Language Models , 2023, FINNLP.

[23]  Senlin Zhang,et al.  Physics-informed Neural Network Combined with Characteristic-Based Split for Solving Navier-Stokes Equations , 2023, Engineering Applications of Artificial Intelligence.

[24]  Juho Leinonen,et al.  Comparing Code Explanations Created by Students and Large Language Models , 2023, ITiCSE.

[25]  P. Ray ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope , 2023, Internet of Things and Cyber-Physical Systems.

[26]  Wayne Xin Zhao,et al.  A Survey of Large Language Models , 2023, ArXiv.

[27]  P. Kambadur,et al.  BloombergGPT: A Large Language Model for Finance , 2023, ArXiv.

[28]  E. Horvitz,et al.  Capabilities of GPT-4 on Medical Challenge Problems , 2023, ArXiv.

[29]  Philip S. Yu,et al.  A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT , 2023, ArXiv.

[30]  Xinbing Wang,et al.  GeoDeepShovel: A platform for building scientific database from geoscience literature with AI assistance , 2023, Geoscience Data Journal.

[31]  Naman Goyal,et al.  LLaMA: Open and Efficient Foundation Language Models , 2023, ArXiv.

[32]  Lei Zhu,et al.  A Comprehensive Survey on Source-free Domain Adaptation , 2023, IEEE transactions on pattern analysis and machine intelligence.

[33]  Z. Pardos,et al.  Learning gain differences between ChatGPT and human tutor generated algebra hints , 2023, ArXiv.

[34]  Justin D. Weisz,et al.  The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development , 2023, IUI.

[35]  Benjamin Van Durme,et al.  Can GPT-3 Perform Statutory Reasoning? , 2023, ICAIL.

[36]  D. Levine,et al.  The Diagnostic and Triage Accuracy of the GPT-3 Artificial Intelligence Model , 2023, medRxiv.

[37]  J. El-Khoury,et al.  Evaluating the Performance of ChatGPT in Ophthalmology , 2023, medRxiv.

[38]  Hyung Won Chung,et al.  Large language models encode clinical knowledge , 2022, Nature.

[39]  Frank Schilder,et al.  Legal Prompting: Teaching a Language Model to Think Like a Lawyer , 2022, ArXiv.

[40]  Guillem Cucurull,et al.  Galactica: A Large Language Model for Science , 2022, ArXiv.

[41]  Hang Su,et al.  Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications , 2022, ArXiv.

[42]  Xiao Xiang Zhu,et al.  EarthNets: Empowering AI in Earth Observation , 2022, ArXiv.

[43]  A. Hadid,et al.  W-Transformers: A Wavelet-based Transformer Framework for Univariate Time Series Forecasting , 2022, 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA).

[44]  Sizhe Wang,et al.  GeoImageNet: a multi-source natural feature benchmark dataset for GeoAI and supervised machine learning , 2022, GeoInformatica.

[45]  Ming-Hsuan Yang,et al.  Diffusion Models: A Comprehensive Survey of Methods and Applications , 2022, ACM Comput. Surv..

[46]  Brendan Dolan-Gavitt,et al.  Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants , 2022, USENIX Security Symposium.

[47]  O. Winther,et al.  Can large language models reason about medical questions? , 2022, Patterns.

[48]  Lei Dong,et al.  MetroGAN: Simulating Urban Morphology with Generative Adversarial Network , 2022, KDD.

[49]  C. Piech,et al.  The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialogues , 2022, EDM.

[50]  Yisheng Song,et al.  A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities , 2022, ACM Comput. Surv..

[51]  Skylar W. Marvel,et al.  ToxPi*GIS Toolkit: creating, viewing, and sharing integrative visualizations for geospatial data using ArcGIS , 2022, Journal of Exposure Science & Environmental Epidemiology.

[52]  Muhammad Ejaz Ahmed,et al.  Transformer-Based Language Models for Software Vulnerability Detection , 2022, ACSAC.

[53]  Frank F. Xu,et al.  A systematic evaluation of large language models of code , 2022, MAPS@PLDI.

[54]  D. Grana,et al.  Application of Bayesian Generative Adversarial Networks to Geological Facies Modeling , 2022, Mathematical Geosciences.

[55]  Pascale Fung,et al.  Survey of Hallucination in Natural Language Generation , 2022, ACM Comput. Surv..

[56]  H. Gotovac,et al.  Application of physics-informed neural networks to inverse problems in unsaturated groundwater flow , 2021, Georisk: Assessment and Management of Risk for Engineered Systems and Geohazards.

[57]  G. Karniadakis,et al.  Physics‐Informed Neural Networks (PINNs) for Wave Propagation and Full Waveform Inversions , 2021, Journal of Geophysical Research: Solid Earth.

[58]  Yelong Shen,et al.  LoRA: Low-Rank Adaptation of Large Language Models , 2021, ICLR.

[59]  A. Wills,et al.  Physics-informed machine learning , 2021, Nature Reviews Physics.

[60]  Youzuo Lin,et al.  SeismoGen: Seismic Waveform Synthesis Using GAN With Application to Seismic Data Augmentation , 2021, Journal of Geophysical Research: Solid Earth.

[61]  Jun Li,et al.  Solving localized wave solutions of the derivative nonlinear Schrödinger equation using an improved PINN method , 2021, Nonlinear Dynamics.

[62]  Chengkai Zhang,et al.  U-net generative adversarial network for subsurface facies modeling , 2021, Computational Geosciences.

[63]  P. Tahmasebi,et al.  Physics informed machine learning: Seismic wave equation , 2020, Geoscience Frontiers.

[64]  Rowena Rodrigues Legal and human rights issues of AI: Gaps, challenges and vulnerabilities , 2020 .

[65]  Ion Androutsopoulos,et al.  LEGAL-BERT: “Preparing the Muppets for Court’” , 2020, FINDINGS.

[66]  Song Gao,et al.  Multiscale dynamic human mobility flow dataset in the U.S. during the COVID-19 epidemic , 2020, Scientific Data.

[67]  Sue Ellen Haupt,et al.  Open weather and climate science in the digital era , 2020, Geoscience Communication.

[68]  Tariq Alkhalifah,et al.  PINNeik: Eikonal solution using physics-informed neural networks , 2020, Comput. Geosci..

[69]  Qiusheng Wu,et al.  geemap: A Python package for interactive mapping with Google Earth Engine , 2020, J. Open Source Softw..

[70]  Mehdi P. Heris,et al.  A rasterized building footprint dataset for the United States , 2020, Scientific Data.

[71]  T. Nissen‐Meyer,et al.  Solving the wave equation with physics-informed deep learning , 2020, 2006.11894.

[72]  Jiagen Hou,et al.  Geological Facies modeling based on progressive growing of generative adversarial networks (GANs) , 2020, Computational Geosciences.

[73]  Marcelo Kehl de Souza,et al.  Evaluation of machine learning methods for lithology classification using geophysical data , 2020, Comput. Geosci..

[74]  Chris Hill,et al.  DiscretizationNet: A Machine-Learning based solver for Navier-Stokes Equations using Finite Volume Discretization , 2020, Computer Methods in Applied Mechanics and Engineering.

[75]  G. Karniadakis,et al.  B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data , 2020, J. Comput. Phys..

[76]  George Em Karniadakis,et al.  NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations , 2020, J. Comput. Phys..

[77]  G. Karniadakis,et al.  Physics-informed neural networks for high-speed flows , 2020, Computer Methods in Applied Mechanics and Engineering.

[78]  Hui Xiong,et al.  A Comprehensive Survey on Transfer Learning , 2019, Proceedings of the IEEE.

[79]  Thomas Esch,et al.  Outlining where humans live, the World Settlement Footprint 2015 , 2019, Scientific Data.

[80]  Dogu Araci,et al.  FinBERT: Financial Sentiment Analysis with Pre-trained Language Models , 2019, ArXiv.

[81]  Zhiping Mao,et al.  DeepXDE: A Deep Learning Library for Solving Differential Equations , 2019, AAAI Spring Symposium: MLPS.

[82]  Lin Liang,et al.  Generating geologically realistic 3D reservoir facies models using deep learning of sedimentary architecture with generative adversarial networks , 2019, Petroleum Science.

[83]  Maarten V. de Hoop,et al.  Machine learning for data-driven discovery in solid Earth geoscience , 2019, Science.

[84]  Begüm Demir,et al.  Bigearthnet: A Large-Scale Benchmark Archive for Remote Sensing Image Understanding , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[85]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[86]  Michael. Horswell,et al.  GIS has changed! Exploring the potential of ArcGIS online , 2018 .

[87]  Emanuele Strano,et al.  Modeling Urbanization Patterns with Generative Adversarial Networks , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[88]  Anuj Karpatne,et al.  Machine Learning for the Geosciences: Challenges and Opportunities , 2017, IEEE Transactions on Knowledge and Data Engineering.

[89]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[90]  Siu Cheung Hui,et al.  Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture , 2017, SIGIR.

[91]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[92]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[93]  Vladimir Vovk,et al.  Nonparametric predictive distributions based on conformal prediction , 2017, Machine Learning.

[94]  Ben Evans,et al.  The Australian Geoscience Data Cube - foundations and lessons learned , 2017 .

[95]  Martin J. Blunt,et al.  Reconstruction of three-dimensional porous media using generative adversarial neural networks , 2017, Physical review. E.

[96]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[97]  Kevin Gimpel,et al.  Gaussian Error Linear Units (GELUs) , 2016, 1606.08415.

[98]  Surya Ganguli,et al.  Deep Unsupervised Learning using Nonequilibrium Thermodynamics , 2015, ICML.

[99]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[100]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[101]  Shaowen Wang,et al.  CyberGIS software: a synthetic review and integration roadmap , 2013, Int. J. Geogr. Inf. Sci..

[102]  R. Kays,et al.  The Movebank data model for animal tracking , 2011, Environ. Model. Softw..

[103]  Xing Xie,et al.  GeoLife: A Collaborative Social Networking Service among User, Location and Trajectory , 2010, IEEE Data Eng. Bull..

[104]  V. Jayaraman,et al.  Remote sensing applications : An overview , 2007 .

[105]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[106]  Jiangtao Peng,et al.  DBCTNet: Double Branch Convolution-Transformer Network for Hyperspectral Image Classification , 2024, IEEE Transactions on Geoscience and Remote Sensing.

[107]  Xile Zhao,et al.  GraphGST: Graph Generative Structure-Aware Transformer for Hyperspectral Image Classification , 2024, IEEE Transactions on Geoscience and Remote Sensing.

[108]  Xu Sun,et al.  DAAN: A Deep Autoencoder-Based Augmented Network for Blind Multilinear Hyperspectral Unmixing , 2024, IEEE Transactions on Geoscience and Remote Sensing.

[109]  Lars Hornuf,et al.  Using GPT-4 for Financial Advice , 2023, SSRN Electronic Journal.

[110]  Xinbing Wang,et al.  Learning A Foundation Language Model for Geoscience Knowledge Understanding and Utilization , 2023, ArXiv.

[111]  Yijie Zhang,et al.  Seismic Inversion Based on Acoustic Wave Equations Using Physics-Informed Neural Network , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[112]  Jian Wang,et al.  Residual Learning of Cycle-GAN for Seismic Data Denoising , 2021, IEEE Access.

[113]  Qingqi Pei,et al.  Remote Sensing Data Augmentation Through Adversarial Training , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[114]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[115]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .

[116]  Ryan J. Tibshirani Conformal Prediction , 2017 .

[117]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[118]  Vladimir Vovk,et al.  Kernel Ridge Regression , 2013, Empirical Inference.

[119]  Kevin P. Murphy Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.