Zero-Shot Learning of Aerosol Optical Properties with Graph Neural Networks

Aerosols sourced from combustion such as black carbon (BC) are important short-lived climate forcers whose direct radiative forcing and atmospheric lifetime depend on their morphology. These aerosols' complex morphology makes modeling their optical properties difficult, contributing to uncertainty in both their direct and indirect climate effects. Accurate and fast calculations of BC optical properties are needed for remote sensing inversions and for radiative forcing calculations in atmospheric models, but current methods to accurately calculate the optical properties of these aerosols are computationally expensive and are compiled in extensive databases off-line to be used as a look-up table. Recent advances in machine learning approaches have shown the potential of graph neural networks (GNN's) for various physical science applications, demonstrating skill in generalizing beyond initial training data by learning internal properties and small-scale interactions defining the emergent behavior of the larger system. Here we demonstrate that a GNN trained to predict the optical properties of numerically-generated BC fractal aggregates can accurately generalize to arbitrarily shaped particles, even over much larger (10x) aggregates than in the training dataset, providing a fast and accurate method to calculate aerosol optical properties in models and for observational retrievals. This zero-shot learning approach could be integrated into atmospheric models or remote sensing inversions to predict the physical properties of realistically-shaped aerosol and cloud particles. In addition, GNN's can be used to gain physical intuition on the relationship between small-scale interactions (here of the spheres' positions and interactions) and large-scale properties (here of the radiative properties of aerosols).

[1]  Max Welling,et al.  E(n) Equivariant Graph Neural Networks , 2021, ICML.

[2]  Quoc V. Le,et al.  A graph placement methodology for fast chip design , 2021, Nature.

[3]  B. DeAngelo,et al.  Bounding the role of black carbon in the climate system: A scientific assessment , 2013 .

[4]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[5]  Nicholay Topin,et al.  Super-convergence: very fast training of neural networks using large learning rates , 2018, Defense + Commercial Sensing.

[6]  Michael I. Mishchenko,et al.  Calculation of the T matrix and the scattering matrix for ensembles of spheres , 1996 .

[7]  Dantong Liu,et al.  Lifecycle of light-absorbing carbonaceous aerosols in the atmosphere , 2020, npj Climate and Atmospheric Science.

[8]  Kouji Adachi,et al.  Fractal parameters of individual soot particles determined using electron tomography: Implications for optical properties , 2007 .

[9]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[10]  D. Fahey,et al.  Black carbon aerosol size in snow , 2013, Scientific Reports.

[11]  Mark A. Garro,et al.  Simulation of Aggregates with Point-Contacting Monomers in the Cluster–Dilute Regime. Part 2: Comparison of Two- and Three-Dimensional Structural Properties as a Function of Fractal Dimension , 2011 .

[12]  S. Will,et al.  Laser-induced incandescence: Particulate diagnostics for combustion, atmospheric, and industrial applications , 2015 .

[13]  Y. Kondo,et al.  Anthropogenic iron oxide aerosols enhance atmospheric heating , 2017, Nature Communications.

[14]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[15]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[16]  Xiao-Ming Wu,et al.  Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning , 2018, AAAI.

[17]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[18]  Zhen Lin,et al.  Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network , 2018, NeurIPS.

[19]  Demis Hassabis,et al.  Improved protein structure prediction using potentials from deep learning , 2020, Nature.

[20]  C. Sorensen Light Scattering by Fractal Aggregates: A Review , 2001 .

[21]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[22]  A. Middlebrook,et al.  Complex refractive indices in the ultraviolet and visible spectral region for highly absorbing non-spherical biomass burning aerosol , 2020 .

[23]  Pierre Gentine,et al.  Deep learning to represent subgrid processes in climate models , 2018, Proceedings of the National Academy of Sciences.

[24]  T. Onasch,et al.  Radiative absorption enhancements by black carbon controlled by particle-to-particle heterogeneity in composition , 2020, Proceedings of the National Academy of Sciences.

[25]  Tianhai Cheng,et al.  Black carbon radiative forcing at TOA decreased during aging , 2016, Scientific Reports.

[26]  M. Head‐Gordon,et al.  Resonance-stabilized hydrocarbon-radical chain reactions may explain soot inception and growth , 2018, Science.

[27]  Sorensen,et al.  The Prefactor of Fractal Aggregates , 1997, Journal of colloid and interface science.

[28]  P. Ciais,et al.  Estimation of global black carbon direct radiative forcing and its uncertainty constrained by observations , 2016 .

[29]  M. Mishchenko,et al.  A multiple sphere T-matrix Fortran code for use on parallel computer clusters , 2011 .

[30]  R. Bambha,et al.  Effects of aggregate morphology and size on laser-induced incandescence and scattering from black carbon (mature soot) , 2015 .

[31]  Yongming Zhang,et al.  Applying machine learning to estimate the optical properties of black carbon fractal aggregates , 2018 .

[32]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[33]  Pierre Gentine,et al.  Could Machine Learning Break the Convection Parameterization Deadlock? , 2018, Geophysical Research Letters.

[34]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[35]  Rui Xu,et al.  Discovering Symbolic Models from Deep Learning with Inductive Biases , 2020, NeurIPS.

[36]  Stephan Günnemann,et al.  Directional Message Passing for Molecular Graphs , 2020, ICLR.

[37]  Li Li,et al.  Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds , 2018, ArXiv.

[38]  M. Schnaiter,et al.  Black carbon aggregates: A database for optical properties , 2019, Journal of Quantitative Spectroscopy and Radiative Transfer.

[39]  Jeffrey C. Grossman,et al.  Graph dynamical networks for unsupervised learning of atomic scale dynamics in materials , 2019, Nature Communications.

[40]  B. Gustafson,et al.  A generalized multiparticle Mie-solution: further experimental verification , 2001 .

[41]  M. Kahnert Numerically exact computation of the optical properties of light absorbing carbon aggregates for wavelength of 200 nm–12.2 μm , 2010 .

[42]  Roy G. Grainger,et al.  Simplifying the calculation of light scattering properties for black carbon fractal aggregates , 2014 .

[43]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[44]  T. Witten,et al.  Long-range correlations in smoke-particle aggregates , 1979 .

[45]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[46]  P. Barber Absorption and scattering of light by small particles , 1984 .

[47]  D. Mackowski,et al.  Calculation of total cross sections of multiple-sphere clusters , 1994 .

[48]  Mario V. Wuthrich,et al.  Networks, Random Graphs and Percolation , 2014, 1409.2339.

[49]  D. Murphy,et al.  Investigating biomass burning aerosol morphology using a laser imaging nephelometer , 2017 .

[50]  T. Müller,et al.  Radiative properties of coated black carbon aggregates: numerical simulations and radiative forcing estimates , 2021 .

[51]  Benjamin Kurt Miller,et al.  Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties , 2020, ArXiv.

[52]  T. Cheng,et al.  Fractal Dimensions and Mixing Structures of Soot Particles during Atmospheric Processing , 2017 .

[53]  Uli Lemmer,et al.  CELES: CUDA-accelerated simulation of electromagnetic scattering by large ensembles of spheres , 2017, 1706.02145.

[54]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[55]  E. Purcell,et al.  Scattering and Absorption of Light by Nonspherical Dielectric Grains , 1973 .

[56]  G. Mie Beiträge zur Optik trüber Medien, speziell kolloidaler Metallösungen , 1908 .

[57]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[58]  Xi Zhang,et al.  Aerosol influence on energy balance of the middle atmosphere of Jupiter , 2015, Nature Communications.

[59]  Pushmeet Kohli,et al.  Unveiling the predictive power of static structure in glassy systems , 2020 .

[60]  S. M. Moosavi,et al.  The Role of Machine Learning in the Understanding and Design of Materials , 2020, Journal of the American Chemical Society.

[61]  U. Lohmann,et al.  Future warming exacerbated by aged-soot effect on cloud formation , 2020, Nature Geoscience.

[62]  M. A. Yurkina,et al.  The discrete dipole approximation : An overview and recent developments , 2007 .

[63]  D. E. Rosner,et al.  Fractal-like Aggregates: Relation between Morphology and Physical Properties. , 2000, Journal of colloid and interface science.

[64]  T. Cheng,et al.  The role of biomass burning states in light absorption enhancement of carbonaceous aerosols , 2020, Scientific Reports.

[65]  Mark A. Garro,et al.  Simulation of Aggregates with Point-Contacting Monomers in the Cluster–Dilute Regime. Part 1: Determining the Most Reliable Technique for Obtaining Three-Dimensional Fractal Dimension from Two-Dimensional Images , 2011 .

[66]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[67]  Nicola De Cao,et al.  MolGAN: An implicit generative model for small molecular graphs , 2018, ArXiv.

[68]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[69]  Y L Xu,et al.  Electromagnetic scattering by an aggregate of spheres. , 1995, Applied optics.

[70]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[71]  M. Kahnert,et al.  Modelling optical properties of atmospheric black carbon aerosols , 2020, Journal of Quantitative Spectroscopy and Radiative Transfer.