Applying Convolutional Neural Networks to Data on Unstructured Meshes with Space-Filling Curves

This paper presents the first classical Convolutional Neural Network (CNN) that can be applied directly to data from unstructured finite element meshes or control volume grids. CNNs have been hugely influential in the areas of image classification and image compression, both of which typically deal with data on structured grids. Unstructured meshes are frequently used to solve partial differential equations and are particularly suitable for problems that require the mesh to conform to complex geometries or for problems that require variable mesh resolution. Central to the approach are space-filling curves, which traverse the nodes or cells of a mesh tracing out a path that is as short as possible (in terms of numbers of edges) and that visits each node or cell exactly once. The space-filling curves (SFCs) are used to find an ordering of the nodes or cells that can transform multi-dimensional solutions on unstructured meshes into a one-dimensional (1D) representation, to which 1D convolutional layers can then be applied. Although developed in two dimensions, the approach is applicable to higher dimensional problems. To demonstrate the approach, the network we choose is a convolutional autoencoder (CAE) although other types of CNN could be used. The approach is tested by applying CAEs to data sets that have been reordered with an SFC. Sparse layers are used at the input and output of the autoencoder, and the use of multiple SFCs is explored. We compare the accuracy of the SFC-based CAE with that of a classical CAE applied to two idealised problems on structured meshes, and then apply the approach to solutions of flow past a cylinder obtained using the finite-element method and an unstructured mesh.

[1]  Qiang Lu,et al.  PointNGCNN: Deep convolutional networks on 3D point clouds with neighborhood graph filters , 2020, Comput. Graph..

[2]  Kookjin Lee,et al.  Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders , 2018, J. Comput. Phys..

[3]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Christopher C. Pain,et al.  An autoencoder-based reduced-order model for eigenvalue problems with application to neutron diffusion , 2020, ArXiv.

[6]  Mohammad Saifur Rahman,et al.  CNN Model With Hilbert Curve Representation of DNA Sequence For Enhancer Prediction , 2019, bioRxiv.

[7]  Frédéric Alauzet,et al.  A decade of progress on anisotropic mesh adaptation for computational fluid dynamics , 2016, Comput. Aided Des..

[8]  Hans-Joachim Bungartz,et al.  A Parallel Adaptive Cartesian PDE Solver Using Space-Filling Curves , 2006, Euro-Par.

[9]  J. Percival,et al.  A balanced-force control volume finite element method for interfacial flows with surface tension using adaptive anisotropic unstructured meshes , 2016 .

[10]  James M. Kang,et al.  Space-Filling Curves , 2017, Encyclopedia of GIS.

[11]  Oubay Hassan,et al.  Advances in co-volume mesh generation and mesh optimisation techniques , 2017 .

[12]  Wenjie Zhang,et al.  Data-driven reduced order model with temporal convolutional neural network , 2020 .

[13]  Geoffrey E. Hinton,et al.  Transforming Auto-Encoders , 2011, ICANN.

[14]  Karthik Duraisamy,et al.  Multi-level Convolutional Autoencoder Networks for Parametric Prediction of Spatio-temporal Dynamics , 2019, Computer Methods in Applied Mechanics and Engineering.

[15]  Silvia Crivelli,et al.  A Spatial Mapping Algorithm with Applications in Deep Learning-Based Structure Classification , 2018, ArXiv.

[16]  Eirikur Agustsson,et al.  High-Fidelity Generative Image Compression , 2020, NeurIPS.

[17]  I. Moulitsas,et al.  Multilevel Algorithms for Generating Coarse Grids for Multigrid Methods , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[18]  S. Dargaville,et al.  A comparison of element agglomeration algorithms for unstructured geometric multigrid , 2020, ArXiv.

[19]  Brendan Kochunas,et al.  Spatial decomposition of structured grids for nuclear reactor simulations , 2019, Annals of Nuclear Energy.

[20]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[21]  Francisco J. Gonzalez,et al.  Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems , 2018, ArXiv.

[22]  Jens Zimmermann,et al.  Parallelizing an Unstructured Grid Generator with a Space-Filling Curve Approach , 2000, Euro-Par.

[23]  Meire Fortunato,et al.  Learning Mesh-Based Simulation with Graph Networks , 2020, ArXiv.

[24]  Daniel Cohen-Or,et al.  MeshCNN: a network with an edge , 2019, ACM Trans. Graph..

[25]  Christopher C. Pain,et al.  A domain decomposition non-intrusive reduced order model for turbulent flows , 2019, Computers & Fluids.

[26]  Jure Leskovec,et al.  Learning to Simulate Complex Physics with Graph Networks , 2020, ICML.

[27]  Claudia Plant,et al.  A Novel Hilbert Curve for Cache-Locality Preserving Loops , 2021, IEEE Transactions on Big Data.

[28]  J. Hesthaven,et al.  Non-intrusive reduced order modeling of nonlinear problems using neural networks , 2018, J. Comput. Phys..

[29]  Yike Guo,et al.  A reduced order model for turbulent flows in the urban environment using machine learning , 2019, Building and Environment.

[30]  Tomoyuki Obuchi,et al.  Mean-field theory of graph neural networks in graph partitioning , 2018, NeurIPS.

[31]  Junmo Kim,et al.  Image Classification Using Convolutional Neural Networks With Multi-stage Feature , 2014, RiTA.

[32]  D. Hilbert.  Ueber die stetige Abbildung einer Line auf ein Flächenstück , 1891 .

[33]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[34]  George Karypis,et al.  Multilevel k-way Partitioning Scheme for Irregular Graphs , 1998, J. Parallel Distributed Comput..

[35]  Rossella Arcucci,et al.  Attention-based Convolutional Autoencoders for 3D-Variational Data Assimilation , 2020, Computer Methods in Applied Mechanics and Engineering.

[36]  Michael Bader,et al.  Space-Filling Curves - An Introduction with Applications in Scientific Computing , 2012, Texts in Computational Science and Engineering.

[37]  C.R.E. de Oliveira,et al.  Tetrahedral mesh optimisation and adaptivity for steady-state and transient finite element calculations , 2001 .

[38]  Ewa Skubalska-Rafajłowicz,et al.  Applications of the space-filling curves with data driven measure-preserving property , 1997 .

[39]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[40]  G. Peano.  Sur une courbe, qui remplit toute une aire plane , 1890 .

[41]  Nikos Papamarkos,et al.  Color reduction using local features and a kohonen self-organized feature map neural network , 1999, Int. J. Imaging Syst. Technol..

[42]  C. C. Pain,et al.  A neural network graph partitioning procedure for grid-based domain decomposition , 1999 .

[43]  Sander M. Bohte,et al.  An image representation based convolutional network for DNA classification , 2018, ICLR.

[44]  A. Muggeridge,et al.  Dynamic adaptive mesh optimisation for immiscible viscous fingering , 2020, Computational Geosciences.

[45]  Omer San,et al.  Memory embedded non-intrusive reduced order modeling of non-ergodic flows , 2019 .

[46]  Sorin Draghici,et al.  Space-filling curves and Kolmogorov superposition-based neural networks , 2002, Neural Networks.

[47]  Peter Sanders,et al.  Recent Advances in Graph Partitioning , 2013, Algorithm Engineering.