Knowledge-Driven Transfer Learning for Tree Species Recognition

Deep learning methods on remote sensing data are an attractive approach in place of human observation for automating recognition of hundreds of thousands of tree species in nature. However, this approach requires a large amount of training data for each species, while actual data are scarce - only a small subset of tree species data can be acquired, notwithstanding the unknown, new species. To overcome the data scarcity challenge and to enable versatile recognition of known and unknown species, we propose a knowledge-driven transfer learning framework for tree species profiling, where a base model of multitasking graph neural network is trained on synthetic species data, which are generated from the universal botany domain knowledge and limited field measurement data. This base model is then transferred to a new multitasking graph neural network model to train on real tree data of limited availability. Our proposed species recognition framework was tested for profiling tree species by classifying a few species profile parameters and showed a significant improvement in the prediction accuracy in comparison to deep learning models trained on just real tree data.

[1]  Laura von Rueden,et al.  Informed Machine Learning – A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems , 2019, IEEE Transactions on Knowledge and Data Engineering.

[2]  Guoqi Chai,et al.  ACE R-CNN: An Attention Complementary and Edge Detection-Based Instance Segmentation Algorithm for Individual Tree Species Identification Using UAV RGB Images and LiDAR Data , 2022, Remote. Sens..

[3]  Fei Gao,et al.  Theory‐Guided Machine Learning to Predict the Performance of Noble Metal Catalysts in the Water‐Gas Shift Reaction , 2022, ChemCatChem.

[4]  Lorenzo Livi,et al.  Graph Neural Networks With Convolutional ARMA Filters , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  L. Bruzzone,et al.  An Approach Based on Deep Learning for Tree Species Classification in LiDAR Data Acquired in Mixed Forest , 2022, IEEE Geoscience and Remote Sensing Letters.

[6]  Le Song,et al.  Graph Neural Networks , 2021, Deep Learning on Graphs.

[7]  Yong Wang,et al.  Recent advances in the application of deep learning methods to forestry , 2021, Wood Science and Technology.

[8]  Sakari Tuominen,et al.  Tree species classification from airborne hyperspectral and LiDAR data using 3D convolutional neural networks , 2021, Remote Sensing of Environment.

[9]  Zhengjun Liu,et al.  Tree species classification of LiDAR data based on 3D deep learning , 2021 .

[10]  Jacek Rapinski,et al.  A Review of Tree Species Classification Based on Airborne LiDAR Data and Applied Classifiers , 2021, Remote. Sens..

[11]  Cesare Alippi,et al.  Graph Neural Networks in TensorFlow and Keras with Spektral , 2020, IEEE Comput. Intell. Mag..

[12]  Hui Xiong,et al.  A Comprehensive Survey on Transfer Learning , 2019, Proceedings of the IEEE.

[13]  Like Gobeawan,et al.  Tree Species Modelling for Digital Twin Cities , 2021, IFIP International Conference on Theoretical Computer Science.

[14]  Alejandro Barredo Arrieta,et al.  Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI , 2019, Inf. Fusion.

[15]  Hee Joo Poh,et al.  Generation of Tree Surface Mesh Models from Point Clouds using Skin Surfaces , 2020, VISIGRAPP.

[16]  Patrick Mäder,et al.  Image-based classification of plant genus and family for trained and untrained plant species , 2019, BMC Bioinformatics.

[17]  Hamid Hamraz,et al.  Deep learning for conifer/deciduous classification of airborne LiDAR 3D point clouds representing individual trees , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[18]  Pietro Liò,et al.  Towards Sparse Hierarchical Graph Classifiers , 2018, ArXiv.

[19]  Xiaojing Huang,et al.  MODELING TREES FOR VIRTUAL SINGAPORE: FROM DATA ACQUISITION TO CITYGML MODELS , 2018, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[20]  Chong Wang,et al.  Attention-based Graph Neural Network for Semi-supervised Learning , 2018, ArXiv.

[21]  Ming Cheng,et al.  Tree Classification in Complex Forest Point Clouds Based on Deep Learning , 2017, IEEE Geoscience and Remote Sensing Letters.

[22]  K. H. Soon,et al.  CITYGML MODELLING FOR SINGAPORE 3D NATIONAL MAPPING , 2017 .

[23]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[24]  Nagiza F. Samatova,et al.  Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data , 2016, IEEE Transactions on Knowledge and Data Engineering.

[25]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[26]  M. Christenhusz,et al.  The number of known plants species in the world and its annual increase , 2016 .

[27]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[28]  Qi Zhang,et al.  Deep learning-based tree classification using mobile LiDAR data , 2015 .