What is the Point? Evaluating the Structure, Color, and Semantic Traits of Computer Vision Point Clouds of Vegetation

Remote sensing of the structural and spectral traits of vegetation is being transformed by structure from motion (SFM) algorithms that combine overlapping images to produce three-dimensional (3D) red-green-blue (RGB) point clouds. However, much remains unknown about how these point clouds are used to observe vegetation, limiting the understanding of the results and future applications. Here, we examine the content and quality of SFM point cloud 3D-RGB fusion observations. An SFM algorithm using the Scale Invariant Feature Transform (SIFT) feature detector was applied to create the 3D-RGB point clouds of a single tree and forest patches. The fusion quality was evaluated using targets placed within the tree and was compared to fusion measurements from terrestrial LIDAR (TLS). K-means clustering and manual classification were used to evaluate the semantic content of SIFT features. When targets were fully visible in the images, SFM assigned color in the correct place with a high accuracy (93%). The accuracy was lower when targets were shadowed or obscured (29%). Clustering and classification revealed that the SIFT features highlighted areas that were brighter or darker than their surroundings, showing little correspondence with canopy objects like leaves or branches, though the features showed some relationship to landscape context (e.g., canopy, pavement). Therefore, the results suggest that feature detectors play a critical role in determining how vegetation is sampled by SFM. Future research should consider developing feature detectors that are optimized for vegetation mapping, including extracting elements like leaves and flowers. Features should be considered the fundamental unit of SFM mapping, like the pixel in optical imaging and the laser pulse of LIDAR. Under optimal conditions, SFM fusion accuracy exceeded that of TLS, and the two systems produced similar representations of the overall tree shape. SFM is the lower-cost solution for obtaining accurate 3D-RGB fusion measurements of the outer surfaces of vegetation, the critical zone of interaction between vegetation, light, and the atmosphere from leaf to canopy scales.

[1]  N. Coops,et al.  Extracting urban vegetation characteristics using spectral mixture analysis and decision tree classifications. , 2009 .

[2]  Sébastien Lefèvre,et al.  Morphological Description of Color Images for Content-Based Image Retrieval , 2009, IEEE Transactions on Image Processing.

[3]  Daniel P. Huttenlocher,et al.  Location Recognition Using Prioritized Feature Matching , 2010, ECCV.

[4]  L. Monika Moskal,et al.  Fusion of LiDAR and imagery for estimating forest canopy fuels , 2010 .

[5]  Davide Marenchino,et al.  Performance Analysis of the SIFT Operator for Automatic Feature Extraction and Matching in Photogrammetric Applications , 2009, Sensors.

[6]  Erle C. Ellis,et al.  Remote Sensing of Vegetation Structure Using Computer Vision , 2010, Remote. Sens..

[7]  G. Parker,et al.  Structure and microclimate of forest canopies. , 1995 .

[8]  Erle C. Ellis,et al.  Using lightweight unmanned aerial vehicles to monitor tropical forest recovery , 2015 .

[9]  Kenji Omasa,et al.  Estimation and Error Analysis of Woody Canopy Leaf Area Density Profiles Using 3-D Airborne and Ground-Based Scanning Lidar Remote-Sensing Techniques , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Michael A. Lefsky,et al.  Volume estimates of trees with complex architecture from terrestrial laser scanning , 2008 .

[11]  S. Popescu,et al.  Seeing the Trees in the Forest: Using Lidar and Multispectral Data Fusion with Local Filtering and Variable Window Size for Estimating Tree Height , 2004 .

[12]  Maurizio Mencuccini,et al.  The relationship between carbon dioxide uptake and canopy colour from two camera systems in a deciduous forest in southern England , 2012, Functional Ecology.

[13]  Karen Anderson,et al.  Lightweight unmanned aerial vehicles will revolutionize spatial ecology , 2013 .

[14]  Chengquan Huang,et al.  Automated masking of cloud and cloud shadow for forest change analysis using Landsat images , 2010 .

[15]  W. Cohen,et al.  Integration of lidar and Landsat ETM+ data for estimating and mapping forest canopy height , 2002 .

[16]  C. Glennie Rigorous 3D error analysis of kinematic scanning LIDAR systems , 2007 .

[17]  Peter Reinartz,et al.  Applicability of the SIFT operator to geometric SAR image registration , 2010 .

[18]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Arko Lucieer,et al.  Assessing the Accuracy of Georeferenced Point Clouds Produced via Multi-View Stereopsis from Unmanned Aerial Vehicle (UAV) Imagery , 2012, Remote. Sens..

[20]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[21]  R. Dubayah,et al.  Integrating waveform lidar with hyperspectral imagery for inventory of a northern temperate forest , 2008 .

[22]  Carl Seielstad,et al.  Deriving Fuel Mass by Size Class in Douglas-fir (Pseudotsuga menziesii) Using Terrestrial Laser Scanning , 2011, Remote. Sens..

[23]  Marc Olano,et al.  Optimal Altitude, Overlap, and Weather Conditions for Computer Vision UAV Estimates of Forest Structure , 2015, Remote. Sens..

[24]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Anil K. Jain Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[26]  M. Maltamo,et al.  A Two Stage Method to Estimate Species-specific Growing Stock , 2009 .

[27]  Joachim M. Buhmann,et al.  Stability-Based Validation of Clustering Solutions , 2004, Neural Computation.

[28]  David J. Kriegman,et al.  Automated annotation of coral reef survey images , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  David E. Knapp,et al.  Operational Tree Species Mapping in a Diverse Tropical Forest with Airborne Imaging Spectroscopy , 2015, PloS one.

[30]  Brian R. Johnson,et al.  NEON: the first continental-scale ecological observatory with airborne remote sensing of vegetation canopy biochemistry and structure , 2010 .

[31]  Shawn D. Newsam,et al.  Geographic Image Retrieval Using Local Invariant Features , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Erle C. Ellis,et al.  High spatial resolution three-dimensional mapping of vegetation spectral dynamics using computer vision , 2013 .

[33]  J. Clevers,et al.  Classification of floodplain vegetation by data fusion of spectral (CASI) and LiDAR data , 2007 .

[34]  J. Brasington,et al.  Modeling the topography of shallow braided rivers using Structure-from-Motion photogrammetry , 2014 .

[35]  Cindy E. Hauser,et al.  Quantifying Plant Colour and Colour Difference as Perceived by Humans Using Digital Images , 2013, PloS one.

[36]  Yinghai Ke,et al.  A review of methods for automatic individual tree-crown detection and delineation from passive remote sensing , 2011 .

[37]  H. Olff,et al.  Mapping Tropical Forest Trees Using High‐Resolution Aerial Digital Photographs , 2013 .

[38]  Josechu J. Guerrero,et al.  Photogrammetric Methodology for the Production of Geomorphologic Maps: Application to the Veleta Rock Glacier (Sierra Nevada, Granada, Spain) , 2009, Remote. Sens..

[39]  Russell G. Congalton,et al.  A review of assessing the accuracy of classifications of remotely sensed data , 1991 .

[40]  Roberta E. Martin,et al.  Airborne spectranomics: mapping canopy chemical and taxonomic diversity in tropical forests , 2009 .

[41]  M. Westoby,et al.  ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications , 2012 .

[42]  Daniel J. Isaak,et al.  Improving Stream Studies With a Small‐Footprint Green Lidar , 2009 .

[43]  M. Lefsky,et al.  Urban forest biomass estimates: is it important to use allometric relationships developed specifically for urban trees? , 2009, Urban Ecosystems.

[44]  Xiao Cheng,et al.  Improving Measurement of Forest Structural Parameters by Co-Registering of High Resolution Aerial Imagery and Low Density LiDAR Data , 2009, Sensors.

[45]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[46]  David J. Hawkes,et al.  Voxel similarity measures for 3-D serial MR brain image registration , 1999, IEEE Transactions on Medical Imaging.

[47]  Jacob T. Mundt,et al.  Mapping Sagebrush Distribution Using Fusion of Hyperspectral and Lidar Classifications , 2006 .

[48]  David J. Harding,et al.  A portable LIDAR system for rapid determination of forest canopy structure , 2004 .

[49]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[50]  Richard Szeliski,et al.  Computer Vision , 2010 .

[51]  M. Pierrot-Deseilligny,et al.  A Photogrammetric Workflow for the Creation of a Forest Canopy Height Model from Small Unmanned Aerial System Imagery , 2013 .

[52]  M. Friedl,et al.  Tracking forest phenology and seasonal physiology using digital repeat photography: a critical assessment. , 2014, Ecological applications : a publication of the Ecological Society of America.

[53]  Peter Vitousek,et al.  Landscape-level variation in forest structure and biogeochemistry across a substrate age gradient in Hawaii. , 2009, Ecology.

[54]  Justin Morgenroth,et al.  Assessment of tree structure using a 3D image analysis technique—A proof of concept , 2014 .