Multi-view object-based classification of wetland land covers using unmanned aircraft system images

Abstract Traditionally, the multiple images collected by cameras mounted on Unmanned Aircraft Systems (UAS) are mosaicked into a single orthophoto on which Object-Based Image Analysis (OBIA) is conducted. This approach does not take advantage of the Multi-View (MV) information of the individual images. In this study, we introduce a new OBIA approach utilizing multi-view information of original UAS images and compare its performance with that of traditional OBIA, which uses only the orthophoto (Ortho-OBIA). The proposed approach, called multi-view object-based image analysis (MV-OBIA), classifies multi-view object instances on UAS images corresponding to each orthophoto object and utilizes a voting procedure to assign a final label to the orthophoto object. The proposed MV-OBIA is also compared with the classification approaches based on Bidirectional Reflectance Distribution Function (BRDF) simulation. Finally, to reduce the computational burden of multi-view object-based data generation for MV-OBIA and make the proposed approach more operational in practice, this study proposes two window-based implementations of MV-OBIA that utilize a window positioned at the geometric centroid of the object instance, instead of the object instance itself, to extract features. The first window-based MV-OBIA adopts a fixed window size (denoted as FWMV-OBIA), while the second window-based MV-OBIA uses an adaptive window size (denoted as AWMV-OBIA). Our results show that the MV-OBIA substantially improves the overall accuracy compared with Ortho-OBIA, regardless of the features used for classification and types of wetland land covers in our study site. Furthermore, the MV-OBIA also demonstrates a much higher efficiency in utilizing the multi-view information for classification based on its considerably higher overall accuracy compared with BRDF-based methods. Lastly, FWMV-OBIA and AWMV-OBIA both show potential in generating an equal if not higher overall accuracy compared with MV-OBIA at substantially reduced computational costs.

[1]  F. R. Schiebe,et al.  Canopy attributes of desert grassland and transition communities derived from multiangular airborne imagery , 2003 .

[2]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[3]  Tao Liu,et al.  Object-based classification of wetland vegetation using very high-resolution unmanned air system imagery , 2017 .

[4]  William J. Emery,et al.  Very High Resolution Multiangle Urban Classification Analysis , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[5]  J. Ross.  The radiation regime and architecture of plant stands , 1981, Tasks for vegetation sciences 3.

[6]  Françoise Viénot,et al.  Wheat leaf bidirectional reflectance measurements: Description and quantification of the volume, specular and hot-spot scattering features , 2012 .

[7]  Jungho Im,et al.  Forest Biomass and Carbon Stock Quantification Using Airborne LiDAR Data: A Case Study Over Huntington Wildlife Forest in the Adirondack Park , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Geoffrey A. Hollinger,et al.  Aerial Vehicle Path Planning for Monitoring Wildfire Frontiers , 2015, FSR.

[9]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[10]  A. Rango,et al.  Image Processing and Classification Procedures for Analysis of Sub-decimeter Imagery Acquired with an Unmanned Aircraft over Arid Rangelands , 2011 .

[11]  Neha Gupta,et al.  Object based Information Extraction from High Resolution Satellite Imagery using eCognition , 2014 .

[12]  A. Strahler,et al.  Classification of ASAS multiangle and multispectral measurements using artificial neural networks , 1996 .

[13]  Jungho Im,et al.  Forest biomass estimation from airborne LiDAR data using machine learning approaches , 2012 .

[14]  Lindi J. Quackenbush,et al.  Optimum Scale in Object‐Based Image Analysis , 2014 .

[15]  F. López-Granados,et al.  Multi-temporal mapping of the vegetation fraction in early-season wheat fields using images from UAV , 2014 .

[16]  Faith R. Kearns,et al.  Classification of the wildland-urban interface: A comparison of pixel- and object-based classifications using high-resolution aerial photography , 2008, Comput. Environ. Urban Syst..

[17]  P. Wolf,et al.  Elements of Photogrammetry(with Applications in GIS) , 2000 .

[18]  Tae Ho Lee,et al.  A Study on Land Cover Map of UAV Imagery using an Object-based Classification Method , 2015 .

[19]  S. Sandmeier,et al.  Structure Analysis and Classification of Boreal Forests Using Airborne Hyperspectral Brdf Data from Asas Imagery and Processing Techniques Have Also Been Used Potential for Combining Both High Spectral Resolution And , 2022 .

[20]  Elizabeth H. Boughton,et al.  Trade-Offs Among Ecosystem Services and Disservices on a Florida Ranch , 2013 .

[21]  Dongmei Chen,et al.  Change detection from remotely sensed images: From pixel-based to object-based approaches , 2013 .

[22]  Bing Zhang,et al.  Measurement and Analysis of Bidirectional SIF Emissions in Wheat Canopies , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Taro Suzuki,et al.  Generation of Large Mosaic Images for Vegetation Monitoring Using a Small Unmanned Aerial Vehicle , 2010, J. Robotics Mechatronics.

[24]  Changchun Li,et al.  The Application of Unmanned Aerial Vehicle Remote Sensing in Quickly Monitoring Crop Pests , 2012, Intell. Autom. Soft Comput..

[25]  J. Townshend,et al.  Bidirectional effects in Landsat reflectance estimates: Is there a problem to solve? , 2015 .

[26]  Fabio Del Frate,et al.  Hyperspectral and Multiangle CHRIS–PROBA Images for the Generation of Land Cover Maps , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Alessandro Matese,et al.  A flexible unmanned aerial vehicle for precision agriculture , 2012, Precision Agriculture.

[28]  Patrick R. Harrison,et al.  Learning class descriptions from a data base of spectral reflectance with multiple view angles , 1992, IEEE Trans. Geosci. Remote. Sens..

[29]  Michael D. King,et al.  Airborne spectral BRDF of various surface types (ocean, vegetation, snow, desert, wetlands, cloud decks, smoke layers) for remote sensing applications , 2016 .

[30]  Korehisa Kaneko,et al.  Review of Effective Vegetation Mapping Using the UAV (Unmanned Aerial Vehicle) Method , 2014 .

[31]  Jungho Im,et al.  A novel transferable individual tree crown delineation model based on Fishing Net Dragging and boundary classification , 2015 .

[32]  O. Csillik,et al.  Automated parameterisation for multi-scale image segmentation on multiple layers , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[33]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[34]  Giles M. Foody,et al.  Using mixed objects in the training of object-based image classifications , 2017 .

[35]  Dharma P. Agrawal,et al.  Communication and networking of UAV-based systems: Classification and associated architectures , 2017, J. Netw. Comput. Appl..

[36]  Bai Zhang,et al.  Comparison of object-based and pixel-based Random Forest algorithm for wetland vegetation mapping using high spatial resolution GF-1 and SAR data , 2017 .

[37]  S. Luke Flory,et al.  Cogongrass (Imperata cylindrica) invasions in the US: Mechanisms, impacts, and threats to biodiversity , 2015 .

[38]  Tim M. O'Brien.  Small Unmanned Aerial Vehicles as Remote Sensors: An Effective Data Gathering Tool for Wetland Mapping , 2016 .

[39]  C. Atzberger,et al.  Evaluation of semi-empirical BRDF models inverted against multi-angle data from a digital airborne frame camera for enhancing forest type classification , 2014 .

[40]  Tao Liu,et al.  Evaluating the potential of multi-view data extraction from small Unmanned Aerial Systems (UASs) for object-based classification for Wetland land covers , 2018, GIScience & Remote Sensing.

[41]  Paolo Gamba,et al.  Improvements to urban area characterization using multitemporal and multiangle SAR images , 2003, IEEE Trans. Geosci. Remote. Sens..

[42]  Koichi Amaha,et al.  Canopy height measurement by photogrammetric analysis of aerial images: Application to buckwheat (Fagopyrum esculentum Moench) lodging evaluation , 2012 .

[43]  Amr H. Abd-Elrahman,et al.  An Object-Based Image Analysis Method for Enhancing Classification of Land Covers Using Fully Convolutional Networks and Multi-View Images of Small Unmanned Aerial System , 2018, Remote. Sens..

[44]  E. Milton,et al.  The use of the empirical line method to calibrate remotely sensed data to reflectance , 1999 .

[45]  Li Minzan,et al.  Precision management of winter wheat based on aerial images and hyperspectral data obtained by unmanned aircraft , 2005, Proceedings. 2005 IEEE International Geoscience and Remote Sensing Symposium, 2005. IGARSS '05..

[46]  F. J. García-Haro,et al.  Airborne measurement of hot spot reflectance signatures , 2004 .

[47]  H. Rahman,et al.  Coupled surface-atmosphere reflectance (CSAR) model: 2. Semiempirical surface model usable with NOAA advanced very high resolution radiometer data , 1993 .

[48]  David J. Selkowitz,et al.  A comparison of multi-spectral, multi-angular, and multi-temporal remote sensing datasets for fractional shrub canopy mapping in Arctic Alaska , 2010 .

[49]  Juan López,et al.  Architecture for a helicopter-based unmanned aerial systems wildfire surveillance system , 2011 .

[50]  F. M. Roka,et al.  Integrating Ranch Forage Production, Cattle Performance, and Economics in Ranch Management Systems for Southern Florida , 2007 .

[51]  H. Franklin Percival,et al.  Use of Unmanned Aircraft Systems to Delineate Fine-Scale Wetland Vegetation Communities , 2014, Wetlands.

[52]  Thomas Blaschke,et al.  Geographic Object-Based Image Analysis – Towards a new paradigm , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[53]  J. Roujean,et al.  A bidirectional reflectance model of the Earth's surface for the correction of remote sensing data , 1992 .

[54]  Eric C. Brown de Colstoun,et al.  Improving global scale land cover classifications with multi-directional POLDER data and a decision tree classifier , 2006 .

[55]  Michael S. Ross,et al.  Vegetation Classification for South Florida Natural Areas , 2006 .

[56]  Anne W. Nolin,et al.  Classification of glacier zones in western Greenland using albedo and surface roughness from the Multi-angle Imaging SpectroRadiometer (MISR) , 2007 .

[57]  Michael Ol,et al.  Small UAV Research and Evolution in Long Endurance Electric Powered Vehicles , 2007 .

[58]  Debra P. C. Peters,et al.  Support vector machines for recognition of semi-arid vegetation types using MISR multi-angle imagery , 2007 .

[59]  Alan H. Strahler,et al.  Improving MODIS land cover classification by combining MODIS spectral and angular signatures in a Canadian boreal forest , 2011 .

[60]  Jianya Gong,et al.  Angular difference feature extraction for urban scene classification using ZY-3 multi-angle high-resolution satellite imagery , 2018 .

[61]  T. Schmugge,et al.  Research Article: Using Unmanned Aerial Vehicles for Rangelands: Current Applications and Future Potentials , 2006 .

[62]  Yuhong He,et al.  Species classification using Unmanned Aerial Vehicle (UAV)-acquired high spatial resolution imagery in a heterogeneous grassland , 2017 .

[63]  Tao Liu,et al.  Deep convolutional neural network training enrichment using multi-view object-based analysis of Unmanned Aerial systems imagery for wetlands classification , 2018 .

[64]  Tao Liu,et al.  Comparing fully convolutional networks, random forest, support vector machine, and patch-based deep convolutional neural networks for object-based wetland mapping using images from small unmanned aircraft system , 2018 .

[65]  V. N. Sridhar,et al.  Impact of Surface Anisotropy on Classification Accuracy of Selected Vegetation Classes: An Evaluation Using Multidate Multiangular MISR Data Over Parts of Madhya Pradesh, India , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[66]  K. McGwire,et al.  Multiscale assessment of green leaf cover in a semi-arid rangeland with a small unmanned aerial vehicle , 2013 .

[67]  L. G. Holm,et al.  The world's worst weeds. , 1977 .

[68]  Lindi J. Quackenbush,et al.  Impact of training and validation sample selection on classification accuracy and accuracy assessment when using reference polygons in object-based classification , 2013 .

[69]  Philip Moore,et al.  ACCURACY ASSESSMENT OF A UAV-BASED LANDSLIDE MONITORING SYSTEM , 2016 .

[70]  Jorge Delgado,et al.  Analysis of Landslide Evolution Affecting Olive Groves Using UAV and Photogrammetric Techniques , 2016, Remote. Sens..

[71]  V. Salokhe,et al.  Application of low altitude remote sensing (LARS) platform for monitoring crop growth and weed infestation in a soybean plantation , 2012, Precision Agriculture.

[72]  Clement Atzberger,et al.  Potential of Multi-Angular Data Derived From a Digital Aerial Frame Camera for Forest Classification , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[73]  P. Gong,et al.  Object-based Detailed Vegetation Classification with Airborne High Spatial Resolution Remote Sensing Imagery , 2006 .