DecSolNet: A noise resistant missing information recovery framework for daily satellite NO2 columns

Abstract A novel statistical method (hereafter referred to as DecSolNet) for reconstructing satellite NO2 columns is introduced. The method has been developed and evaluated by comparing its performance with four benchmark models in three scenarios. When the amount of satellite data is limited, DecSolNet outperforms the benchmark models and its performance does not degrade with noisy inputs. The implementation of DecSolNet consists of: (1) feature extraction, sequential data decomposition in both temporal and frequency domains; (2) NO2 columns reconstruction by training a deep neural network. In three cross-validations, the averaged R 2 score of DecSolNet reaches 0.9, which is better than that of the most benchmark models. The multi-layer perceptron (MLP) has a higher R 2 score, but it degrades greatly with noisy inputs, while the performance of DecSolNet remains robust with an R 2 of ~ 0.8. The bias of DecSolNet is small with an average of 1.61 μ g / m 3 . In addition, DecSolNet is a reliable learning machine, the averaged loss and standard deviation are 0.42 μ g / m 3 and 0.04 μ g / m 3 , respectively.

[1]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[2]  W. Verhoef,et al.  Reconstructing cloudfree NDVI composites using Fourier analysis of time series , 2000 .

[3]  Liangpei Zhang,et al.  Recovering Reflectance of AQUA MODIS Band 6 Based on Within-Class Local Fitting , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[4]  I. D. Smedt,et al.  Validation of OMI, GOME-2A and GOME-2B tropospheric NO 2 , SO 2 and HCHO products using MAX-DOAS observations from 2011 to 2014 in Wuxi, China: investigation of the effects of priori profiles and aerosols on the satellite products , 2017 .

[5]  William J. Blackwell,et al.  Neural Network Applications in High-Resolution Atmospheric Remote Sensing , 2005 .

[6]  R. Spurr,et al.  Volcanic SO2 plume height retrieval from UV sensors using a full-physics inverse learning machine algorithm , 2017 .

[7]  Chao Zeng,et al.  Missing Data Reconstruction in Remote Sensing Image With a Unified Spatial–Temporal–Spectral Deep Convolutional Neural Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[8]  S. Correa A Review of NOx Formation Under Gas-Turbine Combustion Conditions , 1993 .

[9]  J. Thepaut,et al.  The ERA5 global reanalysis , 2020, Quarterly Journal of the Royal Meteorological Society.

[10]  K. F. Boersma,et al.  Near-real time retrieval of tropospheric NO 2 from OMI , 2006 .

[11]  Nianzeng Che,et al.  A new method for retrieving band 6 of aqua MODIS , 2006, IEEE Geoscience and Remote Sensing Letters.

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  Nikos Komodakis,et al.  Image Completion Using Global Optimization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[14]  Chao Zeng,et al.  Recovering missing pixels for Landsat ETM + SLC-off imagery using multi-temporal regression analysis and a regularization method , 2013 .

[15]  M. Gardner,et al.  Neural network modelling and prediction of hourly NOx and NO2 concentrations in urban air in London , 1999 .

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Jin Chen,et al.  A simple method for reconstructing a high-quality NDVI time-series data set based on the Savitzky-Golay filter , 2004 .

[18]  Jicai Ning,et al.  Artificial neural network model for ozone concentration estimation and Monte Carlo analysis , 2018, Atmospheric Environment.

[19]  D. Loyola,et al.  Spatially and temporally coherent reconstruction of tropospheric NO2 over China combining OMI and GOME-2B measurements , 2020, Environmental Research Letters.

[20]  Hao He,et al.  A Changing-Weight Filter Method for Reconstructing a High-Quality NDVI Time Series to Preserve the Integrity of Vegetation Phenology , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Lei Zhang,et al.  An edge-guided image interpolation algorithm via directional filtering and data fusion , 2006, IEEE Transactions on Image Processing.

[22]  S. Howard,et al.  An Evaluation of Gap-Filled Landsat SLC-Off Imagery for Wildland Fire Burn Severity Mapping , 2004 .

[23]  Martin Casdagli,et al.  Nonlinear prediction of chaotic time series , 1989 .

[24]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[25]  Liang Li,et al.  Nonlinear adaptive prediction of nonstationary signals , 1995, IEEE Trans. Signal Process..

[26]  Xiaoliang Lu,et al.  Removal of Noise by Wavelet Method to Generate High Quality Temporal Data of Terrestrial MODIS Products , 2007 .

[27]  F. Veroustraete,et al.  Reconstructing pathfinder AVHRR land NDVI time-series data for the Northwest of China , 2006 .

[28]  Petros Koutrakis,et al.  Daily ambient NO2 concentration predictions using satellite ozone monitoring instrument NO2 data and land use regression. , 2014, Environmental science & technology.

[29]  S. S. Shen,et al.  Applications of Hilbert–Huang transform to non‐stationary financial time series analysis , 2003 .

[30]  Qi Ying,et al.  Investigation of relationships between meteorological conditions and high PM10 pollution in a megacity in the western Yangtze River Delta, China , 2017, Air Quality, Atmosphere & Health.

[31]  Farid Melgani,et al.  Contextual reconstruction of cloud-contaminated multitemporal multispectral images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Chao-Hung Lin,et al.  Cloud Removal From Multitemporal Satellite Images Using Information Cloning , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[33]  I. Uno,et al.  An evaluation of the CMAQ reproducibility of satellite tropospheric NO 2 column observations at different local times over East Asia , 2013 .

[34]  E. Wiegant Improving satellite-based estimations of UV index and dose and first assessment of UV in a world-avoided , 2016 .

[35]  Tianshu Zhang,et al.  A synchronous observation of enhanced aerosol and NO2 over Beijing, China, in winter 2015. , 2017, The Science of the total environment.

[36]  Jos van Geffen,et al.  TEMIS UV product validation using NILU-UV ground-based measurements in Thessaloniki, Greece , 2017 .

[37]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[38]  Jordi Inglada,et al.  Land-cover maps from partially cloudy multi-temporal image series: Optimal temporal sampling and cloud removal , 2010, 2010 IEEE International Geoscience and Remote Sensing Symposium.

[39]  Norden E. Huang,et al.  A review on Hilbert‐Huang transform: Method and its applications to geophysical studies , 2008 .

[40]  N. Watkins,et al.  Seasonal and diurnal analysis of NO2 concentrations from a long-duration study conducted in Las Vegas, Nevada , 2013, Journal of the Air & Waste Management Association.

[41]  Jian Xu,et al.  Applying FP_ILM to the retrieval of geometry-dependent effective Lambertian equivalent reflectivity (GE_LER) daily maps from UVN satellite measurements , 2020 .

[42]  Gang Yang,et al.  Recovering Quantitative Remote Sensing Products Contaminated by Thick Clouds and Shadows Using Multitemporal Dictionary Learning , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Shanshan Wang,et al.  Long-distance mobile MAX-DOAS observations of NO2 and SO2 over the North China Plain and identification of regional transport and power plant emissions , 2020 .

[44]  Steffen Beirle,et al.  MAX-DOAS measurements and satellite validation of tropospheric NO2 and SO2 vertical column densities at a rural site of North China , 2016 .

[45]  Hongliang Zhang,et al.  Investigation of relationships between meteorological conditions and high PM 10 pollution in a megacity in the western Yangtze River Delta , China , 2017 .

[46]  Liangpei Zhang,et al.  Dead Pixel Completion of Aqua MODIS Band 6 Using a Robust M-Estimator Multiregression , 2014, IEEE Geoscience and Remote Sensing Letters.

[47]  Adrian Doicu,et al.  A Novel Ozone Profile Shape Retrieval Using Full-Physics Inverse Learning Machine (FP-ILM) , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[48]  Jun Zhang,et al.  Missing Data and Regression Models for Spatial Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[49]  H. Hersbach,et al.  The ERA5 Atmospheric Reanalysis. , 2016 .

[50]  H. Eskes,et al.  Surface UV radiation monitoring based on gome and sciamachy , 2005 .

[51]  Farid Melgani,et al.  Missing-Area Reconstruction in Multispectral Images Under a Compressive Sensing Perspective , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Jefersson Alex dos Santos,et al.  Towards better exploiting convolutional neural networks for remote sensing scene classification , 2016, Pattern Recognit..

[53]  Jun Zhang,et al.  Functional Concurrent Linear Regression Model for Spatial Images , 2011 .

[54]  R. C. Robbins,et al.  Gaseous nitrogen compound pollutants from urban and natural sources , 1970 .

[55]  Mahdi H. Al-Badrawi,et al.  An Optimized De-Noising Scheme Based on the Null Hypothesis of Intrinsic Mode Functions , 2019, IEEE Signal Processing Letters.

[56]  R. Spurr,et al.  SO2 Layer Height retrieval from Sentinel-5 Precursor/TROPOMI using FP_ILM , 2019 .

[57]  Ulrich Platt,et al.  Differential optical absorption spectroscopy (DOAS) , 1994 .

[58]  Liangpei Zhang,et al.  Cloud removal for remotely sensed images by similar pixel replacement guided with a spatio-temporal MRF model , 2014 .

[59]  Reinhard Beer,et al.  Earth Observing System missions benefit atmospheric research , 2004 .

[60]  Chao-Hung Lin,et al.  Patch-Based Information Reconstruction of Cloud-Contaminated Multitemporal Images , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[61]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[62]  Bin Chen,et al.  Stable classification with limited sample: transferring a 30-m resolution sample set collected in 2015 to mapping 10-m resolution global land cover in 2017. , 2019, Science bulletin.

[63]  P. Beck,et al.  Improved monitoring of vegetation dynamics at very high latitudes: A new method using MODIS NDVI , 2006 .

[64]  H. Budzinski,et al.  Relative rate constants for the heterogeneous reactions of NO2 and OH radicals with polycyclic aromatic hydrocarbons adsorbed on carbonaceous particles. Part 2: PAHs adsorbed on diesel particulate exhaust SRM 1650a , 2006 .

[65]  Piet Stammes,et al.  An improved retrieval of tropospheric NO₂columns from the Ozone Monitoring Instrument , 2010 .

[66]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[67]  Gang Yang,et al.  Missing Information Reconstruction of Remote Sensing Data: A Technical Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[68]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[69]  José A. Sobrino,et al.  Comparison of cloud-reconstruction methods for time series of composite NDVI data , 2010 .

[70]  Jian Xu,et al.  Neural network aided fast pointing information determination approach for occultation payloads from in-flight measurements: Algorithm design and assessment , 2019, Advances in Space Research.

[71]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[72]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[73]  B. Holben Characteristics of maximum-value composite images from temporal AVHRR data , 1986 .

[74]  Luis Guanter,et al.  Nonlinear Statistical Retrieval of Atmospheric Profiles From MetOp-IASI and MTG-IRS Infrared Sounding Data , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[75]  Adam B. Milstein,et al.  Neural network temperature and moisture retrieval algorithm validation for AIRS/AMSU and CrIS/ATMS , 2015 .

[76]  R. D'Agostino An omnibus test of normality for moderate and large size samples , 1971 .

[77]  F. Molteni,et al.  The ECMWF Ensemble Prediction System: Methodology and validation , 1996 .

[78]  Huanfeng Shen,et al.  Spatially continuous mapping of daily global ozone distribution (2004–2014) with the Aura OMI sensor , 2016 .

[79]  A. Lapedes,et al.  Nonlinear signal processing using neural networks: Prediction and system modelling , 1987 .

[80]  K. F. Boersma,et al.  Validation of OMI tropospheric NO2 observations during INTEX-B and application to constrain NOx emissions over the eastern United States and Mexico , 2008 .

[81]  J. Seinfeld,et al.  Atmospheric Chemistry and Physics: From Air Pollution to Climate Change , 1997 .

[82]  Jian Sun,et al.  Image Completion Approaches Using the Statistics of Similar Patches , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[83]  Per Jönsson,et al.  Seasonality extraction by function fitting to time-series of satellite sensor data , 2002, IEEE Trans. Geosci. Remote. Sens..

[84]  Yuqi Bai,et al.  Mapping essential urban land use categories in China (EULUC-China): preliminary results for 2018. , 2020, Science bulletin.

[85]  Wataru Takeuchi,et al.  Restoration of Aqua MODIS Band 6 Using Histogram Matching and Local Least Squares Fitting , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[86]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[87]  Lei Huang,et al.  Development of land use regression models for PM2.5, SO2, NO2 and O3 in Nanjing, China , 2017, Environmental research.