Unsharp Masking Layer: Injecting Prior Knowledge in Convolutional Networks for Image Classification

Image enhancement refers to the enrichment of certain image features such as edges, boundaries, or contrast. The main objective is to process the original image so that the overall performance of visualization, classification and segmentation tasks is considerably improved. Traditional techniques require manual fine-tuning of the parameters to control enhancement behavior. To date, recent Convolutional Neural Network (CNN) approaches frequently employ the aforementioned techniques as an enriched pre-processing step. In this work, we present the first intrinsic CNN pre-processing layer based on the well-known unsharp masking algorithm. The proposed layer injects prior knowledge about how to enhance the image, by adding high frequency information to the input, to subsequently emphasize meaningful image features. The layer optimizes the unsharp masking parameters during model training, without any manual intervention. We evaluate the network performance and impact on two applications: CIFAR100 image classification, and the PlantCLEF identification challenge. Results obtained show a significant improvement over popular CNNs, yielding 9.49% and 2.42% for PlantCLEF and general-purpose CIFAR100, respectively. The design of an unsharp enhancement layer plainly boosts the accuracy with negligible performance cost on simple CNN models, as prior knowledge is directly injected to improve its robustness.

[1]  Juan Carlos Cruz,et al.  A first glance on the enhancement of digital cell activity videos from glioblastoma cells with nuclear staining , 2016, 2016 IEEE 36th Central American and Panama Convention (CONCAPAN XXXVI).

[2]  Jenny Lee,et al.  Fully Automated Deep Learning System for Bone Age Assessment , 2017, Journal of Digital Imaging.

[3]  Peter V. Gehler,et al.  Learning Sparse High Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[5]  Pierre Soille,et al.  Mathematical Morphology and Its Applications to Image Processing , 1994, Computational Imaging and Vision.

[6]  Arun Kumar Sunaniya,et al.  An adaptive image sharpening scheme based on local intensity variations , 2017, Signal Image Video Process..

[7]  Erick Mata-Montero,et al.  Automated Plant Species Identification: Challenges and Opportunities , 2016, WITFOR.

[8]  Sanjit K. Mitra,et al.  Quadratic filters for image contrast enhancement , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[9]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[10]  Tony F. Chan,et al.  Image processing and analysis - variational, PDE, wavelet, and stochastic methods , 2005 .

[11]  Gabriel B. Paranhos da Costa,et al.  Deep Convolutional Neural Networks and Noisy Images , 2017, CIARP.

[12]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[13]  Anil K. Jain Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[14]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[15]  Giovanni Ramponi,et al.  Image enhancement via adaptive unsharp masking , 2000, IEEE Trans. Image Process..

[16]  Pierre Bonnet,et al.  Plant Identification Based on Noisy Web Data: the Amazing Performance of Deep Learning (LifeCLEF 2017) , 2017, CLEF.

[17]  Blaz Meden,et al.  Assessing the Impact of the Deceived Non Local Means Filter as a Preprocessing Stage in a Convolutional Neural Network Based Approach for Age Estimation Using Digital Hand X-Ray Images , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[18]  T. S. R. Raja,et al.  Impact of applying pre-processing techniques for improving classification accuracy , 2014, Signal Image Video Process..

[19]  Wei Ye,et al.  Blurriness-Guided Unsharp Masking , 2018, IEEE Transactions on Image Processing.

[20]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[21]  João Batista Neto,et al.  An empirical study on the effects of different types of noise in image classification tasks , 2016, ArXiv.

[22]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[23]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[24]  Zohair Al-Ameen,et al.  Sharpness Improvement for Medical Images Using a New Nimble Filter , 2018 .

[25]  Lina J. Karam,et al.  Understanding how image quality affects deep neural networks , 2016, 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX).

[26]  Thomas M. Deserno,et al.  Fundamentals of Biomedical Image Processing , 2010 .

[27]  Peter V. Gehler,et al.  Superpixel Convolutional Networks Using Bilateral Inceptions , 2015, ECCV.

[28]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Moon Gi Kang,et al.  Edge enhancement algorithm for low-dose X-ray fluoroscopic imaging , 2017, Comput. Methods Programs Biomed..

[30]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Andy Wai Kan Yeung,et al.  Review of Radiology: App for Android by the Radiological Society of North America (RSNA) , 2016, Journal of Digital Imaging.

[32]  Sang Won Yoon,et al.  A Dual-Tree Complex Wavelet Transform Based Convolutional Neural Network for Human Thyroid Medical Image Segmentation , 2018, 2018 IEEE International Conference on Healthcare Informatics (ICHI).

[33]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[34]  Jean-Michel Morel,et al.  Neighborhood filters and PDE’s , 2006, Numerische Mathematik.