Artifact Detection Maps Learned using Shallow Convolutional Networks

Automatically identifying the locations and severities of video artifacts is a difficult problem. We have developed a general method for detecting local artifacts by learning differences between distorted and pristine video frames. Our model, which we call the Video Impairment Mapper (VID-MAP), produces a full resolution map of artifact detection probabilities based on comparisons of exitatory and inhibatory convolutional responses. Validation on a large database shows that our method outperforms the previous state-of-the-art. A software release of VID-MAP that was trained to produce upscaling and combing detection probability maps is available online: http://live.ece.utexas.edu/research/quality/VIDMAP release.zip for public use and evaluation.

[1]  Heung-Kyu Lee,et al.  Detection of Upscale-Crop and Partial Manipulation in Surveillance Video Based on Sensor Pattern Noise , 2013, Sensors.

[2]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  David Vazquez-Padin,et al.  An SVD approach to forensic image resampling detection , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[4]  Alan C. Bovik,et al.  Automatic Prediction of Perceptual Image and Video Quality , 2013, Proceedings of the IEEE.

[5]  Andrew C. Gallagher.  Detection of linear and cubic interpolation in JPEG compressed images , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[6]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[7]  Alan C. Bovik,et al.  Visual quality assessment algorithms: what does the future hold? , 2010, Multimedia Tools and Applications.

[8]  D. Ruderman.  The statistics of natural images , 1994 .

[9]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[10]  Matthias Kirchner,et al.  Spectral methods to determine the exact scaling factor of resampled digital images , 2012, 2012 5th International Symposium on Communications, Control and Signal Processing.

[11]  Alan C. Bovik,et al.  Feature maps driven no-reference image quality prediction of authentically distorted images , 2015, Electronic Imaging.

[12]  Xinbo Gao,et al.  Image scaling factor estimation based on normalized energy density and learning to rank , 2014, Proceedings 2014 IEEE International Conference on Security, Pattern Analysis, and Cybernetics (SPAC).

[13]  David M. Baylon.  On the Detection of Temporal Field Order in Interlaced Video Data , 2007, 2007 IEEE International Conference on Image Processing.

[14]  Babak Mahdian,et al.  Ieee Transactions on Information Forensics and Security 1 Blind Authentication Using Periodic Properties of Interpolation , 2022 .

[15]  Yong Jae Lee,et al.  Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-Supervised Object and Action Localization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Heung-Kyu Lee,et al.  Estimation of linear transformation by analyzing the periodicity of interpolation , 2014, Pattern Recognit. Lett..

[17]  Dragomir Anguelov,et al.  Self-taught object localization with deep networks , 2014, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[18]  Hany Farid,et al.  Exposing digital forgeries by detecting traces of resampling , 2005 .

[19]  Alan C. Bovik,et al.  Blind Picture Upscaling Ratio Prediction , 2016, IEEE Signal Processing Letters.

[20]  Christophe Charrier,et al.  Blind Prediction of Natural Video Quality , 2014, IEEE Transactions on Image Processing.

[21]  Ingemar J. Cox,et al.  Normalized Energy Density-Based Forensic Detection of Resampled Images , 2012, IEEE Transactions on Multimedia.

[22]  Michal Irani,et al.  Detecting Irregularities in Images and in Video , 2005, ICCV.

[23]  Matthias Kirchner,et al.  Fast and reliable resampling detection by spectral analysis of fixed linear predictor residue , 2008, MM&Sec '08.

[24]  David Vazquez-Padin,et al.  Prefilter design for forensic resampling estimation , 2011, 2011 IEEE International Workshop on Information Forensics and Security.

[25]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[26]  S. Prasad,et al.  On Resampling Detection and its Application to Detect Image Tampering , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[27]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[28]  Anne Aaron,et al.  Native Resolution Detection of Video Sequences , 2015 .