Efficient embedded context-based surveillance image and video analysis

• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers.

[1]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Paulo Mateus,et al.  On the complexity of minimizing probabilistic and quantum automata , 2012, Inf. Comput..

[3]  Mike McHenry,et al.  Detecting water hazards for autonomous off-road navigation , 2003, SPIE Defense + Commercial Sensing.

[4]  Josiane Zerubia,et al.  Texture feature analysis using a gauss-Markov model in hyperspectral image classification , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[5]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[6]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Patrick Haffner,et al.  Support vector machines for histogram-based image classification , 1999, IEEE Trans. Neural Networks.

[8]  D. Sagi,et al.  Gabor filters as texture discriminator , 1989, Biological Cybernetics.

[9]  Daphne Koller,et al.  Support Vector Machine Active Learning with Applications to Text Classification , 2000, J. Mach. Learn. Res..

[10]  Zhiyu Xiang,et al.  Robust water hazard detection for autonomous off-road navigation , 2009 .

[11]  Peter H. N. de With,et al.  Flexible Multi-modal Graph-Based Segmentation , 2013, ACIVS.

[12]  Lihi Zelnik-Manor,et al.  Context-aware saliency detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  R. Manmatha,et al.  IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops: Preface , 2003, CVPR 2003.

[14]  Sabine Süsstrunk,et al.  Salient Region Detection and Segmentation , 2008, ICVS.

[15]  Joni-Kristian Kämäräinen,et al.  Fundamental frequency Gabor filters for object recognition , 2002, Object recognition supported by user interaction for service robots.

[16]  Svitlana Zinger,et al.  Fast sky and road detection for video context analysis , 2012 .

[17]  P. Nagabhushan,et al.  A simple and robust line detection algorithm based on small eigenvalue analysis , 2004, Pattern Recognit. Lett..

[18]  Jake K. Aggarwal,et al.  Robust Vehicle Detection for Tracking in Highway Surveillance Videos Using Unsupervised Learning , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[19]  Hideyuki Tamura,et al.  Textural Features Corresponding to Visual Perception , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[20]  Adel M. Alimi,et al.  Event Detection from Video Surveillance Data Based on Optical Flow Histogram and High-level Feature Extraction , 2009, 2009 20th International Workshop on Database and Expert Systems Application.

[21]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[22]  Svitlana Zinger,et al.  Moving ship detection based on context modeling and motion analysis , 2014 .

[23]  Shih-Fu Chang,et al.  Single color extraction and image query , 1995, Proceedings., International Conference on Image Processing.

[24]  Zi-Quan Hong,et al.  Algebraic feature extraction of image for recognition , 1991, Pattern Recognit..

[25]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[26]  A. Broggi,et al.  Lateral vehicles detection using monocular high resolution cameras on TerraMax™ , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[27]  Zhongde Wang.  Fast algorithms for the discrete W transform and for the discrete Fourier transform , 1984 .

[28]  Oswald Lanz,et al.  Approximate Bayesian multibody tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  P. Yip,et al.  Discrete Cosine Transform: Algorithms, Advantages, Applications , 1990 .

[31]  Vincent Charvillat,et al.  Context modeling in computer vision: techniques, implications, and applications , 2010, Multimedia Tools and Applications.

[32]  Rafael C. González,et al.  Digital image processing using MATLAB , 2006 .

[33]  Retantyo Wardoyo,et al.  Time Complexity Analysis of Support Vector Machines (SVM) in LibSVM , 2015 .

[34]  Rob G. J. Wijnhoven,et al.  Online learning for ship detection in maritime surveillance , 2010 .

[35]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[36]  Lutz Priese,et al.  Sky Detection in CSC-segmented Color Images , 2009, VISAPP.

[37]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Svitlana Zinger,et al.  Automatic generic Region-Of-Interest selection for video surveillance applications , 2014 .

[39]  Peter H. N. de With,et al.  Robust classification system with reliability prediction for semi-automatic traffic-sign inventory systems , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[40]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Peyman Milanfar,et al.  Nonparametric bottom-up saliency detection by self-resemblance , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[42]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[43]  Thomas Mauthner,et al.  Semantic Image Classification using Consistent Regions and Individual Context , 2009, BMVC.

[44]  R. Maini.  Study and Comparison of Various Image Edge Detection Techniques , 2004 .

[45]  Amanpreet Kaur.  Comparison between YCbCr Color Space and CIELab Color Space for Skin Color Segmentation , 2012 .

[46]  Qu Ying-Dong,et al.  A fast subpixel edge detection method using Sobel-Zernike moments operator , 2005, Image Vis. Comput..

[47]  Liang-Gee Chen,et al.  Survey on Block Matching Motion Estimation Algorithms and Architectures with New Results , 2006, J. VLSI Signal Process..

[48]  Yunsong Guo,et al.  Comparisons of sequence labeling algorithms and extensions , 2007, ICML '07.

[49]  Martin D. Levine,et al.  Visual Saliency Based on Scale-Space Analysis in the Frequency Domain , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  .. K. Kanagalakshmi,et al.  Frequency Domain Enhancement Algorithm Based on Log -Gabor Filter in FFT Domain , 2012 .

[51]  S. Herman,et al.  Locally-adaptive processing of television images based on real-time image segmentation , 2002, 2002 Digest of Technical Papers. International Conference on Consumer Electronics (IEEE Cat. No.02CH37300).

[52]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[53]  Fons van der Sommen,et al.  Real-time semantic context labeling for image understanding , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[54]  Laurence Likforman-Sulem,et al.  A Hough based algorithm for extracting text lines in handwritten documents , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[55]  Peter H. N. de With,et al.  Fast semantic region analysis for surveillance & video databases , 2017, 2017 IEEE International Conference on Consumer Electronics (ICCE).

[56]  Cheng Li,et al.  Detection and Recognition of Road Markings in Panoramic Images , 2014, ACCV Workshops.

[57]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[58]  Martin Jägersand,et al.  Saliency Maps and Attention Selection in Scale and Spatial Coordinates: An Information Theoretic Approach , 1995, ICCV.

[59]  Jitendra Malik,et al.  Recognizing action at a distance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[60]  Allam S. Hassanein,et al.  A Survey on Hough Transform, Theory, Techniques and Applications , 2015, ArXiv.

[61]  Hai-Yan Zhang,et al.  Multiple moving objects detection and tracking based on optical flow in polar-log images , 2010, 2010 International Conference on Machine Learning and Cybernetics.

[62]  Joachim Weickert,et al.  Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods , 2005, International Journal of Computer Vision.

[63]  A.H.R. Albers,et al.  Modeling and control of image processing for interventional X-ray , 2007 .

[64]  Gijs Dubbelman,et al.  Training of Convolutional Networks on Multiple Heterogeneous Datasets for Street Scene Semantic Segmentation , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[65]  Serge J. Belongie,et al.  Context based object categorization: A critical survey , 2010, Comput. Vis. Image Underst..

[66]  Simon C. K. Shiu,et al.  Monogenic Binary Coding: An Efficient Local Feature Extraction Approach to Face Recognition , 2012, IEEE Transactions on Information Forensics and Security.

[67]  Esa Rahtu,et al.  Segmenting Salient Objects from Images and Videos , 2010, ECCV.

[68]  Mubarak Shah,et al.  Visual attention detection in video sequences using spatiotemporal cues , 2006, MM '06.

[69]  Jong-Nam Kim,et al.  Multiple Ship Detection and Tracking Using Background Registration and Morphological Operations , 2010, FGIT-SIP/MulGraB.

[70]  Pooja Kamavisdar,et al.  A Survey on Image Classification Approaches and Techniques , 2013 .

[71]  Guangming Xiong,et al.  Road detection using support vector machine based on online learning and evaluation , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[72]  Ivo M. Creusen,et al.  Automatic recognition system for surveying of traffic signs and road markings from street-level panoramic images , 2016 .

[73]  S. Zinger,et al.  Context-based region labeling for event detection in surveillance video , 2014, 2014 International Conference on Information Science, Electronics and Electrical Engineering.

[74]  Yuan-Kai Wang,et al.  A robust vehicle detection approach , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[75]  Larry S. Davis,et al.  Learning What and How of Contextual Models for Scene Labeling , 2010, ECCV.

[76]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[77]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[78]  Michael Brady,et al.  Saliency, Scale and Image Description , 2001, International Journal of Computer Vision.

[79]  Irfan A. Essa,et al.  Expectation grammars: leveraging high-level expectations for activity recognition , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[80]  Mingjing Li,et al.  Color texture moments for content-based image retrieval , 2002, Proceedings. International Conference on Image Processing.

[81]  Fuhui Long,et al.  Fundamentals of Content-Based Image Retrieval , 2003 .

[82]  Gabriel Cristóbal,et al.  Self-Invertible 2D Log-Gabor Wavelets , 2007, International Journal of Computer Vision.

[83]  Anton van den Hengel,et al.  Wider or Deeper: Revisiting the ResNet Model for Visual Recognition , 2016, Pattern Recognit..

[84]  Wei Wang,et al.  Design and implementation of Log-Gabor filter in fingerprint image enhancement , 2008, Pattern Recognit. Lett..

[85]  Peter H. N. de With,et al.  Context-based object-of-interest detection for a generic traffic surveillance analysis system , 2014, 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[86]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[87]  Svitlana Zinger,et al.  Adding context information to video analysis for surveillance applications , 2015 .

[88]  Isabelle Bloch,et al.  Using relative spatial relationships to improve individual region recognition , 2005 .

[89]  Svitlana Zinger,et al.  Context analysis : sky, water and motion , 2011 .

[90]  Neil A. Thacker,et al.  Tutorial: Computing 2D and 3D Optical Flow. , 2004 .

[91]  Yali Amit,et al.  Shape Quantization and Recognition with Randomized Trees , 1997, Neural Computation.

[92]  D. Geman,et al.  Randomized Inquiries About Shape: An Application to Handwritten Digit Recognition. , 1994 .

[93]  Jörg Arndt.  The Haar transform , 2011 .

[94]  M. Strintzis,et al.  COMPRESSED-DOMAIN OBJECT DETECTION FOR VIDEO UNDERSTANDING , 2004 .

[95]  Mateus Beck Fonseca,et al.  Design of pipelined butterflies from Radix-2 FFT with Decimation in Time algorithm using efficient adder compressors , 2011, 2011 IEEE Second Latin American Symposium on Circuits and Systems (LASCAS).

[96]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[97]  Larry Matthies,et al.  Daytime water detection based on color variation , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[98]  Bahman Zafarifar.  Adaptive modeling of sky for video processing and coding applications , 2006 .

[99]  Alexei A. Efros,et al.  An empirical study of context in object detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[100]  Peter H. N. de With,et al.  Fast Training of Object Detection Using Stochastic Gradient Descent , 2010, 2010 20th International Conference on Pattern Recognition.

[101]  David Zhang,et al.  Palmprint feature extraction using 2-D Gabor filters , 2003, Pattern Recognit..

[102]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[103]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[104]  Rama Chellappa,et al.  View Invariance for Human Action Recognition , 2005, International Journal of Computer Vision.

[105]  Peter H. N. de With,et al.  Ship detection in port surveillance based on context and motion saliency analysis , 2013, Electronic Imaging.

[106]  Vittorio Murino,et al.  Decentralized particle filter for joint individual-group tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[107]  Csaba Benedek,et al.  Study on color space selection for detecting cast shadows in video surveillance , 2007, Int. J. Imaging Syst. Technol..

[108]  Alexei A. Efros,et al.  Recovering Surface Layout from an Image , 2007, International Journal of Computer Vision.

[109]  Peter H. N. de With,et al.  Context modeling combined with motion analysis for moving ship detection in port surveillance , 2013, J. Electronic Imaging.

[110]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[111]  Peter H. N. de With,et al.  Fast scene analysis for surveillance & video databases , 2017, IEEE Transactions on Consumer Electronics.

[112]  Svitlana Zinger,et al.  Fast abnormal event detection from video surveillance , 2012 .

[113]  Jungho Im,et al.  ISPRS Journal of Photogrammetry and Remote Sensing , 2022 .

[114]  Anil K. Jain,et al.  Image retrieval using color and shape , 1996, Pattern Recognit..

[115]  Thomas B. Moeslund,et al.  Real-time Multiple Abnormality Detection in Video Data , 2013, VISAPP.

[116]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[117]  F. Bremond,et al.  Crowd event recognition using HOG tracker , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[118]  Md. Monirul Islam,et al.  A geometric method to compute directionality features for texture images , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[119]  P. Varshney,et al.  Multisource Classification Using Support Vector Machines: An Empirical Comparison with Decision Tree and Neural Network Classifiers , 2008 .

[120]  Rgj Rob Wijnhoven,et al.  Object categorization and detection and their application in surveillance , 2013 .

[121]  Sos S. Agaian,et al.  Transform Coefficient Histogram-Based Image Enhancement Algorithms Using Contrast Entropy , 2007, IEEE Transactions on Image Processing.

[122]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[123]  P.H.N. de With,et al.  Blue Sky Detection for Content-based Television Picture Quality Enhancement , 2007, 2007 Digest of Technical Papers International Conference on Consumer Electronics.

[124]  Peter H. N. de With,et al.  Water Region Detection Supporting Ship Identification in Port Surveillance , 2012, ACIVS.

[125]  Antonio Criminisi,et al.  TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context , 2007, International Journal of Computer Vision.

[126]  Homayoun Mahdavi-Nasab,et al.  Optical Flow Based Moving Object Detection and Tracking for Traffic Surveillance , 2013 .

[127]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[128]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[129]  Yue Zhang,et al.  A New Feature of Uniformity of Image Texture Directions Coinciding with the Human Eyes Perception , 2005, FSKD.

[130]  Pushmeet Kohli,et al.  Graph Cut Based Inference with Co-occurrence Statistics , 2010, ECCV.

[131]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[132]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[133]  Yong Zhou,et al.  A robust lane detection and tracking method based on computer vision , 2006 .

[134]  Joost van de Weijer,et al.  Fusing Global and Local Scale for Semantic Image Segmentation , 2011 .

[135]  Giorgio Gambosi,et al.  Complexity and approximation: combinatorial optimization problems and their approximability properties , 1999 .

[136]  Henri Maître,et al.  Entropy and multiscale analysis: a new feature extraction algorithm for aerial images , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[137]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[138]  Manfred H. Hueckel.  An Operator Which Locates Edges in Digitized Pictures , 1971, J. ACM.

[139]  Y. Zhang,et al.  A REVIEW ON IMAGE SEGMENTATION TECHNIQUES WITH REMOTE SENSING PERSPECTIVE , 2010 .

[140]  Chabane Djeraba,et al.  Real-time crowd motion analysis , 2008, 2008 19th International Conference on Pattern Recognition.

[141]  Ramin Zabih,et al.  Histogram refinement for content-based image retrieval , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[142]  Yo-Ping Huang,et al.  Image Retrieval Based on Dominant Texture Features , 2006, 2006 IEEE International Symposium on Industrial Electronics.

[143]  Nuno Vasconcelos,et al.  Analysis of Crowded Scenes using Holistic Properties , 2009 .

[144]  Svitlana Zinger,et al.  Region labeling for surveillance video : techniques and application , 2014 .

[145]  Md. Monirul Islam,et al.  A review on automatic image annotation techniques , 2012, Pattern Recognit..

[146]  Tony Lindeberg,et al.  Discrete derivative approximations with scale-space properties: A basis for low-level feature extraction , 1993, Journal of Mathematical Imaging and Vision.

[147]  Kuo-Chin Fan,et al.  Lane detection using directional random walks , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[148]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[149]  Peter H. N. de With,et al.  ViCoMo: visual context modeling for scene understanding in video surveillance , 2013, J. Electronic Imaging.

[150]  Antonio Criminisi,et al.  Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning , 2012, Found. Trends Comput. Graph. Vis..

[151]  Jiebo Luo,et al.  Improved blue sky detection using polynomial model fit , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[152]  Shane Torbert,et al.  Applied Computer Science , 2012, Springer New York.

[153]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.