Real time expert system for anomaly detection of aerators based on computer vision technology and existing surveillance cameras

Aerators are essential and crucial auxiliary devices in intensive culture, especially in industrial culture in China. The traditional methods cannot accurately detect abnormal condition of aerators in time. Surveillance cameras are widely used as visual perception modules of the Internet of Things, and then using these widely existing surveillance cameras to realize real-time anomaly detection of aerators is a cost-free and easy-to-promote method. However, it is difficult to develop such an expert system due to some technical and applied challenges, e.g., illumination, occlusion, complex background, etc. To tackle these aforementioned challenges, we propose a real-time expert system based on computer vision technology and existing surveillance cameras for anomaly detection of aerators, which consists of two modules, i.e., object region detection and working state detection. First, it is difficult to detect the working state for some small object regions in whole images, and the time complexity of global feature comparison is also high, so we present an object region detection method based on the region proposal idea. Moreover, we propose a novel algorithm called reference frame Kanade-Lucas-Tomasi (RF-KLT) algorithm for motion feature extraction in fixed regions. Then, we present a dimension reduction method of time series for establishing a feature dataset with obvious boundaries between classes. Finally, we use machine learning algorithms to build the feature classifier. The experimental results in both the actual video dataset and the augmented video dataset show that the accuracy for detecting object region and working state of aerators is 100% and 99.9% respectively, and the detection speed is 77-333 frames per second (FPS) according to the different types of surveillance cameras.

[1]  Luis Miguel Bergasa,et al.  Expert video-surveillance system for real-time detection of suspicious behaviors in shopping malls , 2015, Expert Syst. Appl..

[2]  P. KaewTrakulPong,et al.  An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection , 2002 .

[3]  Debi Prosad Dogra,et al.  Queuing Theory Guided Intelligent Traffic Scheduling through Video Analysis using Dirichlet Process Mixture Model , 2018, Expert Syst. Appl..

[4]  Yichen Wei,et al.  Deep Feature Flow for Video Recognition , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[6]  Mohammed Anbar,et al.  A Preliminary Performance Evaluation of K-means, KNN and EM Unsupervised Machine Learning Methods for Network Flow Classification , 2016 .

[7]  Arti Khaparde,et al.  Agriculture pest detection using video processing technique , 2015, 2015 International Conference on Information Processing (ICIP).

[8]  Mark W. Schmidt,et al.  Minimizing finite sums with the stochastic average gradient , 2013, Mathematical Programming.

[9]  Andreas Geiger,et al.  Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art , 2017, Found. Trends Comput. Graph. Vis..

[10]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[11]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[12]  Chong-Min Kyung,et al.  Rejecting Motion Outliers for Efficient Crowd Anomaly Detection , 2019, IEEE Transactions on Information Forensics and Security.

[13]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Yujie Wang,et al.  Flow-Guided Feature Aggregation for Video Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Agnès Bardon-Albaret,et al.  Effects of hypoxia and elevated ammonia concentration on the viability of red snapper embryos and early larvae , 2016 .

[16]  Stan Z. Li,et al.  Normalized LDA for semi-supervised learning , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[17]  Tie Qiu,et al.  Recurrent Broad Learning Systems for Time Series Prediction , 2020, IEEE Transactions on Cybernetics.

[18]  Patrick Mäder,et al.  Plant Species Identification Using Computer Vision Techniques: A Systematic Literature Review , 2017, Archives of Computational Methods in Engineering.

[19]  Garrison W. Cottrell,et al.  A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction , 2017, IJCAI.

[20]  Frode Oppedal,et al.  Dissolved oxygen variability in a commercial sea-cage exposes farmed Atlantic salmon to growth limiting conditions , 2018 .

[21]  Tao Li,et al.  Deep belief echo-state network and its application to time series prediction , 2017, Knowl. Based Syst..

[22]  Nassir Navab,et al.  Revisiting NARX Recurrent Neural Networks for Long-Term Dependencies , 2017, ArXiv.

[23]  Natapon Pantuwong,et al.  A comparative study of feature point matching versus foreground detection for computer detection of dairy cows in video frames , 2015, Artificial Life and Robotics.

[24]  Christopher. Simons,et al.  Machine learning with Python , 2017 .

[25]  Jake K. Aggarwal,et al.  Tracking and Segmentation of Highway Vehicles in Cluttered and Crowded Scenes , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[26]  Matti Pietikäinen,et al.  Deep Learning for Generic Object Detection: A Survey , 2018, International Journal of Computer Vision.

[27]  Antoine Vacavant,et al.  A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos , 2014, Comput. Vis. Image Underst..

[28]  Huihui Yu,et al.  Three-Dimensional Short-Term Prediction Model of Dissolved Oxygen Content Based on PSO-BPANN Algorithm Coupled with Kriging Interpolation , 2016 .

[29]  Jian Liu,et al.  Hierarchical Attention-Based Recurrent Highway Networks for Time Series Prediction , 2018, ArXiv.

[30]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[31]  Z. Zivkovic.  Improved adaptive Gaussian mixture model for background subtraction , 2004, ICPR 2004.

[32]  Gábor J. Székely,et al.  Hierarchical Clustering via Joint Between-Within Distances: Extending Ward's Minimum Variance Method , 2005, J. Classif..

[33]  Yu Zheng,et al.  GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction , 2018, IJCAI.

[34]  Xiaoou Tang,et al.  Surpassing Human-Level Face Verification Performance on LFW with GaussianFace , 2014, AAAI.

[35]  이창기,et al.  Convolutional Neural Network를 이용한 한국어 영화평 감성 분석 , 2016 .

[36]  S. Agnes Shifani,et al.  Security system using raspberry Pi , 2017, 2017 Third International Conference on Science Technology Engineering & Management (ICONSTEM).

[37]  Nassir Navab,et al.  Analyzing and Exploiting NARX Recurrent Neural Networks for Long-Term Dependencies , 2017, ICLR.

[38]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[39]  Weng-Kin Lai,et al.  ArchCam: Real time expert system for suspicious behaviour detection in ATM site , 2018, Expert Syst. Appl..

[40]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[41]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[43]  Farhat Fnaiech,et al.  Convolutional neural network for video fire and smoke detection , 2016, IECON 2016 - 42nd Annual Conference of the IEEE Industrial Electronics Society.

[44]  Jenq-Neng Hwang,et al.  Underwater Fish Tracking for Moving Cameras Based on Deformable Multiple Kernels , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[45]  Yong Wang,et al.  Bayesian Classifiers for Positive Unlabeled Learning , 2011, WAIM.

[46]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[47]  Jiahai Wang,et al.  Financial time series prediction using a dendritic neuron model , 2016, Knowl. Based Syst..

[48]  Ha Young Kim,et al.  Forecasting the volatility of stock price index: A hybrid model integrating LSTM with multiple GARCH-type models , 2018, Expert Syst. Appl..

[49]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[50]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.