Optical Flow Estimation and Denoising of Video Images Based on Deep Learning Models

In order to effectively extract image features highly related to visual perception quality, improve the image quality evaluation method, under the framework of deep learning, combining the optical flow method and the edge detection algorithm, a multi-feature fusion motion based on improved optical flow is proposed target detection algorithm. First, a video fluid model is proposed. The fluid model decomposes the video object area changes into sub-area zoom, rotation and translation movements. The rigid body area and the area hierarchy describe the spatial relationship of pixels, and the rigid body motion describes the time domain relationship of pixels. It provides a region-based video processing. The associated spatiotemporal is association method. Secondly, a video fluid model is proposed. The video fluid model treats all pixels of the same surface imaged in the video as a fluid, using streamlines to represent the regional motion of the object, and streamlines to represent the pixel motion of the video object, using rotators and translation lines to simplify the streamlines when necessary. The streamline of the same fluid is smooth in the time domain, and the flow pattern is smooth in both the time domain and the space domain. Finally, the top-down deep learning generation model conversion is carried out, and finally through continuous adjustment between different levels, the generation model can reconstruct the original sample with lower error, so that the essential characteristics of this sample are obtained, namely the highest abstract representation of the depth model. After processing the deep learning model, the sample features after dimensionality reduction can be obtained, and the recognition module is used on this basis. Experiments show that the optical flow estimation method based on deep learning and multi-grid, optical flow field estimation method based on variational model and desiccation method proposed in this paper are effective, and it is suitable for moving image analysis, target tracking and 3D reconstruction Such research has certain theoretical significance and practical application value.

[1]  Jan Kautz,et al.  Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Chao Xu,et al.  Particle Image Velocimetry Based on a Deep Learning Motion Estimator , 2020, IEEE Transactions on Instrumentation and Measurement.

[3]  Jinjun Tang,et al.  Real-Time Traffic Flow Parameter Estimation From UAV Video Based on Ensemble Classifier and Optical Flow , 2019, IEEE Transactions on Intelligent Transportation Systems.

[4]  Yuanyuan Wang,et al.  Cine MRI analysis by deep learning of optical flow: Adding the temporal dimension , 2019, Comput. Biol. Medicine.

[5]  Qingtian Wu,et al.  Real‐time running detection system for UAV imagery based on optical flow and deep convolutional networks , 2020, IET Intelligent Transport Systems.

[6]  Yiannis Andreopoulos,et al.  Video Classification With CNNs: Using the Codec as a Spatio-Temporal Activity Sensor , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Chuan-Zhi Dong,et al.  Structural displacement monitoring using deep learning-based full field optical flow methods , 2020, Structure and Infrastructure Engineering.

[8]  Lianghai Jin,et al.  Denoising Color Images Based on Local Orientation Estimation and CNN Classifier , 2020, Journal of Mathematical Imaging and Vision.

[9]  Xin Xu,et al.  Large-scale gesture recognition with a fusion of RGB-D data based on optical flow and the C3D model , 2019, Pattern Recognit. Lett..

[10]  Ariana Moncada,et al.  Deep Learning to Forecast Solar Irradiance Using a Six-Month UTSA SkyImager Dataset , 2018, Energies.

[11]  Sung Wook Baik,et al.  Activity Recognition Using Temporal Optical Flow Convolutional Features and Multilayer LSTM , 2019, IEEE Transactions on Industrial Electronics.

[12]  Guixiong Liu,et al.  Blind Deblurring of Saturated Images Based on Optimization and Deep Learning for Dynamic Visual Inspection on the Assembly Line , 2019, Symmetry.

[13]  Navdeep Jaitly,et al.  Deep Learning for Automated Occlusion Edge Detection in RGB-D Frames , 2016, Journal of Signal Processing Systems.

[14]  Shengqi Lu,et al.  Uniform regularity for a Keller-Segel-Navier-Stokes system , 2020, Appl. Math. Lett..

[15]  Kishanprasad G. Gunale,et al.  Deep Learning with a Spatiotemporal Descriptor of Appearance and Motion Estimation for Video Anomaly Detection , 2018, J. Imaging.

[16]  Ahmad B. A. Hassanat,et al.  Predicting Compressive Strength of Cement-Stabilized Rammed Earth Based on SEM Images Using Computer Vision and Deep Learning , 2019, Applied Sciences.

[17]  Fathi E. Abd El-Samie,et al.  Gait identification by convolutional neural networks and optical flow , 2019, Multimedia Tools and Applications.

[18]  Qingmin Liao,et al.  Depth-images-based pose estimation using regression forests and graphical models , 2015, Neurocomputing.

[19]  Antony J. Hodgson,et al.  A deep learning framework for segmentation and pose estimation of pedicle screw implants based on C-arm fluoroscopy , 2018, International Journal of Computer Assisted Radiology and Surgery.

[20]  Bin Zhou,et al.  A regional adaptive variational PDE model for computed tomography image reconstruction , 2019, Pattern Recognit..

[21]  Ting-Lan Lin,et al.  Reconstruction Algorithm for Lost Frame of Multiview Videos in Wireless Multimedia Sensor Network Based on Deep Learning Multilayer Perceptron Regression , 2018, IEEE Sensors Journal.

[22]  Xianzhi Zhang,et al.  Manufacturing cost estimation based on the machining process and deep-learning method , 2020 .

[23]  Robertas Damasevicius,et al.  Multi-sink distributed power control algorithm for Cyber-physical-systems in coal mine tunnels , 2019, Comput. Networks.

[24]  Kai Cui,et al.  Research on prediction model of geotechnical parameters based on BP neural network , 2018, Neural Computing and Applications.

[25]  Gabriele Costante,et al.  LS-VO: Learning Dense Optical Subspace for Robust Visual Odometry Estimation , 2017, IEEE Robotics and Automation Letters.

[26]  Jose Luis Lisani,et al.  Patch-Based Video Denoising With Optical Flow Estimation , 2016, IEEE Transactions on Image Processing.

[27]  Guy Nir,et al.  Deep Learning-Based Gleason Grading of Prostate Cancer From Histopathology Images—Role of Multiscale Decision Aggregation and Data Augmentation , 2020, IEEE Journal of Biomedical and Health Informatics.

[28]  Jian Zhang,et al.  Learning deep facial expression features from image and optical flow sequences using 3D CNN , 2018, The Visual Computer.

[29]  Xuezhi Xiang,et al.  Deep Optical Flow Supervised Learning With Prior Assumptions , 2018, IEEE Access.

[30]  Naixue Xiong,et al.  Abnormal event detection in crowded scenes based on deep learning , 2016, Multimedia Tools and Applications.

[31]  Yuxin Peng,et al.  Two-Stream Collaborative Learning With Spatial-Temporal Attention for Video Classification , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  Nouar AlDahoul,et al.  Real-Time Human Detection for Aerial Captured Video Sequences via Deep Models , 2018, Comput. Intell. Neurosci..

[33]  Ying Yu,et al.  Automatic extraction of impervious surfaces from high resolution remote sensing images based on deep learning , 2019, J. Vis. Commun. Image Represent..

[34]  Ling Shao,et al.  Video Salient Object Detection via Fully Convolutional Networks , 2017, IEEE Transactions on Image Processing.

[35]  Jianye Liu,et al.  Research of optical flow aided MEMS navigation based on convex optimization and ROF denoising , 2018 .

[36]  Wei Li,et al.  A fused deep learning architecture for viewpoint classification of echocardiography , 2017, Inf. Fusion.