A Novel Two-stream Architecture Fusing Static And Dynamic Features for Human Action Recognition

—Action recognition in real-world videos is di ffi cult because of factors such as scenery muddle, scale alter, dynamic standpoint, and sharp motion.This paper proposes a novel two-stream architecture fusing static and dynamic features to recognize human actions in videos. Firstly, the original image (single frame and optical flow fields) is extracted by Convolutional Neural Network (CNN) to obtain feature maps. Secondly, we extract the obtained feature maps via a 3 (cid:2) 3 convolution over all the neighbor features, leading to a static representations of features. Then we concatenate these static features with input feature maps to produce the dynamic attention matrix through two 1 (cid:2) 1 convolutions. All of the generated feature maps are then aggregated using the learnt attention matrix, producing a dynamic representation. Thirdly, we take the interaction of the static and dynamic presentations as final outputs. Finally, we utilize Long Short-Term Memory (LSTM) to catch time sequence information among dense optical flow.The experimental results on the three hard datasets UCF101, HMDB51, and Kinetics400 have shown that the method works better than other state-of-the-art methods.