Human behaviour recognition with mid-level representations for crowd understanding and analysis

Funding information National Natural Science Foundation of China, Grant/Award Number: 62076199; China Postdoctoral Science Foundation, Grant/Award Number: 2019M653784; National Key R∖&D Program of China, Grant/Award Number: 2017YFB0502900; CAS Light of West China Program, Grant/Award Number: XAB2017B15; Key Laboratory of Spectral Imaging Technology of Chinese Academy of Sciences, Grant/Award Number: LSIT201801D Abstract Crowd understanding and analysis have received increasing attention for couples of decades, and development of human behaviour recognition strongly supports the application of crowd understanding and analysis. Human behaviour recognition usually seeks to automatically analyse ongoing movements and actions in different camera views by using various machine learning methodologies in unknown video clips or image sequences. Compared to other data modalities such as documents and images, processing video data demands much higher computational and storage resources. The idea of using middle level semantic concepts to represent human actions from videos is explored and it is argued that these semantic attributes enable the construction of more descriptive methods for human action recognition. The mid-level attributes, initialized by a cluster processing, are built upon low level features and fully utilize the discrepancies in different action classes, which can capture the importance of each attribute for each action class. In this way, the representation is constructed to be semantically rich and capable of highly discriminative performance even paired with simple linear classifiers. The method is verified on three challenging datasets (KTH, UCF50 and HMDB51), and the experimental results demonstrate that our method achieves better results than the baseline methods on human action recognition.