Structure Function Based Transform Features for Behavior-Oriented Social Media Image Classification

Social media has become an essential part of people to reflect their day to day activities including emotions, feelings, threatening and so on. This paper presents a new method for the automatic classification of behavior-oriented images like Bullying, Threatening, Neuroticism-Depression, Neuroticism-Sarcastic, Psychopath and Extraversion of a person from social media images. The proposed method first finds facial key points for extracting features based on a face detection algorithm. Then the proposed method labels face regions as foreground and other than face region as background to define context between foreground and background information. To extract context, the proposed method explores Structural Function based Transform (SFBT) features, which study variations on pixel values. To increase discriminating power of the context features, the proposed method performs clustering to integrate the strength of the features. The extracted features are then fed to Support Vector Machines (SVM) for classification. Experimental results on a dataset of six classes show that the proposed method outperforms the existing methods in terms of confusion matrix and classification rate.

[1]  Hua-Tsung Chen,et al.  Accelerating Vanishing Point-Based Line Sampling Scheme for Real-Time People Localization , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Qiang Ji,et al.  Thermal Augmented Expression Recognition , 2018, IEEE Transactions on Cybernetics.

[3]  Giuliano Arru,et al.  Exploiting Visual Behaviour for Autism Spectrum Disorder Identification , 2019, 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[4]  Lyle H. Ungar,et al.  Analyzing Personality through Social Media Profile Picture Choice , 2016, ICWSM.

[5]  Diana Inkpen,et al.  Natural Language Processing for Social Media , 2015, Natural Language Processing for Social Media.

[6]  Madasu Hanmandlu,et al.  Suspicious Face Detection based on Eye and other facial features movement monitoring , 2015, 2015 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[7]  Palaiahnakote Shivakumara,et al.  Rough-fuzzy based scene categorization for text detection and recognition in video , 2018, Pattern Recognit..

[8]  Chung-Lin Huang,et al.  A video-based abnormal human behavior detection for psychiatric patient monitoring , 2018, 2018 International Workshop on Advanced Image Technology (IWAIT).

[9]  Jian Sun,et al.  Social multi-modal event analysis via knowledge-based weighted topic model , 2019, J. Vis. Commun. Image Represent..

[10]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[11]  Anima Majumder,et al.  Automatic Facial Expression Recognition System Using Deep Network-Based Data Fusion , 2018, IEEE Transactions on Cybernetics.

[12]  Ezzeddine Zagrouba,et al.  Abnormal behavior recognition for intelligent video surveillance systems: A review , 2018, Expert Syst. Appl..

[13]  Toon Goedemé,et al.  Abnormal behavior detection in LWIR surveillance of railway platforms , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[14]  Madasu Hanmandlu,et al.  The fusion of multispectral palmprints using the information set based features and classifier , 2018, Eng. Appl. Artif. Intell..

[15]  Vania Bogorny,et al.  Toward Abnormal Trajectory and Event Detection in Video Surveillance , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Jefersson Alex dos Santos,et al.  Towards better exploiting convolutional neural networks for remote sensing scene classification , 2016, Pattern Recognit..

[17]  Youren Wang,et al.  Medical image fusion using discrete fractional wavelet transform , 2016, Biomed. Signal Process. Control..

[18]  K. Unnikrishnan,et al.  Fraud Detection by Facial Expression Analysis Using Intel RealSense and Augmented Reality , 2018, 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS).

[19]  Nazil Perveen,et al.  Spontaneous Expression Recognition Using Universal Attribute Model , 2018, IEEE Transactions on Image Processing.