Deep supervised feature selection for social relationship recognition

Abstract Social relationships link everyone in human society. Exploring social relationships in still images promotes researches of behaviors or characteristics among persons. Previous literature has discovered that face and body attributes can provide effective semantic information for social relationship recognition. However, they ignore that attributes contribute much differently to the recognition accuracy, and these multi-source attributes may contain redundancies and noises. This work aims to promote social relationship recognition accuracy by abstracting multi-source attribute features more efficiently. To this end, we propose a novel Deep Supervised Feature Selection (DSFS) framework to recognize social relationships in photos, which fuses the deep learning algorithm with l2,1-norm to learn a discriminative feature subset from multi-source features by leveraging the face and body attributes. Experimental results on PIPA-relation dataset qualitatively demonstrate the effectiveness of the proposed DSFS framework.

[1]  MengChu Zhou,et al.  An Efficient Second-Order Approach to Factorize Sparse Matrices in Recommender Systems , 2015, IEEE Transactions on Industrial Informatics.

[2]  MengChu Zhou,et al.  An Effective Scheme for QoS Estimation via Alternating Direction Method-Based Matrix Factorization , 2019, IEEE Transactions on Services Computing.

[3]  Jiwen Lu,et al.  Neighborhood repulsed metric learning for kinship verification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Ming Shao,et al.  RFIW: Large-Scale Kinship Recognition Challenge , 2017, ACM Multimedia.

[5]  D. Bugental,et al.  Acquisition of the algorithms of social life: a domain-based approach. , 2000, Psychological bulletin.

[6]  Haibin Yan,et al.  Semantic three-stream network for social relation recognition , 2019, Pattern Recognit. Lett..

[7]  Shuai Li,et al.  Symmetric and Nonnegative Latent Factor Models for Undirected, High-Dimensional, and Sparse Networks in Industrial Applications , 2017, IEEE Transactions on Industrial Informatics.

[8]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[9]  Hui Cheng,et al.  Deep Reasoning with Knowledge Graph for Social Relationship Understanding , 2018, IJCAI.

[10]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[11]  Mohan S. Kankanhalli,et al.  Dual-Glance Model for Deciphering Social Relationships , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Bernt Schiele,et al.  A Domain Based Approach to Social Relation Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Xiaogang Wang,et al.  Multi-task Recurrent Neural Network for Immediacy Prediction , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Greg Mori,et al.  Social roles in hierarchical models for human activity recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Ming Shao,et al.  What Do You Do? Occupation Recognition in a Photo via Social Context , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  Zhongfu Ye,et al.  A deep learning approach for face recognition based on angularly discriminative features , 2019, Pattern Recognit. Lett..

[17]  Jinhui Tang,et al.  Unsupervised Feature Selection via Nonnegative Spectral Analysis and Redundancy Control , 2015, IEEE Transactions on Image Processing.

[18]  MengChu Zhou,et al.  Generating Highly Accurate Predictions for Missing QoS Data via Aggregating Nonnegative Latent Factor Models , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Song-Chun Zhu,et al.  Joint inference of groups, events and human roles in aerial videos , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Tao Mei,et al.  Social Relation Recognition From Videos via Multi-Scale Spatial-Temporal Reasoning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Keng Teck Ma,et al.  An End-To-End Network for Generating Social Relationship Graphs , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Jing Liu,et al.  Clustering-Guided Sparse Structural Learning for Unsupervised Feature Selection , 2014, IEEE Transactions on Knowledge and Data Engineering.

[23]  MengChu Zhou,et al.  A Nonnegative Latent Factor Model for Large-Scale Sparse Matrices in Recommender Systems via Alternating Direction Method , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Fei-Fei Li,et al.  Social Role Discovery in Human Events , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Xiaoou Tang,et al.  From Facial Expression Recognition to Interpersonal Relation Prediction , 2016, International Journal of Computer Vision.

[26]  Xiaoou Tang,et al.  Learning Social Relation Traits from Face Images , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).