Real-Time Streaming Image Based PP2LFA-CRNN Model for Facial Sentiment Analysis

In modern society, the real-time emotion adaptive driving system for providing safety to drivers and emotion-based services are being researched. However, in the service process have problem of personal information might get leaked. Therefore, a robust personal information protection method is required for face recognition services based on real-time images. In this study, we propose a real-time streaming image based PingPong256 (PP2) algorithm, line-segment feature analysis (LFA), convolutional recurrent neural network (CRNN) model for facial sentiment analysis. The proposed method applied the PP2 algorithm to images for encryption and decryption for the security of the real-time images collected by image devices. For transmitting images to a server, LFA, as a dimensionality reduction algorithm, is used to extract facial information. PP2 encrypts and decrypts an image through a linear feedback shift register with a different length and sets a random value other than 0 so that inferring the initial value of encryption becomes difficult, and then executes the random operations approximately 1,000 times. The LFA analyzes the line segments of an image, assigns a different unique number depending on its type, and cumulatively adds them to generate a Line-Segment map (LS-map) with a size of $16\times16$ . The LS-map is used as an input of the CRNN model designed in this study, and the facial expressions are classified. Performance evaluation compares the accuracy of face recognition by using the proposed method with the loss rate for other models. Performance evaluation renders excellence to the accuracy of face recognition and loss rate comparison.