Reconfigurable Cyber-Physical System for Lifestyle Video-Monitoring via Deep Learning

Indoor monitoring of people at their homes has become a popular application in Smart Health. With the advances in Machine Learning and hardware for embedded devices, new distributed approaches for Cyber-Physical Systems (CPSs) are enabled. Also, changing environments and need for cost reduction motivate novel reconfigurable CPS architectures. In this work, we propose an indoor monitoring reconfigurable CPS that uses embedded local nodes (Nvidia Jetson TX2). We embed Deep Learning architectures to address Human Action Recognition. Local processing at these nodes let us tackle some common issues: reduction of data bandwidth usage and preservation of privacy (no raw images are transmitted). Also real-time processing is facilitated since optimized nodes compute only its local video feed. Regarding the reconfiguration, a remote platform monitors CPS qualities and a Quality and Resource Management (QRM) tool sends commands to the CPS core to trigger its reconfiguration. Our proposal is an energy-aware system that triggers reconfiguration based on energy consumption for battery-powered nodes. Reconfiguration reduces up to 22% the local nodes energy consumption extending the device operating time, preserving similar accuracy with respect to the alternative with no reconfiguration.

[1]  Yuqing Chen,et al.  A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[2]  Akikazu Takeuchi,et al.  STAIR Actions: A Video Dataset of Everyday Home Actions , 2018, ArXiv.

[3]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[4]  Rached Tourki,et al.  Optimized spatio-temporal descriptors for real-time fall detection: comparison of support vector machine and Adaboost-based classification , 2013, J. Electronic Imaging.

[5]  Abdul-Rahman Al-Ali,et al.  Cyber physical systems role in manufacturing technologies , 2018 .

[6]  Meikang Qiu,et al.  Health-CPS: Healthcare Cyber-Physical System Assisted by Cloud and Big Data , 2017, IEEE Systems Journal.

[7]  Qing-Long Han,et al.  A Survey on Model-Based Distributed Control and Filtering for Industrial Cyber-Physical Systems , 2019, IEEE Transactions on Industrial Informatics.

[8]  Twan Basten,et al.  Interface Modeling for Quality and Resource Management , 2020, ArXiv.

[9]  Thomas Serre,et al.  HMDB: A large video database for human motion recognition , 2011, 2011 International Conference on Computer Vision.

[10]  James Demmel,et al.  Scaling Deep Learning on GPU and Knights Landing clusters , 2017, SC17: International Conference for High Performance Computing, Networking, Storage and Analysis.

[11]  Mauro Mazzolini,et al.  CPS-based hierarchical and self-similar automation architecture for the control and verification of reconfigurable manufacturing systems , 2014, Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA).

[12]  Eduardo Tovar,et al.  CAP: Context-Aware Programming for Cyber Physical Systems , 2019, 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).

[13]  Fabio Viola,et al.  The Kinetics Human Action Video Dataset , 2017, ArXiv.

[14]  Martín Barrère,et al.  CPS-MT: A Real-Time Cyber-Physical System Monitoring Tool for Security Research , 2018, 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA).

[15]  Antonio Torralba,et al.  Recognizing indoor scenes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Andrew Zisserman,et al.  Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Twan Basten,et al.  The FitOptiVis ECSEL project: highly efficient distributed embedded image/video processing in cyber-physical systems , 2019, CF.

[18]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Eduardo Ros,et al.  Real-Time Visual Saliency Architecture for FPGA With Top-Down Attention Modulation , 2014, IEEE Transactions on Industrial Informatics.

[20]  Tao Lei,et al.  A review of Convolutional-Neural-Network-based action recognition , 2019, Pattern Recognit. Lett..

[21]  Jun Zhou,et al.  TSE-CNN: A Two-Stage End-to-End CNN for Human Activity Recognition , 2020, IEEE Journal of Biomedical and Health Informatics.

[22]  Gil Gonçalves,et al.  Self-adaptive Cobots in Cyber-Physical Production Systems , 2019, 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).

[23]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[24]  Andrew Zisserman,et al.  A Short Note about Kinetics-600 , 2018, ArXiv.

[25]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[26]  Miguel A. Labrador,et al.  A Survey on Human Activity Recognition using Wearable Sensors , 2013, IEEE Communications Surveys & Tutorials.

[27]  Edward A. Lee.  Cyber Physical Systems: Design Challenges , 2008, 2008 11th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (ISORC).

[28]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Ali Farhadi,et al.  Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding , 2016, ECCV.

[30]  Dario Pompili,et al.  Towards A Reconfigurable Cyber Physical System , 2014, 2014 IEEE 11th International Conference on Mobile Ad Hoc and Sensor Systems.

[31]  Sebastian Thrun,et al.  Learning to Learn: Introduction and Overview , 1998, Learning to Learn.

[32]  Marco Taisch,et al.  Functional requirements for reconfigurable and flexible cyber-physical system , 2016, IECON 2016 - 42nd Annual Conference of the IEEE Industrial Electronics Society.

[33]  Nuno Pereira,et al.  Cyber-physical systems clouds: A survey , 2016, Comput. Networks.

[34]  Kok Kiong Tan,et al.  Non-intrusive fall detection monitoring for the elderly based on fuzzy logic , 2018 .

[35]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[36]  Mubarak Shah,et al.  UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild , 2012, ArXiv.

[37]  Eduardo Tovar,et al.  Design and Implementation of Secret Key Agreement for Platoon-based Vehicular Cyber-physical Systems , 2019, ACM Trans. Cyber Phys. Syst..

[38]  Cordelia Schmid,et al.  PoTion: Pose MoTion Representation for Action Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Ranjan Pal,et al.  The STREAM Mechanism for CPS Security The Case of the Smart Grid , 2017, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[40]  Alessandro Carbonari,et al.  A cyber-physical system approach for building efficiency monitoring , 2019, Automation in Construction.

[41]  Yann LeCun,et al.  A Closer Look at Spatiotemporal Convolutions for Action Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.