Context-Based Rider Assistant System for Two Wheeled Self-Balancing Vehicles

Kim J., Sato K., Hashimoto N., Kashevnik A., Tomita K., Miyakoshi S., Takinami Y., Matsumoto O., Boyali A. Context-Based Rider Assistant System for Two Wheeled SelfBalancing Vehicles. Abstract. Personal mobility devises become more and more popular last years. Gyroscooters, two wheeled self-balancing vehicles, wheelchair, bikes, and scooters help people to solve the first and last mile problems in big cities. To help people with navigation and to increase their safety the intelligent rider assistant systems can be utilized that are used the rider personal smartphone to form the context and provide the rider with the recommendations. We understand the context as any information that characterize current situation. So, the context represents the model of current situation. We assume that rider mounts personal smartphone that allows it to track the rider face using the front-facing camera. Modern smartphones allow to track current situation using such sensors as: GPS / GLONASS, accelerometer, gyroscope, magnetometer, microphone, and video cameras. The proposed rider assistant system uses these sensors to capture the context information about the rider and the vehicle and generates context-oriented recommendations. The proposed system is aimed at dangerous situation detection for the rider, we are considering two dangerous situations: drowsiness and distraction. Using the computer vision methods, we determine parameters of the rider face (eyes, nose, mouth, head pith and rotation angles) and based on analysis of this parameters detect the dangerous situations. The paper presents a comprehensive related work analysis in the topic of intelligent driver assistant systems and recommendation generation, an approach to dangerous situation detection and recommendation generation is proposed, and evaluation of the distraction dangerous state determination for personal mobility device riders.

[1]  Franklin M. Silva,et al.  Real Time Driver Drowsiness Detection Based on Driver's Face Image Behavior Using a System of Human Computer Interaction Implemented in a Smartphone , 2018, ICITS.

[2]  Jürgen Schmidt,et al.  Eye blink detection for different driver states in conditionally automated driving and manual driving using EOG and a driver camera , 2017, Behavior Research Methods.

[3]  George K. Hung,et al.  Drowsy driver mobile application: Development of a novel scleral-area detection method , 2017, Comput. Biol. Medicine.

[4]  Zahid Halim,et al.  Clustering the driving features based on data streams , 2013, INMIC.

[5]  Alexey Kashevnik,et al.  Decision Support System for Drivers & Passengers: Smartphone-Based Reference Model and Evaluation , 2018, 2018 23rd Conference of Open Innovations Association (FRUCT).

[6]  Alexey Kashevnik,et al.  Context-based driver support system development: Methodology and case study , 2017, 2017 21st Conference of Open Innovations Association (FRUCT).

[7]  S. Chandrakala,et al.  Android OpenCV based effective driver fatigue and distraction monitoring system , 2015, 2015 International Conference on Computing and Communications Technologies (ICCCT).

[8]  Alexander V. Smirnov,et al.  Smartphone-based identification of dangerous driving situations: Algorithms and implementation , 2016, 2016 18th Conference of Open Innovations Association and Seminar on Information Security and Protection of Information Technology (FRUCT-ISPIT).

[9]  Yizhi Wang,et al.  A System of Driving Fatigue Detection Based on Machine Vision and Its Application on Smart Device , 2015, J. Sensors.

[10]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.

[11]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[12]  Jean Underwood,et al.  Driving Experience and Situation Awareness in Hazard Detection , 2013 .

[13]  Chalermpol Saiprasert,et al.  Detecting Driving Events using Smartphone , 2013 .

[14]  Alexander V. Smirnov,et al.  Human-Smartphone Interaction for Dangerous Situation Detection and Recommendation Generation While Driving , 2016, SPECOM.

[15]  Alice Caplier,et al.  Driver Head Movements While Using a Smartphone in a Naturalistic Context , 2017 .

[16]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[17]  Kai Zeng,et al.  A smartphone-based driver fatigue detection using fusion of multiple real-time facial features , 2016, 2016 13th IEEE Annual Consumer Communications & Networking Conference (CCNC).

[18]  Daniele Ruscio,et al.  Limitations and automation: the role of information about device-specific features in ADAS acceptability , 2016 .

[19]  Rainer Stark,et al.  How to Consider Emotional Reactions of the Driver within the Development of Advanced Driver Assistance Systems (ADAS) , 2014 .

[20]  Alexander V. Smirnov,et al.  Smartphone-based two-wheeled self-balancing vehicles rider assistant , 2015, 2015 17th Conference of Open Innovations Association (FRUCT).

[21]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  A. D. Khomonenko,et al.  Software for Mobile Indoor Navigation using Neural Networks , 2016 .

[23]  Alexey Kashevnik,et al.  Web-Service for Drive Safely System User Analysis: Architecture and Implementation , 2018, 2018 22nd Conference of Open Innovations Association (FRUCT).

[24]  M Kutila.  Methods for machine vision based driver monitoring applications , 2006 .

[25]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[26]  Anirban Dasgupta,et al.  A Smartphone-Based Drowsiness Detection and Warning System for Automotive Drivers , 2019, IEEE Transactions on Intelligent Transportation Systems.

[27]  Lamiaa Fattouh Ibrahim,et al.  Mobile Platform Detect and Alerts System for Driver Fatigue , 2015, SCSE.

[28]  Alice Caplier,et al.  Sleep Deprivation Detection for Real-Time Driver Monitoring Using Deep Learning , 2018, ICIAR.