Driver Drowsiness Detection Using Multi-Channel Second Order Blind Identifications

It is well known that blink, yawn, and heart rate changes give clue about a human’s mental state, such as drowsiness and fatigue. In this paper, image sequences, as the raw data, are captured from smart phones which serve as non-contact optical sensors. Video streams containing subject’s facial region are analyzed to identify the physiological sources that are mixed in each image. We then propose a method to extract blood volume pulse and eye blink and yawn signals as multiple independent sources simultaneously by multi-channel second-order blind identification (SOBI) without any other sophisticated processing, such as eye and mouth localizations. An overall decision is made by analyzing the separated source signals in parallel to determine the driver’s driving state. The robustness of the proposed method is tested under various illumination contexts and a variety of head motion modes. Experiments on 15 subjects show that the multi-channel SOBI presents a promising framework to accurately detect drowsiness by merging multi-physiological information in a less complex way.

[1]  Margrit Betke,et al.  Real Time Eye Tracking and Blink Detection with USB Cameras , 2005 .

[2]  Drew Dawson,et al.  Look before you (s)leep: evaluating the use of fatigue detection technologies within a fatigue risk management system for the road transport industry. , 2014, Sleep medicine reviews.

[3]  Serge Boverie,et al.  Driver vigilance diagnostic based on eyelid movement observation , 2008 .

[4]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[5]  Umakant P. Kulkarni,et al.  Detection of Drowsiness Using Fusion of Yawning and Eyelid Movements , 2013 .

[6]  Yufei Huang,et al.  Prediction of driver's drowsy and alert states from EEG signals with deep learning , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[7]  Antoine Picot,et al.  On-Line Detection of Drowsiness Using Brain and Visual Information , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[8]  Hang-Bong Kang,et al.  Various Approaches for Driver and Driving Behavior Monitoring: A Review , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[9]  Kaigui Bian,et al.  Sober-Drive: A smartphone-assisted drowsy driving detection system , 2014, 2014 International Conference on Computing, Networking and Communications (ICNC).

[10]  Mubarak Shah,et al.  Determining driver visual attention with one camera , 2003, IEEE Trans. Intell. Transp. Syst..

[11]  Kenneth Sundaraj,et al.  Detecting Driver Drowsiness Based on Sensors: A Review , 2012, Sensors.

[12]  Zhao Lv,et al.  Simultaneous detection of blink and heart rate using multi-channel ICA from smart phone videos , 2017, Biomed. Signal Process. Control..

[13]  Pablo Laguna,et al.  Drowsiness detection using heart rate variability , 2016, Medical & Biological Engineering & Computing.

[14]  Bo Gao,et al.  Driving Style Recognition for Intelligent Vehicle Control and Advanced Driver Assistance: A Survey , 2018, IEEE Transactions on Intelligent Transportation Systems.

[15]  Sepideh Hajipour Sardouie,et al.  An Efficient Jacobi-Like Deflationary ICA Algorithm: Application to EEG Denoising , 2015, IEEE Signal Processing Letters.

[16]  Shahram Azadi,et al.  Data Fusion to Develop a Driver Drowsiness Detection System with Robustness to Signal Loss , 2014, Sensors.

[17]  Atsuo Murata,et al.  Evaluation of Drowsiness by HRV Measures : Basic Study for Drowsy Driver Detection , 2008 .

[18]  Jay D. Fuletra.  A Survey on Driver’s Drowsiness Detection Techniques , 2013 .

[19]  Wen-Zhong Tang,et al.  A Review on Fatigue Driving Detection , 2017 .

[20]  Xuesong Wang,et al.  Driver drowsiness detection based on non-intrusive metrics considering individual specifics. , 2016, Accident; analysis and prevention.

[21]  Cataldo Guaragnella,et al.  A visual approach for driver inattention detection , 2007, Pattern Recognit..

[22]  K. Banitsas,et al.  A novel method to detect Heart Beat Rate using a mobile phone , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[23]  Wen Jun Jiang,et al.  Real-time quantifying heart beat rate from facial video recording on a smart phone using Kalman filters , 2014, 2014 IEEE 16th International Conference on e-Health Networking, Applications and Services (Healthcom).

[24]  Xun Zhang,et al.  Traffic accidents involving fatigue driving and their extent of casualties. , 2016, Accident; analysis and prevention.

[25]  É. Moulines,et al.  Second Order Blind Separation of Temporally Correlated Sources , 1993 .

[26]  T. Åkerstedt,et al.  Transport and industrial safety, how are they affected by sleepiness and sleep restriction? , 2006, Sleep medicine reviews.

[27]  Abdulmotaleb El Saddik,et al.  Heart Rate Variability Extraction From Videos Signals: ICA vs. EVM Comparison , 2017, IEEE Access.

[28]  Chng Eng Siong,et al.  Foreground motion detection by difference-based spatial temporal entropy image , 2004, 2004 IEEE Region 10 Conference TENCON 2004..

[29]  Seong G. Kong,et al.  Visual Analysis of Eye State and Head Pose for Driver Alertness Monitoring , 2013, IEEE Transactions on Intelligent Transportation Systems.

[30]  Roy Kalawsky,et al.  Noncontact imaging photoplethysmography to effectively access pulse rate variability , 2012, Journal of biomedical optics.

[31]  Bae Jin Yong,et al.  The Core Technical Trends of TESLA EV(Electric Vehicle) Motors , 2017 .

[32]  Aditya Shah,et al.  Drowsiness Detection based on Eye Movement , Yawn Detection and Head Rotation , 2012 .

[33]  Neil Cooke,et al.  VOG-enhanced ICA for SSVEP response detection from consumer-grade EEG , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[34]  Izabela Rejer,et al.  Benefits of ICA in the Case of a Few Channel EEG , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[35]  Daniel McDuff,et al.  Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam , 2011, IEEE Transactions on Biomedical Engineering.

[36]  Shervin Shirmohammadi,et al.  Driver drowsiness monitoring based on yawning detection , 2011, 2011 IEEE International Instrumentation and Measurement Technology Conference.

[37]  Daniel McDuff,et al.  Remote Detection of Photoplethysmographic Systolic and Diastolic Peaks Using a Digital Camera , 2014, IEEE Transactions on Biomedical Engineering.

[38]  Kazunori Shidoji,et al.  Detecting drowsiness while driving by measuring eye movement - a pilot study , 2002, Proceedings. The IEEE 5th International Conference on Intelligent Transportation Systems.

[39]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[40]  Giuseppe Mancia,et al.  Sympathetic, parasympathetic and non‐autonomic contributions to cardiovascular spectral powers in unanesthetized spontaneously hypertensive rats , 1995, Journal of hypertension.

[41]  C. Takano,et al.  Heart rate measurement based on a time-lapse image. , 2007, Medical engineering & physics.

[42]  R Stojanovic,et al.  A LED-LED-based photoplethysmography sensor. , 2007, Physiological measurement.

[43]  Toshio Fukuda,et al.  Simultaneous Measurement of Heart Rate Variability and Blinking Duration to Predict Sleep Onset and Drowsiness in Drivers , 2015 .

[44]  Erhan Akin,et al.  Estimating driving behavior by a smartphone , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[45]  Sijung Hu,et al.  The preliminary investigation of imaging photoplethysmographic system , 2007 .

[46]  Shinobu Tanaka,et al.  Comparison between red, green and blue light reflection photoplethysmography for heart rate monitoring during motion , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[47]  Aapo Hyvärinen,et al.  Fast ICA for noisy data using Gaussian moments , 1999, ISCAS.

[48]  Jian-Ping Li,et al.  Eye behaviour based drowsiness Detection System , 2015, 2015 12th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP).

[49]  Ayoub Al-Hamadi,et al.  A color based approach for eye blink detection in image sequences , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[50]  John Allen.  Photoplethysmography and its application in clinical physiological measurement , 2007, Physiological measurement.

[51]  Wan-Young Chung,et al.  Wearable driver drowsiness detection system based on biomedical and motion sensors , 2015, 2015 IEEE SENSORS.

[52]  B. Li,et al.  Non-contact detection of oxygen saturation based on visible light imaging device using ambient light. , 2013, Optics express.

[53]  Yue-Der Lin,et al.  Wavelet-based embedded algorithm for respiratory rate estimation from PPG signal , 2017, Biomed. Signal Process. Control..

[54]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[55]  Vasilis Ntziachristos,et al.  Looking and listening to light: the evolution of whole-body photonic imaging , 2005, Nature Biotechnology.

[56]  Aouatif Amine,et al.  Driver's fatigue detection based on yawning extraction , 2014 .

[57]  Tomáš Štula.  Evaluation of Heart Rate Variability from ECG Signal , 2003 .

[58]  J. Verster,et al.  Prolonged nocturnal driving can be as dangerous as severe alcohol‐impaired driving , 2011, Journal of sleep research.

[59]  Stefanos Zafeiriou,et al.  A survey on face detection in the wild: Past, present and future , 2015, Comput. Vis. Image Underst..

[60]  Luciano Boquete,et al.  EOG-based eye movements codification for human computer interaction , 2012, Expert Syst. Appl..

[61]  Chih-Peng Fan,et al.  Near-infrared-ray and side-view video based drowsy driver detection system: Whether or not wearing glasses , 2016, 2016 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS).

[62]  Mehrdad Tanha,et al.  Morphological drowsy detection , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[63]  Koji Oguri,et al.  Estimation of drowsiness level based on eyelid closure and heart rate variability , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[64]  Lamiaa Fattouh Ibrahim,et al.  Using Mobile Platform to Detect and Alerts Driver Fatigue , 2015 .

[65]  Hagen Malberg,et al.  Improved heart rate detection for camera-based photoplethysmography by means of Kalman filtering , 2015, 2015 IEEE 35th International Conference on Electronics and Nanotechnology (ELNANO).