Multi-sensor driver drowsiness monitoring

Abstract A system for driver drowsiness monitoring is proposed, using multi-sensor data acquisition and investigating two decision-making algorithms, namely a fuzzy inference system (FIS) and an artificial neural network (ANN), to predict the drowsiness level of the driver. Drowsiness indicator signals are selected allowing non-intrusive measurements. The experimental set-up of a driver-drowsiness-monitoring system is designed on the basis of the sought-after indicator signals. These selected signals are the eye closure via pupil area measurement, gaze vector and head motion acquired by a monocular computer vision system, steering wheel angle, vehicle speed, and force applied to the steering wheel by the driver. It is believed that, by fusing these signals, driver drowsiness can be detected and drowsiness level can be predicted. For validation of this hypothesis, 30 subjects, in normal and sleep-deprived conditions, are involved in a standard highway simulation for 1.5h, giving a data set of 30 pairs. For designing a feature space to be used in decision making, several metrics are derived using histograms and entropies of the signals. An FIS and an ANN are used for decision making on the drowsiness level. To construct the rule base of the FIS, two different methods are employed and compared in terms of performance: first, linguistic rules from experimental studies in literature and, second, mathematically extracted rules by fuzzy subtractive clustering. The drowsiness levels belonging to each session are determined by the participants before and after the experiment, and videos of their faces are assessed to obtain the ground truth output for training the systems. The FIS is able to predict correctly 98 per cent of determined drowsiness states (training set) and 89 per cent of previously unknown test set states, while the ANN has a correct classification rate of 90 per cent for the test data. No significant difference is observed between the FIS and the ANN; however, the FIS might be considered better since the rule base can be improved on the basis of new observations.