Tractor Assistant Driving Control Method Based on EEG Combined With RNN-TL Deep Learning Algorithm

Nowadays, fieldwork is often accompanied by tight schedules, which tends to strain the shoulder muscles due to high-intensity work. Moreover, it is difficult and stressful for the disabled to drive agricultural machinery. Besides, current artificial intelligence technology could not fully realize tractor autonomous driving because of a high uncertain filed environment and short interruptions of satellite navigation signal shaded by trees. To reduce manual operations, a tractor assistant driving control method was proposed based on the human-machine interface utilizing the electroencephalographic (EEG) signal. First, the EEG signals of the tractor drivers were collected by a low-cost brain-computer interface (BCI), followed by denoising using a wavelet packet. Then the spectral features of EEG signals were calculated and extracted as the input of Recurrent Neural Network (RNN). Finally, the EEG-aided RNN driving model was trained for tractor driving robot control such as straight going, brake, left turn, and right turn operations, which control accuracy was 94.5% and time cost was 0.61 ms. Also, 8 electrodes were selected by the PCA algorithm for the design of a portable EEG controller. And the control accuracy reached 93.1% with the time cost of 0.48 ms. To solve the incomplete driving data set in the actual world because some driving manners may cause dangerous or even death, RNN-TL algorithm was employed by creating the complete driving data in the virtual environment followed by transferring the driving control experience to the actual world with small actual driving data set in the field, which control accuracy was 93.5% and time consumption was 0.48 ms. The experimental results showed the feasibility of the proposed tractor driving control method based on EEG signal combined with RNN-TL deep learning algorithm which can work with the displacement error less than 6.7 mm when the tractor speed is less than 50 km/h.

[1]  João Paulo Papa,et al.  EEG-based Person Authentication Using Multi-objective Flower Pollination Algorithm , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[2]  Xiangwen Sun,et al.  Harmonic Frequency Estimation Based on Modified-MUSIC Algorithm in Power System , 2015 .

[3]  J. Vanderplas.  Understanding the Lomb–Scargle Periodogram , 2017, 1703.09824.

[4]  Ruoyu Du,et al.  Optimal Feature Selection and Deep Learning Ensembles Method for Emotion Recognition From Human Brain EEG Sensors , 2017, IEEE Access.

[5]  Mohammed Azmi Al-Betar,et al.  The effects of EEG feature extraction using multi-wavelet decomposition for mental tasks classification , 2019, ICICT '19.

[6]  T. Martin McGinnity,et al.  EEG-Based Mobile Robot Control Through an Adaptive Brain–Robot Interface , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[7]  Humera Farooq,et al.  Investigating EEG Patterns for Dual-Stimuli Induced Human Fear Emotional State , 2019, Sensors.

[8]  Marius Leordeanu,et al.  Reading into the mind's eye: Boosting automatic visual recognition with EEG signals , 2020, Neurocomputing.

[9]  Hyo Jong Lee,et al.  Exploration of Prominent Frequency Wave in EEG Signals from Brain Sensors Network , 2015, Int. J. Distributed Sens. Networks.

[10]  Ahamad Tajudin Khader,et al.  EEG Signals Denoising Using Optimal Wavelet Transform Hybridized With Efficient Metaheuristic Methods , 2020, IEEE Access.

[11]  Mohammed Azmi Al-Betar,et al.  EEG Feature Extraction for Person Identification Using Wavelet Decomposition and Multi-Objective Flower Pollination Algorithm , 2018, IEEE Access.

[12]  Wei-Dong Dang,et al.  Multiplex Limited Penetrable Horizontal Visibility Graph from EEG Signals for Driver Fatigue Detection , 2019, Int. J. Neural Syst..

[13]  Mousa K. Wali,et al.  Wavelet Packet Transform Based Driver Distraction Level Classification Using EEG , 2013 .

[14]  Sung Wook Baik,et al.  Multiple Electric Energy Consumption Forecasting Using a Cluster-Based Strategy for Transfer Learning in Smart Building , 2020, Sensors.

[15]  Osama Ahmad Alomari,et al.  Person identification using EEG channel selection with hybrid flower pollination algorithm , 2020, Pattern Recognit..

[16]  Gabriel Lodewijks,et al.  Detecting fatigue in car drivers and aircraft pilots by using non-invasive measures: The value of differentiation of sleepiness and mental fatigue. , 2020, Journal of safety research.

[17]  Zhen Li,et al.  Tractor path tracking control based on binocular vision , 2018 .

[18]  Maysam F. Abbod,et al.  Spectrum Analysis of EEG Signals Using CNN to Model Patient’s Consciousness Level Based on Anesthesiologists’ Experience , 2019, IEEE Access.

[19]  Soomaayeh Heysieattalab,et al.  Changes of the brain’s bioelectrical activity in cognition, consciousness, and some mental disorders , 2017, Medical journal of the Islamic Republic of Iran.

[20]  Henryk Palus,et al.  Prediction of Pilot's Reaction Time Based on EEG Signals , 2020, Frontiers in Neuroinformatics.

[21]  Kimihiko Nakano,et al.  A surface electromyography controlled steering assistance interface , 2019, Journal of Intelligent and Connected Vehicles.

[22]  Yisong Yue,et al.  Long-term Forecasting using Higher Order Tensor RNNs , 2017 .

[23]  Jakub Nalepa,et al.  Selecting training sets for support vector machines: a review , 2018, Artificial Intelligence Review.

[24]  M. Hallett,et al.  Prediction of human voluntary movement before it occurs , 2011, Clinical Neurophysiology.

[25]  Yasuharu Koike,et al.  Hybrid Control of a Vision-Guided Robot Arm by EOG, EMG, EEG Biosignals and Head Movement Acquired via a Consumer-Grade Wearable Device , 2016, IEEE Access.

[26]  Hongyu Zheng,et al.  Analysis of vehicle static steering torque based on tire–road contact patch sliding model and variable transmission ratio , 2016 .

[27]  William C. Stacey,et al.  Redaction of false high frequency oscillations due to muscle artifact improves specificity to epileptic tissue , 2019, Clinical Neurophysiology.

[28]  Lei Yang,et al.  A Human ECG Identification System Based on Ensemble Empirical Mode Decomposition , 2013, Sensors.

[29]  Mohammed Azmi Al-Betar,et al.  An Efficient Optimization Technique of EEG Decomposition for User Authentication System , 2018, 2018 2nd International Conference on BioSignal Analysis, Processing and Systems (ICBAPS).

[30]  Long Liu.  Recognition and Analysis of Motor Imagery EEG Signal Based on Improved BP Neural Network , 2019, IEEE Access.

[31]  Yong-Joo Kim,et al.  Path planning for autonomous lawn mower tractor , 2015 .

[32]  João Paulo Papa,et al.  Classification of EEG mental tasks using Multi-Objective Flower Pollination Algorithm for Person Identification , 2018 .

[33]  Yun-Hui Liu,et al.  Robust Path Following of the Tractor-Trailers System in GPS-Denied Environments , 2020, IEEE Robotics and Automation Letters.

[34]  Xun Chen,et al.  Classification of EEG Signals Using a Multiple Kernel Learning Support Vector Machine , 2014, Sensors.

[35]  Maria V. Sanchez-Vives,et al.  Quantification of Signal-to-Noise Ratio in Cerebral Cortex Recordings Using Flexible MEAs With Co-localized Platinum Black, Carbon Nanotubes, and Gold Electrodes , 2018, Front. Neurosci..

[36]  Krzysztof A. Cyran,et al.  A Machine Learning Approach to the Detection of Pilot's Reaction to Unexpected Events Based on EEG Signals , 2018, Comput. Intell. Neurosci..

[37]  Pradip Sircar,et al.  Automated emotion recognition based on higher order statistics and deep learning algorithm , 2020, Biomed. Signal Process. Control..

[38]  Jaime Gómez Gil,et al.  Steering a Tractor by Means of an EMG-Based Human-Machine Interface , 2011, Sensors.