Detecting Human Trust Calibration in Automation: A Deep Learning Approach

INTRODUCTION Complicated automated systems are misused and disused by humans, even though the systems provide probable advantages by aiding them in satisfying their tasks (Lee & See, 2004). The misuse (over-reliance) or disuse (under-reliance) of automation was considered as trust of people to automation. For instance, the automated systems tend to be used by people who trust the system than those who do not trust that. More importantly, misuse and disuse of automation often occur when trust of operator in automation is poorly adjusted (Merrit & Ilgen, 2008). Trust calibration is the case by which the operator’s trust of automated system toward automation is adjusted based on the physical system performance and capabilities (McGuirl & Sarter, 2006). Several factors that influence trust calibration have been found, including reliability (Dzindolet et al., 2003), automation errors (Wiegmann et al., 2001), individual differences in the causal attribution of automation errors (Pop et al., 2015), and task difficulty (Madhavan et al., 2006). However, there is still a lack of studies that monitor and detect the trust adjustment of an operator in an automated system by a general quantitative model during their task. There are several reasons for this. First, studies have revealed that changed trust of an operator towards automation can be found (e.g., Dzindolet et al., 2003; Madhavan et al., 2006; Pop et al., 2015; Wiegmann et al., 2001), the adjusted trust was decided only after the task is finished. Second, several models related with prediction of human trust have been suggested mainly using discrete, self-reported behavioral measurements (e.g., Jonker & Treur, 1999) and continuous cognitive signals (e.g., Riedl & Javor, 2012). Neural correlates of trust between humans (e.g., Krueger et al., 2007) and between human and machine (e.g., Desmet et al., 2014; Goodyear et al., 2017) were also studied. But only a few studies investigated neural correlates of trust calibration in automation, in particular using electroencephalograms (EEGs) that have better temporal dynamics than other neuroimaging methods (e.g., fMRI) (e.g., de Visser et al., 2018). Finally, Deep Learning (DL) has been widely applied in many challenging domains owing to its higher success in many applications than existing machine learning. For example, Convolutional Neural Networks (CNN) has been showing promising performance in neuroergonomics fields (e.g., mental workload (Zhang et al., 2017), motor imagery (Yang et al., 2015)). However, there was almost no research on developing an EEG-based DL method that can detect an operator’s trust calibration in automation dynamically. One of the main goals of the present study was to propose a CNN-based framework to detect human trust calibration in automation using image features of EEG preserving spectral and spatial information. The effectiveness of the proposed framework was validated against a Multi-Layer Perceptron (MLP) and Deep Neural Network (DNN).