Detecting cognitive hacking in visual inspection with physiological measurements.

Cyber threats are targeting vulnerabilities of human workers performing tasks in manufacturing processes, including visual inspection to bias their decision-making, thereby sabotaging product quality. This article examines the use of priming as a form of "cognitive hacking" to adversely affect quality inspection decisions in manufacturing, and investigates physiological measurements as means to detect such intrusion. In a within-subject design experiment, twenty participants inspected surface roughness of a manufactured component with and without exposure to priming on the display of an inspection logging system. The results show that the presence of primes impacted accuracy on surface roughness, cortical activities at parietal lobe P4, and eye gaze for inspecting components. The experiment provides supporting evidence that basic hacking of a worker display can be an effective method to alter decision making in inspection. The findings also illustrate that cortical activities and eye gaze can be useful indicators of cognitive hacking. A major implication of the study results is that physiological indicators can be effective at revealing unconscious cognitive influence in visual inspection.

[1]  K. Forster,et al.  Masked priming with graphemically related forms: Repetition or partial activation? , 1987 .

[2]  C. G. Drury,et al.  An Industrial Study of the Effects of Feedback and Fault Density on Inspection Performance , 1973 .

[3]  Rolf Zon,et al.  Eye Movements as an Indicator of Situation Awareness in a Flight Simulator Experiment , 2012 .

[4]  Robert Oostenveld,et al.  Enhanced EEG gamma-band activity reflects multisensory semantic matching in visual-to-auditory object priming , 2008, NeuroImage.

[5]  J. Grainger,et al.  Priming word recognition with orthographic neighbors: effects of relative prime-target frequency. , 1990 .

[6]  C G Drury,et al.  Inspection of Sheet Materials — Model and Data , 1975, Human factors.

[7]  T. Chartrand,et al.  The mind in the middle: A practical guide to priming and automaticity research. , 2000 .

[8]  R. Hague *,et al.  Material and design considerations for rapid manufacturing , 2004 .

[9]  Glyn W. Humphreys,et al.  Event perception and the word repetition effect , 1988 .

[10]  Jules White,et al.  Bad Parts: Are Our Manufacturing Systems at Risk of Silent Cyberattacks? , 2015, IEEE Security & Privacy.

[11]  Anil Mital,et al.  Manual, Hybrid and Automated Inspection Literature and Current Research , 1993 .

[12]  Gernot G. Supp,et al.  Oscillatory MEG gamma band activity dissociates perceptual and conceptual aspects of visual object processing: A combined repetition/conceptual priming study , 2012, NeuroImage.

[13]  Claude Ghaoui,et al.  Encyclopedia of Human Computer Interaction , 2005 .

[14]  J. Bargh,et al.  Automaticity of social behavior: direct effects of trait construct and stereotype-activation on action. , 1996, Journal of personality and social psychology.

[15]  Reena Master,et al.  Measurement of human trust in a hybrid inspection system based on signal detection theory measures , 2004 .

[16]  B. Hallowell,et al.  Validity of eye movement methods and indices for capturing semantic (associative) priming effects. , 2009, Journal of speech, language, and hearing research : JSLHR.

[17]  J. Doyle,et al.  Lateral specialization of cognitive mode: II. EEG frequency analysis. , 1974, Psychophysiology.

[18]  Glenn F. Wilson,et al.  Real-Time Assessment of Mental Workload Using Psychophysiological Measures and Artificial Neural Networks , 2003, Hum. Factors.

[19]  K. Nakayama,et al.  Priming of popout: III. A short-term implicit memory system beneficial for rapid target selection , 2000 .

[20]  Y. Lin,et al.  Using eye movement parameters for evaluating human-machine interface frameworks under normal control operation and fault detection situations , 2003, Int. J. Hum. Comput. Stud..

[21]  Christopher C. Yang,et al.  Spatial quantization errors in active vision inspection , 1994, Proceedings of IEEE International Conference on Systems, Man and Cybernetics.

[22]  Kilseop Ryu,et al.  Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic , 2005 .

[23]  Mohsen A. Jafari,et al.  Signature analysis and defect detection in layered manufacturing of ceramic sensors and actuators , 2003, Machine Vision and Applications.

[24]  Daniel L. Schacter,et al.  Priming effects in word-fragment completion are independent of recognition memory. , 1982 .

[25]  R. Zajonc,et al.  Affect, cognition, and awareness: affective priming with optimal and suboptimal stimulus exposures. , 1993, Journal of personality and social psychology.

[26]  I. Hooge,et al.  Goal-directed visual attention drives health goal priming: An eye-tracking experiment , 2016, Appetite.

[27]  Stefano Federici,et al.  Gaze and eye-tracking solutions for psychological research , 2012, Cognitive Processing.

[28]  J. Hoffmann,et al.  Conscious control over the content of unconscious cognition , 2003, Cognition.

[29]  C G Drury,et al.  Inspection of Sheet Materials—Test of Model Predictions , 1978, Human factors.

[30]  M. Damian Congruity effects evoked by subliminally presented primes: automaticity rather than semantic processing. , 2001 .

[31]  Colin G. Drury Human Factors and Automation in Test and Inspection , 2007 .

[32]  Prasad V. Prabhu,et al.  A review of human error in aviation maintenance and inspection , 2000 .

[33]  Shinsuke Shimojo,et al.  Sound-induced illusory flash perception: role of gamma band responses , 2002, Neuroreport.

[34]  J. Rueckl,et al.  Similarity effects in word and pseudoword repetition priming. , 1990, Journal of experimental psychology. Learning, memory, and cognition.

[35]  V. Mountcastle,et al.  Parietal lobe mechanisms for directed visual attention. , 1977, Journal of neurophysiology.

[36]  William Bradley Glisson,et al.  Implications of Malicious 3D Printer Firmware , 2017, HICSS.

[37]  Anand K. Gramopadhye,et al.  Use of eye movements as feedforward training for a synthetic aircraft inspection task , 2005, CHI.

[38]  Matthias M. Müller,et al.  Modulation of oscillatory brain activity and evoked potentials in a repetition priming task in the human EEG , 2004, The European journal of neuroscience.

[39]  Jia Liu,et al.  Online Real-Time Quality Monitoring in Additive Manufacturing Processes Using Heterogeneous Sensors , 2015 .

[40]  Ran Jin,et al.  Functional Quantitative and Qualitative Models for Quality Modeling in a Fused Deposition Modeling Process , 2018, IEEE Transactions on Automation Science and Engineering.

[41]  S. Dehaene,et al.  Levels of processing during non-conscious perception: a critical review of visual masking , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[42]  David Z. Zhang,et al.  Additive manufacturing: A framework for implementation , 2014 .

[43]  Colin G. Drury,et al.  Training strategies for visual inspection , 1997 .

[44]  E D Megaw,et al.  Factors affecting visual inspection accuracy. , 1979, Applied ergonomics.

[45]  Jules White,et al.  Cyber-physical security challenges in manufacturing systems , 2014 .

[46]  Atsushi Matsumoto,et al.  Subliminal Semantic Priming Changes the Dynamic Causal Influence between the Left Frontal and Temporal Cortex , 2014, Journal of Cognitive Neuroscience.

[47]  Reinhold Kliegl,et al.  Semantic preview benefit in eye movements during reading: A parafoveal fast-priming study. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[48]  John D. Gould,et al.  Field of View and Target Uncertainty in Visual Search and Inspection , 1973 .

[49]  Anand K. Gramopadhye,et al.  A comparison of three levels of training designed to promote systematic search behavior in visual inspection , 2003 .

[50]  Mark P. Zanna,et al.  Subliminal priming and persuasion: Striking while the iron is hot , 2002 .

[51]  Päivi Majaranta,et al.  Eye Tracking and Eye-Based Human–Computer Interaction , 2014 .

[52]  Paul H.P. Yeow,et al.  Ergonomics Improvements of the Visual Inspection Process in a Printed Circuit Assembly Factory , 2004, International journal of occupational safety and ergonomics : JOSE.

[53]  B. Lowery,et al.  Long-term Effects of Subliminal Priming on Academic Performance , 2007 .

[54]  W. Klimesch EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[55]  Anand K. Gramopadhye,et al.  Evaluation of best system performance: Human, automated, and hybrid inspection systems , 2003 .

[56]  R. Yerkes,et al.  The relation of strength of stimulus to rapidity of habit‐formation , 1908 .

[57]  Stephan Konz,et al.  The Effect of Lighting and Low Power Magnification on Inspection Performance , 1978 .

[58]  Steven L. Johnson,et al.  An Analysis of Human Reliability Measures in Visual Inspection , 1980 .

[59]  Louise Venables,et al.  The influence of task demand and learning on the psychophysiological response. , 2005, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[60]  Chongxun Zheng,et al.  Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. , 2012, Accident; analysis and prevention.

[61]  Anthony Skjellum,et al.  Using 3D printers as weapons , 2016, Int. J. Crit. Infrastructure Prot..

[62]  Maxim Bazhenov,et al.  Perceptual priming leads to reduction of gamma frequency oscillations , 2010, Proceedings of the National Academy of Sciences.

[63]  Glenn F. Wilson,et al.  Operator Functional State Classification Using Multiple Psychophysiological Features in an Air Traffic Control Task , 2003, Hum. Factors.

[64]  Ran Jin,et al.  Statistical modeling for visualization evaluation through data fusion. , 2017, Applied ergonomics.

[65]  Judi E. See,et al.  Visual inspection : a review of the literature. , 2012 .

[66]  Michael E. Smith,et al.  Monitoring Task Loading with Multivariate EEG Measures during Complex Forms of Human-Computer Interaction , 2001, Hum. Factors.

[67]  S. Slobounov,et al.  Neurophysiological and behavioral indices of time pressure effects on visuomotor task performance. , 2000, Brain research. Cognitive brain research.

[68]  Ran Jin,et al.  Quality Modeling of Printed Electronics in Aerosol Jet Printing Based on Microscopic Images , 2017 .

[69]  Douglas H. Harris,et al.  The Nature of Industrial Inspection1 , 1969 .

[70]  J B Poline,et al.  Cerebral mechanisms of word masking and unconscious repetition priming , 2001, Nature Neuroscience.

[71]  Lisa D. Ordóñez,et al.  Conscience without cognition: The effects of subconscious priming on ethical behavior , 2014 .

[72]  D. Salisbury,et al.  Semantic priming increases left hemisphere theta power and intertrial phase synchrony. , 2012, Psychophysiology.

[73]  S. Grand,et al.  Recovery in the absence of recall: an investigation of color-word interference. , 1966, Journal of experimental psychology.

[74]  Jim Nixon,et al.  Measuring mental workload using physiological measures: A systematic review. , 2019, Applied ergonomics.

[75]  Maciej Tomczak,et al.  The need to report effect size estimates revisited. An overview of some recommended measures of effect size , 2014 .

[76]  M. Hautus Corrections for extreme proportions and their biasing effects on estimated values ofd′ , 1995 .

[77]  Agnieszka Kujawińska,et al.  Human factors in visual quality control , 2015 .

[78]  Jochen Kaiser,et al.  Human γ-band activity and behavior. , 2011, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[79]  K. Forster,et al.  REPETITION PRIMING AND FREQUENCY ATTENUATION IN LEXICAL ACCESS , 1984 .

[80]  Helen L Jamison,et al.  Morphology and the internal structure of words. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[81]  O. Bock,et al.  The effects of priming with age stereotypes on a PC-based mail-sorting task , 2017, Ergonomics.

[82]  Colin G. Drury,et al.  Training Programs for Inspection , 1981 .

[83]  Glenn F. Wilson,et al.  Performance Enhancement in an Uninhabited Air Vehicle Task Using Psychophysiologically Determined Adaptive Aiding , 2007, Hum. Factors.

[84]  Tung-Hsu Hou,et al.  An empirical study of hybrid inspection systems and allocation of inspection functions , 1993 .

[85]  Douglas C. Schmidt,et al.  Taxonomies for Reasoning About Cyber-physical Attacks in IoT-based Manufacturing Systems , 2017, Int. J. Interact. Multim. Artif. Intell..

[86]  S. Dehaene,et al.  Imaging unconscious semantic priming , 1998, Nature.

[87]  Martin Eimer,et al.  Masked prime stimuli can bias “free” choices between response alternatives , 2004, Psychonomic bulletin & review.

[88]  T. Sejnowski,et al.  Early Cross-Modal Interactions in Auditory and Visual Cortex Underlie a Sound-Induced Visual Illusion , 2007, The Journal of Neuroscience.

[89]  George Cybenko,et al.  Cognitive Hacking: A Battle for the Mind , 2002, Computer.

[90]  M. Eimer,et al.  Effects of masked stimuli on motor activation: behavioral and electrophysiological evidence. , 1998, Journal of experimental psychology. Human perception and performance.

[91]  H Stanislaw,et al.  Calculation of signal detection theory measures , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[92]  Anil Mital,et al.  A comparison between manual and hybrid methods in parts inspection , 1998 .

[93]  Michael E. Smith,et al.  Monitoring Working Memory Load during Computer-Based Tasks with EEG Pattern Recognition Methods , 1998, Hum. Factors.