Application of hidden Markov models to eye tracking data analysis of visual quality inspection operations

Visual inspection is used in many areas due to the potential high costs of inspection error such as injury, fatality, loss of expensive equipment, scrapped items, rework, or failure to procure repeat business. This study presents an application of hidden Markov models (HMM) to fixations’ sequences analysis during visual inspection of front panels in a home appliance facility. The eye tracking data are gathered when quality control operators perform their tasks. The results support the difference between expert and novice operator. Moreover, the article demonstrates four HMMs with two and three hidden states both for novice and experienced operators and provides analysis and discussion of the outcomes.

[1]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[2]  Neil D. B. Bruce,et al.  Predicting task from eye movements: On the importance of spatial distribution, dynamics, and image features , 2016, Neurocomputing.

[3]  Rafał Michalski,et al.  Information presentation compatibility in a simple digital control panel design: eye-tracking study , 2018, International journal of occupational safety and ergonomics : JOSE.

[4]  Colin G. Drury,et al.  Training strategies for visual inspection , 1997 .

[5]  J. Findlay,et al.  Active Vision: The Psychology of Looking and Seeing , 2003 .

[6]  Ann Speed.  Visual Search in Operational Environments - Balancing Operational Constraints with Experimental Control , 2015, HCI.

[7]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[8]  Keith S. Karn,et al.  Commentary on Section 4. Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. , 2003 .

[9]  Kerry Hourigan,et al.  Wake transition of a rolling sphere , 2011, J. Vis..

[10]  Berna Haktanirlar Ulutas,et al.  D7-3 From the Point of a Tower Crane Operator's View: Use of an Eye-Tracker in Construction Sites , 2017 .

[11]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[12]  Guang-Zhong Yang,et al.  A hidden markov model-based analysis framework using eye-tracking data to characterise re-orientation strategies in minimally invasive surgery , 2010, Cognitive Processing.

[13]  Berna Ulutas,et al.  Assessing visual control activities in ceramic tile surface defect detection: an eye-tracking study , 2017, Int. J. Intell. Eng. Informatics.

[14]  Judi E. See,et al.  Visual inspection : a review of the literature. , 2012 .

[15]  Vittorio Rampa,et al.  Leveraging RF signals for human sensing: Fall detection and localization in human-machine shared workspaces , 2015, 2015 IEEE 13th International Conference on Industrial Informatics (INDIN).

[16]  M. Gucma,et al.  Application of eye-tracking techniques in human factor research in marine operations. Challenges and methodology , 2013 .

[17]  Tim Chuk,et al.  Understanding eye movements in face recognition using hidden Markov models. , 2014, Journal of vision.

[18]  Colin G. Drury,et al.  The Role of Visual Inspection in the 21st Century , 2017 .

[19]  Michelle R. Greene,et al.  Reconsidering Yarbus: A failure to predict observers’ task from eye movement patterns , 2012, Vision Research.

[20]  C. Eriksen,et al.  Visual attention within and around the field of focal attention: A zoom lens model , 1986, Perception & psychophysics.

[21]  Jerzy Grobelny,et al.  Applying Hidden Markov Models to Visual Activity Analysis for Simple Digital Control Panel Operations , 2016, ISAT.

[22]  Miwa Hayashi.  Hidden Markov Models to identify pilot instrument scanning and attention patterns , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[23]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[24]  S R Ellis,et al.  Statistical Dependency in Visual Scanning , 1986, Human factors.

[25]  Kai Puolamäki,et al.  Inferring Intent and Action from Gaze in Naturalistic Behavior: A Review , 2017, Int. J. Mob. Hum. Comput. Interact..

[26]  M. Posner,et al.  Attention and the detection of signals. , 1980, Journal of experimental psychology.

[27]  Ilpo Kojo,et al.  Using hidden Markov model to uncover processing states from eye movements in information search tasks , 2008, Cognitive Systems Research.

[28]  D. Ballard,et al.  Eye guidance in natural vision: reinterpreting salience. , 2011, Journal of vision.

[29]  Jerzy Grobelny,et al.  An Eye Tracking Based Examination of Visual Attention During Pairwise Comparisons of a Digital Product's Package , 2016, HCI.

[30]  Michel Wedel,et al.  Global and local covert visual attention: Evidence from a bayesian hidden markov model , 2003 .

[31]  Judi E. See,et al.  Visual Inspection Reliability for Precision Manufactured Parts , 2015, Hum. Factors.

[32]  Dmitry Kit,et al.  Classifying mobile eye tracking data with hidden Markov models , 2016, MobileHCI Adjunct.

[33]  C W Stiles.  PROPOSITIONS FOR CHANGES IN THE INTERNATIONAL CODE OF ZOOLOGICAL NOMENCLATURE. , 1912, Science.

[34]  Debasish Biswas,et al.  Visualization of unsteady viscous flow around turbine blade , 2008, J. Vis..

[35]  Antoine Coutrot,et al.  Scanpath modeling and classification with hidden Markov models , 2017, Behavior Research Methods.

[36]  T. J. Gallwey,et al.  Selection tests for visual inspection on a multiple fault type task , 1982 .