User-interfaces layout optimization using eye-tracking, mouse movements and genetic algorithms.

Establishing the best layout configuration for software-generated interfaces and control panels is a complex problem when they include many controls and indicators. Several methods have been developed for arranging the interface elements; however, the results are usually conceptual designs that must be manually adjusted to obtain layouts valid for real situations. Based on these considerations, in this work we propose a new automatized procedure to obtain optimal layouts for software-based interfaces. Eye-tracking and mouse-tracking data collected during the use of the interface is used to obtain the best configuration for its elements. The solutions are generated using a slicing-trees based genetic algorithm. This algorithm is able to obtain really applicable configurations that respect the geometrical restrictions of elements in the interface. Results show that this procedure increases effectiveness, efficiency and satisfaction of the users when they interact with the obtained interfaces.

[1]  Srdjan Kovacevic,et al.  UIDE—an intelligent user interface design environment , 1991 .

[2]  Christopher D. Wickens,et al.  Effects of Data-Link Modality and Display Redundancy on Pilot Performance: An Attentional Perspective , 2003 .

[3]  S Raeisi,et al.  Ergonomic Redesign of an Industrial Control Panel , 2016, The international journal of occupational and environmental medicine.

[4]  Jose Antonio Diego-Mas,et al.  Slicing tree's geometric potential: an indicator for layout problems based on slicing tree structure , 2008 .

[5]  Mickaël Causse,et al.  Embedded eye tracker in a real aircraft: new perspectives on pilot/aircraft interaction monitoring , 2008 .

[6]  Cornell Juliano,et al.  The hunt for usability: tracking eye movements , 1999, CHI Extended Abstracts.

[7]  H Ujike,et al.  Serine racemase binds to PICK1: potential relevance to schizophrenia , 2006, Molecular Psychiatry.

[8]  Nursel Öztürk,et al.  Two-stage optimisation method for material flow and allocation management in cross-docking networks , 2017, Int. J. Prod. Res..

[9]  Kris Luyten,et al.  Familiarisation: Restructuring Layouts with Visual Learning Models , 2018, IUI.

[10]  Andrew Kusiak,et al.  The facility layout problem , 1987 .

[11]  T Stewart,et al.  Ergonomics standards concerning human-system interaction: visual displays, controls and environmental requirements. , 1995, Applied ergonomics.

[12]  Michel Wedel,et al.  Improving Ad Interfaces with Eye Tracking , 2017 .

[13]  Jacek Gwizdka,et al.  What Can Searching Behavior Tell Us About the Difficulty of Information Tasks? A Study of Web Navigation , 2007, ASIST.

[14]  Daniel S. Weld,et al.  Exploring the design space for adaptive graphical user interfaces , 2006, AVI '06.

[15]  Friedhelm Nachreiner,et al.  Human factors in process control systems: The design of human–machine interfaces ☆ , 2006 .

[16]  Kenneth Holmqvist,et al.  Eye tracking: a comprehensive guide to methods and measures , 2011 .

[17]  Jose Antonio Diego-Mas,et al.  Using RGB-D sensors and evolutionary algorithms for the optimization of workstation layouts. , 2017, Applied ergonomics.

[18]  Christopher D. Wickens,et al.  The Proximity Compatibility Principle: Its Psychological Foundation and Relevance to Display Design , 1995, Hum. Factors.

[19]  James D. Foley,et al.  DON: user interface presentation design assistant , 1990, UIST '90.

[20]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Karl H.E. Kroemer,et al.  Ergonomics: How to Design for Ease and Efficiency , 1993 .

[22]  Fred W. Glover,et al.  Multistart Tabu Search and Diversification Strategies for the Quadratic Assignment Problem , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[23]  Alphonse Chapanis,et al.  Human factors in systems engineering , 1996 .

[24]  Sheau-Farn Max Liang,et al.  Evaluation of human factors in interface design in main control rooms , 2009 .

[25]  Abdullah Konak,et al.  A large-scale hybrid simulated annealing algorithm for cyclic facility layout problems , 2015 .

[26]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[27]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[28]  C. Wickens,et al.  Applied Attention Theory , 2007 .

[29]  Antti Oulasvirta,et al.  Improvements to keyboard optimization with integer programming , 2014, UIST.

[30]  Myriam Arrue,et al.  Automatic Generation of Tailored Accessible User Interfaces for Ubiquitous Services , 2015, IEEE Transactions on Human-Machine Systems.

[31]  Nigan Bayazit,et al.  An ergonomics based design research method for the arrangement of helicopter flight instrument panels. , 2015, Applied ergonomics.

[32]  M. Adel El-Baz,et al.  A genetic algorithm for facility layout problems of different manufacturing environments , 2004, Comput. Ind. Eng..

[33]  Peter Sedlmeier,et al.  Comparing eye trackers by correlating their eye-metric data , 2017, Behavior Research Methods.

[34]  Anna L. Cox,et al.  The Role of Mouse Movements in Interactive Search , 2006 .

[35]  Ignace T. C. Hooge,et al.  The area-of-interest problem in eyetracking research: A noise-robust solution for face and sparse stimuli , 2016, Behavior research methods.

[36]  Christopher D. Wickens,et al.  Pilot Maneuver Choice and Workload in Free Flight , 2002, Hum. Factors.

[37]  Jacek Gwizdka,et al.  Predicting task difficulty for different task types , 2010, ASIST.

[38]  K. Y. Tam,et al.  Genetic algorithms, function optimization, and facility layout design , 1992 .

[39]  Kenneth C. Scott-Brown,et al.  Looking for trouble: a description of oculomotor search strategies during live CCTV operation , 2013, Front. Hum. Neurosci..

[40]  Joseph H. Goldberg,et al.  Computer interface evaluation using eye movements: methods and constructs , 1999 .

[41]  B. Freisleben,et al.  A comparison of memetic algorithms, tabu search, and ant colonies for the quadratic assignment problem , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[42]  Honghai Liu,et al.  Visual Focus of Attention Estimation Using Eye Center Localization , 2017, IEEE Systems Journal.

[43]  B B Morgan,et al.  Engineering Psychology and Human Performance , 1976 .

[44]  Antti Oulasvirta,et al.  User Interface Design with Combinatorial Optimization , 2017, Computer.

[45]  J. Diego-Mas,et al.  Solving facility layout problems with strict geometric constraints using a two-phase genetic algorithm , 2009 .

[46]  Philippe Averty,et al.  Analysis of Mental Workload during En-route Air Traffic Control Task Execution Based on Eye-Tracking Technique , 2011, HCI.

[47]  Andrea Lockerd Thomaz,et al.  Cheese: tracking mouse movement activity on websites, a tool for user modeling , 2001, CHI Extended Abstracts.

[48]  Kasper Hornbæk,et al.  Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? , 2000, CHI.

[49]  Hasan Hosseini-Nasab,et al.  A hybrid particle swarm optimisation for dynamic facility layout problem , 2013 .

[50]  Christopher D. Wickens,et al.  An introduction to human factors engineering , 1997 .

[51]  Mark S. Sanders,et al.  Human Factors in Engineering and Design , 2016 .

[52]  Thomas Stützle,et al.  Tabu search vs. simulated annealing as a function of the size of quadratic assignment problem instances , 2014, Comput. Oper. Res..

[53]  Antti Oulasvirta.  Optimizing User Interfaces for Human Performance , 2017, IHCI.

[54]  Jeffrey Nichols,et al.  Mobilization by demonstration: using traces to re-author existing web sites , 2008, IUI '08.

[55]  Yili Liu,et al.  Development and evaluation of an ergonomic software package for predicting multiple-task human performance and mental workload in human-machine interface design and evaluation , 2009, Comput. Ind. Eng..

[56]  Barry H. Kantowitz,et al.  Human Factors: Understanding People-System Relationships , 1983 .

[57]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[58]  Tim Horberry,et al.  Controls and displays , 2010 .

[59]  Alan R. McKendall,et al.  New Tabu search heuristics for the dynamic facility layout problem , 2012 .

[60]  Changxu Wu,et al.  Improved link analysis method for user interface design – modified link table and optimisation-based algorithm , 2010, Behav. Inf. Technol..

[61]  Chun Hung Cheng,et al.  Recent advances in dynamic facility layout research , 2018, INFOR Inf. Syst. Oper. Res..

[62]  Jeffrey Nichols,et al.  UNIFORM: automatically generating consistent remote control user interfaces , 2006, CHI.

[63]  Gregory Francis,et al.  Designing Multifunction Displays: An Optimization Approach , 2000 .

[64]  Chris Baber,et al.  Visual Sampling in A Road Traffic Management Control Room Task , 2015 .

[65]  Jeonghyun Kim,et al.  Task difficulty as a predictor and indicator of web searching interaction , 2006, CHI Extended Abstracts.

[66]  Marcos R. S. Borges,et al.  Human factors approach for evaluation and redesign of human-system interfaces of a nuclear power plant simulator , 2008, Displays.

[67]  Krzysztof Z. Gajos,et al.  Automatically generating custom user interfaces for users with physical disabilities , 2006, Assets '06.

[68]  Brian J. Carnahan,et al.  Using Linear Programming to Optimize Control Panel Design from an Ergonomics Perspective , 2003 .

[69]  Lalit M. Patnaik,et al.  Genetic algorithms: a survey , 1994, Computer.

[70]  Asaf Degani,et al.  “Soft” Controls for Hard Displays: Still a Challenge , 1992 .

[71]  Ryen W. White,et al.  No clicks, no problem: using cursor movements to understand and improve search , 2011, CHI.

[72]  Michael Burch,et al.  State-of-the-Art of Visualization for Eye Tracking Data , 2014, EuroVis.

[73]  Umut Rifat Tuzkaya,et al.  A particle swarm optimization algorithm for the multiple-level warehouse layout design problem , 2008, Comput. Ind. Eng..