Less is more! Support of Parallel and Time-critical Assembly Tasks with Augmented Reality

Manual assembly tasks require workers to precisely assemble parts in 3D space in parallel steps. Often, additional time pressure further increases the complexity of these parallel tasks (e.g. in adhesive bonding processes). The performance in parallel tasks is heavily influenced by the capacity of the working memory, which is often overwhelmed if rules and states of too many tasks have to be remembered or perceived. Therefore, we propose to use Augmented Reality (AR) to investigate how visual assistance with parallel tasks can affect worker performance in time and spatial dependent process steps. In a user study, we compare three conditions: AR instructions presented (a) One task, (b) Two tasks, and (c) Four tasks per step on a tablet. For instructions we used selected work steps from a standardized adhesive bonding process as a representative for common time-critical assembly tasks. Our results show that instructions with multiple displayed tasks simultaneously per step can improve the process time but also increase the error rate and task load. The work instructions with less displayed tasks per work step showed better subjective results among participants, which may increase motivation, especially among less experienced workers. Our results help designers and developers to design assistance systems for time-critical and simultaneously executable assembly tasks, while considering process times, error rate and task load.

[1]  Anna Syberfeldt,et al.  Support Systems on the Industrial Shop-floors of the Future – Operators’ Perspective on Augmented Reality☆ , 2016 .

[2]  Sule Serubugo,et al.  Comparison of Wearable Optical See-through and Handheld Devices as Platform for an Augmented Reality Museum Guide , 2017, VISIGRAPP.

[3]  H. Pashler The Psychology of Attention , 1997 .

[4]  Melynda Hoover,et al.  An evaluation of the Microsoft HoloLens for a manufacturing-guided assembly task , 2018 .

[5]  Gordon D. Logan,et al.  Parallel memory retrieval in dual-task situations: II. Episodic memory. , 2001 .

[6]  David E. Breen,et al.  Annotating Real-World Objects Using Augmented Reality , 1995, Computer Graphics.

[7]  Tao Yang,et al.  Eye-Wearable Technology for Machine Maintenance: Effects of Display Position and Hands-free Operation , 2015, CHI.

[8]  Adam Gazzaley,et al.  Neural Suppression of Irrelevant Information Underlies Optimal Working Memory Performance , 2009, The Journal of Neuroscience.

[9]  Dario D. Salvucci,et al.  Threaded cognition: an integrated theory of concurrent multitasking. , 2008, Psychological review.

[10]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[11]  P. Jolicoeur,et al.  A central capacity sharing model of dual-task performance. , 2003, Journal of experimental psychology. Human perception and performance.

[12]  Eliot Winer,et al.  Comparing Visual Assembly Aids for Augmented Reality Work Instructions , 2017 .

[13]  Max Pfingsthorn,et al.  Time is money! Evaluating Augmented Reality Instructions for Time-Critical Assembly Tasks , 2020, MUM.

[14]  Kiyoshi Kiyokawa,et al.  The effectiveness of an AR-based context-aware assembly support system in object assembly , 2014, 2014 IEEE Virtual Reality (VR).

[15]  Yann Laurillau,et al.  Tabletop AR with HMD and Tablet: A Comparative Study for 3D Selection , 2019, ISS.

[16]  Eleonora Bottani,et al.  Augmented reality technology in the manufacturing industry: A review of the last decade , 2019, IISE Trans..

[17]  M. Sigman,et al.  Dynamics of the Central Bottleneck: Dual-Task and Task Uncertainty , 2006, PLoS biology.

[18]  Rico Fischer,et al.  Efficient multitasking: parallel versus serial processing of multiple tasks , 2015, Front. Psychol..

[19]  Junfeng Wang,et al.  Research on the visual elements of augmented reality assembly processes , 2019, Virtual Real. Intell. Hardw..

[20]  H. Pashler Dual-task interference in simple tasks: data and theory. , 1994, Psychological bulletin.

[21]  A. Welford THE ‘PSYCHOLOGICAL REFRACTORY PERIOD’ AND THE TIMING OF HIGH‐SPEED PERFORMANCE—A REVIEW AND A THEORY , 1952 .

[22]  Jessie Y. C. Chen,et al.  Virtual, Augmented and Mixed Reality: Applications in Health, Cultural Heritage, and Industry , 2018, Lecture Notes in Computer Science.

[23]  Frank Biocca,et al.  Comparative effectiveness of augmented reality in object assembly , 2003, CHI '03.

[24]  Daniel G Bobrow,et al.  On data-limited and resource-limited processes , 1975, Cognitive Psychology.

[25]  Torsten Schubert,et al.  Evidence for parallel semantic memory retrieval in dual tasks , 2007, Memory & cognition.

[26]  Wolfgang Prinz,et al.  Process interference and code overlap in dual-task performance , 2002 .

[27]  Eliot Winer,et al.  Augmented reality via expert demonstration authoring (AREDA) , 2019, Comput. Ind..

[28]  Keith Case,et al.  Experimental study of cognitive aspects affecting human performance in manual assembly , 2017 .

[29]  Timo Engelke,et al.  Controlling and Filtering Information Density with Spatial Interaction Techniques via Handheld Augmented Reality , 2013, HCI.

[30]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[31]  Harold Bekkering,et al.  Dissociating restart cost and mixing cost in task switching , 2009, Psychological research.

[32]  Siam Charoenseang,et al.  Augmented reality for skill transfer in assembly task , 2005, ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005..

[33]  Steve Bourgeois,et al.  An interactive Augmented Reality system: A prototype for industrial maintenance training applications , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[34]  B. Hommel Automatic stimulus-response translation in dual-task performance. , 1998, Journal of experimental psychology. Human perception and performance.

[35]  James H. Oliver,et al.  Augmented Reality-Based Manual Assembly Support With Visual Features for Different Degrees of Difficulty , 2015, Int. J. Hum. Comput. Interact..

[36]  Paulo Dias,et al.  Comparing Spatial and Mobile Augmented Reality for Guiding Assembling Procedures with Task Validation , 2019, 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC).

[37]  Michael J. Daly,et al.  Surgeons blinded by enhanced navigation: the effect of augmented reality on attention , 2013, Surgical Endoscopy.

[38]  Markus Funk,et al.  Interactive worker assistance: comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions , 2016, UbiComp.

[39]  D. Navon,et al.  Role of outcome conflict in dual-task interference. , 1987, Journal of experimental psychology. Human perception and performance.

[40]  J. Duncan Divided attention: the whole is more than the sum of its parts. , 1979, Journal of experimental psychology. Human perception and performance.

[41]  S. T. Klapp,et al.  The bottleneck of the psychological refractory period effect involves timing of response initiation rather than response selection , 2018, Psychonomic Bulletin & Review.

[42]  N. Meiran,et al.  On the origins of the task mixing cost in the cuing task-switching paradigm. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[43]  A. Elliot,et al.  Is red the colour of danger? Testing an implicit red–danger association , 2014, Ergonomics.

[44]  D. Kahneman,et al.  Attention and Effort , 1973 .

[45]  Thies Pfeiffer,et al.  Attention guiding techniques using peripheral vision and eye tracking for feedback in augmented-reality-based assistance systems , 2017, 2017 IEEE Symposium on 3D User Interfaces (3DUI).

[46]  C. L. M. The Psychology of Attention , 1890, Nature.

[47]  Timo Engelke,et al.  Augmented Reality Views: Discussing the Utility of Visual Elements by Mediation Means in Industrial AR from a Design Perspective , 2018, HCI.

[48]  Henry Been-Lirn Duh,et al.  Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[49]  Andrea Kiesel,et al.  New perspectives on human multitasking , 2018, Psychological Research.

[50]  Thies Pfeiffer,et al.  Comparing Conventional and Augmented Reality Instructions for Manual Assembly Tasks , 2017, PETRA.

[51]  Jeff Miller,et al.  Backward crosstalk effects in psychological refractory period paradigms: effects of second-task response types on first-task response latencies , 2006, Psychological research.

[52]  Adam Gazzaley,et al.  Practice-related improvement in working memory is modulated by changes in processing external interference. , 2009, Journal of neurophysiology.

[53]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[54]  Steven K. Feiner,et al.  Information filtering for mobile augmented reality , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[55]  D. Navon,et al.  Queuing or Sharing? A Critical Evaluation of the Single-Bottleneck Notion , 2002, Cognitive Psychology.

[56]  Ronald Hübner,et al.  Strategic capacity sharing between two tasks: evidence from tasks with the same and with different task sets , 2009, Psychological research.

[57]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[58]  Andrea Kiesel,et al.  Cognitive Structure, Flexibility, and Plasticity in Human Multitasking—An Integrative Review of Dual-Task and Task-Switching Research , 2018, Psychological bulletin.

[59]  Markus A. Maier,et al.  Color and psychological functioning: the effect of red on performance attainment. , 2007, Journal of experimental psychology. General.

[60]  N. Kanwisher,et al.  Functional Magnetic Resonance Imaging Provides New Constraints on Theories of the Psychological Refractory Period , 2004, Psychological science.

[61]  G D Logan,et al.  Parallel memory retrieval in dual-task situations: I. Semantic memory. , 2000, Journal of experimental psychology. Human perception and performance.

[62]  J. Benders,et al.  Head-worn displays and job content: A systematic literature review. , 2020, Applied ergonomics.

[63]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[64]  Xiangyu Wang,et al.  Using Augmented Reality to Facilitate Piping Assembly: An Experiment-Based Evaluation , 2015, J. Comput. Civ. Eng..