The effects of augmented reality on improving spatial problem solving for object assembly

Abstract The capability of Augmented Reality (AR) technology to track and visualize relations of objects in space has led to diverse industry applications to support complex engineering tasks. Object assembly is one of them. For an AR aid to support Ready-to-Assemble (RTA) furniture particularly, the challenge is to effectively design the visual features and mode of interaction, so that the first-time users can quickly conceive spatial relations of its parts. However, AR developers and engineers do not have sufficient guidelines to achieve such performance-driven goals. The scientific evidence and account of how one could cognitively benefit in object assembly can be useful to guide them. This experimental research developed an Augmented Reality (AR) application on the Microsoft HoloLens™ headset, and tested it on the first-time users of RTA furniture. The controlled experiments and behavioral analyses of fourteen participants in working out the two RTA furniture with different assembly complexity showed that, the application was effective to improve spatial problem-solving abilities. Especially, the positive effects of the AR-based supports stood out against the assembly of higher complexity. The findings have implications for the information design of advanced AR applications to support assembly tasks with high demands of spatial knowledge. In addition, the extensive analyses of the participants’ performance, behaviors, cognitive workload, and subjective responses helped identify usability issues with the Microsoft HoloLens™.

[1]  M. Hegarty,et al.  A dissociation between mental rotation and perspective-taking spatial abilities , 2004 .

[2]  Juval Portugali,et al.  Synergetics, Inter-Representation Networks and Cognitive Maps , 1996 .

[3]  Martin Helander,et al.  Design for Human Assembly (DHA) , 2003 .

[4]  Mark Torrance,et al.  Identifying the Task Variables That Predict Object Assembly Difficulty , 2006, Hum. Factors.

[5]  Timo Hartmann,et al.  Implementing information systems with project teams using ethnographic-action research , 2009, Adv. Eng. Informatics.

[6]  H. J. Whitford,et al.  How to Use the Two Sample t‐Test , 1986 .

[7]  David B. Kaber,et al.  The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task , 2004 .

[8]  Holger Regenbrecht,et al.  Augmented reality projects in the automotive and aerospace industries , 2005, IEEE Computer Graphics and Applications.

[9]  Gordon Baxter,et al.  Human factors and ergonomics in consumer product design: methods and techniques , 2012, Ergonomics.

[10]  Edgar Erdfelder,et al.  GPOWER: A general power analysis program , 1996 .

[11]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[12]  Steven K. Feiner,et al.  Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair , 2011, IEEE Transactions on Visualization and Computer Graphics.

[13]  A. Paivio.  Mental Representations: A Dual Coding Approach , 1986 .

[14]  Hitendra K. Pillay,et al.  Cognitive load and assembly tasks : effect of instructional formats on learning assembly procedures , 1997 .

[15]  Aloke Dey,et al.  Optimal Crossover Designs , 2009 .

[16]  Phillip S. Dunston,et al.  Identification of application areas for Augmented Reality in industrial construction based on technology suitability , 2008 .

[17]  David M. Lane,et al.  Effects of academic instruction on spatial visualization , 1980 .

[18]  U. Neisser.  Cognition and reality: principles and implications , 1976 .

[19]  Brett E. Shelton,et al.  Exploring a Cognitive Basis for Learning Spatial Relationships with Augmented Reality , 2003 .

[20]  Jaideep Motwani,et al.  Improving competitiveness of ready‐to‐assemble manufacturers through information technology , 2000 .

[21]  J. Geoffrey Chase,et al.  Human-Robot Collaboration: A Literature Review and Augmented Reality Approach in Design , 2008 .

[22]  Joseph L. Gabbard,et al.  Behind the Glass: Driver Challenges and Opportunities for AR Automotive Applications , 2014, Proceedings of the IEEE.

[23]  S. Kita,et al.  The nature of gestures ' beneficial role in spatial problem solving , 2013 .

[24]  Valerie L. Shalin,et al.  A cognitive perspective on manual assembly , 1996 .

[25]  Delinda van Garderen,et al.  Spatial Visualization, Visual Imagery, and Mathematical Problem Solving of Students With Varying Abilities , 2006 .

[26]  Mary A. LaRussa,et al.  A STUDY OF TWO MEASURES OF SPATIAL ABILITY AS PREDICTORS OF SUCCESS IN DIFFERENT LEVELS OF GENERAL CHEMISTRY , 1987 .

[27]  F. Rousset,et al.  AN EXACT TEST FOR POPULATION DIFFERENTIATION , 1995, Evolution; international journal of organic evolution.

[28]  David H. Jonassen,et al.  Toward a design theory of problem solving , 2000 .

[29]  R. Wood.  Task complexity: Definition of the construct , 1986 .

[30]  F. Paas,et al.  Instructional control of cognitive load in the training of complex cognitive tasks , 1994 .

[31]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[32]  Alex Morgan,et al.  Representations gone mental , 2013, Synthese.

[33]  M. Hegarty,et al.  Individual Differences in Spatial Abilities , 2005 .

[34]  Antonija Mitrovic,et al.  Intelligent Augmented Reality Training for Motherboard Assembly , 2015, International Journal of Artificial Intelligence in Education.

[35]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[36]  S. Olkun.  Making connections: improving spatial abilities with engineering drawing activities , 2003 .

[37]  Woodrow Barfield,et al.  Boeing’s Wire Bundle Assembly Project , 2001 .

[38]  Paco Calvo,et al.  Augmented reality: An ecological blend , 2017, Cognitive Systems Research.

[39]  Alvin I. Goldman,et al.  The Bodily Formats Approach to Embodied Cognition , 2013 .

[40]  Juval Portugali.  INTER-REPRESENTATION NETWORKS AND COGNITIVE MAPS , 1996 .

[41]  Hsiu-Fang Hsieh,et al.  Three Approaches to Qualitative Content Analysis , 2005, Qualitative health research.

[42]  D. W. F. van Krevelen,et al.  A Survey of Augmented Reality Technologies, Applications and Limitations , 2010, Int. J. Virtual Real..

[43]  Irwin N. Jankovic,et al.  Principles of spatial problem solving. , 1982 .

[44]  Clark C. Presson,et al.  Orientation specificity in spatial memory: what makes a path different from a map of the path? , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[45]  David Benyon,et al.  Presence in blended spaces , 2012, Interact. Comput..

[46]  D. Lohman.  Spatial Ability: A Review and Reanalysis of the Correlational Literature. , 1979 .

[47]  John B. Carroll,et al.  PSYCHOMETRIC TESTS AS COGNITIVE TASKS: A NEW "STRUCTURE OF INTELLECT" , 1974 .

[48]  Richard T. Houang,et al.  The Effect of Instruction on Spatial Visualization Skills of Middle School Boys and Girls , 1988 .

[49]  Didier Stricker,et al.  Augmented reality for construction tasks: doorlock assembly , 1999 .

[50]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[51]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .