Affordances for Capturing and Re-enacting Expert Performance with Wearables

The WEKIT.one prototype is a platform for immersive procedural training with wearable sensors and Augmented Reality. Focusing on capture and re-enactment of human expertise, this work looks at the unique affordances of suitable hard- and software technologies. The practical challenges of interpreting expertise, using suitable sensors for its capture and specifying the means to describe and display to the novice are of central significance here. We link affordances with hardware devices, discussing their alternatives, including Microsoft Hololens, Thalmic Labs MYO, Alex Posture sensor, MyndPlay EEG headband, and a heart rate sensor. Following the selection of sensors, we describe integration and communication requirements for the prototype. We close with thoughts on the wider possibilities for implementation and next steps.

[1]  Limin Sun,et al.  Performance monitoring and evaluation in dance teaching with mobile sensing technology , 2014, Personal and Ubiquitous Computing.

[2]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[3]  Mikhail Fominykh,et al.  An Overview of Capturing Live Experience with Virtual and Augmented Reality , 2015, Intelligent Environments.

[4]  Michael Rohs,et al.  On-skin technologies for muscle sensing and actuation , 2016, UbiComp Adjunct.

[5]  Jorge Bacca,et al.  Augmented Reality Trends in Education: A Systematic Review of Research and Applications , 2014, J. Educ. Technol. Soc..

[6]  Sebastian Bodenstedt,et al.  Development and validation of a sensor- and expert model-based training system for laparoscopic surgery: the iSurgeon , 2017, Surgical Endoscopy.

[7]  Fridolin Wild,et al.  Requirement analysis and sensor specifications – First version , 2016 .

[8]  Fengfeng Ke,et al.  Teaching training in a mixed-reality integrated learning environment , 2016, Comput. Hum. Behav..

[9]  Giuseppe Sartori,et al.  Attention Training with an Easy-to-Use Brain Computer Interface , 2014, HCI.

[10]  Woontack Woo,et al.  AiRSculpt: A Wearable Augmented Reality 3D Sculpting System , 2014, HCI.

[11]  Matt Bower,et al.  What are the educational affordances of wearable technologies? , 2015, Comput. Educ..

[12]  Robert J. Sternberg,et al.  Practical intelligence in real-world pursuits: The role of tacit knowledge. , 1985 .

[13]  Fridolin Wild,et al.  Supporting Training of Expertise with Wearable Technologies: The WEKIT Reference Framework , 2018 .

[14]  Antonio Rizzo.  The origin and design of intentional affordances , 2006, DIS '06.

[15]  J. J. Gibson.  The theory of affordances , 1977 .

[16]  Heng Li,et al.  Proactive training system for safe and efficient precast installation , 2015 .

[17]  R. Shaw,et al.  Perceiving, Acting and Knowing : Toward an Ecological Psychology , 1978 .

[18]  M. Yao,et al.  Comparison of the performance of experienced and novice surgeons: measurement of gripping force during laparoscopic surgery performed on pigs using forceps with pressure sensors , 2017, Surgical Endoscopy.

[19]  Rui Rodrigues,et al.  Natural User Interfaces in the Motor Development of Disabled Children , 2014 .

[20]  Anind K. Dey,et al.  Augmenting human senses to improve the user experience in cars: applying augmented reality and haptics approaches to reduce cognitive distances , 2015, Multimedia Tools and Applications.

[21]  K. Ericsson,et al.  Prospects and limits of the empirical study of expertise: an introduction , 1991 .

[22]  Vinayak Ashok Prabhu,et al.  Digitisation of manual composite layup task knowledge using gaming technology , 2017 .

[23]  Filippo Sanfilippo,et al.  A multi-sensor fusion framework for improving situational awareness in demanding maritime training , 2017, Reliab. Eng. Syst. Saf..

[24]  Ross E. Willis,et al.  Is current surgery resident and GI fellow training adequate to pass FES? , 2016, Surgical Endoscopy.

[25]  Kurt Debattista,et al.  Visuohaptic augmented feedback for enhancing motor skills acquisition , 2017, The Visual Computer.