Augmented Reality and the Internet of Things

This paper presents a novel smart eyewear that uses embedded photo reflect ve sensors and machine learni g to r cognize a wearer’s facial expressions in daily life. We leverage the skin deformation when wearers change their facial expressions. With small photo reflective sensors, we measure the proximity between the skin surface on a face and the eyewear frame where 17 sensors are integrated. A Support Vector Machine (SVM) algorithm was applied for the sensor informatio . The sensors can cover var ous facial muscle movements and can be integrated into everyday glasses. The main contributions of our work are as follows. (1) The eyewear recognizes eight facial expressions (92.8% accuracy for one time use and 78.1% for use on 3 different days). (2) It is designed and implemented considering social acceptability. The device looks like normal eyewear, so users can wear it a ytime, anywhere. (3) Initi l field trials in daily life were undertaken. Our work is one of the first attempts to recognize and evaluate a variety of facial expressions in th form of an unobtrusive wearable device. Author

[1]  Alexander Ilic,et al.  A lightweight user tracking method for app providers , 2016, Conf. Computing Frontiers.

[2]  Paul Lukowicz,et al.  Opportunistic human activity and context recognition , 2013, Computer.

[3]  Alexander Ilic,et al.  A Secure Shopping Experience Based on Blockchain and Beacon Technology , 2016, RecSys Posters.

[4]  Andreas Holzinger,et al.  Reasoning Under Uncertainty: Towards Collaborative Interactive Machine Learning , 2016, Machine Learning for Health Informatics.

[5]  Sébastien Gambs,et al.  Show me how you move and I will tell you who you are , 2010, SPRINGL '10.

[6]  Kai Kunze,et al.  Facial Expression Recognition in Daily Life by Embedded Photo Reflective Sensors on Smart Eyewear , 2016, IUI.

[7]  Vijay Kumar,et al.  An automated system for semantic object labeling with soft object recognition and dynamic programming segmentation , 2015, 2015 IEEE International Conference on Automation Science and Engineering (CASE).

[8]  Tieniu Tan,et al.  Affective Computing: A Review , 2005, ACII.

[9]  Kai Kunze,et al.  Estimation of english skill with a mobile eye tracker , 2016, UbiComp Adjunct.

[10]  Maria E. Burke,et al.  Low-cost Internet of Things digital technology adoption in SMEs , 2013 .

[11]  P. Hartvigsen.  The Computer for the 21st Century (1991) , 2014 .

[12]  Thomas B. Sheridan,et al.  Musings on Telepresence and Virtual Presence , 1992, Presence: Teleoperators & Virtual Environments.

[13]  Wijnand A. IJsselsteijn,et al.  Presence: concept, determinants, and measurement , 2000, Electronic Imaging.

[14]  Chris Arney.  Nudge: Improving Decisions about Health, Wealth, and Happiness , 2015 .

[15]  Felix Wortmann,et al.  Business Models and the Internet of Things , 2014 .

[16]  Alex Pentland,et al.  Enigma: Decentralized Computation Platform with Guaranteed Privacy , 2015, ArXiv.

[17]  Alexander Ilic,et al.  Reality-Mining with Smartphones: Detecting and Predicting Life Events based on App Installation Behavior , 2015, ICIS.

[18]  K. Strimbu,et al.  What are biomarkers? , 2010, Current opinion in HIV and AIDS.

[19]  Alexander Ilic,et al.  Product as a Service: Enabling Physical Products as Service End-Points , 2014, ICIS.

[20]  Manfred Tscheligi,et al.  Persuasive Technology , 2016, Lecture Notes in Computer Science.

[21]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[22]  Kai Kunze,et al.  Making Regular Eyeglasses Smart , 2015, IEEE Pervasive Computing.

[23]  Steve Benford,et al.  Ambiguity as a resource for design , 2003, CHI '03.

[24]  Tadayoshi Kohno,et al.  How to Safely Augment Reality: Challenges and Directions , 2016, HotMobile.

[25]  Elgar Fleisch,et al.  Understanding the impact of personality traits on mobile app adoption - Insights from a large-scale field study , 2016, Comput. Hum. Behav..

[26]  Jonathan B Freeman,et al.  MouseTracker: Software for studying real-time mental processing using a computer mouse-tracking method , 2010, Behavior research methods.

[27]  D. Kahneman,et al.  Attention and Effort , 1973 .

[28]  Thomas V. Wiecki,et al.  Eye tracking and pupillometry are indicators of dissociable latent decision processes. , 2014, Journal of experimental psychology. General.

[29]  J. P. Higgins,et al.  Smartphone Applications for Patients' Health and Fitness. , 2016, The American journal of medicine.

[30]  Brian P. Bailey,et al.  On the need for attention-aware systems: Measuring effects of interruption on task performance, error rate, and affective state , 2006, Comput. Hum. Behav..

[31]  Elgar Fleisch,et al.  WHAT IS THE INTERNET OF THINGS? AN ECONOMIC PERSPECTIVE , 2010 .

[32]  David B. Kaber,et al.  Telepresence , 1998, Hum. Factors.

[33]  Thomas B. Sheridan,et al.  Recollections on Presence Beginnings, and Some Challenges for Augmented and Virtual Reality , 2016, PRESENCE: Teleoperators and Virtual Environments.

[34]  Henrik I. Christensen,et al.  "My Roomba Is Rambo": Intimate Home Appliances , 2007, UbiComp.