Enhancing education through natural interaction with physical paper

Abstract Pervasive computing environments have permeated current research and practice, unobtrusively augmenting existing environments with digital content. The present work, following a pervasive computing approach, proposes a framework to augment an educational environment, being a typical classroom or any studying environment. In this context, the work presented in this paper investigates unobtrusive interaction and support of active educational or studying activities through appropriate context-sensitive information. To this end, passive visual sensing is employed in order to unobtrusively perceive the current context and users’ actions, thus providing novel ways to implement natural interaction. The suitability of the proposed interaction technologies and overall approach has been demonstrated through three interactive applications integrated in the framework, each one supporting different interaction techniques and addressing different educational activities. Finally, a user experience evaluation of the three test-bed applications has been carried out, aiming to assess the applicability of the approach and the suitability of each of the proposed technologies to the educational tasks in hand.

[1]  A. Bus,et al.  How Well Suited are Electronic Books to Supporting Literacy? , 2003 .

[2]  Constantine Stephanidis,et al.  A Student-Centric Intelligent Classroom , 2011, AmI.

[3]  John Krumm,et al.  Ubiquitous Computing Fundamentals , 2009 .

[4]  Cheol-Hoon Lee,et al.  Live Book: A mixed reality book using a projection system , 2012, 2012 IEEE International Conference on Consumer Electronics (ICCE).

[5]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[6]  Constantine Stephanidis,et al.  Ambient educational mini-games , 2012, AVI.

[7]  Antonis A. Argyros,et al.  PaperView: augmenting physical surfaces with location-aware digital information , 2010, TEI.

[8]  Sriram Subramanian,et al.  Talking about tactile experiences , 2013, CHI.

[9]  James D. Hollan,et al.  Papiercraft: A gesture-based command system for interactive paper , 2008, TCHI.

[10]  Mark Billinghurst,et al.  The mixed reality book: a new multimedia reading experience , 2007, CHI Extended Abstracts.

[11]  Constantine Stephanidis,et al.  The book of Ellie: An interactive book for teaching the alphabet to children , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[12]  Pierre David Wellner,et al.  Interacting with paper on the DigitalDesk , 1993, CACM.

[13]  X. Zabulis,et al.  Augmenting physical books towards education enhancement , 2013, 2013 1st IEEE Workshop on User-Centered Computer Vision (UCCV).

[14]  Albrecht Schmidt,et al.  The drift table: designing for ludic engagement , 2004, CHI EA '04.

[15]  Tom Drummond,et al.  Localisation and interaction for augmented maps , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[16]  Constantine Stephanidis,et al.  Ambient Intelligence in the Classroom: an Augmented School Desk , 2010 .

[17]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[18]  Beat Signer,et al.  Only touching the surface: creating affinities between digital content and paper , 2004, CSCW.

[19]  Diane J. Cook,et al.  How smart are our environments? An updated look at the state of the art , 2007, Pervasive Mob. Comput..

[20]  Haris Baltzakis,et al.  Multiview 3D Pose Estimation of a Wand for Human-Computer Interaction , 2011, ISVC.

[21]  Gregory D. Abowd,et al.  What next, ubicomp?: celebrating an intellectual disappearing act , 2012, UbiComp.

[22]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[23]  Carrie Spencer,et al.  Research on Learners' Preferences for Reading from a Printed Text or from a Computer Screen. , 2006 .

[24]  Kenton O'Hara,et al.  A comparison of reading paper and on-line documents , 1997, CHI.

[25]  Marc Dymetman,et al.  Intelligent Paper , 1998, Annual Conference on Evolutionary Programming.

[26]  J. Cooperstock.  The Classroom of the Future : Enhancing Education through Augmented Reality , 2001 .

[27]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[28]  Andrew D. Wilson.  PlayAnywhere: a compact interactive tabletop projection-vision system , 2005, UIST.

[29]  Ivan Poupyrev,et al.  MagicBook: transitioning between reality and virtuality , 2001, CHI Extended Abstracts.

[30]  Francisco Abad,et al.  Using an Augmented Reality game to find matching pairs , 2011 .

[31]  Shih-Fu Chang,et al.  VisualSEEk: a fully automated content-based image query system , 1997, MULTIMEDIA '96.

[32]  Chris Harrison,et al.  OmniTouch: wearable multitouch interaction everywhere , 2011, UIST.

[33]  Constantine Stephanidis,et al.  Augmented interaction with physical books in an Ambient Intelligence learning environment , 2013, Multimedia Tools and Applications.

[34]  Joseph J. LaViola,et al.  ErgoDesk: A Framework for Two- and Three-Dimensional Interaction at the ActiveDesk , 1998 .

[35]  Stefan Winkler,et al.  A Tangible Game Interface Using Projector-Camera Systems , 2007, HCI.

[36]  Norbert A. Streitz,et al.  Roomware: Toward the Next Generation of Human- Computer Interaction Based on an Integrated Design of Real and Virtual Worlds , 2001 .

[37]  Brian P. Bailey,et al.  Build your world and play in it: Interacting with surface particles on complex objects , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[38]  Jun Rekimoto,et al.  SmartSkin: an infrastructure for freehand manipulation on interactive surfaces , 2002, CHI.

[39]  Tan-Hsu Tan,et al.  2D Barcode and Augmented Reality Supported English Learning System , 2007, 6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007).

[40]  David Goldberg,et al.  Touch-typing with a stylus , 1993, INTERCHI.

[41]  A. J. Bernheim Brush.  Ubiquitous Computing Field Studies , 2010, Ubicomp 2010.

[42]  Ivan Poupyrev,et al.  The MagicBook - Moving Seamlessly between Reality and Virtuality , 2001, IEEE Computer Graphics and Applications.

[43]  Jefferson Y. Han.  Low-cost multi-touch sensing through frustrated total internal reflection , 2005, UIST.

[44]  Andrew D. Wilson.  Using a depth camera as a touch sensor , 2010, ITS '10.

[45]  Claudio S. Pinhanez.  Using a steerable projector and a camera to transform surfaces into interactive displays , 2001, CHI Extended Abstracts.

[46]  Dieter Schmalstieg,et al.  The Studierstube Augmented Reality Project , 2002, Presence: Teleoperators & Virtual Environments.

[47]  A. Conchon,et al.  Applications and services , 2001 .

[48]  Xenophon Zabulis,et al.  Augmented Multitouch Interaction upon a 2-DOF Rotating Disk , 2012, ISVC.

[49]  T. J. Reynolds,et al.  Laddering theory, method, analysis, and interpretation. , 2001 .

[50]  Constantine Stephanidis,et al.  Towards Ambient Intelligence in the Classroom , 2011, HCI.

[51]  Arthur Gelb,et al.  Applied Optimal Estimation , 1974 .

[52]  Hideki Koike,et al.  EnhancedDesk: integrating paper documents and digital documents , 1998, Proceedings. 3rd Asia Pacific Computer Human Interaction (Cat. No.98EX110).

[53]  Constantine Stephanidis,et al.  A Glimpse into the Ambient Classroom , 2012 .

[54]  F. Bookstein,et al.  Morphometric Tools for Landmark Data: Geometry and Biology , 1999 .

[55]  Claudio S. Pinhanez,et al.  Interacting with steerable projected displays , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[56]  Hannes Kaufmann,et al.  Collaborative Augmented Reality in Education , 2002 .

[57]  Fred L. Bookstein,et al.  Morphometric Tools for Landmark Data. , 1998 .

[58]  Tom Gross,et al.  The cuetable: cooperative and competitive multi-touch interaction on a tabletop , 2008, CHI Extended Abstracts.

[59]  Darren Leigh,et al.  DiamondTouch: a multi-user touch technology , 2001, UIST '01.

[60]  Maiga Chang,et al.  Designing a Trading Card Game as Educational Reward System to Improve Students' Learning Motivations , 2009, Edutainment.

[61]  Florian Michahelles,et al.  SwissPeaks-Mobile augmented reality to identify mountains , 2009 .

[62]  Brett E. Shelton,et al.  Using augmented reality for teaching Earth-Sun relationships to undergraduate geography students , 2002, The First IEEE International Workshop Agumented Reality Toolkit,.

[63]  Mahadev Satyanarayanan,et al.  Pervasive computing: vision and challenges , 2001, IEEE Wirel. Commun..