Interactive Display System based on Adaptive Image Projection to a Deformable Tangible Screen

We propose a variant of the multi-touch display technology that introduces an original way of manipulating three-dimensional data. The underlying metaphor is that of a deformable screen that acts as a boundary surface between the real and the virtual worlds. By doing so, the interface can create the illusion of continuity between the user’s real space and the virtual three-dimensional space. The developed system presented here enables this by employing three key technologies: a tangible and deformable projection screen, a real-time three-dimensional sensing mechanism, and an algorithm for dynamic compensation for anamorphic projection. This paper introduces the concept of the deformable tangible workspace, and describes the required technologies for implementing it. Also, several applications developed on a prototype system are detailed and demonstrated.

[1]  William Ribarsky,et al.  Evaluation of the effects of frame time variation on VR task performance , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[2]  Greg Welch,et al.  The office of the future: a unified approach to image-based modeling and spatially immersive displays , 1998, SIGGRAPH.

[3]  Hiroaki Nishino,et al.  Making 3D objects through bimanual actions , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[4]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Naokazu Yokoya,et al.  SeamlessDesign for 3D Object Creation , 2000, IEEE Multim..

[6]  Peter Schröder,et al.  Surface drawing: creating organic 3D shapes with the hand and tangible tools , 2001, CHI.

[7]  Jun Rekimoto,et al.  SmartSkin: an infrastructure for freehand manipulation on interactive surfaces , 2002, CHI.

[8]  Gerard Jounghyun Kim,et al.  Modeling virtual object behavior within virtual environment , 2002, VRST '02.

[9]  Elsayed E. Hemayed,et al.  A survey of camera self-calibration , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[10]  Y Wang,et al.  Bringing Clay and Sand into Digital Design — Continuous Tangible user Interfaces , 2004 .

[11]  Zachary Booth Simpson Novel infrared touch-screen technology and associated artwork , 2004, SIGGRAPH '04.

[12]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[13]  Masatoshi Ishikawa,et al.  Khronos projector , 2005, SIGGRAPH '05.

[14]  Masatoshi Ishikawa,et al.  955-fps Real-time Shape Measurement of a Moving/Deforming Object using High-speed Vision for Numerous-point Analysis , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[15]  Koichi Hirota,et al.  Cross-section Projector: Interactive and Intuitive Presentation of 3D Volume Data using a Handheld Screen , 2007, 2007 IEEE Symposium on 3D User Interfaces.

[16]  David Kim,et al.  Creating malleable interactive surfaces using liquid displacement sensing , 2008, 2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems.

[17]  Masatoshi Ishikawa,et al.  The deformable workspace: A membrane between real and virtual space , 2008, 2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems.

[18]  Jonathan Hook,et al.  A reconfigurable ferromagnetic input device , 2009, UIST '09.

[19]  Philip L. Davidson,et al.  A screen-space formulation for 2D and 3D direct manipulation , 2009, UIST '09.

[20]  Ravin Balakrishnan,et al.  EverybodyLovesSketch: 3D sketching for a broader audience , 2009, UIST '09.