BiDi screen: a thin, depth-sensing LCD for 3D interaction using light fields

An LCD screen is transformed into a BiDi (bidirectional) screen to support 2D multi-touch and walk-up 3D gesture interaction. An image sensor placed a small distance behind an LCD forms a mask-based light-field camera, allowing passive depth estimation. The BiDi screen also supports novel mixed-reality rendering with external light-emitting widgets that light a virtual scene.

[1]  Dan Rosenfeld,et al.  Going beyond the display: a surface technology with an electronically switchable diffuser , 2008, UIST '08.

[2]  E E Fenimore,et al.  New family of binary arrays for coded aperture imaging. , 1989, Applied optics.

[3]  Steven K. Feiner,et al.  Cross-dimensional gestural interaction techniques for hybrid immersive environments , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[4]  T. M. Cannon,et al.  Coded aperture imaging with uniformly redundant arrays. , 1978, Applied optics.

[5]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[6]  Shree K. Nayar,et al.  Shape from Focus , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Kenneth C. Smith,et al.  A multi-touch three dimensional touch-sensitive tablet , 1985, CHI '85.

[8]  Hans-Peter Seidel,et al.  Towards passive 6D reflectance field displays , 2008, ACM Trans. Graph..

[9]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, ACM Trans. Graph..

[10]  Douglas Lanman,et al.  Shield fields: modeling and capturing 3D occluders , 2008, ACM Trans. Graph..

[11]  Hrvoje Benko,et al.  Using Depth-Sensing Camera to Enable Freehand Interactions On and Above the Interactive Surface , 2008 .

[12]  Andrew D. Wilson.  TouchLight: an imaging touch screen and display for gesture-based interaction , 2004, ICMI '04.

[13]  Tom E. Bishop,et al.  Light field superresolution , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[14]  W. Daniel Hillis,et al.  A High-Resolution Imaging Touch Sensor , 1982 .

[15]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[16]  Shree K. Nayar,et al.  Rational Filters for Passive Depth from Defocus , 1998, International Journal of Computer Vision.

[17]  Chia-Kai Liang,et al.  Programmable aperture photography: multiplexed light field acquisition , 2008, SIGGRAPH 2008.

[18]  Jefferson Y. Han.  Low-cost multi-touch sensing through frustrated total internal reflection , 2005, UIST.

[19]  H. Kato,et al.  A Continuous-Grain Silicon-System LCD With Optical Input Function , 2007, IEEE Journal of Solid-State Circuits.

[20]  Shahzad Malik,et al.  Visual touchpad: a two-handed gestural input device , 2004, ICMI '04.

[21]  Alan Hedge,et al.  Multi-Touch: A New Tactile 2-D Gesture Interface for Human-Computer Interaction , 2001 .

[22]  Marc Levoy,et al.  Reconstructing Occluded Surfaces Using Synthetic Apertures: Stereo, Focus and Robust Measures , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[23]  Shree K. Nayar,et al.  Lighting sensitive display , 2004, ACM Trans. Graph..

[24]  Andrew Lumsdaine,et al.  The focused plenoptic camera , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[25]  Tsuhan Chen,et al.  Light field capturing with lensless cameras , 2005, IEEE International Conference on Image Processing 2005.

[26]  E. Hecht.  Optics 4th Edition , 1998 .

[27]  Clifton Forlines,et al.  DTLens: multi-user tabletop spatial data exploration , 2005, UIST.

[28]  Shree K. Nayar,et al.  Light field transfer: global illumination between real and synthetic objects , 2008, ACM Trans. Graph..

[29]  Douglas Lanman,et al.  BiDi screen: a thin, depth-sensing LCD for 3D interaction using light fields , 2009, SIGGRAPH 2009.

[30]  Jun Rekimoto,et al.  HoloWall: designing a finger, hand, body, and object sensitive wall , 1997, UIST '97.

[31]  Jun Rekimoto,et al.  SmartSkin: an infrastructure for freehand manipulation on interactive surfaces , 2002, CHI.

[32]  Ren Ng.  Fourier slice photography , 2005, ACM Trans. Graph..

[33]  Shree K. Nayar,et al.  Lensless Imaging with a Controllable Aperture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[34]  Darren Leigh,et al.  DiamondTouch: a multi-user touch technology , 2001, UIST '01.

[35]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[36]  William Buxton,et al.  ThinSight: integrated optical multi-touch sensing through thin form-factor displays , 2007, EDT '07.

[37]  Eero P. Simoncelli,et al.  Range estimation by optical differentiation. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.