A low-cost 3D human interface device using GPU-based optical flow algorithms

Except for a few cases, nowadays it is very common to find a camera embedded in a consumer grade laptop, notebook, mobile internet device MID, mobile phone or handheld game console. Some of them also have a Graphic Processing Unit GPU to handle 3D graphics and other related tasks. This trend will probably continue in the next future and the pair camera+GPU will be more and more frequent in the market. Because of this, the proposal of this work is to use these resources in order to build a low-cost software-based 3D Human Interface Device 3D HID able to run in this kind of devices, in real time without degrading the overall performance. This is achieved implementing a parallel version of an existing Optical Flow Algorithm that runs fully in the GPU without using it at full power. In this way, usual graphic processes coexist with Optical Flow computations. To the best of author's knowledge, this approach a software-based 3D HID that runs fully in a GPU is not found in academic research nor in commercial products prototypes. Indeed, this is the salient contribution of this paper. The performance of the proposal is good enough to achieve real time in low grade computers.

[1]  Alberto Del Bimbo,et al.  A Robust Algorithm for Optical Flow Estimation , 1995, Comput. Vis. Image Underst..

[2]  John D. Owens,et al.  GPU Computing , 2008, Proceedings of the IEEE.

[3]  Jörn Loviscach.  Playing with All Senses: Human-Computer Interface Devices for Games , 2009, Adv. Comput..

[4]  Alexander Zelinsky,et al.  Visual gesture interfaces for virtual environments , 2002, Interact. Comput..

[5]  Unai Bidarte,et al.  Hardware implementation of optical flow constraint equation using FPGAs , 2005, Comput. Vis. Image Underst..

[6]  Janne Heikkilä,et al.  Vision-based motion estimation for interaction with mobile devices , 2007, Comput. Vis. Image Underst..

[7]  Seong-Whan Lee,et al.  Qualitative estimation of camera motion parameters from the linear composition of optical flow , 2004, Pattern Recognit..

[8]  Cristina Manresa-Yee,et al.  User experience to improve the usability of a vision-based interface , 2010, Interact. Comput..

[9]  Carlos A. Pomalaza-Raez,et al.  On-chip body posture detection for medical care applications using low-cost CMOS cameras , 2010, Integr. Comput. Aided Eng..

[10]  Gamini Dissanayake,et al.  Optical flow based analyses to detect emotion from human facial image data , 2010, Expert Syst. Appl..

[11]  David W. Murray,et al.  Experiments in the machine interpretation of visual motion , 1990 .

[12]  Javier Díaz,et al.  Superpipelined high-performance optical-flow computation architecture , 2008, Comput. Vis. Image Underst..

[13]  Peter D. Lawrence,et al.  A non-contact device for tracking gaze in a human computer interface , 2005, Comput. Vis. Image Underst..

[14]  James Demmel,et al.  Benchmarking GPUs to tune dense linear algebra , 2008, HiPC 2008.

[15]  Brian G. Schunck,et al.  Image Flow Segmentation and Estimation by Constraint Line Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[17]  Andy C. Downton,et al.  Evaluating optical-flow algorithms on a parallel machine , 2001, Image Vis. Comput..

[18]  Luciano Sánchez,et al.  3D motion estimation of bubbles of gas in fluid glass, using an optical flow gradient technique extended to a third dimension , 2003, Machine Vision and Applications.

[19]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[20]  Chern-Sheng Lin,et al.  A novel device for head gesture measurement system in combination with eye-controlled human-machine interface , 2006 .

[21]  Luc Van Gool,et al.  Range determination for mobile robots using an omnidirectional camera , 2006, Integr. Comput. Aided Eng..

[22]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[23]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[24]  J. Otero,et al.  Mode based hierarchical optical flow estimation , 2001 .

[25]  Jae Wook Jeon,et al.  Readjusting Unstable Regions to Improve the Quality of High Accuracy Optical Flow , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Xavier Armangué,et al.  A review on egomotion by means of differential epipolar geometry applied to the movement of a mobile robot , 2003, Pattern Recognit..

[27]  David J. Heeger,et al.  Optical flow using spatiotemporal filters , 2004, International Journal of Computer Vision.

[28]  Klaus H. Hinrichs,et al.  A multi-touch enabled human-transporter metaphor for virtual 3D traveling , 2010, 2010 IEEE Symposium on 3D User Interfaces (3DUI).