A Light-Powered Smart Camera With Compressed Domain Gesture Detection

This paper presents an ultralow power smart camera with gesture detection. Low power is achieved by directly extracting gesture features from the compressed measurements, which are the block averages and the linear combinations of the image sensor’s pixel values. We present two classifier techniques to allow low computational and storage requirements. The system has been implemented on an analog devices BlackFin ULP vision processor. By enabling ultralow energy consumption, we demonstrate that the system is powered by ambient light harvested through photovoltaic cells whose output is regulated by TI’s dc–dc buck converter with maximum power point tracking. Measured data reveals that with only 400 compressed measurements (<inline-formula> <tex-math notation="LaTeX">$768\times $ </tex-math></inline-formula> compression ratio) per frame, the system is able to recognize key wake-up gestures with greater than 80% accuracy and only <inline-formula> <tex-math notation="LaTeX">$95mJ$ </tex-math></inline-formula> of energy per frame. Owing to its fully self-powered operation, the proposed system can find wide applications in “always-on” vision systems, such as in surveillance, robotics, and consumer electronics with touch-less operation.

[1]  Fan Zhang,et al.  A Batteryless 19 $\mu$W MICS/ISM-Band Energy Harvesting Body Sensor Node SoC for ExG Applications , 2013, IEEE Journal of Solid-State Circuits.

[2]  Joan Climent,et al.  A Performance Evaluation of HMM and DTW for Gesture Recognition , 2012, CIARP.

[3]  Yu Shi,et al.  An FPGA-Based Smart Camera for Gesture Recognition in HCI Applications , 2007, ACCV.

[4]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Wang Ran,et al.  Real-time visual static hand gesture recognition system and its FPGA-based hardware implementation , 2014, 2014 12th International Conference on Signal Processing (ICSP).

[6]  Shahrokh Valaee,et al.  Accelerometer-based gesture recognition via dynamic-time warping, affinity propagation, & compressive sensing , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  ChaoTang Li,et al.  A Novel FPGA-based Hand Gesture Recognition System , 2012 .

[8]  Bing Luo,et al.  A real-time dynamic hand gesture recognition system using kinect sensor , 2015, 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[9]  Richard G. Baraniuk,et al.  Random Projections of Smooth Manifolds , 2009, Found. Comput. Math..

[10]  Shyamnath Gollakota,et al.  Bringing Gesture Recognition to All Devices , 2014, NSDI.

[11]  Ling Shao,et al.  Learning Discriminative Representations from RGB-D Video Data , 2013, IJCAI.

[12]  Arijit Raychowdhury,et al.  Appearance-based gesture recognition in the compressed domain , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Arijit Raychowdhury,et al.  A SAR Pipeline ADC Embedding Time Interleaved DAC Sharing for Ultra-low Power Camera Front Ends , 2015, VLSI-SoC.

[14]  M. Reinders,et al.  Multi-Dimensional Dynamic Time Warping for Gesture Recognition , 2007 .

[15]  Pierre Gançarski,et al.  A global averaging method for dynamic time warping, with applications to clustering , 2011, Pattern Recognit..

[16]  Justin K. Romberg,et al.  Compressive Matched-Field Processing , 2012, The Journal of the Acoustical Society of America.

[17]  Richard G. Baraniuk,et al.  The smashed filter for compressive classification and target recognition , 2007, Electronic Imaging.

[18]  Arijit Raychowdhury,et al.  An Ultra-Low Power, “Always-On” Camera Front-End for Posture Detection in Body Worn Cameras Using Restricted Boltzman Machines , 2015, IEEE Transactions on Multi-Scale Computing Systems.

[19]  Edgar Sanchez-Sinencio,et al.  A Highly Efficient Ultralow Photovoltaic Power Harvesting System With MPPT for Internet of Things Smart Nodes , 2015, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[20]  Shree K. Nayar,et al.  Towards Self-Powered Cameras , 2015, 2015 IEEE International Conference on Computational Photography (ICCP).

[21]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[22]  Jaehyuk Choi,et al.  Always-On CMOS Image Sensor for Mobile and Wearable Devices , 2016, IEEE Journal of Solid-State Circuits.

[23]  Daeho Lee,et al.  Vision-based remote control system by motion detection and open finger counting , 2009, IEEE Transactions on Consumer Electronics.

[24]  Meinard Müller,et al.  Dynamic Time Warping , 2008 .