Wide-angle micro sensors for vision on a tight budget

Achieving computer vision on micro-scale devices is a challenge. On these platforms, the power and mass constraints are severe enough for even the most common computations (matrix manipulations, convolution, etc.) to be difficult. This paper proposes and analyzes a class of miniature vision sensors that can help overcome these constraints. These sensors reduce power requirements through template-based optical convolution, and they enable a wide field-of-view within a small form through a novel optical design. We describe the trade-offs between the field of view, volume, and mass of these sensors and we provide analytic tools to navigate the design space. We also demonstrate milli-scale prototypes for computer vision tasks such as locating edges, tracking targets, and detecting faces.

[1]  J. Goodman.  Introduction to Fourier optics , 1969 .

[2]  Richard G. Baraniuk,et al.  Signal Processing With Compressive Measurements , 2010, IEEE Journal of Selected Topics in Signal Processing.

[3]  Klaus D. Mielenz,et al.  On the Diffraction Limit for Lensless Imaging , 1999, Journal of Research of the National Institute of Standards and Technology.

[4]  António de Sousa.  Smart Cameras in Embedded Systems , 2020 .

[5]  Steven W. Zucker,et al.  Local Scale Control for Edge Detection and Blur Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, ACM Trans. Graph..

[7]  Davide Brunelli,et al.  Wireless Sensor Networks , 2012, Lecture Notes in Computer Science.

[8]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[9]  Paolo Fiorini,et al.  Human++: autonomous wireless sensors for body area networks , 2005, Proceedings of the IEEE 2005 Custom Integrated Circuits Conference, 2005..

[10]  Y.Y. Schechner,et al.  Flat refractive geometry , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Shree K. Nayar,et al.  Towards a true spherical camera , 2009, Electronic Imaging.

[12]  J. Pate.  Introduction to Optics , 1937, Nature.

[13]  Martin Edge,et al.  The Underwater Photographer , 1996 .

[14]  Nathan A. Goodman,et al.  Optically multiplexed imaging with superposition space tracking , 2008, Optical Engineering + Applications.

[15]  Dario Floreano,et al.  Flying Insects and Robots , 2010 .

[16]  Takeo Kanade,et al.  Computational sensor for visual tracking with attention , 1998, IEEE J. Solid State Circuits.

[17]  Ramesh Raskar,et al.  Krill-eye : Superposition compound eye for wide-angle imaging via GRIN lenses , 2009, ICCV Workshops.

[18]  Yann LeCun,et al.  An FPGA-based stream processor for embedded real-time vision with Convolutional Networks , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[19]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[20]  Robert Williams Wood.  Optical and Physical Effects of High Explosives , 1936 .

[21]  Matthew O'Toole,et al.  Optical computing for fast light transport analysis , 2010, ACM Trans. Graph..

[22]  Nicholas F. Borrelli,et al.  Microoptics Technology: Fabrication and Applications of Lens Arrays and Devices , 1999 .

[23]  Ren Ng.  Fourier slice photography , 2005, ACM Trans. Graph..

[24]  Naveen Verma,et al.  Micropower wireless sensors , 2006 .

[25]  J. Tanida,et al.  Thin Observation Module by Bound Optics (TOMBO): Concept and Experimental Verification. , 2001, Applied optics.

[26]  Shree K. Nayar,et al.  Lensless Imaging with a Controllable Aperture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[27]  R.S. Fearing,et al.  Dynamometer Power Output Measurements of Miniature Piezoelectric Actuators , 2009, IEEE/ASME Transactions on Mechatronics.

[28]  Luke P. Lee,et al.  Biologically Inspired Artificial Compound Eyes , 2006, Science.

[29]  Kenro Miyamoto,et al.  Fish Eye Lens , 1964 .

[30]  Robert J. Wood,et al.  Milligram-scale high-voltage power electronics for piezoelectric microrobots , 2009, 2009 IEEE International Conference on Robotics and Automation.

[31]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[32]  Shree K. Nayar,et al.  Caustics of catadioptric cameras , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[33]  G. R. Noakes.  Physical Optics , 1958, Nature.

[34]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[35]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[36]  Shai Avidan,et al.  Support Vector Tracking , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[37]  Sing H. Lee.  Optical Pattern Recognition , 1985, COMPCON.

[38]  Kiriakos N. Kutulakos,et al.  Optical computing for fast light transport analysis , 2010, SIGGRAPH 2010.

[39]  A. Wilhelm,et al.  Evaluation of a micro fuel cell as applied to a mobile robot , 2005, IEEE International Conference Mechatronics and Automation, 2005.

[40]  Shree K. Nayar,et al.  Programmable Imaging: Towards a Flexible Camera , 2006, International Journal of Computer Vision.

[41]  Heung Cho Ko,et al.  Curvilinear electronics formed using silicon membrane circuits and elastomeric transfer elements. , 2009, Small.

[42]  Kenneth J. Weible,et al.  Miniaturized imaging systems , 2003 .

[43]  Visesh Chari,et al.  Multi-View Geometry of the Refractive Plane , 2009, BMVC.

[44]  H. Herzig.  Micro-Optics : Elements, Systems And Applications , 1997 .

[45]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.