GLIMPSE: Continuous, Real-Time Object Recognition on Mobile Devices

Excerpted from "Glimpse: Continuous, Real-Time Object Recognition on Mobile Devices" from Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems with permission. http://dx.doi.org/10.1145/2809695.2809711 © ACM 2015. Glimpse is a continuous, real-time object recognition system for camera-equipped mobile devices. Glimpse captures full-motion video, locates objects of interest, recognizes and labels them, and tracks them from frame to frame for the user. Because the algorithms for object recognition entail significant computation, Glimpse runs them on server machines. When the latency between the server and mobile device is higher than a frame-time, this approach lowers object recognition accuracy. To regain accuracy, Glimpse uses an active cache of video frames on the mobile device. A subset of the frames in the active cache are used to track objects on the mobile, using (stale) hints about objects that arrive from the server from time to time. To reduce network bandwidth usage, Glimpse computes trigger frames to send to the server for recognizing.

[1]  Ramesh K. Sitaraman,et al.  The Akamai network: a platform for high-performance internet applications , 2010, OPSR.

[2]  Chong-Wah Ngo,et al.  Video summarization and scene detection by graph modeling , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Alan Jay Smith,et al.  Cache Memories , 1982, CSUR.

[4]  Alec Wolman,et al.  MAUI: making smartphones last longer with code offload , 2010, MobiSys '10.

[5]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[6]  Sivan Toledo,et al.  Wishbone: Profile-based Partitioning for Sensornet Applications , 2009, NSDI.

[7]  Brad Fitzpatrick,et al.  Distributed caching with memcached , 2004 .

[8]  Yueting Zhuang,et al.  Adaptive key frame extraction using unsupervised clustering , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[9]  Mahadev Satyanarayanan,et al.  Towards wearable cognitive assistance , 2014, MobiSys.

[10]  Xi Yang,et al.  Advance in vision-based driver assistance , 2011, 2011 International Conference on Electric Technology and Civil Engineering (ICETCE).

[11]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[12]  Jonathan Foote,et al.  Discriminative techniques for keyframe selection , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[13]  Bernd Girod,et al.  Outdoors augmented reality on mobile phone using loxel-based visual feature organization , 2008, MIR '08.

[14]  Ramesh Govindan,et al.  Odessa: enabling interactive perception applications on mobile devices , 2011, MobiSys '11.

[15]  Luc Van Gool,et al.  Augmented faces , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).