Poster Abstract: Intuitive Appliance Identification using Image Matching in Smart Buildings

Identifying an appliance for interaction in commercial buildings becomes non-trivial as the number of smart appliances explodes. We present a system for users to intuitively "look up" appliances using image matching-based technique on a pre-constructed and annotated visual model of building interiors. It matched 98% images on a public robot-collected dataset and achieved 100% recall and precision. Our lab experiments with human captured videos and images also show the feasibility of real world deployments.

[1]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[2]  QuanLong,et al.  Linear N-Point Camera Pose Determination , 1999 .

[3]  Jun Rekimoto,et al.  CyberCode: designing augmented reality environments with visual tags , 2000, DARE '00.

[4]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.

[5]  George Vosselman,et al.  Visibility analysis of point cloud in close range photogrammetry , 2014 .

[6]  David E. Culler,et al.  sMAP: a simple measurement and actuation profile for physical information , 2010, SenSys '10.

[7]  Justin Manweiler,et al.  OverLay: Practical Mobile Augmented Reality , 2015, MobiSys.

[8]  H. Kato,et al.  2D barcodes for mobile phones , 2005 .

[9]  François Michaud,et al.  Online global loop closure detection for large-scale multi-session graph-based SLAM , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[11]  Jie Liu,et al.  A realistic evaluation and comparison of indoor location technologies: experiences and lessons learned , 2015, IPSN.

[12]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[13]  François Michaud,et al.  Memory management for real-time appearance-based loop closure detection , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Geoff Wyvill,et al.  SIFT and SURF Performance Evaluation against Various Image Deformations on Benchmark Dataset , 2011, 2011 International Conference on Digital Image Computing: Techniques and Applications.

[15]  Simon Mayer,et al.  Device recognition for intuitive interaction with the web of things , 2013, UbiComp.

[16]  Pattie Maes,et al.  Smarter objects: using AR technology to program physical objects and their interactions , 2013, CHI Extended Abstracts.

[17]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Ronen Basri,et al.  Direct visibility of point sets , 2007, ACM Trans. Graph..

[19]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Andrew Owens,et al.  SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels , 2013, 2013 IEEE International Conference on Computer Vision.

[21]  Edward A. Lee,et al.  HOBS: head orientation-based selection in physical spaces , 2014, SUI.

[22]  Dieter Fox,et al.  Interactive 3D modeling of indoor environments with a consumer depth camera , 2011, UbiComp '11.

[23]  Simon Mayer,et al.  User interfaces for smart things -- A generative approach with semantic interaction descriptions , 2014, TCHI.

[24]  Niloy J. Mitra,et al.  Visibility of noisy point cloud data , 2010, Comput. Graph..

[25]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[26]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[27]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[28]  Jitendra Malik,et al.  Aligning 3D models to RGB-D images of cluttered scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  David E. Culler,et al.  BOSS: Building Operating System Services , 2013, NSDI.

[30]  Jianxiong Xiao,et al.  SUN RGB-D: A RGB-D scene understanding benchmark suite , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Gerhard Tröster,et al.  Wearable EOG goggles: Seamless sensing and context-awareness in everyday environments , 2009, J. Ambient Intell. Smart Environ..

[32]  Zhe Xu,et al.  A point-and-click interface for the real world: Laser designation of objects for mobile manipulation , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[33]  Long Quan,et al.  Linear N-Point Camera Pose Determination , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Gregory D. Abowd,et al.  A 2-Way Laser-Assisted Selection Scheme for Handhelds in a Physical Environment , 2003, UbiComp.

[35]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.