Feature selection for low bit rate mobile augmented reality applications

Mobile augmented reality applications rely on automatically matching a captured visual scene to an image in a database. This is typically achieved by deriving a set of features from the captured image, transmitting them through a network and then matching with features derived from a database of reference images. A fundamental problem is to select as few and robust features as possible such that the matching accuracy is invariant to distortions caused by camera capture whilst minimizing the bit rate required for their transmission. In this paper, novel feature selection methods are proposed, based on the entropy of the image content, entropy of extracted features and the Discrete Cosine Transformation (DCT) coefficients. The methods proposed in the descriptor domain and DCT domain achieve better retrieval accuracy under low bit rate transmission than state-of-the-art peak based feature selection used within the MPEG-7 Compact Descriptor for Visual Search (CDVS). The robustness of the proposed methods is evaluated under controlled single distortion and the retrieval performance is verified from image retrieval experiments and results for a realistic dataset with complex real world capturing distortion. Results show that the proposed method can improve the matching accuracy for various detectors. On the one side, feature selection can achieve low bit rate transmission, on the other side, when coping with distorted images, it can result in a higher matching accuracy than using all features. Hence, even if all the features can be transmitted in high transmission bandwidth scenarios, feature selection should still be applied to the distorted query image to ensure high matching accuracy. Novel relevance-based feature selection for low bit rate MAR applications.Proposed relevance metrics select significant features for content retrieval.Proposed methods achieve superior retrieval performance under low bit rate.Proposed methods improve retrieval accuracy for different detectors.Feature selection should be applied to the distorted image for high accuracy.

[1]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[2]  Vincent Lepetit,et al.  Appearance-based keypoint clustering , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Jiri Matas,et al.  Efficient Symmetry Detection Using Local Affine Frames , 2007, SCIA.

[4]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[5]  Michael Brady,et al.  Saliency, Scale and Image Description , 2001, International Journal of Computer Vision.

[6]  Panu Turcot,et al.  Better matching with fewer features: The selection of useful features in large database recognition problems , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[7]  Bernd Girod,et al.  Memory-Efficient Image Databases for Mobile Visual Search , 2014, IEEE MultiMedia.

[8]  Sanmay Das,et al.  Filters, Wrappers and a Boosting-Based Hybrid for Feature Selection , 2001, ICML.

[9]  Christian Ritz,et al.  Ensuring Quality of Experience for markerless image recognition applied to print media content , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[10]  S. S. Iyengar,et al.  An Evaluation of Filter and Wrapper Methods for Feature Selection in Categorical Clustering , 2005, IDA.

[11]  Changming Sun,et al.  Fast Reflectional Symmetry Detection Using Orientation Histograms , 1999, Real Time Imaging.

[12]  Aggelos K. Katsaggelos,et al.  Robust feature selection with self-matching score , 2013, 2013 IEEE International Conference on Image Processing.

[13]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[14]  Christian Ritz,et al.  Image compression and retrieval for Mobile Visual Search , 2012, 2012 International Symposium on Communications and Information Technologies (ISCIT).

[15]  Massimo Balestri,et al.  Selection of local features for visual search , 2013, Signal Process. Image Commun..

[16]  Yoel Shkolnisky,et al.  An algebraic approach to symmetry detection , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[17]  Masaaki Fujiyoshi,et al.  Fast Method for Joint Retrieval and Identification of JPEG Coded Images Based on DCT Sign , 2007, 2007 IEEE International Conference on Image Processing.

[18]  Yannis Avrithis,et al.  SymCity: feature selection by symmetry for large scale image retrieval , 2012, ACM Multimedia.

[19]  Chong-Wah Ngo,et al.  Exploiting image indexing techniques in DCT domain , 2001, Pattern Recognit..

[20]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[21]  Krystian Mikolajczyk,et al.  Evaluation of local detectors and descriptors for fast feature matching , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[22]  J. Biemond,et al.  Basic Methods for Image Restoration and Identification , 2009 .

[23]  Kuldip K. Paliwal,et al.  Fast K-dimensional tree algorithms for nearest neighbor search with application to vector quantization encoding , 1992, IEEE Trans. Signal Process..

[24]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[25]  Seungjae Lee,et al.  Extensive analysis of feature selection for compact descriptor , 2013, The 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision.

[26]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[27]  Huizhong Chen,et al.  The stanford mobile visual search data set , 2011, MMSys.

[28]  Michael I. Jordan,et al.  A Unified Probabilistic Model for Global and Local Unsupervised Feature Selection , 2011, ICML.

[29]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[30]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[31]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[32]  Yoel Shkolnisky,et al.  An algebraic approach to symmetry detection , 2004, ICPR 2004.

[33]  J. Preston.  Ξ-filters , 1983 .

[34]  G. Schaefer,et al.  An overview and evaluation of JPEG compressed domain retrieval techniques , 2012, Proceedings ELMAR-2012.