Evaluation of color descriptors for projector-camera systems

We study and evaluate invariance properties of local descriptors.We introduce a new Local Histogram Equalization (LHE) descriptor.We show the robustness of LHE for various color and geometric distortions.LHE is recommended for feature matching in Camera-Projector systems. Spatial Augmented Reality applications generally use projector-camera systems to control the visual projection appearance by comparing the initial projected and the acquired images. To obtain an accurate geometric compensation, a non-intrusive feature-point matching approach can be exploited which must handle complex photometric distortions due to the spectral devices responses, complex illumination and the mixing of the projected image with the projection surface.This paper first discusses the invariance properties of existing color descriptors in that application for non-intrusive geometric compensation. Their performance is evaluated using the framework of Setkov et al. (2013) extended by adding the several new test cases: modeled synthetic projections, real-world projections under various illuminants on one and two planar surfaces. Our experimental results show two main conclusions: (1) classical color vision models are hardly suitable to model the distortions in a projector-camera system, and (2) the LHE-based descriptor (Local Histogram Equalization) is the most reliable to compensate real-projections.

[1]  Shree K. Nayar,et al.  What is the space of camera response functions? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[2]  De Xu,et al.  Color Constant Descriptors Combining Image Derivative Structures , 2012, Comput. Informatics.

[3]  Mauro Barni,et al.  On the effectiveness of local warping against SIFT-based copy-move detection , 2012, 2012 5th International Symposium on Communications, Control and Signal Processing.

[4]  N. Hurtós,et al.  Plane-based calibration of a projector-camera system , 2008 .

[5]  Julie Delon,et al.  Combining color and geometry for local image matching , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[6]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[7]  Markus Gross,et al.  Towards a continuous, unified calibration of projectors and cameras , 2007 .

[8]  Shree K. Nayar,et al.  A Projector-Camera System with Real-Time Photometric Adaptation for Dynamic Environments , 2005, CVPR.

[9]  Yongbeom Lee,et al.  Simultaneous Geometric and Radiometric Adaptation to Dynamic Surfaces With a Mobile Projector-Camera System , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Shree K. Nayar,et al.  Making one object look like another: controlling appearance using a projector-camera system , 2004, CVPR 2004.

[11]  Xu Chen,et al.  Multimodal Video Indexing and Retrieval Using Directed Information , 2012, IEEE Transactions on Multimedia.

[12]  Christian Jacquemin,et al.  Color invariant feature matching for image geometric correction , 2013, MIRAGE '13.

[13]  Oliver Bimber,et al.  Passive-Active Geometric Calibration for View-Dependent Projections onto Arbitrary Surfaces , 2007, J. Virtual Real. Broadcast..

[14]  Pierre-Marc Jodoin,et al.  Camera-projector matching using an unstructured video stream , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[15]  Takashi Yamanaka,et al.  Adaptive Image Projection onto Non-planar Screen Using Projector-Camera Systems , 2010, 2010 20th International Conference on Pattern Recognition.

[16]  Gordon Wetzstein,et al.  Enabling view-dependent stereoscopic projection in real environments , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[17]  John J. McCann,et al.  Retinex in Matlab , 2000, CIC.

[18]  Ruixia Xu,et al.  Convex programming colour constancy with a diagonal-offset model , 2005, IEEE International Conference on Image Processing 2005.

[19]  Arnold W. M. Smeulders,et al.  Color Invariance , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Christophe Renaud,et al.  Radiometric compensation for a low-cost immersive projection system , 2008, VRST '08.

[21]  Cordelia Schmid,et al.  Coloring Local Feature Extraction , 2006, ECCV.

[22]  Arnold W. M. Smeulders,et al.  Color Based Object Recognition , 1997, ICIAP.

[23]  Alberto Del Bimbo,et al.  A Set of Selected SIFT Features for 3D Facial Expression Recognition , 2010, 2010 20th International Conference on Pattern Recognition.

[24]  Jong-Il Park,et al.  Surface-Independent Direct-Projected Augmented Reality , 2006, ACCV.

[25]  Gerald Schaefer,et al.  Illuminant and device invariant colour using histogram equalisation , 2005, Pattern Recognit..

[26]  Pascal Fua,et al.  Linear Local Models for Monocular Reconstruction of Deformable Surfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  A. Föhrenbach,et al.  SIMPLE++ , 2000, OR Spectr..

[28]  Ruigang Yang,et al.  Automatic and Continuous Projector Display Surface Estimation Using Everyday Imagery , 2001, WSCG.

[29]  Garrison W. Cottrell,et al.  Color Constancy Algorithms for Object and Face Recognition , 2010, ISVC.

[30]  Joost van de Weijer,et al.  Boosting color saliency in image feature detection , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Gabriel Taubin,et al.  Simple, Accurate, and Robust Projector-Camera Calibration , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[32]  Oliver Bimber,et al.  Projector-Based Augmentation , 2007 .

[33]  Tian-Tsong Ng,et al.  From the Rendering Equation to Stratified Light Transport Inversion , 2011, International Journal of Computer Vision.

[34]  Tony Q. S. Quek,et al.  Radiometric compensation using stratified inverses , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[35]  Chengjun Liu,et al.  A New Color SIFT Descriptor and Methods for Image Category Classification , 2010 .

[36]  Markus H. Gross,et al.  Embedding imperceptible patterns into projected images for simultaneous acquisition and display , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[37]  Andrew Zisserman,et al.  Scene Classification Using a Hybrid Generative/Discriminative Approach , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Joost van de Weijer,et al.  Generalized Gamut Mapping using Image Derivative Structures for Color Constancy , 2008, International Journal of Computer Vision.

[39]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[40]  Tian-Tsong Ng,et al.  On the Duality of Forward and Inverse Light Transport , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[42]  Yuncai Liu,et al.  Monocular 3-D Tracking of Inextensible Deformable Surfaces Under $L_2$ -Norm , 2010, IEEE Transactions on Image Processing.

[43]  Aly A. Farag,et al.  CSIFT: A SIFT Descriptor with Color Invariant Characteristics , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[44]  Jong-Il Park,et al.  Contrast Enhancement in Direct-Projected Augmented Reality , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[45]  Gertjan J. Burghouts,et al.  Performance evaluation of local colour invariants , 2009, Comput. Vis. Image Underst..

[46]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[48]  S. Govindarajulu,et al.  A Comparison of SIFT, PCA-SIFT and SURF , 2012 .

[49]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[50]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[51]  Shree K. Nayar,et al.  A Projection System with Radiometric Compensation for Screen Imperfections , 2003 .

[52]  Luo Juan,et al.  A comparison of SIFT, PCA-SIFT and SURF , 2009 .

[53]  Adrien Bartoli,et al.  Monocular Template-based Reconstruction of Inextensible Surfaces , 2011, International Journal of Computer Vision.

[54]  João Pedro Barreto,et al.  sRD-SIFT: Keypoint Detection and Matching in Images With Radial Distortion , 2012, IEEE Transactions on Robotics.