Improving visual recognition using color normalization in digital video applications

Color variations in video imagery caused by imperfections of acquisition devices from analog sources severely penalize the performance of visual recognition tasks, which are very important for smart video indexing, annotation and browsing, for instance. We propose a color correction technique based on modeling the distribution in RGB space of the colors generated by a particular device using a mixture of Gaussian probability density functions, in order to automatically obtain transformations that let us have a normalized representation of color images. It has been tested within the framework of appearance based recognition of video key-frames. In this case, the representation of key-frames consists of their projection into a low dimensional linear subspace given by principal component analysis (PCA), where the matching is done using minimum Euclidean distance. The application of color correction significantly improves the recognition rate, giving nearly optimum results.

[1]  Bernt Schiele,et al.  Comprehensive Colour Image Normalization , 1998, ECCV.

[2]  Bartlett W. Mel.  SEEMORE: Combining Color, Shape, and Texture Histogramming in a Neurally Inspired Approach to Visual Object Recognition , 1997, Neural Computation.

[3]  John S. Boreczky,et al.  Comparison of video shot boundary detection techniques , 1996, J. Electronic Imaging.

[4]  Andrea Salgian,et al.  Improving appearance-based object recognition in cluttered backgrounds , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[5]  John R. Kender,et al.  Video summaries and cross-referencing , 2002 .

[6]  John R. Kender,et al.  Video scene segmentation via continuous video coherence , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[7]  Alan F. Smeaton,et al.  Evaluation of automatic shot boundary detection on a large video test suite , 1999 .

[8]  M. Ibrahim Sezan,et al.  A computational approach to semantic event detection , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[9]  Yueting Zhuang,et al.  Adaptive key frame extraction using unsupervised clustering , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[10]  Shih-Fu Chang,et al.  Structural and semantic analysis of video , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[11]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[12]  Riccardo Leonardi,et al.  Semantic Indexing of Multimedia Documents , 2002, IEEE Multim..

[13]  Ming-Ting Sun,et al.  Finding Structure in Consumer Videos by Probabilistic Hierarchical Clustering , 2002 .

[14]  Arnold W. M. Smeulders,et al.  Color Based Object Recognition , 1997, ICIAP.

[15]  Brian V. Funt,et al.  Color Constant Color Indexing , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Irena Koprinska,et al.  Temporal video segmentation: A survey , 2001, Signal Process. Image Commun..

[17]  Tomaso A. Poggio,et al.  A pattern classification approach to dynamical object detection , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[18]  Randal C. Nelson,et al.  Qualitative recognition of motion using temporal texture , 1992, CVGIP Image Underst..

[19]  Shih-Fu Chang,et al.  Structure analysis of sports video using domain models , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[20]  Shih-Fu Chang,et al.  Clustering methods for video browsing and annotation , 1996, Electronic Imaging.

[21]  Patrick Bouthemy,et al.  From video shot clustering to sequence segmentation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[22]  Stefan Eickeler,et al.  Content-based video indexing of TV broadcast news using hidden Markov models , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[23]  M. S. Drew,et al.  Color constancy - Generalized diagonal transforms suffice , 1994 .

[24]  Thomas S. Huang,et al.  Constructing table-of-content for videos , 1999, Multimedia Systems.

[25]  Minerva M. Yeung,et al.  Efficient matching and clustering of video shots , 1995, Proceedings., International Conference on Image Processing.

[26]  Martin Szummer,et al.  Temporal texture modeling , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[27]  Xavier Binefa,et al.  An EM algorithm for video summarization, generative model approach , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[28]  Andreas Girgensohn,et al.  Time-Constrained Keyframe Selection Technique , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[29]  Patrick Pérez,et al.  Non Parametric Statistical Analysis of Scene Activity for Motion-Based Video Indexing and Retrieval , 2000 .

[30]  Sushil K. Bhattacharjee,et al.  Shot Detection Tools In Digital Video , 1998, NMBIA.

[31]  Donald A. Adjeroh,et al.  A distance measure for video sequence similarity matching , 1998, Proceedings International Workshop on Multi-Media Database Management Systems (Cat. No.98TB100249).

[32]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[33]  Patrick Bouthemy,et al.  Motion characterization from temporal cooccurrences of local motion-based measures for video indexing , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[34]  Arun Hampapur,et al.  Semantic video indexing: approach and issues , 1999, SGMD.

[35]  Boon-Lock Yeo,et al.  Time-constrained clustering for segmentation of video into story units , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[36]  Alberto Del Bimbo,et al.  Commercial video retrieval by induced semantics , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[37]  Ramin Zabih,et al.  Comparing images using color coherence vectors , 1997, MULTIMEDIA '96.

[38]  Chong-Wah Ngo,et al.  On clustering and retrieval of video shots , 2001, MULTIMEDIA '01.

[39]  D H Brainard,et al.  Bayesian color constancy. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[40]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[41]  Rosalind W. Picard,et al.  Texture orientation for sorting photos "at a glance" , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[42]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[43]  Mahdi Nezamabadi,et al.  Color Appearance Models , 2014, J. Electronic Imaging.

[44]  M.,et al.  Statistical and Structural Approaches to Texture , 2022 .

[45]  John R. Kender,et al.  On the structure and analysis of home videos , 2000 .

[46]  Jordi Vitrià,et al.  Local Color Analysis for Scene Break Detection Applied to TV Commercials Recognition , 1999, VISUAL.

[47]  Wolfgang Effelsberg,et al.  On the detection and recognition of television commercials , 1997, Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[48]  Alberto Del Bimbo,et al.  Sensations and psychological effects in color image database , 1997, Proceedings of International Conference on Image Processing.

[49]  Xavier Binefa,et al.  AudiCom: a video analysis system for auditing commercial broadcasts , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[50]  Milind R. Naphade,et al.  Multimodal pattern matching for audio-visual query and retrieval , 2001, IS&T/SPIE Electronic Imaging.

[51]  Shih-Fu Chang,et al.  Multimedia Knowledge Integration, Summarization And Evaluation , 2002, MDM/KDD.

[52]  Ullas Gargi,et al.  Performance characterization of video-shot-change detection methods , 2000, IEEE Trans. Circuits Syst. Video Technol..

[53]  Tiecheng Liu,et al.  A hidden Markov model approach to the structure of documentaries , 2000, 2000 Proceedings Workshop on Content-based Access of Image and Video Libraries.

[54]  Ramin Zabih,et al.  Comparing images using joint histograms , 1999, Multimedia Systems.

[55]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[56]  Wayne H. Wolf,et al.  Key frame selection by motion analysis , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[57]  Milind R. Naphade,et al.  Supporting audiovisual query using dynamic programming , 2001, MULTIMEDIA '01.

[58]  Giridharan Iyengar,et al.  Content-based browsing and editing of unstructured video , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[59]  Brian V. Funt,et al.  Is Machine Colour Constancy Good Enough? , 1998, ECCV.

[60]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[61]  Stephen W. Smoliar,et al.  An integrated system for content-based video retrieval and browsing , 1997, Pattern Recognit..

[62]  Shimon Ullman,et al.  Combining Class-Specific Fragments for Object Classification , 1999, BMVC.

[63]  Ali N. Akansu,et al.  Low-level motion activity features for semantic characterization of video , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[64]  Xin Liu,et al.  Generating optimal video summaries , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[65]  Anil K. Jain,et al.  Content-based hierarchical classification of vacation images , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[66]  Ramin Zabih,et al.  A feature-based algorithm for detecting and classifying scene breaks , 1995, MULTIMEDIA '95.

[67]  Xavier Binefa,et al.  Color Normalization for Digital Video Processing , 2000, VISUAL.

[68]  Patrick Bouthemy,et al.  Motion-Based Feature Extraction and Ascendant Hierarchical Classification for Video Indexing and Retrieval , 1999, VISUAL.

[69]  Brian V. Funt,et al.  Color constancy under varying illumination , 1995, Proceedings of IEEE International Conference on Computer Vision.

[70]  Tomaso A. Poggio,et al.  A general framework for object detection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[71]  Philippe Jean Salembier Clairon,et al.  Analysis of video sequences: table of content and index creation , 1999 .

[72]  Xavier Binefa,et al.  Bayesian Video Mosaicing with Moving Objects , 2002, Int. J. Pattern Recognit. Artif. Intell..

[73]  Shih-Fu Chang,et al.  A knowledge engineering approach for image classification based on probabilistic reasoning systems , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[74]  Philippe Aigrain,et al.  Medium knowledge-based macro-segmentation of video into sequences , 1997 .

[75]  Ruud M. Bolle,et al.  Comparison of distance measures for video copy detection , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[76]  Anil K. Jain,et al.  On image classification: city images vs. landscapes , 1998, Pattern Recognit..

[77]  Nuno Vasconcelos,et al.  Statistical models of video structure for content analysis and characterization , 2000, IEEE Trans. Image Process..