Object segmentation and tracking using video locales

In this paper, we present a new technique based on feature localization for segmenting and tracking objects in videos. A video locale is a sequence of image feature locales that share similar features (color, texture, shape, and motion) in the spatio-temporal domain of videos. Image feature locales are grown from tiles (blocks of pixels) and can be non-disjoint and non-connected. To exploit the temporal redundancy in digital videos, two algorithms (intra-frame and inter-frame) are used to grow locales efficiently. Multiple motion tracking is achieved by tracking and performing tile-based dominant motion estimation for each locale separately.

[1]  Jian Wang,et al.  Kernel-based multiple-cue algorithm for object segmentation , 2000, Electronic Imaging.

[2]  Aaron F. Bobick,et al.  Closed-world tracking , 1995, Proceedings of IEEE International Conference on Computer Vision.

[3]  Ze-Nian Li,et al.  Spatial-temporal joint probability images for video segmentation , 2002, Pattern Recognit..

[4]  Trevor Darrell,et al.  Integrated Person Tracking Using Stereo, Color, and Pattern Detection , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[5]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[6]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[7]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[8]  Ian D. Reid,et al.  Active tracking of foveated feature clusters using affine structure , 1996, International Journal of Computer Vision.

[9]  Frédéric Dufaux,et al.  Key Frame Selection to Represent a Video , 2000, ICIP.

[10]  Michael G. Strintzis,et al.  A novel rigid object segmentation method based on multiresolution 3-D motion and luminance analysis , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[11]  Rangachar Kasturi,et al.  Machine vision , 1995 .

[12]  Frederic Dufaux,et al.  MPEG-4 Natural Video Coding - Part II , 2000 .

[13]  Jie Wei,et al.  Spatio-temporal joint probability images for video segmentation , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[14]  Rama Chellappa,et al.  Vehicle detection and tracking in video , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[15]  David J. Fleet,et al.  Robustly Estimating Changes in Image Appearance , 2000, Comput. Vis. Image Underst..

[16]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Zinovi Tauber,et al.  VISUAL OBJECT RETRIEVAL BASED ON LOCALES , 2000 .

[18]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[19]  Michal Irani,et al.  Video indexing based on mosaic representations , 1998, Proc. IEEE.

[20]  Wee Kheng Leow,et al.  Color segmentation and figure-ground segregation of natural images , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[21]  Stephen M. Smith,et al.  ASSET-2: Real-Time Motion Segmentation and Shape Tracking , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Jie Wei,et al.  Illumination-invariant video segmentation by hierarchical robust thresholding , 1997, Electronic Imaging.

[23]  Qian Huang,et al.  Multimedia Search and Retrieval , 1999 .

[24]  Sridhar Lakshmanan,et al.  CLARK: a heterogeneous sensor fusion method for finding lanes and obstacles , 2000, Image Vis. Comput..

[25]  Prentice Reeves,et al.  The Response of the Average Pupil to Various Intensities of Light , 1920 .

[26]  Demin Wang.  Unsupervised video segmentation based on watersheds and temporal tracking , 1998, IEEE Trans. Circuits Syst. Video Technol..

[27]  Azriel Rosenfeld,et al.  Compact Region Extraction Using Weighted Pixel Linking in a Pyramid , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Touradj Ebrahimi,et al.  Object-Based Video Coding , 2000 .

[29]  Jitendra Malik,et al.  Color- and texture-based image segmentation using EM and its application to content-based image retrieval , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[30]  Jitendra Malik,et al.  Image Retrieval in Digital Libraries , 2000 .

[31]  Jian Wang,et al.  Locale-based multiple cue algorithm for object segmentation , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[32]  Shih-Fu Chang,et al.  An integrated approach for content-based video object segmentation and retrieval , 1999, IEEE Trans. Circuits Syst. Video Technol..

[33]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[34]  David J. Fleet,et al.  Probabilistic detection and tracking of motion discontinuities , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[35]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[36]  Xuemin Chen,et al.  MPEG-4 Natural Video Coding—Part I , 2000 .

[37]  David G. Lowe,et al.  Robust model-based motion tracking through the integration of search and estimation , 1992, International Journal of Computer Vision.

[38]  Mark S. Drew,et al.  Video keyframe production by efficient clustering of compressed chromaticity signatures (poster session) , 2000, ACM Multimedia.

[39]  Jie Wei,et al.  Illumination-invariant color object recognition via compressed chromaticity histograms of color-channel-normalized images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[40]  B. S. Manjunath,et al.  Edge flow: A framework of boundary detection and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Ferran Marqués,et al.  Region-based representations of image and video: segmentation tools for multimedia services , 1999, IEEE Trans. Circuits Syst. Video Technol..

[42]  Sankar K. Pal,et al.  A review on image segmentation techniques , 1993, Pattern Recognit..

[43]  Ze-Nian Li,et al.  Illumination Invariance and Object Model in Content-Based Image and Video Retrieval , 1999, J. Vis. Commun. Image Represent..

[44]  Ze-Nian Li,et al.  Locale-based visual object retrieval under illumination change , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[45]  Fernando Pereira,et al.  MPEG-7: Status and Directions , 2000 .

[46]  R. Deriche,et al.  Geodesic active regions for motion estimation and tracking , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[47]  Wenjun Zeng,et al.  Integrated image and speech analysis for content-based video indexing , 1996, Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems.

[48]  A. Murat Tekalp,et al.  Efficient Filtering and Clustering Methods for Temporal Video Segmentation and Visual Summarization , 1998, J. Vis. Commun. Image Represent..

[49]  Shih-Fu Chang,et al.  VideoQ: an automated content based video search system using visual cues , 1997, MULTIMEDIA '97.