Multimedia news exploration and retrieval by integrating keywords, relations and visual features

Multimedia news may be organized by the keywords and categories for exploration and retrieval applications, but it is very difficult to integrate the relation and visual information into the traditional category browsing and keyword-based search framework. This paper propose a new semantic model that can integrate keyword, relation and visual information in a uniform framework. Based on this semantic representation framework, the news exploration and retrieval applications can be organized by not only keywords and categories but also relations and visual properties. We also proposed a set of algorithms to automatically extract the proposed semantic model automatically from large collection of multimedia news reports.

[1]  Amarnath Gupta,et al.  Visual information retrieval , 1997, CACM.

[2]  Bo Zhang,et al.  Learning concepts from large scale imbalanced data sets using support cluster machines , 2006, MM '06.

[3]  Jitendra Malik,et al.  Blobworld: A System for Region-Based Image Indexing and Retrieval , 1999, VISUAL.

[4]  Ramana Rao,et al.  The Hyperbolic Browser: A Focus + Context Technique for Visualizing Large Hierarchies , 1996, J. Vis. Lang. Comput..

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  Helge J. Ritter,et al.  On interactive visualization of high-dimensional data using the hyperbolic plane , 2002, KDD.

[7]  David Jensen,et al.  TimeMines: Constructing Timelines with Statistical Models of Word Usage , 2000, KDD 2000.

[8]  Jianping Fan,et al.  Concept-oriented video skimming and adaptation via semantic classification , 2004, MIR '04.

[9]  Jianping Fan,et al.  Incorporating feature hierarchy and boosting to achieve more effective classifier training and concept-oriented video summarization and skimming , 2008, TOMCCAP.

[10]  Derek Hoiem,et al.  Object-based image retrieval using the statistical structure of images , 2004, CVPR 2004.

[11]  Yixin Chen,et al.  A Region-Based Fuzzy Feature Matching Approach to Content-Based Image Retrieval , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Danushka Bollegala,et al.  A Bottom-Up Approach to Sentence Ordering for Multi-Document Summarization , 2006, ACL.

[13]  Marcel Worring,et al.  Learning rich semantics from news video archives by style analysis , 2006, TOMCCAP.

[14]  Shih-Fu Chang,et al.  Semantic visual templates: linking visual features to semantics , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[15]  Nitin Madnani,et al.  Measuring Variability in Sentence Ordering for News Summarization , 2007, ENLG.

[16]  James J. Thomas,et al.  Visualizing the non-visual: spatial analysis and interaction with information from text documents , 1995, Proceedings of Visualization 1995 Conference.

[17]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[18]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Nozha Boujemaa,et al.  Region-based image retrieval: fast coarse segmentation and fine color description , 2004, J. Vis. Lang. Comput..

[20]  Regina Barzilay,et al.  Inferring Strategies for Sentence Ordering in Multidocument News Summarization , 2002, J. Artif. Intell. Res..

[21]  Paul Whitney,et al.  Multi-faceted insight through interoperable visual information analysis paradigms , 1998, Proceedings IEEE Symposium on Information Visualization (Cat. No.98TB100258).

[22]  Tony McEnery,et al.  The Lancaster Corpus of Mandarin Chinese , 2003 .

[23]  Edward Y. Chang,et al.  Semantics and feature discovery via confidence-based ensemble , 2005, TOMCCAP.

[24]  Karen Spärck Jones.  A statistical interpretation of term specificity and its application in retrieval , 2021, J. Documentation.

[25]  Jianping Fan,et al.  Extracting informative images from web news pages via imbalanced classification , 2009, MM '09.

[26]  Taghi M. Khoshgoftaar,et al.  Experimental perspectives on learning from imbalanced data , 2007, ICML '07.

[27]  Alan L. Yuille,et al.  Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Dragomir R. Radev,et al.  NewsInEssence: summarizing online news topics , 2005, Commun. ACM.

[29]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[30]  Carlo Tomasi,et al.  Texture-based image retrieval without segmentation , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[31]  Tatiana Louchnikova,et al.  Flexible image decomposition for multimedia indexing and retrieval , 2001, IS&T/SPIE Electronic Imaging.

[32]  Jianping Fan,et al.  Analyzing Large-Scale News Video Databases to Support Knowledge Visualization and Intuitive Retrieval , 2007, 2007 IEEE Symposium on Visual Analytics Science and Technology.

[33]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[34]  Jianping Fan,et al.  Integrating Concept Ontology and Multitask Learning to Achieve More Effective Classifier Training for Multilevel Image Annotation , 2008, IEEE Transactions on Image Processing.

[35]  Steven Skiena,et al.  Spatial Analysis of News Sources , 2006, IEEE Transactions on Visualization and Computer Graphics.

[36]  Edward Y. Chang,et al.  Confidence-based dynamic ensemble for image annotation and semantics discovery , 2003, MULTIMEDIA '03.

[37]  James Ze Wang,et al.  Unsupervised Multiresolution Segmentation for Images with Low Depth of Field , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Christopher D. Manning,et al.  Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling , 2005, ACL.

[39]  Wei Li,et al.  Early results for Named Entity Recognition with Conditional Random Fields, Feature Induction and Web-Enhanced Lexicons , 2003, CoNLL.

[40]  Lucy T. Nowell,et al.  ThemeRiver: Visualizing Thematic Changes in Large Document Collections , 2002, IEEE Trans. Vis. Comput. Graph..

[41]  Yixin Chen,et al.  Image Categorization by Learning and Reasoning with Regions , 2004, J. Mach. Learn. Res..

[42]  Thorsten Joachims,et al.  Learning to classify text using support vector machines - methods, theory and algorithms , 2002, The Kluwer international series in engineering and computer science.

[43]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.