Enhancing Multi-lingual Information Extraction via Cross-Media Inference and Fusion

We describe a new information fusion approach to integrate facts extracted from cross-media objects (videos and texts) into a coherent common representation including multi-level knowledge (concepts, relations and events). Beyond standard information fusion, we exploited video extraction results and significantly improved text Information Extraction. We further extended our methods to multi-lingual environment (English, Arabic and Chinese) by presenting a case study on cross-lingual comparable corpora acquisition based on video comparison.

[1]  Rudolf Kruse,et al.  Fusion: General concepts and characteristics , 2001, Int. J. Intell. Syst..

[2]  Tao Mei,et al.  Correlative multi-label video annotation , 2007, ACM Multimedia.

[3]  Heng Ji,et al.  Mining Name Translations from Comparable Corpora by Creating Bilingual Information Networks , 2009, BUCC@ACL/IJCNLP.

[4]  Pavel Praks,et al.  Information extraction from HTML product catalogues: from source code and images to RDF , 2005, The 2005 IEEE/WIC/ACM International Conference on Web Intelligence (WI'05).

[5]  MarcuDaniel,et al.  Improving Machine Translation Performance by Exploiting Non-Parallel Corpora , 2005 .

[6]  Tao Mei,et al.  Multi-Layer Multi-Instance Learning for Video Concept Detection , 2008, IEEE Transactions on Multimedia.

[7]  Tao Mei,et al.  Multi-layer multi-instance kernel for video concept detection , 2007, ACM Multimedia.

[8]  Joseph Olive,et al.  Handbook of Natural Language Processing and Machine Translation: DARPA Global Autonomous Language Exploitation , 2011 .

[9]  Pascale Fung,et al.  Sentence Alignment in Paralllel, Comparable and Quasi-comparable Corpora , 2004 .

[10]  Heng Ji,et al.  Refining Event Extraction through Cross-Document Inference , 2008, ACL.

[11]  Heng Ji,et al.  Gender and Animacy Knowledge Discovery from Web-Scale N-Grams for Unsupervised Person Mention Detection , 2009, PACLIC.

[12]  Fabio Ciravegna,et al.  Exploring multimedia in a keyword space , 2008, ACM Multimedia.

[13]  Dragos Stefan Munteanu,et al.  Improving Machine Translation Performance by Exploiting Non-Parallel Corpora , 2005, CL.

[14]  Éric Grégoire,et al.  An unbiased approach to iterated fusion by weakening , 2006, Inf. Fusion.

[15]  Flora Amato,et al.  Information Extraction from Multimedia Documents for e-Government Applications , 2009 .

[16]  Pavel Praks,et al.  Multimedia information extraction from HTML product catalogues , 2005, DATESO.

[17]  Yansong Feng,et al.  Automatic Image Annotation Using Auxiliary Text Information , 2008, ACL.

[18]  Marie-Francine Moens,et al.  Text Analysis for Automatic Image Annotation , 2007, ACL.

[19]  Shane Bergsma,et al.  Automatic Acquisition of Gender Information for Anaphora Resolution , 2005, Canadian Conference on AI.

[20]  Mohammad Rahmati,et al.  A novel multimedia data mining framework for information extraction of a soccer video stream , 2009, Intell. Data Anal..

[21]  Antonella De Angeli,et al.  Integration and synchronization of input modes during multimodal human-computer interaction , 1997, CHI.

[22]  Kalina Bontcheva,et al.  Multimedia indexing through multi-source and multi-language information extraction: the MUMIS project , 2004, Data Knowl. Eng..

[23]  Shumeet Baluja,et al.  Boosting Sex Identification Performance , 2005, International Journal of Computer Vision.

[24]  Fei Wang,et al.  Label Propagation through Linear Neighborhoods , 2006, IEEE Transactions on Knowledge and Data Engineering.

[25]  Avideh Zakhor,et al.  Efficient video similarity measurement and search , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[26]  Dilek Z. Hakkani-Tür,et al.  Information extraction to improve cross-lingual document retrieval , 2007 .