Application Potential of Multimedia Information Retrieval

This paper will first briefly survey the existing impact of multimedia information retrieval (MIR) in applications. It will then analyze the current trends of MIR research which can have an influence on future applications. It will then detail the future possibilities and bottlenecks in applying the MIR research results in the main target application areas, such as the consumer (e.g., personal video recorders, web information retrieval), public safety (e.g., automated smart surveillance systems), and professional world (e.g., automated meeting capture and summarization). In particular, recommendations will be made to the research community regarding the challenges that need to be met to make the knowledge transfer towards the applications more efficient and effective. It will also attempt to study the trends in the applications which can inform the MIR community on directing intellectual resources towards MIR problems which can have a maximal real-world impact.

[1]  Ramesh C. Jain,et al.  Knowledge-guided parsing in video databases , 1993, Electronic Imaging.

[2]  Manuel Blum,et al.  Peekaboom: a game for locating objects in images , 2006, CHI.

[3]  Ben Gerson.  The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed Our Culture , 2005 .

[4]  Shi-Kuo Chang,et al.  Image information systems , 1985, Proc. IEEE.

[5]  L. KherfiM.,et al.  Image Retrieval from the World Wide Web , 2004 .

[6]  Anil K. Jain,et al.  Shape-Based Retrieval: A Case Study With Trademark Image Databases , 1998, Pattern Recognit..

[7]  Thomas A. Funkhouser,et al.  Shape-based retrieval and analysis of 3d models , 2005, CACM.

[8]  Ioannis T. Pavlidis,et al.  A video-based surveillance solution for protecting the air-intakes of buildings from chem-bio attacks , 2002, Proceedings. International Conference on Image Processing.

[9]  Naphtali Rishe,et al.  Content-based image retrieval , 1995, Multimedia Tools and Applications.

[10]  Qi Tian,et al.  Semantic Retrieval of Video , 2006 .

[11]  Zhe Wang,et al.  VFerret: content-based similarity search tool for continuous archived video , 2006, CARPE '06.

[12]  Wei Niu,et al.  Human activity detection and recognition for video surveillance , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[13]  Anoop Gupta,et al.  Distributed meetings: a meeting capture and broadcasting system , 2002, MULTIMEDIA '02.

[14]  Janto Skowronek,et al.  Automatic surveillance of the acoustic activity in our living environment , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[15]  M. Shah,et al.  KNIGHT M : A REAL TIME SURVEILLANCE SYSTEM FOR MULTIPLE OVERLAPPING AND NON-OVERLAPPING CAMERAS , 2003 .

[16]  Ying Liu,et al.  A survey of content-based image retrieval with high-level semantics , 2007, Pattern Recognit..

[17]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[18]  Alan F. Smeaton,et al.  Towards event detection in an audio-based sensor network , 2005, VSSN@MM.

[19]  Gio Wiederhold,et al.  Semantics-sensitive integrated matching for picture libraries and biomedical image databases , 2000 .

[20]  Thomas S. Huang,et al.  Content-based image retrieval with relevance feedback in MARS , 1997, Proceedings of International Conference on Image Processing.

[21]  Fausto Pellandini,et al.  Automatic sound detection and recognition for noisy environment , 2000, 2000 10th European Signal Processing Conference.

[22]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[23]  Manuele Bicego,et al.  On-line adaptive background modelling for audio surveillance , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[24]  Chloé Clavel,et al.  Events Detection for an Audio-Based Surveillance System , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[25]  Chng Eng Siong,et al.  Automatic generation of personalized music sports video , 2005, MULTIMEDIA '05.

[26]  M. Grgic,et al.  A survey of biometric recognition methods , 2004, Proceedings. Elmar-2004. 46th International Symposium on Electronics in Marine.

[27]  Sharath Pankanti,et al.  Biometrics: a tool for information security , 2006, IEEE Transactions on Information Forensics and Security.

[28]  Bernt Schiele,et al.  Automatic Detection and Tracking of Abandoned Objects , 2003 .

[29]  Tanveer F. Syeda-Mahmood,et al.  Content-based retrieval in gene expression databases , 2004, MULTIMEDIA '04.

[30]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Sergio A. Velastin,et al.  Intelligent distributed surveillance systems: a review , 2005 .

[32]  Noboru Babaguchi,et al.  Event based indexing of broadcasted sports video by intermodal collaboration , 2002, IEEE Trans. Multim..

[33]  Yoshinao Aoki,et al.  Indexing of baseball telecast for content-based video retrieval , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[34]  Witold Pedrycz,et al.  Face recognition: A study in information fusion using fuzzy integral , 2005, Pattern Recognit. Lett..

[35]  Djemel Ziou,et al.  Image Retrieval from the World Wide Web: Issues, Techniques, and Systems , 2004, CSUR.

[36]  Qi Tian,et al.  Semantic retrieval of video - review of research on video retrieval in meetings, movies and broadcast news, and sports , 2006, IEEE Signal Processing Magazine.

[37]  Anoop Gupta,et al.  Automatically extracting highlights for TV Baseball programs , 2000, ACM Multimedia.

[38]  Manuele Bicego,et al.  On-line adaptive background modelling for audio surveillance , 2004, ICPR 2004.

[39]  Mubarak Shah,et al.  KNIGHT/spl trade/: a real time surveillance system for multiple and non-overlapping cameras , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[40]  Shih-Fu Chang,et al.  Structure analysis of soccer video with hidden Markov models , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[41]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[42]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[43]  Marcel Worring,et al.  Multimodal Video Indexing : A Review of the State-ofthe-art , 2001 .

[44]  Leonidas J. Guibas,et al.  Counting people in crowds with a real-time network of simple image sensors , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[45]  ChengXiang Zhai,et al.  Active feedback in ad hoc information retrieval , 2005, SIGIR '05.

[46]  Luca Benini,et al.  An integrated multi-modal sensor network for video surveillance , 2005, VSSN '05.

[47]  Chng Eng Siong,et al.  Event detection based on non-broadcast sports video , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[48]  R. Cucchiara,et al.  Multimedia surveillance: content-based retrieval with multicamera people tracking , 2006, VSSN '06.

[49]  James Ze Wang,et al.  Content-based image retrieval: approaches and trends of the new age , 2005, MIR '05.

[50]  Mohan S. Kankanhalli,et al.  Information assimilation framework for event detection in multimedia surveillance systems , 2006, Multimedia Systems.

[51]  Atsuo Yoshitaka,et al.  A Survey on Content-Based Retrieval for Multimedia Databases , 1999, IEEE Trans. Knowl. Data Eng..

[52]  Edward Y. Chang,et al.  Multi-camera spatio-temporal fusion and biased sequence-data learning for security surveillance , 2003, MULTIMEDIA '03.

[53]  Cyrus Shahabi,et al.  An experimental study of alternative shape-based image retrieval techniques , 2006, Multimedia Tools and Applications.

[54]  Gian Luca Foresti,et al.  A distributed sensor network for video surveillance of outdoor environments , 2002, Proceedings. International Conference on Image Processing.

[55]  Mingjing Li,et al.  Automated annotation of human faces in family albums , 2003, MULTIMEDIA '03.

[56]  Gerard Salton,et al.  Automatic Information Organization And Retrieval , 1968 .

[57]  J. O. Peralta,et al.  Security PIDS with physical sensors, real-time pattern recognition, and continuous patrol , 2002, IEEE Trans. Syst. Man Cybern. Part C.

[58]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[59]  Yong Rui,et al.  Real-time speaker tracking using particle filter sensor fusion , 2004, Proceedings of the IEEE.

[60]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  Alberto Del Bimbo,et al.  Content-based retrieval of 3D models , 2006, TOMCCAP.

[62]  Yong Rui,et al.  Time delay estimation in the presence of correlated noise and reverberation , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[63]  Takeo Kanade,et al.  Rotation Invariant Neural Network-Based Face Detection , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[64]  Thomas S. Huang,et al.  Exploring video structure beyond the shots , 1998, Proceedings. IEEE International Conference on Multimedia Computing and Systems (Cat. No.98TB100241).

[65]  Chng Eng Siong,et al.  Automatic replay generation for soccer video broadcasting , 2004, MULTIMEDIA '04.