Dynamic Data-driven Application System (DDDAS) for Video Surveillance User Support

Human-machine interaction mixed initiatives require a pragmatic coordination between different systems. Context understanding is established from the content, analysis, and guidance from query-based coordination between users and machines. Inspired by Level 5 Information Fusion user refinement, a live-video computing (LVC) structure is presented for user-based query access of a data-base management of information. Information access includes multimedia fusion of query-based text, images, and exploited tracks which can be utilized for context assessment, content-based information retrieval (CBIR), and situation awareness. In this paper, we explore new developments in dynamic data-driven application systems (DDDAS) of context analysis for user support. Using a common image processing data set, a system-level time savings is demonstrated using a query-based approach in a context, control, and semantic-aware information fusion design.

[1]  Alex Aved,et al.  Scene Understanding For Real Time Processing Of Queries Over Big Data Streaming Video , 2013 .

[2]  Mihaela van der Schaar,et al.  Bandit Framework for Systematic Learning in Wireless Video-Based Face Recognition , 2015, IEEE Journal of Selected Topics in Signal Processing.

[3]  Zheng Liu,et al.  Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement in Night Vision: A Comparative Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Li Bai,et al.  Evaluation of visual tracking in extremely low frame rate wide area motion imagery , 2011, 14th International Conference on Information Fusion.

[5]  Erik Blasch,et al.  Multi-scale decomposition tool for Content Based Image Retrieval , 2013, 2013 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[6]  Hanan Samet,et al.  Foundations of multidimensional and metric data structures , 2006, Morgan Kaufmann series in data management systems.

[7]  Genshe Chen,et al.  Information fusion in a cloud computing era: A systems-level perspective , 2014, IEEE Aerospace and Electronic Systems Magazine.

[8]  Genshe Chen,et al.  A container-based elastic cloud architecture for real-time full-motion video (FMV) target tracking , 2014, 2014 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[9]  Erik Blasch,et al.  Revisiting the JDL model for information exploitation , 2013, Proceedings of the 16th International Conference on Information Fusion.

[10]  Ying Liu,et al.  A survey of content-based image retrieval with high-level semantics , 2007, Pattern Recognit..

[11]  R. Brunelli,et al.  A Survey on the Automatic Indexing of Video Data, , 1999, J. Vis. Commun. Image Represent..

[12]  Li Bai,et al.  Efficient Minimum Error Bounded Particle Resampling L1 Tracker With Occlusion Detection , 2013, IEEE Transactions on Image Processing.

[13]  Jonathan Foote,et al.  An overview of audio information retrieval , 1999, Multimedia Systems.

[14]  Kien A. Hua,et al.  BroadCatch: a periodic broadcast technique for heterogeneous video-on-demand , 2004, IEEE Transactions on Broadcasting.

[15]  Erik Blasch,et al.  Automatic Association of Chats and Video Tracks for Activity Learning and Recognition in Aerial Video Surveillance , 2014, Sensors.

[16]  Erik Blasch,et al.  Static Versus Dynamic Data Information Fusion Analysis Using DDDAS for Cyber Security Trust , 2014, ICCS.

[17]  Qian Huang,et al.  Content-based indexing and retrieval-by-example in audio , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[18]  Larry S. Davis,et al.  Real-time foreground-background segmentation using codebook model , 2005, Real Time Imaging.

[19]  Nhan Nguyen,et al.  Context Aware Data Acquisition Framework for Dynamic Data Driven Applications Systems (DDDAS) , 2013, MILCOM 2013 - 2013 IEEE Military Communications Conference.

[20]  Amarnath Gupta,et al.  Virage image search engine: an open framework for image management , 1996, Electronic Imaging.

[21]  Erik Blasch,et al.  Sensor, User, Mission (SUM) Resource Management and Their Interaction with Level 2/3 Fusion , 2006, 2006 9th International Conference on Information Fusion.

[22]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..

[23]  Li Bai,et al.  Minimum error bounded efficient ℓ1 tracker with occlusion detection , 2011, CVPR 2011.

[24]  Ichiro Ide,et al.  Automatic Video Indexing Based on Shot Classification , 1998, AMCP.

[25]  Erik Blasch,et al.  Dynamic Data Driven Applications Systems (DDDAS) modeling for automatic target recognition , 2013, Defense, Security, and Sensing.

[26]  Shih-Fu Chang,et al.  Next-generation content representation, creation, and searching for new-media applications in education , 1998 .

[27]  Mark S. Nixon,et al.  Feature Extraction & Image Processing for Computer Vision, Third Edition , 2012 .

[28]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[29]  Erik Blasch,et al.  Dynamic Data Driven Applications System Concept for Information Fusion , 2013, ICCS.

[30]  Zhu Liu,et al.  Multimedia content analysis-using both audio and visual clues , 2000, IEEE Signal Process. Mag..

[31]  Erik Blasch,et al.  Multi-source Multi-modal Activity Recognition in Aerial Video Surveillance , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[32]  Erik Blasch,et al.  Cognitive fusion analysis based on context , 2004, SPIE Defense + Commercial Sensing.

[33]  James Llinas,et al.  High Level Information Fusion (HLIF): Survey of models, issues, and grand challenges , 2012, IEEE Aerospace and Electronic Systems Magazine.

[34]  Ramesh C. Jain,et al.  Metadata in video databases , 1994, SGMD.

[35]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[36]  A. Blaser,et al.  Data Base Techniques for Pictorial Applications , 1980, Lecture Notes in Computer Science.

[37]  Bin Chen,et al.  Feature Matching and Adaptive Prediction Models in an Object Tracking DDDAS , 2013, ICCS.

[38]  Jose B. Cruz,et al.  Game Theoretic Approach to Threat Prediction and Situation Awareness , 2006, 2006 9th International Conference on Information Fusion.

[39]  I. Kadar,et al.  Resource management coordination with level 2/3 fusion issues and challenges [Panel Report] , 2008, IEEE Aerospace and Electronic Systems Magazine.

[40]  Ruofei Zhang,et al.  Multimedia Data Mining: A Systematic Introduction to Concepts and Theory , 2008 .

[41]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[43]  Vaidy S. Sunderam,et al.  Predict: Privacy and Security Enhancing Dynamic Information Collection and Monitoring , 2013, ICCS.

[44]  Douglas Keislar,et al.  Content-Based Classification, Search, and Retrieval of Audio , 1996, IEEE Multim..

[45]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Dimitris N. Metaxas,et al.  Image-Based Stress Recognition Using a Model-Based Dynamic Face Tracking System , 2004, International Conference on Computational Science.

[47]  J. Grimes,et al.  What is multimedia? , 1991, IEEE Computer Graphics and Applications.

[48]  Takeo Kanade,et al.  Video OCR: indexing digital news libraries by recognition of superimposed captions , 1999, Multimedia Systems.

[49]  Chandrika Kamath,et al.  Robust techniques for background subtraction in urban traffic video , 2004, IS&T/SPIE Electronic Imaging.

[50]  Kannan Ramchandran,et al.  Multimedia Analysis and Retrieval System (MARS) Project , 1996, Data Processing Clinic.

[51]  Marcel Worring,et al.  Multimodal Video Indexing : A Review of the State-ofthe-art , 2001 .

[52]  Michael Hunter,et al.  A Dynamic Data Driven Application System for Vehicle Tracking , 2014, ICCS.

[53]  Daben Liu,et al.  Speech and language technologies for audio indexing and retrieval , 2000, Proceedings of the IEEE.

[54]  Boon-Lock Yeo,et al.  Video query: Research directions , 1998, IBM J. Res. Dev..

[55]  William I. Grosky,et al.  Negotiating the semantic gap: from feature maps to semantic landscapes , 2002, Pattern Recognit..

[56]  King-Sun Fu,et al.  Query-by-Pictorial-Example , 1980, IEEE Trans. Software Eng..

[57]  Alex Pentland,et al.  Photobook: Content-based manipulation of image databases , 1996, International Journal of Computer Vision.

[58]  Vasumathi Narayanan,et al.  A Survey of Content-Based Video Retrieval , 2008 .

[59]  Sidney S. Fels,et al.  Evaluation of Background Subtraction Algorithms with Post-Processing , 2008, 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance.

[60]  Steve Hoberman,et al.  Data Modeling Made Simple: A Practical Guide for Business and IT Professionals, , 2009 .

[61]  Zhonghai Wang,et al.  Video-based activity analysis using the L1 tracker on VIRAT data , 2013, 2013 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[62]  Jinoh Kim,et al.  DDDAS/ITR: A Data Mining and Exploration Middleware for Grid and Distributed Computing , 2007, International Conference on Computational Science.

[63]  Li Bai,et al.  Multiple source data fusion via sparse representation for robust visual tracking , 2011, 14th International Conference on Information Fusion.

[64]  Mark S. Nixon,et al.  Feature extraction & image processing for computer vision , 2012 .

[65]  Zhonghai Wang,et al.  Pattern of life from WAMI objects tracking based on visual context-aware tracking and infusion network models , 2013, Defense, Security, and Sensing.

[66]  Erik Blasch,et al.  Extraction of Semantic Activities from Twitter Data , 2013, STIDS.