An end-to-end eChronicling System for Mobile Human Surveillance

Rapid advances in mobile computing devices and sensor technologies are enabling the capture of unprecedented volumes of data by individuals involved in field operations in a variety of applications. As capture becomes ever more rich and pervasive the biggest challenge is in developing information processing and representation tools that maximize the utility of the captured multi-sensory data. The right tools hold the promise of converting captured data into actionable intelligence resulting in improved memory, enhanced situational understanding, and more efficient execution of operations. These tools need to be at least as rich and diverse as the sensors used for capture, and need to be unified within an effective system architecture. This paper presents our initial attempt at such a system and architecture that combines several emerging sensor technologies, state of the art analytic engines, and multi-dimensional navigation tools, into an end-to-end electronic chronicling solution for mobile surveillance by humans.

[1]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[2]  David R. Karger,et al.  Haystack: A Platform for Creating, Organizing and Visualizing Information Using RDF , 2002, Semantic Web Workshop.

[3]  Jun Rekimoto,et al.  TimeScape: a time machine for the desktop environment , 1999, CHI Extended Abstracts.

[4]  Q. Summerfield.  Book Review: Auditory Scene Analysis: The Perceptual Organization of Sound , 1992 .

[5]  Kiyoharu Aizawa,et al.  Capturing life-log and retrieval based on contexts , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[6]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[7]  Pilho Kim,et al.  Personal chronicling tools for enhancing information archival and collaboration in enterprises , 2004, CARPE'04.

[8]  Ramesh C. Jain,et al.  Electronic Chronicles: Empowering Individuals, Groups, and Organizations , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[9]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[11]  Scott Axelrod,et al.  Discriminative estimation of subspace precision and mean (SPAM) models , 2003, INTERSPEECH.

[12]  Chalapathy Neti,et al.  Recent advances in the automatic recognition of audiovisual speech , 2003, Proc. IEEE.

[13]  Geoffrey Zweig,et al.  fMPE: discriminatively trained features for speech recognition , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[14]  Antonio G. Ruzzelli.  Proceedings of the First ACM Workshop on Embedded Sensing Systems for Energy-Efficiency in Buildings , 2009 .

[15]  Christine Collet,et al.  Towards a Semantic Event Service for Distributed Active Database Applications , 1998, DEXA.

[16]  Dorin Comaniciu,et al.  Design, analysis, and engineering of video monitoring systems: an approach and a case study , 2001, Proc. IEEE.

[17]  William M. Campbell,et al.  Advanced Language Recognition using Cepstra and Phonotactics: MITLL System Performance on the NIST 2005 Language Recognition Evaluation , 2006, 2006 IEEE Odyssey - The Speaker and Language Recognition Workshop.

[18]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Jirí Navrátil,et al.  Recent advances in phonotactic language recognition using binary-decision trees , 2006, INTERSPEECH.

[22]  A. Hampapur,et al.  Smart video surveillance: exploring the concept of multiscale spatiotemporal tracking , 2005, IEEE Signal Processing Magazine.

[23]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Harry Shum,et al.  Statistical Learning of Multi-view Face Detection , 2002, ECCV.

[25]  Paul A. Viola,et al.  Fast Multi-view Face Detection , 2003 .

[26]  David A. Ferrucci,et al.  Building an example application with the Unstructured Information Management Architecture , 2004, IBM Syst. J..

[27]  Felix C. Freiling,et al.  A modular approach to build structured event-based systems , 2002, SAC '02.

[28]  Vannevar Bush,et al.  As we may think , 1945, INTR.

[29]  Hironobu Fujiyoshi,et al.  A System for Video Surveillance and Monitoring CMU VSAM Final Report , 1999 .

[30]  Eric Freeman,et al.  Lifestreams: Organizing your Electronic Life* , 1995 .

[31]  Takeo Kanade,et al.  A statistical method for 3D object detection applied to faces and cars , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[32]  James G. Shanahan,et al.  Boosting support vector machines for text classification through parameter-free threshold relaxation , 2003, CIKM '03.

[33]  N. Paragios,et al.  Video-Based Surveillance Systems: Computer Vision and Distributed Processing , 2001 .

[34]  Jim Gemmell,et al.  Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences , 2004, MM 2004.

[35]  Ben Shneiderman,et al.  LifeLines: visualizing personal histories , 1996, CHI.

[36]  Thierry Coupaye,et al.  A Visualization Service for Event-Based Systems , 1999, Proc. 15èmes Journées Bases de Données Avancées, BDA.

[37]  P. Jonathon Phillips,et al.  Face Recognition Vendor Test 2002 Performance Metrics , 2003, AVBPA.

[38]  Alex Pentland,et al.  Wearable computing and contextual awareness , 1999 .

[39]  Yves Jean,et al.  LucentVision: converting real world events into multimedia experiences , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[40]  J. Scott McCarley,et al.  Infrastructure and Systems for Adaptive Speech and Text Analytics , 2022 .

[41]  J. William Murdock,et al.  Exploiting pervasive enterprise chronicles using unstructured information management , 2005, ICPS '05. Proceedings. International Conference on Pervasive Services, 2005..

[42]  Brendan J. Frey,et al.  Event-coupled hidden Markov models , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[43]  John R. Smith,et al.  Semantic Indexing of Multimedia Content Using Visual, Audio, and Text Cues , 2003, EURASIP J. Adv. Signal Process..

[44]  Thad Starner,et al.  Remembrance Agent: A Continuously Running Automated Information Retrieval System , 1996, PAAM.