Embedded Middleware for Smart Camera Networks and Sensor Fusion

Smart cameras represent an interesting research field that has evolved over the last decade. In this chapter we focus on the integration of multiple, potentially heterogeneous smart cameras into a distributed system for computer vision and sensor fusion. Because an important aspect of distributed systems is the system-level software, also called middleware, we discuss the middleware requirements of distributed smart cameras and the services the middleware must provide. In our opinion, a middleware following the agent-oriented paradigm allows us to build flexible and self-organizing applications that encourage a modular design.

[1]  Mubarak Shah,et al.  Consistent Labeling of Tracked Objects in Multiple Cameras with Overlapping Fields of View , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Bernhard Rinner,et al.  Autonomous Multicamera Tracking on Embedded Smart Cameras , 2007, EURASIP J. Embed. Syst..

[3]  David E. Culler,et al.  TinyOS: An Operating System for Sensor Networks , 2005, Ambient Intelligence.

[4]  James Llinas,et al.  An introduction to multi-sensor data fusion , 1998, ISCAS '98. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (Cat. No.98CH36187).

[5]  Bernhard Rinner,et al.  Distributed embedded smart cameras for surveillance applications , 2006, Computer.

[6]  Manolis Koubarakis,et al.  Agents in decentralised information ecosystems:the diet approach , 2001 .

[7]  Wei Hong,et al.  TinyDB: an acquisitional query processing system for sensor networks , 2005, TODS.

[8]  Li-Der Chou,et al.  Implementation of Mobile-Agent-Based Network Management Systems for National Broadband Experimental Networks in Taiwan , 2003, HoloMAS.

[9]  Sheikh Iqbal Ahamed,et al.  A survey of middleware for sensor network and challenges , 2006, 2006 International Conference on Parallel Processing Workshops (ICPPW'06).

[10]  Esmond Pitt,et al.  java.rmi: The Remote Method Invocation Guide , 2001 .

[11]  Douglas C. Schmidt.  An architectural overview of the ACE framework: a case study of successful cross-platform systems software reuse , 1999 .

[12]  Douglas C. Schmidt,et al.  Middleware for real-time and embedded systems , 2002, CACM.

[13]  Erwin Bonsma,et al.  Core specification and experiments in DIET: a decentralised ecosystem-inspired mobile agent system , 2002, AAMAS '02.

[14]  Wayne H. Wolf,et al.  A peer-to-peer architecture for distributed real-time gesture recognition , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[15]  Makoto Yokoo,et al.  Trends of internet auctions and agent-mediated Web commerce , 2001, New Generation Computing.

[16]  Giovanni Vigna,et al.  Understanding Code Mobility , 1998, IEEE Trans. Software Eng..

[17]  Oscar de Bruijn,et al.  Living memory: agent-based information management for connected local communities , 2002, Interact. Comput..

[18]  Bernhard Rinner,et al.  Distributed Multilevel Data Fusion for Networked Embedded Systems , 2008, IEEE Journal of Selected Topics in Signal Processing.

[19]  George Cybenko,et al.  D'Agents: Applications and performance of a mobile‐agent system , 2002, Softw. Pract. Exp..

[20]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[21]  Ram Dantu,et al.  Collaborative vision using networked sensors , 2004, International Conference on Information Technology: Coding and Computing, 2004. Proceedings. ITCC 2004..

[22]  Ian F. Akyildiz,et al.  A survey on wireless multimedia sensor networks , 2007, Comput. Networks.

[23]  Luigi Carro,et al.  Distributed real-time embedded systems: Recent advances, future trends and their impact on manufactu , 2006 .

[24]  Jim White,et al.  Telescript technology: mobile agent , 1999 .

[25]  Lisa M. Brown,et al.  IBM smart surveillance system (S3): a open and extensible framework for event based surveillance , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[26]  Luigi Carro,et al.  DISTRIBUTED REAL-TIME EMBEDDED SYSTEMS: RECENT ADVANCES, FUTURE TRENDS AND THEIR IMPACT ON MANUFACTURING PLANT CONTROL , 2006 .

[27]  Wei Zhang,et al.  Video Compass , 2002, ECCV.

[28]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[29]  Dejan S. Milojicic,et al.  MASIF: The OMG mobile agent system interoperability facility , 1998, Personal Technologies.

[30]  Keith J. Blow,et al.  Making motes intelligent: an agent-based approach to wireless sensor networks , 2006 .

[31]  Chenyang Lu,et al.  Rapid Development and Flexible Deployment of Adaptive Wireless Sensor Network Applications , 2005, 25th IEEE International Conference on Distributed Computing Systems (ICDCS'05).

[32]  Behzad Moshiri,et al.  Pseudo information measure: a new concept for extension of Bayesian fusion in robotic map building , 2002, Inf. Fusion.

[33]  Bernhard Rinner,et al.  Embedded Middleware on Distributed Smart Cameras , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[34]  Chih-Lin Hu,et al.  A mobile agent-based active network architecture for intelligent network control , 2002, Inf. Sci..

[35]  Douglas C. Schmidt,et al.  Research Advances in Middleware for Distributed Systems , 2002, Communication Systems: The State of the Art.

[36]  Bernhard Rinner,et al.  A novel software framework for power-aware reconfiguration in distributed embedded smart cameras , 2006, 12th International Conference on Parallel and Distributed Systems - (ICPADS'06).

[37]  Jake K. Aggarwal,et al.  Tracking Human Motion in Structured Environments Using a Distributed-Camera System , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Tuomas Sandholm.  eMediator: A Next Generation Electronic Commerce Server , 2002, Comput. Intell..

[39]  Richard P. Kleihorst,et al.  Architecture and Applications of wireless Smart Cameras (Networks) , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[40]  James Llinas,et al.  Handbook of Multisensor Data Fusion , 2001 .

[41]  Thomas Magedanz,et al.  Grasshopper - A Mobile Agent Platform for Active Telecommunication , 1999, IATA.

[42]  Theodorus J.A. de Vries,et al.  An agent-based framework for designing multi-controller systems , 2000 .

[43]  Viktor K. Prasanna,et al.  Issues in designing middleware for wireless sensor networks , 2004, IEEE Network.

[44]  Belur V. Dasarathy,et al.  Information Fusion - what, where, why, when, and how? Editorial , 2001, Inf. Fusion.

[45]  Roger Sessions.  COM and DCOM - Microsoft's vision for distributed objects , 1997 .

[46]  Douglas C. Schmidt,et al.  The design of the TAO real-time object request broker , 1998, Comput. Commun..

[47]  Horst Bischof,et al.  Online Auto-Calibration in Man-Made Worlds , 2005, Digital Image Computing: Techniques and Applications (DICTA'05).

[48]  Michael Wooldridge,et al.  Intelligent Agents: The Key Concepts , 2001, Multi-Agent-Systems and Applications.

[49]  Nicholas R. Jennings,et al.  Agent-based control systems: Why are they suited to engineering complex systems? , 2003 .

[50]  Bernhard Rinner,et al.  An Introduction to Distributed Smart Cameras , 2008, Proceedings of the IEEE.

[51]  Luigi Carro,et al.  Distributed real-time embedded systems: Recent advances, future trends and their impact on manufacturing plant control , 2007, Annu. Rev. Control..

[52]  Bernhard Rinner,et al.  Integrating multi-camera tracking into a dynamic task allocation system for smart cameras , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[53]  Mitsuru Oshima,et al.  Infrastructure for Mobile Agents: Requirements and Design , 1998, Mobile Agents.

[54]  Wayne H. Wolf,et al.  Smart Cameras as Embedded Systems , 2002, Computer.

[55]  Bo Chen,et al.  Mobile-C: a mobile agent platform for mobile C-C++ agents , 2006 .

[56]  Alan Pope,et al.  The CORBA reference guide - understanding the common object request broker architecture , 1998 .

[57]  Douglas C. Schmidt,et al.  An overview of the Real-Time CORBA specification , 2000, Computer.

[58]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[59]  W. Grimson,et al.  Ground Plane Rectification by Tracking Moving Objects , 2003 .

[60]  Bernhard Rinner,et al.  A novel software framework for embedded multiprocessor smart cameras , 2009, TECS.

[61]  Philippe Bonnet,et al.  Towards Sensor Database Systems , 2001, Mobile Data Management.

[62]  Grady Booch,et al.  Essential COM , 1998 .