FireFly Mosaic: A Vision-Enabled Wireless Sensor Networking System

With the advent of CMOS cameras, it is now possible to make compact, cheap and low-power image sensors capable of on-board image processing. These embedded vision sensors provide a rich new sensing modality enabling new classes of wireless sensor networking applications. In order to build these applications, system designers need to overcome challenges associated with limited bandwidth, limited power, group coordination and fusing of multiple camera views with various other sensory inputs. Real-time properties must be upheld if multiple vision sensors are to process data, communicate with each other and make a group decision before the measured environmental feature changes. In this paper, we present FireFly Mosaic, a wireless sensor network image processing framework with operating system, networking and image processing primitives that assist in the development of distributed vision-sensing tasks. Each FireFly Mosaic wireless camera consists of a FireFly (Rowe et al., 2006) node coupled with a CMUcam3 (Rowe et al., 2007) embedded vision processor. The FireFly nodes run the nano-RK (Eswaran et al., 2005) real-time operating system and communicate using the RT-link (Rowe et al., 2006) collision-free TDMA link protocol. Using FireFly Mosaic, we demonstrate an assisted living application capable of fusing multiple cameras with overlapping views to discover and monitor daily activities in a home. Using this application, we show how an integrated platform with support for time synchronization, a collision-free TDMA link layer, an underlying RTOS and an interface to an embedded vision sensor provides a stable framework for distributed real-time vision processing. To the best of our knowledge, this is the first wireless sensor networking system to integrate multiple coordinating cameras performing local processing.

[1]  Anthony Rowe,et al.  RT-Link: A Time-Synchronized Link Protocol for Energy- Constrained Multi-hop Wireless Networks , 2006, 2006 3rd Annual IEEE Communications Society on Sensor and Ad Hoc Communications and Networks.

[2]  Deborah Estrin,et al.  Cyclops: in situ image sensing and interpretation in wireless sensor networks , 2005, SenSys '05.

[3]  Alan Burns,et al.  Guest Editorial: A Review of Worst-Case Execution-Time Analysis , 2000, Real-Time Systems.

[4]  Gregory Dudek,et al.  Topology inference for a vision-based sensor network , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[5]  Rolf Ernst,et al.  Integrated analysis of communicating tasks in MPSoCs , 2006, Proceedings of the 4th International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS '06).

[6]  Anthony Rowe,et al.  Nano-RK: an energy-aware resource-centric RTOS for sensor networks , 2005, 26th IEEE International Real-Time Systems Symposium (RTSS'05).

[7]  T. Hamalainen,et al.  Overview of bus-based system-on-chip interconnections , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[8]  Luca Benini,et al.  A multiprocessor system-on-chip for real-time biomedical monitoring and analysis: architectural design space exploration , 2006, 2006 43rd ACM/IEEE Design Automation Conference.

[9]  Rolf Ernst,et al.  Scheduling analysis of real-time systems with precise modeling of cache related preemption delay , 2005, 17th Euromicro Conference on Real-Time Systems (ECRTS'05).

[10]  Zhi Lü.  Involutions fixing $\mathbb{RP}^{\text{odd}}\sqcup P(h,i)$, I , 2002 .

[11]  Chenyang Lu,et al.  RAP: a real-time communication architecture for large-scale wireless sensor networks , 2002, Proceedings. Eighth IEEE Real-Time and Embedded Technology and Applications Symposium.

[12]  Henrik Theiling,et al.  Fast and Precise WCET Prediction by Separated Cache and Path Analyses , 2000, Real-Time Systems.

[13]  Tarek F. Abdelzaher,et al.  On real-time capacity limits of multihop wireless sensor networks , 2004, 25th IEEE International Real-Time Systems Symposium.

[14]  Giorgio C. Buttazzo,et al.  Real-time resource reservation protocol for wireless mobile ad hoc networks , 2004, 25th IEEE International Real-Time Systems Symposium.

[15]  Chenyang Lu,et al.  SPEED: a stateless protocol for real-time communication in sensor networks , 2003, 23rd International Conference on Distributed Computing Systems, 2003. Proceedings..

[16]  Hamid K. Aghajan,et al.  Application Development in Vision-Enabled Wireless Sensor Networks , 2006, 2006 International Conference on Systems and Networks Communications (ICSNC'06).

[17]  Ronald L. Graham,et al.  Optimal scheduling for two-processor systems , 1972, Acta Informatica.

[18]  Rolf Ernst,et al.  Worst case timing analysis of input dependent data cache behavior , 2006, 18th Euromicro Conference on Real-Time Systems (ECRTS'06).

[19]  Eugenio Culurciello,et al.  A Lightweight Camera Sensor Network Operating on Symbolic Information , 2006 .

[20]  Anthony Rowe,et al.  CMUcam3: An Open Programmable Embedded Vision Sensor , 2007 .

[21]  Zafer Sahinoglu,et al.  Image Transmission over IEEE 802 . 15 . 4 and ZigBee Networks , 2005 .

[22]  Sharad Malik,et al.  Cache modeling for real-time software: beyond direct mapped instruction caches , 1996, 17th IEEE Real-Time Systems Symposium.

[23]  Anthony Rowe,et al.  FireFly: A Time Synchronized Real-Time Sensor Networking Platform , 2007 .

[24]  Reinhard Wilhelm,et al.  The influence of processor architecture on the design and the results of WCET tools , 2003, Proceedings of the IEEE.

[25]  Anthony Rowe,et al.  FireFly: a cross-layer platform for real-time embedded wireless networks , 2007, Real-Time Systems.

[26]  Lothar Thiele,et al.  Design for Timing Predictability , 2004, Real-Time Systems.

[27]  Nikil D. Dutt,et al.  Fast exploration of bus-based on-chip communication architectures , 2004, International Conference on Hardware/Software Codesign and System Synthesis, 2004. CODES + ISSS 2004..

[28]  Petru Eles,et al.  Analysis and optimization of distributed real-time embedded systems , 2004, ACM Trans. Design Autom. Electr. Syst..