Exploiting the Interplay Between Memory and Flash Storage for Sensor Data Management

Although memory is an important constraint in embedded sensor nodes, existing sensor applications and systems are typically designed to work under the memory constraints of a single platform and do not consider the interplay between memory and flash storage. In this paper, we present the design of a memory-adaptive flash-based sensor system that allows an application to exploit the presence of flash and adapt to different amounts of RAM on the embedded device. We describe how such a system can be exploited by sensor data management applications. Our design involves several novel features: flash and memory-efficient storage and indexing, techniques for efficient storage reclamation, and intelligent buffer management to maximize write coalescing. Our results show that our system is highly energy-efficient under different workloads, and can be configured for sensor platforms with memory constraints ranging from a few kilobytes to hundreds of kilobytes.

[1]  Mendel Rosenblum,et al.  The design and implementation of a log-structured file system , 1991, SOSP '91.

[2]  Dimitrios Gunopulos,et al.  Microhash: an efficient index structure for fash-based sensor devices , 2005, FAST'05.

[3]  Sang-Won Lee,et al.  Design of flash-based DBMS: an in-page logging approach , 2007, SIGMOD '07.

[4]  Kyriakos Mouratidis,et al.  Continuous monitoring of top-k queries over sliding windows , 2006, SIGMOD Conference.

[5]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Fred Douglis,et al.  Redundancy Elimination Within Large Collections of Files , 2004, USENIX Annual Technical Conference, General Track.

[7]  M.B. Ahmed,et al.  Conception of a pico-data base management system for a health smart card , 2004, Proceedings. 2004 International Conference on Information and Communication Technologies: From Theory to Applications, 2004..

[8]  Michael Neufeld,et al.  ELF: an efficient log-structured flash file system for micro sensor nodes , 2004, SenSys '04.

[9]  Sivan Toledo,et al.  Algorithms and data structures for flash memories , 2005, CSUR.

[10]  Suman Nath,et al.  FlashDB: Dynamic Self-tuning Database for NAND Flash , 2007, 2007 6th International Symposium on Information Processing in Sensor Networks.

[11]  Tei-Wei Kuo,et al.  An efficient B-tree layer implementation for flash-memory storage systems , 2007, TECS.

[12]  Gerhard Weikum,et al.  The LHAM log-structured history data access method , 2000, The VLDB Journal.

[13]  Peter Desnoyers,et al.  Capsule: an energy-optimized object storage system for memory-constrained sensor devices , 2006, SenSys '06.

[14]  Emery D. Berger,et al.  CRAMM: virtual memory support for garbage-collected applications , 2006, OSDI '06.

[15]  Wendy Hui Wang,et al.  The Threshold Algorithm: From Middleware Systems to the Relational Engine , 2007, IEEE Transactions on Knowledge and Data Engineering.

[16]  Eddie Kohler,et al.  SOS: A Dynamic Operating System for Sensor Networks , 2005 .

[17]  Peter Desnoyers,et al.  Ultra-low power data storage for sensor networks , 2009, TOSN.

[18]  David J. DeWitt,et al.  An evaluation of buffer management strategies for relational database systems , 1986, Algorithmica.

[19]  Mark D. Corner,et al.  Transparent Contribution of Memory , 2006, USENIX Annual Technical Conference, General Track.

[20]  Prashant J. Shenoy,et al.  Rethinking Data Management for Storage-centric Sensor Networks , 2007, CIDR.

[21]  Giovanni Maria Sacco,et al.  Buffer management in relational database systems , 1986, TODS.

[22]  Wolfgang Effelsberg,et al.  Principles of database buffer management , 1984, TODS.

[23]  Patrick E. O'Neil,et al.  Improved query performance with variant indexes , 1997, SIGMOD '97.

[24]  Wei Hong,et al.  TinyDB: an acquisitional query processing system for sensor networks , 2005, TODS.

[25]  Deepak Ganesan,et al.  Distributed Image Search in Sensor Networks , 2008 .

[26]  Andreas Savvides,et al.  XYZ: a motion-enabled, power aware sensor node platform for distributed sensor network applications , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[27]  Heeseung Jo,et al.  A superblock-based flash translation layer for NAND flash memory , 2006, EMSOFT '06.