Design of DRAM-NAND flash hybrid main memory and Q-learning-based prefetching method

Owing to the increased need for machine learning and artificial intelligence in current cloud computing systems, the amount of data that needs to be processed has exponentially increased. Thus, it is important to optimize memory and storage systems to reduce the energy consumption and execution time of applications. This paper proposes a new Q-learning-based prefetching algorithm for DRAM–NAND flash hybrid main memory architecture. To minimize the computational overheads of learning-based schemes, we have designed two learning policies, namely aggressive learning and lazy learning. The proposed system reduces the energy consumption by about 80% of the memory and storage for Redis, OpenStack Swift which is a cloud computing open source framework and Apache Storm workloads. Further, the overall execution time of workloads in cloud computing applications is reduced by almost half. Using a path generator with a Q-learning-based prefetching algorithm, we realize an increased hit rate of about 21% compared to that with a no-prefetching system, compared to non-prefetching system.

[1]  Jihong Kim,et al.  Personalized optimization for android smartphones , 2014, TECS.

[2]  Alan Jay Smith,et al.  Sequential Program Prefetching in Memory Hierarchies , 1978, Computer.

[3]  Abhinav Dutta.  Cheap and Large CAMs for High Performance Data-Intensive Networked Systems , .

[4]  James E. Smith,et al.  Data Cache Prefetching Using a Global History Buffer , 2005, IEEE Micro.

[5]  M.H. Kryder,et al.  After Hard Drives—What Comes Next? , 2009, IEEE Transactions on Magnetics.

[6]  Calvin Lin,et al.  Linearizing irregular memory accesses for improved correlated prefetching , 2013, 2013 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[7]  Li-Pin Chang,et al.  Hybrid solid-state disks: Combining heterogeneous NAND flash in large SSDs , 2008, 2008 Asia and South Pacific Design Automation Conference.

[8]  Rajiv Ranjan,et al.  Survey of Techniques and Architectures for Designing Energy-Efficient Data Centers , 2016, IEEE Systems Journal.

[9]  Brad Fitzpatrick,et al.  Distributed caching with memcached , 2004 .

[10]  Trevor N. Mudge,et al.  Improving NAND Flash Based Disk Caches , 2008, 2008 International Symposium on Computer Architecture.

[11]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[12]  Jing Xu,et al.  CloudCache: On-demand Flash Cache Management for Cloud Computing , 2016, FAST.

[13]  G. Kandiraju,et al.  Going the distance for TLB prefetching: an application-driven study , 2002, Proceedings 29th Annual International Symposium on Computer Architecture.

[14]  Chanik Park,et al.  Energy-aware demand paging on NAND flash-based embedded storages , 2004, Proceedings of the 2004 International Symposium on Low Power Electronics and Design (IEEE Cat. No.04TH8758).

[15]  Tajana Simunic,et al.  PDRAM: A hybrid PRAM and DRAM main memory system , 2009, 2009 46th ACM/IEEE Design Automation Conference.

[16]  Kei Hiraki,et al.  Access Map Pattern Matching for High Performance Data Cache Prefetch , 2011, J. Instr. Level Parallelism.

[17]  Steve Byan,et al.  Mercury: Host-side flash caching for the data center , 2012, 012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST).

[18]  Alexander V. Veidenbaum,et al.  Stride-directed prefetching for secondary caches , 1997, Proceedings of the 1997 International Conference on Parallel Processing (Cat. No.97TB100162).

[19]  Vijayalakshmi Srinivasan,et al.  Scalable high performance main memory system using phase-change memory technology , 2009, ISCA '09.

[20]  Yuanyuan Zhou,et al.  Association Proceedings of the Third USENIX Conference on File and Storage Technologies San Francisco , CA , USA March 31 – April 2 , 2004 , 2004 .

[21]  Trevor N. Mudge,et al.  FlashCache: a NAND flash memory file cache for low power web servers , 2006, CASES '06.

[22]  Joe Arnold,et al.  OpenStack Swift: Using, Administering, and Developing for Swift Object Storage , 2014 .

[23]  Jinchun Kim,et al.  Path confidence based lookahead prefetching , 2016, 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[24]  Dirk Grunwald,et al.  Prefetching Using Markov Predictors , 1997, Conference Proceedings. The 24th Annual International Symposium on Computer Architecture.

[25]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[26]  Pierre Michaud.  Best-offset hardware prefetching , 2016, 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[27]  Tony Tung,et al.  Scaling Memcache at Facebook , 2013, NSDI.

[28]  Youngjae Kim,et al.  FlashSim: A Simulator for NAND Flash-Based Solid-State Drives , 2009, 2009 First International Conference on Advances in System Simulation.

[29]  Yiran Chen,et al.  Utilizing PCM for Energy Optimization in Embedded Systems , 2012, 2012 IEEE Computer Society Annual Symposium on VLSI.

[30]  Mahadev Konar,et al.  ZooKeeper: Wait-free Coordination for Internet-scale Systems , 2010, USENIX ATC.

[31]  Nong Xiao,et al.  A hybrid memory built by SSD and DRAM to support in-memory Big Data analytics , 2013, Knowledge and Information Systems.

[32]  Claus Pahl,et al.  A Comparison of Reinforcement Learning Techniques for Fuzzy Cloud Auto-Scaling , 2017, 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID).

[33]  Charles C. Weems,et al.  A New Memory-Disk Integrated System with HW Optimizer , 2015, ACM Trans. Archit. Code Optim..

[34]  Gabriel H. Loh,et al.  3D-Stacked Memory Architectures for Multi-core Processors , 2008, 2008 International Symposium on Computer Architecture.

[35]  Margaret Martonosi,et al.  TCP: tag correlating prefetchers , 2003, The Ninth International Symposium on High-Performance Computer Architecture, 2003. HPCA-9 2003. Proceedings..

[36]  Adam Silberstein,et al.  Benchmarking cloud serving systems with YCSB , 2010, SoCC '10.

[37]  Fabrice Bellard,et al.  QEMU, a Fast and Portable Dynamic Translator , 2005, USENIX ATC, FREENIX Track.

[38]  Sachin Katti,et al.  Dynacache: Dynamic Cloud Caching , 2015, HotStorage.