Machine Learning for Power, Energy, and Thermal Management on Multicore Processors: A Survey

Due to the high integration density and roadblock of voltage scaling, modern multicore processors experience higher power densities than previous technology scaling nodes. When unattended, this issue might lead to temperature hot spots, that in turn may cause nonuniform aging, accelerate chip failure, impair reliability, and reduce the performance of the system. This paper presents an overview of several research efforts that propose to use machine learning (ML) techniques for power and thermal management on single-core and multicore processors. Traditional power and thermal management techniques rely on a certain a-priori knowledge of the chip’s thermal model, as well as information of the workloads/applications to be executed (e.g., transient and average power consumption). Nevertheless, these a-priori information is not always available, and even if it is, it cannot reflect the spatial and temporal uncertainties and variations that come from the environment, the hardware, or from the workloads/applications. Contrarily, techniques based on ML can potentially adapt to varying system conditions and workloads, learning from past events in order to improve themselves as the environment changes, resulting in improved management decisions.

[1]  Muhammad Shafique,et al.  The EDA challenges in the dark silicon era , 2014, 2014 51st ACM/EDAC/IEEE Design Automation Conference (DAC).

[2]  Ki-Seok Chung,et al.  On-line learning based dynamic thermal management for multicore systems , 2008, 2008 International SoC Design Conference.

[3]  Jing-Yang Jou,et al.  A learning-on-cloud power management policy for smart devices , 2014, 2014 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[4]  Hao Shen,et al.  Learning based DVFS for simultaneous temperature, performance and energy management , 2012, Thirteenth International Symposium on Quality Electronic Design (ISQED).

[5]  Luis Alfonso Maeda-Nunez,et al.  Learning Transfer-Based Adaptive Energy Minimization in Embedded Systems , 2016, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[6]  Russell Tessier,et al.  Reinforcement Learning for Thermal-aware Many-core Task Allocation , 2015, ACM Great Lakes Symposium on VLSI.

[7]  Kai Li,et al.  The PARSEC benchmark suite: Characterization and architectural implications , 2008, 2008 International Conference on Parallel Architectures and Compilation Techniques (PACT).

[8]  Zhao Zhang,et al.  Thermal modeling and management of DRAM memory systems , 2007, ISCA '07.

[9]  Karthick Rajamani,et al.  A performance-conserving approach for reducing peak power consumption in server systems , 2005, ICS '05.

[10]  Hao Yu,et al.  An energy-efficient 2.5D through-silicon interposer I/O with self-adaptive adjustment of output-voltage swing , 2014, 2014 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED).

[11]  Massoud Pedram,et al.  Hierarchical dynamic power management using model-free reinforcement learning , 2013, International Symposium on Quality Electronic Design (ISQED).

[12]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[13]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[14]  Henry Hoffmann,et al.  Self-Aware Adaptation in FPGA-based Systems , 2010, 2010 International Conference on Field Programmable Logic and Applications.

[15]  Samarjit Chakraborty,et al.  LMS-based low-complexity game workload prediction for DVFS , 2010, 2010 IEEE International Conference on Computer Design.

[16]  Anoop Gupta,et al.  The SPLASH-2 programs: characterization and methodological considerations , 1995, ISCA.

[17]  David Blaauw,et al.  Assessing the performance limits of parallelized near-threshold computing , 2012, DAC Design Automation Conference 2012.

[18]  Calvin Lin,et al.  A comprehensive approach to DRAM power management , 2008, 2008 IEEE 14th International Symposium on High Performance Computer Architecture.

[19]  Massoud Pedram,et al.  Reinforcement learning algorithms for dynamic power management , 2014, 2014 World Symposium on Computer Applications & Research (WSCAR).

[20]  John L. Henning.  SPEC CPU2006 benchmark descriptions , 2006, CARN.

[21]  Qiang Xu,et al.  Learning-Based Power Management for Multicore Processors via Idle Period Manipulation , 2014, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[22]  Luca Benini,et al.  Thermal and Energy Management of High-Performance Multicores: Distributed and Self-Calibrating Model-Predictive Controller , 2013, IEEE Transactions on Parallel and Distributed Systems.

[23]  Haoran Li,et al.  Modular reinforcement learning for self-adaptive energy efficiency optimization in multicore system , 2017, 2017 22nd Asia and South Pacific Design Automation Conference (ASP-DAC).

[24]  Tajana Simunic,et al.  System-Level Power Management Using Online Learning , 2009, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[25]  Luca Benini,et al.  A distributed and self-calibrating model-predictive controller for energy and thermal management of high-performance multicores , 2011, 2011 Design, Automation & Test in Europe.

[26]  Jie Huang,et al.  The HiBench benchmark suite: Characterization of the MapReduce-based data analysis , 2010, 2010 IEEE 26th International Conference on Data Engineering Workshops (ICDEW 2010).

[27]  Karthikeyan Sankaralingam,et al.  Dark Silicon and the End of Multicore Scaling , 2012, IEEE Micro.

[28]  Massoud Pedram,et al.  A Reinforcement Learning-Based Power Management Framework for Green Computing Data Centers , 2016, 2016 IEEE International Conference on Cloud Engineering (IC2E).

[29]  Wei Liu,et al.  Enhanced Q-learning algorithm for dynamic power management with performance constraint , 2010, 2010 Design, Automation & Test in Europe Conference & Exhibition (DATE 2010).

[30]  Sally A. McKee,et al.  Design of a parallel vector access unit for SDRAM memory systems , 2000, Proceedings Sixth International Symposium on High-Performance Computer Architecture. HPCA-6 (Cat. No.PR00550).

[31]  Ying Tan,et al.  Achieving autonomous power management using reinforcement learning , 2013, TODE.

[32]  Jianfeng Zhan,et al.  Understanding Big Data Analytics Workloads on Modern Processors , 2015, IEEE Transactions on Parallel and Distributed Systems.

[33]  Rahul Khanna,et al.  Phase-aware predictive thermal modeling for proactive load-balancing of compute clusters , 2012, 2012 International Conference on Energy Aware Computing.

[34]  Michael Bedford Taylor,et al.  Is dark silicon useful? Harnessing the four horsemen of the coming dark silicon apocalypse , 2012, DAC Design Automation Conference 2012.

[35]  Man Lin,et al.  A Framework for Learning Based DVFS Technique Selection and Frequency Scaling for Multi-core Real-Time Systems , 2015, 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conference on Embedded Software and Systems.

[36]  Ali Afzali-Kusha,et al.  Dynamic Voltage and Frequency Scheduling for Embedded Processors Considering Power/Performance Tradeoffs , 2011, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[37]  Chongshen Song,et al.  A Scalable Network-on-Chip Microprocessor With 2.5D Integrated Memory and Accelerator , 2017, IEEE Transactions on Circuits and Systems I: Regular Papers.

[38]  Li Shang,et al.  Power-efficient Interconnection Networks: Dynamic Voltage Scaling with Links , 2002, IEEE Computer Architecture Letters.

[39]  Sai Manoj Pudukotai Dinakarrao.  2.5D and 3D I/O designs for energy-efficient memory-logic integration towards thousand-core on-chip , 2015 .

[40]  Minyi Guo,et al.  Rank-Aware Dynamic Migrations and Adaptive Demotions for DRAM Power Management , 2014, IEEE Transactions on Computers.

[41]  Hao Yu,et al.  Reinforcement learning based self-adaptive voltage-swing adjustment of 2.5D I/Os for many-core microprocessor and memory communication , 2014, 2014 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[42]  Tajana Simunic,et al.  Temperature management in multiprocessor SoCs using online learning , 2008, 2008 45th ACM/IEEE Design Automation Conference.

[43]  Ningmei Yu,et al.  Q-Learning-Based Voltage-Swing Tuning and Compensation for 2.5-D Memory-Logic Integration , 2018, IEEE Design & Test.

[44]  Massoud Pedram,et al.  Fine-grained dynamic voltage and frequency scaling for precise energy and performance tradeoff based on the ratio of off-chip access to on-chip computation times , 2005 .

[45]  Qinru Qiu,et al.  Dynamic thermal management for multimedia applications using machine learning , 2011, 2011 48th ACM/EDAC/IEEE Design Automation Conference (DAC).

[46]  Hao Yu,et al.  3D Many-Core Microprocessor Power Management by Space-Time Multiplexing Based Demand-Supply Matching , 2015, IEEE Transactions on Computers.

[47]  Massoud Pedram,et al.  Optimizing a Reconfigurable Power Distribution Network in a Multicore Platform , 2015, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[48]  Luca Benini,et al.  Dynamic power management using adaptive learning tree , 1999, 1999 IEEE/ACM International Conference on Computer-Aided Design. Digest of Technical Papers (Cat. No.99CH37051).

[49]  Sherief Reda,et al.  Consistent runtime thermal prediction and control through workload phase detection , 2010, Design Automation Conference.

[50]  Jörg Henkel,et al.  Economic learning for thermal-aware power budgeting in many-core architectures , 2011, 2011 Proceedings of the Ninth IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS).

[51]  Massoud Pedram,et al.  Hierarchical power management of a system with autonomously power-managed components using reinforcement learning , 2015, Integr..

[52]  Heba Khdr,et al.  New trends in dark silicon , 2015, 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[53]  Bharadwaj Veeravalli,et al.  Reinforcement learning-based inter- and intra-application thermal optimization for lifetime improvement of multicore systems , 2014, 2014 51st ACM/EDAC/IEEE Design Automation Conference (DAC).

[54]  Santiago Pagani,et al.  Power, Energy, and Thermal Management for Clustered Manycores , 2016 .

[55]  Massoud Pedram,et al.  Supervised Learning Based Power Management for Multicore Processors , 2010, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[56]  Massoud Pedram,et al.  Reinforcement learning based dynamic power management with a hybrid power supply , 2012, 2012 IEEE 30th International Conference on Computer Design (ICCD).

[57]  Axel Jantsch,et al.  Adaptive Power Management for the On-Chip Communication Network , 2006, 9th EUROMICRO Conference on Digital System Design (DSD'06).

[58]  Geoff V. Merrett,et al.  Adaptive energy minimization of embedded heterogeneous systems using regression-based learning , 2015, 2015 25th International Workshop on Power and Timing Modeling, Optimization and Simulation (PATMOS).

[59]  Mahmut T. Kandemir,et al.  DRAM energy management using software and hardware directed power mode control , 2001, Proceedings HPCA Seventh International Symposium on High-Performance Computer Architecture.

[60]  Avesta Sasan,et al.  Machine Learning-Based Approaches for Energy-Efficiency Prediction and Scheduling in Composite Cores Architectures , 2017, 2017 IEEE International Conference on Computer Design (ICCD).

[61]  Pedro Trancoso,et al.  Scalable and Dynamic Global Power Management for Multicore Chips , 2015, PARMA-DITAM '15.

[62]  Massoud Pedram,et al.  Deriving a near-optimal power management policy using model-free reinforcement learning and Bayesian classification , 2011, 2011 48th ACM/EDAC/IEEE Design Automation Conference (DAC).

[63]  Wei Liu,et al.  Adaptive power management using reinforcement learning , 2009, 2009 IEEE/ACM International Conference on Computer-Aided Design - Digest of Technical Papers.

[64]  Kang G. Shin,et al.  Design and Implementation of Power-Aware Virtual Memory , 2003, USENIX ATC, General Track.

[65]  Hao Yu,et al.  Peak power reduction and workload balancing by space-time multiplexing based demand-supply matching for 3D thousand-core microprocessor , 2013, 2013 50th ACM/EDAC/IEEE Design Automation Conference (DAC).

[66]  Hao Yu,et al.  A Q-Learning Based Self-Adaptive I/O Communication for 2.5D Integrated Many-Core Microprocessor and Memory , 2016, IEEE Transactions on Computers.

[67]  Kang G. Shin,et al.  Improving energy efficiency by making DRAM less randomly accessed , 2005, ISLPED '05. Proceedings of the 2005 International Symposium on Low Power Electronics and Design, 2005..

[68]  Kevin Skadron,et al.  Predictive Temperature-Aware DVFS , 2010, IEEE Transactions on Computers.

[69]  Mehdi Kamal,et al.  A heuristic machine learning-based algorithm for power and thermal management of heterogeneous MPSoCs , 2015, 2015 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED).

[70]  Luca Benini,et al.  Dynamic power management for nonstationary service requests , 1999, Design, Automation and Test in Europe Conference and Exhibition, 1999. Proceedings (Cat. No. PR00078).

[71]  Diana Marculescu,et al.  Analysis of dynamic voltage/frequency scaling in chip-multiprocessors , 2007, Proceedings of the 2007 international symposium on Low power electronics and design (ISLPED '07).

[72]  Minyi Guo,et al.  Synergy of Dynamic Frequency Scaling and Demotion on DRAM Power Management: Models and Optimizations , 2015, IEEE Transactions on Computers.