An Efficient and Flexible learning Framework for Dynamic Power and Thermal Co-Management

At the era of Artificial Intelligence and Internet of Things (AIoT), battery-powered mobile devices are required to perform more sophisticated tasks featured with fast varying workloads and constrained power supply, demanding more efficient run-time power management. In this paper, we propose a deep reinforcement learning framework for dynamic power and thermal co-management. We build several machine learning models that incorporate the physical details for an ARM Cortex-A72, with on average 3% and 1% error for power and temperature predictions, respectively. We then build an efficient deep reinforcement learning control incorporating the machine learning models and facilitating the run-time dynamic voltage and frequency scaling (DVFS) strategy selection based on the predicted power, workloads and temperature. We evaluate our proposed framework, and compare the performance with existing management methods. The results suggest that our proposed framework can achieve 6.8% performance improvement compared with other alternatives.

[1]  Mateo Valero,et al.  Power efficient job scheduling by predicting the impact of processor manufacturing variability , 2019, ICS.

[2]  Umit Y. Ogras,et al.  Algorithmic Optimization of Thermal and Power Management for Heterogeneous Mobile Platforms , 2018, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[3]  Samuel Kounev,et al.  Analysis of the Influences on Server Power Consumption and Energy Efficiency for CPU-Intensive Workloads , 2015, ICPE.

[4]  Qingchen Zhang,et al.  Autonomous Power Management With Double-Q Reinforcement Learning Method , 2020, IEEE Transactions on Industrial Informatics.

[5]  Margaret Martonosi,et al.  An Analysis of Efficient Multi-Core Global Power Management Policies: Maximizing Performance for a Given Power Budget , 2006, 2006 39th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO'06).

[6]  Jung Ho Ahn,et al.  McPAT: An integrated power, area, and timing modeling framework for multicore and manycore architectures , 2009, 2009 42nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[7]  Tajana Simunic,et al.  Accurate Direct and Indirect On-Chip Temperature Sensing for Efficient Dynamic Thermal Management , 2010, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[8]  Kevin Skadron,et al.  Temperature-aware microarchitecture: Modeling and implementation , 2004, TACO.

[9]  Yiyu Shi,et al.  A Multi-Level-Optimization Framework for FPGA-Based Cellular Neural Network Implementation , 2018, ACM J. Emerg. Technol. Comput. Syst..

[10]  Ludwik Kurz,et al.  The Kolmogorov-Smirnov tests in signal detection (Corresp.) , 1967, IEEE Trans. Inf. Theory.

[11]  Kai Li,et al.  PARSEC3.0: A Multicore Benchmark Suite with Network Stacks and SPLASH-2X , 2017, CARN.

[12]  Tajana Simunic,et al.  Cool and save: Cooling aware dynamic workload scheduling in multi-socket CPU systems , 2010, 2010 15th Asia and South Pacific Design Automation Conference (ASP-DAC).

[13]  Geoff V. Merrett,et al.  Accurate and stable empirical CPU power modelling for multi- and many-core systems , 2018 .

[14]  Yiyu Shi,et al.  From Layout to System: Early Stage Power Delivery and Architecture Co-Exploration , 2019, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[15]  Zhiguo Shi,et al.  Noise-Aware DVFS for Efficient Transitions on Battery-Powered IoT Devices , 2020, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[16]  Stéphane Rubini,et al.  HyMAD: a hybrid memory-aware DVFS strategy , 2019, SIGBED.

[17]  Josep Torrellas,et al.  Variation-Aware Application Scheduling and Power Management for Chip Multiprocessors , 2008, 2008 International Symposium on Computer Architecture.

[18]  Hassan Ghasemzadeh,et al.  A Dynamic Programming Framework for DVFS-Based Energy-Efficiency in Multicore Systems , 2020, IEEE Transactions on Sustainable Computing.

[19]  Li Shang,et al.  Multi-Optimization power management for chip multiprocessors , 2008, 2008 International Conference on Parallel Architectures and Compilation Techniques (PACT).

[20]  Ivo Pereira,et al.  Negotiation mechanism for self-organized scheduling system with collective intelligence , 2014, Neurocomputing.

[21]  Yu-Min Lee,et al.  XGBIR: An XGBoost-based IR Drop Predictor for Power Delivery Network , 2020, 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[22]  Somayeh Sardashti,et al.  The gem5 simulator , 2011, CARN.

[23]  Kevin Skadron,et al.  HotSpot: a compact thermal modeling methodology for early-stage VLSI design , 2006, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[24]  Efraim Rotem,et al.  Power and thermal constraints of modern system-on-a-chip computer , 2013, 19th International Workshop on Thermal Investigations of ICs and Systems (THERMINIC).

[25]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[26]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[27]  Saurabh Jain,et al.  Integrated Power Management for Battery-Indifferent Systems With Ultra-Wide Adaptation Down to nW , 2020, IEEE Journal of Solid-State Circuits.

[28]  James Charles,et al.  Evaluation of the Intel® Core™ i7 Turbo Boost feature , 2009, 2009 IEEE International Symposium on Workload Characterization (IISWC).

[29]  John L. Henning SPEC CPU2006 benchmark descriptions , 2006, CARN.

[30]  Yiyu Shi,et al.  Dynamic Frequency Scaling Aware Opportunistic Through-Silicon-Via Inductor Utilization in Resonant Clocking , 2020, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[31]  Zhiguo Shi,et al.  Energy-Efficient Real-Time UAV Object Detection on Embedded Platforms , 2020, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[32]  Amit Kumar Singh,et al.  P-EdgeCoolingMode: an agent-based performance aware thermal management unit for DVFS enabled heterogeneous MPSoCs , 2019, IET Comput. Digit. Tech..

[33]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.