- 學位論文
離線元強化學習應用於伺服器風扇控制邏輯
Thermal and Energy Management with Fan Control Through Offline Meta Reinforcement Learning
若您是本文的作者，可授權文章由華藝線上圖書館中協助推廣。
摘要
伺服器散熱系統攸關整體運作效能，由於伺服器體積大、流道劃分複雜，每個流道又同時會受眾多不同風扇之影響，且熱傳與流場特性均為非線性，所以傳統上對於風扇散熱控制需透過工程師進行大規模量測調教找出最適當的風扇曲線，風扇邏輯設計上耗時且不易針對不同客戶的硬體需求與機房環境進行風扇曲線之客製化設計。故我們透過導入離線元強化學習的方法優化伺服器風扇控制邏輯之應用，根據事先蒐集的風扇控制與溫控收益反饋資料訓練模型快速找出最佳的控制策略，我們在穩態控制測試項目中不僅能維持CPU在理想的溫度之下並且模型的風扇曲線與基準模型相比總功率節省了21%，此外在CPU壓力測試環節中我們的模型亦優化了18%的能耗數據。
並列摘要
Reinforcement learning has drawn great attention in various fields such as computer vision, natural language processing, robotics, and so on. In this work, we present an empirical study involving three distinct offline meta-reinforcement learning approaches for the task of fan control, focusing on thermal and energy management. Our fan control models enable adaptive fan speed control that can not only protect the device from heat crash but also effectively reduce power consumption. To better assess the fan control performance, besides the industry-standard steady-state test, we also conduct CPU-stress test to simulate a more general deployment scenario for the server where the workload is more random and dynamic. Compared to the commercially available technique, our solution improves the performance of power consumption by up to 21% reduction for a real 2U-server in the worst thermal setting of the hardware configuration. Our solution can be widely applied to the thermal and energy management of server systems.
參考文獻
[1] B. Acun, E. K. Lee, Y. Park, and L. V. Kale. Neural network-based task scheduling with preemptive fan control. In 2016 4th International Workshop on Energy Efficient Supercomputing (E2SC), pages 77–84, 2016.
[2] W.-X. Chu, Y.-H. Lien, K.-R. Huang, and C.-C. Wang. Energy saving of fans in aircooled server via deep reinforcement learning algorithm. Energy Reports, 7:3437–3448, 2021.
[3] C. Finn. Learning to Learn with Gradients. PhD thesis, EECS Department, University of California, Berkeley, Aug 2018.
[4] C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pages 1126–1135. PMLR, 2017.
[5] C. Finn and S. Levine. Meta-learning and universality: Deep representations and gradient descent can approximate any learning algorithm. In International Conference on Learning Representations, 2018.