結合Q適應學習法之動態電壓緩降調配機制

Q-Learning Based Dynamic Voltage Scaling for Designs with Graceful Degradation

溫婉妤(Wan-Yu Wen)

指導教授 ： 張世杰

國立清華大學/電機資訊學院/資訊工程學系/碩士(2015年)

摘要

現今，動態電壓調配機制(Dynamic Voltage Scaling)被廣泛地用於解決時下電路設計之電源耗電問題。決定每個運行時間下的最佳操作電壓需考量工作負荷量變異、製程變異以及環境因素變異。在晶片設計階段，上述變異議題的預測實屬困難，故過去有許多文獻提出基於增強式學習法的動態電壓調配機制。其中，電壓緩降法則能基於臨界機率容忍時序錯誤，進而達到電源節省。然而，並沒有動態電壓調配機制的研究同時考慮電壓緩降法則。在本論文中，我們提出結合「Q適應學習法(Q-Learning Algorithm)」的動態電壓緩降調配機制，透過關鍵電路路徑監控器和基於時序容錯率的電壓預測，同時考慮溫度、電壓以及工作負荷量的晶片變異議題，達到效能最佳化及電源節省的目的。為評估其機制之優良，我們比較了兩種決定型動態電壓調配機制：步進式機制、統計模型式機制。相較於此兩種傳統機制，實驗結果顯示，針對三種45奈米工業製程晶片設計，我們所提出的Q適應學習法動態電壓緩降調配機制(搭配0.01的時序容忍機率值)可分別節省至多83.9%、29.1%的電源耗損。此研究為首度探討在電壓緩降法則下，基於增強式學習法的電壓緩降調配機制。

關鍵字

電壓緩慢降級 ； 動態電壓調配 ； Q適應學習法

並列摘要

Dynamic voltage scaling (DVS) has been widely used to suppress power consumption in modern designs. The decision of optimal operating voltage at runtime should consider the variations in workload, process as well as environment. As these variations are hard to predict accurately at design time, various reinforcement learning based DVS schemes have been proposed in the literature. However, none of them can be readily applied to designs with graceful degradation, where timing errors are allowed with bounded probability to trade for further power reduction. In this thesis, we propose a Q-learning based DVS scheme dedicated to the designs with graceful degradation. We compare it with two deterministic DVS schemes, i.e., a stepping based scheme and a statistical modeling based scheme. Experimental results on three 45nm industrial designs show that the proposed Q-learning based scheme can achieve up to 83.9% and 29.1% power reduction respectively with 0.01 timing error probability bound. This is the first in-depth work to explore reinforcement learning based DVS schemes for designs with graceful degradation.

並列關鍵字

Graceful Degradation ； Dynamic Voltage Scaling ； Q-Learning

參考文獻

[2] T. Austin, V. Bertacco, D. Blaauw and T. Mudge, "Oppotunities and Challenges for Better Than Worst-Case Design", in Proc. Asia and South Pacific Design Automation Conf., 2005, pp. 2-7.

[4] L. N. Chakrapani, B. E. S. Akgul, S. Cheemalavagu, P. Korkmaz, K. V. Palem and B. Seshasayee, “Ultra-Efficient (Embedded) SOC Architectures Based on Probabilistic CMOS (PCMOS) Technology”, in Proc. Design Automation and Test in Europe, 2006, pp. 1110-1115.

[5] Y.G. Chen, T.Wang, K.Y. Lai, W.Y. Wen, Y. Shi, and S.C. Chang, “Critical path monitor enabled dynamic voltage scaling for graceful degradation in sub-threshold designs,” in Proc. of 51st ACM/EDAC/IEEE Design Automation Conference (DAC), pp.1-6, June 2014.

[6] D. Chinnery, “High performance and low power design techniques for ASIC and custom in nanometer technologies,” in Proc. of the 2013 ACM international symposium on International Symposium on Physical Design (ISPD), pp.25-32. 2013.

[10] M.E. Gomez, V. Santonja, "Self-similarity in I/O workload: analysis