整合使用者行為與深度強化學習之居家即時需量電力管理系統

A Real-time Demand-side Management System Considering User Behavior Using Deep Q-Learning in Home Area Network

戴嘉星(Chia-Shing Tai)

指導教授 ： 傅立成

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2019年)

https://doi.org/10.6342/NTU201903463

摘要

在智慧電網架構中，需量管理系統儼然成為一個重要議題。透過智慧電網與需求端管理，可以在時間電價的機制上通過智能控制和重新調度負載來降低總電力成本，同時降低電力負載的波動性。另一方面，隨著物聯網技術的發展，智慧家庭現在能夠即時監控其家庭狀況並控制能源需求；並且能夠構築自己的家庭區域感測器網路來構建大數據資料庫。隨著近年來電腦計算能力的提高，透過分析大數據資料庫，強化學習等機器學習技術可以很好地應用於需量管理最佳化問題。然而，由於用戶行為和電力消耗的不確定性，仍舊很難確定何為最佳能源管理策略。
在本論文中，提出了一種基於實時多智能體的深度強化學習的方法來解決居家中的需量管理系統問題，並另外考慮了用戶行為以避免干擾用戶舒適度，同時能夠自適應地學習使用者的電器使用偏好並在每日更新中微調系統。在模擬實驗結果中，所提出的需量管理系統提高了智能家居的能源效率，不僅降低了電力成本和峰值，同時降低電力負載的波動性。

關鍵字

需量管理系統 ； 家庭區域感測器網路 ； 強化學習 ； Deep Q-網絡 ； 用戶行為 ； 智慧電網

並列摘要

In smart grids, demand-side management (DSM) has become an important topic since it can reduce the total electricity cost by smart control and rescheduling of loads, meanwhile, reduce the peak-to-average ratio (PAR) under real-time pricing policy. On the other hand, with the growth of IoT technologies, a smart home can nowadays monitor its household status and control the energy demands; besides, construct their own home area network (HAN) and build the big data database. Thanks to the growing computation ability in recent years, the machine learning skills such as reinforcement learning can be well applied into the DSM problem.
However, it is hard to determine a suitable energy management strategy due to the uncertainty of user behavior and the electricity consumption. In the proposed work, a real-time multi-agent deep reinforcement learning based approach has been proposed to solve the DSM problem in HAN, and additionally to consider the user behavior to avoid disturbing user comfort; meanwhile, adaptively learns the appliance usage preference and renew the system day after. The simulation results reveal that the proposed DSM system has improved the energy efficiency in a smart home that not only reduces the electricity cost and peak value but also the PAR value.

並列關鍵字

Demand-side management ； home area network ； reinforcement learning ； Deep Q-Network ； user behavior ； smart grid

參考文獻

[1] B. P. Company, "BP Statistical Review of World Energy," 2018.

[2] Bureau of Energy Ministry of Economic Affairs. (2019, March 23). Energy Statistics Inquiry System. Available: https://www.moeaboe.gov.tw/wesnq/Views/D01/wFrmD0101.aspx

[3] L. Gelazanskas and K. A. Gamage, "Demand side management in smart grid: A review and proposals for future direction," Sustainable Cities Society, vol. 11, pp. 22-30, 2014.

[4] T. Strasser et al., "A review of architectures and concepts for intelligence in future electric energy systems," IEEE Transactions on Industrial electronics, vol. 62, no. 4, pp. 2424-2438, 2014.

[5] C.-L. Wu, Y. Xie, S. K. Pradhan, L.-C. Fu, and Y.-C. Zeng, "Unsupervised context discovery based on hierarchical fusion of heterogeneous features in real smart living environments," in IEEE International Conference on Automation Science and Engineering (CASE), 2016, pp. 1106-1111.