基於多代理人與元預測之強化學習於冰機節能最佳化

Multi-agent and Meta-prediction-based Reinforcement Learning for Energy Saving Optimization in Chiller System

張智鈞(Chih-Chun Chang)

指導教授 ： 李家岩

國立臺灣大學/管理學院/資訊管理學研究所/碩士(2022年)

https://doi.org/10.6342/NTU202203474

摘要

隨著近年來智慧製造的趨勢，除了在生產線或排程上的最佳化，節能與環保的議題在製造現場也越來越受到重視，而在廠房的耗電量中，其中有一大部分的耗電量是來自冷氣機，工廠因為有大量的機台需要散熱，並且生產線又要維持在穩定的溫度，所以有大量的熱需要排出，工廠散熱使用的是冰水系統，由冰水主機、水塔、水泵等元件所組成，不同的元件其中又包含了不同的溫度控制點，本研究希望能透過即時控制多個溫度控制點，並考量其之間的交互作用與不同元件的關係，做到冰機節能最佳化。同時我們發現在做控制點設定時，冰機的狀態值的變化十分的重要，我們考慮控制點與狀態值之間的關係，建立了可以模擬冰機系統的元預測模型，選擇設定點後，模擬系統會更新冰機狀態直到穩定的狀態，最後再預測最終的效率值。建立完模擬模型後，我們使用強化學習去學習對應條件下最好的控制點動作，並且利用多智能體的機制，最佳化多點即時控制，達到節能的效果。本研究貢獻在於利用資料驅動模型來建立出複雜且多變的冰水系統，也說明了多智能體在多設定點下的幫助以及它不同的使用情境，最後我們的模擬模型不只可以用在冰水系統，更可以延伸到其他的多點控制系統，達到同樣的最佳化效果。

關鍵字

冰機最佳化 ； 多控制點 ； 元預測 ； 模擬系統 ； 強化學習 ； 時間序列預測 ； 多智能強化學習

並列摘要

With the trend of smart manufacturing in recent years, except for the optimization of the production lines or scheduling problems. Energy saving and environmental protection are also getting more and more attention in the manufacturing field. In the energy consumption of the factory, a large part of the energy consumption comes from the air conditioner. Because the factory has a large number of machines that need to dissipate heat, and the production line needs to maintain a stable temperature, it has a lot of cooling load. The factory uses a water-cooled chiller system for heat dissipation, which consists of the chiller, water tower, water pump, and other components. Different equipment includes multiple temperature setpoints. This study hopes to control the setpoints in real time and consider the interactive relationship between them and the energy consumption of different components in the system. Meanwhile, we found that the statuses of the chiller are also very important when setting the temperature setpoints. We consider the relationship between the setpoints and the statuses and construct the meta-prediction simulation model that can simulate the chiller system. After constructing the simulation model, we use reinforcement learning to learn the best setpoints action under the corresponding conditions and use the multi-agent technique to optimize the multi-point to achieve the effect of energy saving. The contribution of this research lies in the use of data-driven simulation models to build practical and complex chiller systems, and it also illustrates the help of multi-agents under multiple setpoints. Finally, our model can be used in chiller systems and other multi-point control systems and can be extended in the same way to achieve the effect of optimization.

並列關鍵字

Chiller system optimization ； Multiple setpoints ； Meta-prediction ； Simulation system ； Reinforcement learning ； Time series forecasting ； Multi-agent reinforcement learning

參考文獻

Breiman, L. (2001). Random forests. Machine learning, 45(1), 5-32.

Chien, C.-F., Chen, Y.-J., Han, Y.-T., Wu, Y.-C. (2021). Industry 3.5 for optimizing chiller configuration for energy saving and an empirical study for semiconductor manufacturing. Resources, Conservation and Recycling, 168, 105247.

Chow, T., Zhang, G., Lin, Z., Song, C. (2002). Global optimization of absorption chiller system by genetic algorithm and neural network. Energy and Buildings, 34(1), 103-109.

Evans, R., Gao, J. (2016). Deepmind AI reduces Google data centre cooling bill by 40%. DeepMind blog, 20, 158.

Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of statistics, 1189-1232.