應用於多車種網路之角色導向強化學習

Role-Oriented Reinforcement Learning For Multi-Vehicle-Type Networks

鄭宇恒(Zheng Yu-Heng)

指導教授 ： 陳建隆 ； 黃志煒

國立臺灣師範大學/理學院/數學系/碩士(2022年)

https://doi.org/10.6345/NTNU202201197

摘要

多車種的車聯網環境相較於單車種的環境更為現實和複雜，不同的車種對於通訊的策略
會有所不同。考慮緊急情況下，救護車會更需要傳輸資料給附近的其他車輛，而不是基礎設施或衛星。為了這個目的，我們可以利用先備知識定義每個車種的行為，但在這樣的架構下將會失去對於環境的適應性和彈性。因此我們利用角色導向的 actor-critic 演算法使得相同車種的車輛會有相同的策略，並且學習選擇使用何種傳輸模式、能量以及子頻道去最大化系統效益。根據角色導向的性質，所有車輛可以依照環境和自身的車種做出更好的決定。

關鍵字

車聯網 ； 資源分配 ； 多智能體強化學習

並列摘要

Vehicle-to-everything (V2X) communication including multi-type vehicles is more
realistic and complex than one-type scenario, vehicles with different type need different policy for communication. Considering an emergency, ambulance needs vehicle-to-vehicle(V2V) communication more than vehicle-to-satellite (V2S) and vehicle-to-infrastructure(V2I) to make way for itself. To this purpose, we could use prior domain knowledge to define the behaviors of each type, but this structure will lose adaptability and flexibility to environment. Therefore, we apply role-oriented actor-critic to make the vehicle agent with similar type share similar policy, and learn to arrange their transmission modes, power and sub-channel to maximize the system utility. With role-oriented property, each vehicle agent can make better decision according to environment and its type.

並列關鍵字

V2X ； Resource allocation ； Multi-agent reinforcement Learning

參考文獻

[1] J. Lianghai, A. Weinand, B. Han, and H. D. Schotten, Multi-RATs support to improve V2X communication in 2018 IEEE Wireless Communications and Networking Conference (WCNC), 2018, pp. 1–6. doi:10.1109/WCNC.2018.8377432.

[2] C. W. Huang, and Y. Y. Wu, Intelligent Multi-Connectivity Management for Satellite-aided Vehicular Networks in Department of Communication Engineering, National Central University, Taoyuan, Taiwan, 2021.

[3] T. Wang, H. Dong, V. Lesser, and C. Zhang, Roma: Multi-agent reinforcement learning with emergent roles, in Proceedings of the 37th International Conference on Machine Learning, 2020c.

[4] J. N. T. Vijay R. Konda, Actor-critic algorithms, Advances in neural information processing systems, p. 1008–1014, 2000.

[5] Zhang, R., Cheng, X., Yao, Q., Wang, C. X., Yang Y., Jiao, B. (2013). Interference graph-based resource-sharing schemes for vehicular networks, IEEE transactions on vehicular technology, 62(8), 4028-4039.