在5G網路中達到QoE最大化的基於強化學習換手方法

Reinforcement Learning-Based Handover Approach for QoE Maximization in 5G Networks

張璞剛(Pu-Kang Chang)

指導教授 ： 蔡明哲

國立清華大學/電機資訊學院/資訊工程學系所/碩士(2021年)

摘要

隨著5G網路快速的發展，為了達到更好的可靠性及表現，發展低延遲及高頻寬
為其中的目標，而用戶以及基地台的數量在未來的發展自然也大量的增加，這
便導致了儘管在跟以前網路在相同的範圍之下，遭受到的干擾以及對Quality of
Experience (QoE)的影響相較之前更加劇烈，更加難以保持甚至提升用戶的要求。
再來，小覆蓋範圍的基地台自然導致了更多的換手發生，頻繁以及不佳的換手選擇
更影響了用戶的體驗嚴重甚至斷線，因此，考慮一個對用戶更有效率的換手方式至
關重要。本文中，由於無法獲得完整的環境資訊，不少資訊為未知，例如：排程
器，所以透過使用RL來跟環境互動使幫助用戶做出換手決定，實驗結果也表明在
平均QoE的部分我們的方法可以比傳統的方法更好。

關鍵字

換手 ； 強化學習

並列摘要

With the rapid development of 5G networks, to improve better reliability and performance, low latency and high bandwidth are the key features of 5G, and the number
of user equipmetes (UEs) and base stations (BSs) increase many times. Compared
to previous networks, although a UE moves in the same area, the interference and
the impact on Quality of Experience (QoE) are more severe than before. It is more
challenging to maintain or even improve QoE for UEs. Furthermore, BSs with a small
coverage area leads to more handovers. Frequent and unnecessary handovers affect
the experience even cause radio link failure. Therefore, it is crucial to consider a more
efficient handover for users. In this thesis, since we do not know all information about
the environment, such as the scheduler, we attempt to propose a handover algorithm
using reinforcement learning to interact with the environment. The simulation result
shows our method outperforms two baselines owes in terms of the average QoE of
UEs.

並列關鍵字

Handover ； Reinforcement Learning

參考文獻

[1] 3GPP, “Evolved Universal Terrestrial Radio Access (E-UTRA); Radio

Resource Control (RRC); Protocol specification ,” 3rd Generation Partnership Project (3GPP), Technical Specification (TS) 36.331, 3 2021, version 16.4.0. [Online]. Available: https://portal.3gpp.org/desktopmodules/Specifications/

SpecificationDetails.aspx?specificationId=2440

[2] K. Da Costa Silva, Z. Becvar, and C. R. L. Frances, “Adaptive hysteresis margin based on fuzzy logic for handover in mobile networks with dense small cells,” IEEE Access, vol. 6, pp. 17 178–17 189, 2018.

[3] K. C. Silva, Z. Becvar, E. H. S. Cardoso, and C. R. Francˆes, “Self-tuning handover algorithm based on fuzzy logic in mobile networks with dense small cells,” in 2018 IEEE Wireless Communications and Networking Conference (WCNC), 2018, pp. 1–6.