基於遷移式學習之電競情緒識別於選手表現關聯度之研究

The Research of Relationship between Gaming Emotion Recognition Based on Transfer Learning and eSports Player's Corresponding Performance

林倧琦(Zong-Qi Lin)

指導教授 ： 洪啟舜

國立臺中科技大學/資訊與流通學院/資訊工程系碩士班/碩士(2020年)

摘要

電競教練如果想知道選手在比賽過程中的狀況，往往只能透過賽後觀看選手的比賽影片找出失誤的關鍵時間點，這過程會花費許多的時間。選手的臉部情緒對於比賽的勝負與遊戲操作有著相當的關連性。因此，辨識選手的臉部情緒可以有效的掌握選手在比賽過程中的情緒，透過情緒的起伏知道選手比賽的狀況及提供建議與協助。

臉部情緒辨識的相關研究中，目前最常使用的是卷積神經網路。因為在傳統的機器學習中需要透過人工設計特徵或尋找特徵，而在卷積神經網路則可以從影像中自動學習到紋理特徵，學習的過程中需要從大量的資料中進行學習。本論文透過建立電競臉部情緒資料集與使用Dense_FaceLiveNet之架構在建立電競臉部情緒辨識模型。使用辨識模型分析取得比賽影片中選手出現的情緒與時間點，之後再建立電競影片自動摘要擷取模型取得選手比賽的正面與負面表現片段，給予教練檢討使用，最後完成電競訓練輔助系統。

本論文所收集的電競臉部情緒資料集總共有1275張影像，之後使用提出的多層次的遷移學習在訓練模型上。首先從CK+、FER2013與學習情緒三個情緒辨識模型進行多種不同方式的遷移式學習，最後經實驗結果顯示在三次遷移學習中，資料集由小到大進行遷移同時考慮所要遷移的最後目標資料集類別與相近的資料集需擺放的越靠近，可發揮遷移式學習的最大效用準確率達到87.84%，在電競影片自動摘要擷取模型中透過情緒標記所取得的遊戲表現片段，結果顯示模型可以標記到80%以上請教練標記的片段，教練尋找負面遊戲表現的時間有效節省74.7%，使教練可以快速針對選手失誤或表現不佳的原因提出改善與建議，提升電競選手在比賽上的操作技巧與提高獲勝機會。

關鍵字

電競 ； 電競情緒 ； 卷積神經網路 ； 遷移式學習 ； 臉部情緒辨識

並列摘要

If eSports coaches want to know the status of players during the game, they can often only find the critical point in time where players’ made the mistakes by watching the video recordings after the game, which is quite time-consuming. The emotional expression of a player's face is closely related to the outcome of the game and a player's operation. Therefore, by recognizing the expression of a player's face can effectively grasp a player's emotions and understand the player's situation to provide advice and assistance.

In the related research of facial emotion recognition, the most commonly used is the convolutional neural network, because traditional machine learning requires features found or designed by humans, while convolutional neural networks can automatically learn textural features from images. This process requires learning from a large amount of data. This research establishes a Gaming Facial Emotion dataset and uses the Dense_FaceLiveNet framework to establish a recognition model of eSports players' emotions to analyze and obtain the type of emotions and the point in time where the emotions occur in video recordings. Then, the establishment of an automatic summary extraction model of eSports videos can obtain a player’s positive and negative performance video segments by referring to the previous analysis. Coaches can review these performance fragments and form a complete eSports training assist system.

The Gaming Facial Emotion dataset collected in this research contains 1,275 images. They are used in a multi-level transfer learning to train the model. We use CK+, FER2013, and learning emotions, three emotion recognition models, to carry out various ways of transfer learning. The experimental results show that in the three transfer learning processes, while the data set is transferred from small to large, and the final target data set category is made closer to the similar data set, the maximum utility accuracy of transfer learning can reach 87.84%. The results of the experiment also show that through the automatic summary extraction model, the game performance fragments marked in the gaming video summarization model can cover more than 80% of the marking fragments of the coach, and reduce the time required by 74.7%, allowing the coach to clarify the reasons of a player’s mistake or poor performance faster and offer suggestions to improve eSports players’ skills in order to enhance the chance of winning.

並列關鍵字

eSports ； Gaming emotion ； Convolutional Neural Network ； Transfer Learning ； Facial Emotion Recognition

參考文獻

[1] F. Bányai, M. D. Griffiths, Z. Demetrovics, and O. Király, "The mediating effect of motivations between psychiatric distress and gaming disorder among esport gamers and recreational gamers," Comprehensive Psychiatry, vol. 94, p. 152117, 2019/10/01/ 2019, doi: https://doi.org/10.1016/j.comppsych.2019.152117.

[2] D. C. Funk, A. D. Pizzo, and B. J. Baker, "eSport management: Embracing eSport education and research opportunities," Sport Management Review, vol. 21, no. 1, pp. 7-13, 2018/02/01/ 2018, doi: https://doi.org/10.1016/j.smr.2017.07.008.

[3] M. C. Keiper, R. D. Manning, S. Jenny, T. Olrich, and C. Croft, "No reason to LoL at LoL: the addition of esports to intercollegiate athletic departments," Journal for the Study of Sports and Athletes in Education, vol. 11, no. 2, pp. 143-160, 2017.

[4] M. Mora-Cantallops and M.-Á. Sicilia, "MOBA games: A literature review," Entertainment Computing, vol. 26, pp. 128-138, 2018/05/01/ 2018, doi: https://doi.org/10.1016/j.entcom.2018.02.005.

[5] E. B. Andrade and D. Ariely, "The enduring impact of transient emotions on decision making," Organizational Behavior and Human Decision Processes, vol. 109, no. 1, pp. 1-8, 2009/05/01/ 2009, doi: https://doi.org/10.1016/j.obhdp.2009.02.003.