結合Q-Learning和模糊狀態於足球代理人的學習

Combination of Q-learning and Fuzzy-State for Learning of RoboCup Agent

蘇偉德(Wei-De Su)

指導教授 ： 郭忠義

國立臺北科技大學/電資學院/資訊工程系研究所/碩士(2010年)

https://doi.org/10.6841/NTUT.2010.00307

摘要

人工智慧在電腦科學中一直是令人感興趣的領域，而其中機器學習這個部分更是機器人成功與否的關鍵。機器人世界盃(RoboCup)為近十年來所發展的一種國際競賽，其包含兩足機器人以至電腦模擬的比賽皆有提供完善的規則和機制。在學術上，其為機器學習提供了一個最佳的測試平台。由於足球比賽中，球場上的環境狀態不停的在改變，所以一般規則式或監督式學習較無法應對如此多變的環境，因此本文提出強效式學習(Reinforcement Learning)中的Q-Learning學習方法來應用於足球代理人的學習。而為解決球場上環境狀態量過大，以致學習速度過慢的問題，本文利用將狀態模糊化以及模糊規則的方式，來減少狀態的數量和Q-Learning中狀態-動作表(State-Action Table)的複雜度。

關鍵字

RoboCup ； Q-Learning ； Fuzzy

並列摘要

Artificial intelligence always been interesting in computer science,and in this area machine learning is the key to success, RoboCup(Robot World Cup Tournament) is a competition game which has already become a popular research domain in recent years, includes the real robot as well as computer simulation games and also provide comprehensive rules and mechanisms. In Academic,it provides a best test-bed for machine learning. As the soccer game, the environment states are always changing.Therefor, in this paper, we use the Q-Learning method that is a kind of reinforcement learning to apply for learning of robocup agent. And, in order to solve the environment states of excessive problem which led to slow learning rate, we will use fuzzy-state and fuzzy-rule to decrease the state and state-action table of Q-Learning.

並列關鍵字

RoboCup ； Q-Learning ； Fuzzy

參考文獻

[1] RoboCup Official Site. http://www.robocup.org/.

[2] L. Vig, J. A. Adams. Multi-robot coalition formation, IEEE Transactions on Robotics, Vol. 22, Issue 4, pp. 637-649, 2006.

[5] Toshihiko Watanabe and Yoshiya Takahashi, ” Hierarchical Reinforcement Learning Using A Modular Fuzzy Model for Multi-Agent Problem”, IEEE International Conference on Systems, Man and Cybernetics, pp.1681–1686, 2007.

[6] L. Waltman, U. Kaymak, “A Theoretical Analysis of Cooperative Behavior in Multi-agent Q-learning”, IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 84-91, 2007.

[7] T. Zhou, B. R. Hong, C. X. Shi, H. Y. Zhou, “Cooperative Behavior Acquisition Based Modular Q Learning in Multi-Agent System”, Proceedings of 2005 International Conference on Machine Learning and Cybernetics, Vol. 1, pp. 205-210, 2005.