多代理人系統中的混合學習與合作方法

A Hybrid Learning Approach for multi-agent cooperation in RoboCup

王文凱(Wen-Kai Wang)

指導教授 ： 郭忠義

國立臺北科技大學/電資學院/電資碩士在職專班研究所/碩士(2012年)

https://doi.org/10.6841/NTUT.2012.00251

摘要

在機器學習的議題中,學習效率為一個重要的研究方向。多代理人系統(Multi- Agent system)在動態變化的環境中,能夠透過學習而做出最佳的回應動作是個極具挑 戰性的問題。本篇研究針對多代理人系統問題提出一種混合式的技術架構,目的在於 讓代理人能夠在不需要太多相關背景知識的前提下能夠透過學習達到近似領域專家的 能力。
本研究結合了三種方法來建立一個多代理人的學習系統,利用案例式推論來累積 代理人的經驗,結合基因演算法來最佳化學習效率,並加以基本的規則庫來建立初始 經驗庫以及應付突發狀況。並基於 RoboCup 模擬平台(RoboCup Soccer simulator)建立 此混合技術架構為核心的足球隊。教練代理人透過比賽經驗的累積,能夠針對當下球 場上的動態做出更適合的決策,並建立球員代理人的合作模型讓多個球員代理人互相 分工合作來完成教練所下達的策略。透過多組實驗比較各方法對於代理人學習效率的 影響以及跟其他相關研究的分析與比較。

關鍵字

機器學習 ； 多代理人系統 ； 案例式推論 ； 基因演算法 ； 規則推論

並列摘要

The problem of learning efficiency with multi-agent system is one of the most important tasks for machine learning area. It's a complex challenge to design a multi-agent system and able to make the optimized response through learning in a dynamic environment. In this paper, we propose a hybrid approach that allows agents to learn and react as a domain expert with only little domain knowledge.
In this paper, three notable methods are used to construct a multi-agent learning system. Case Based Reasoning (CBR) is applied to accumulate experiences and Genetic Algorithm (GA) is used to optimize the learning efficiency. Rule Based Reasoning (RBR) is adopted to advance the CBR. A soccer team is built by our learning approach based on RoboCup soccer simulation environment.
The coach agent can make the proper strategies and get smarter as the experiences are accumulated. The cooperate-model allows player agents to work with each other for accomplishing all strategies that were made by coach agent.
Through experiments, we found how each method can affect the learning efficiency and game result.
Finally, we also compare our approach with other related researches.

並列關鍵字

RoboCup ； Machine learning ； Multi-agent System ； Case Based Reasoning ； Genetic Algorithm ； Rule Based Reasoning

參考文獻

[8] X. Chang, G. Jisuanji and Y. Yu, “RoboCup-2D Passing Strategy Based on Joint Reinforcement Learning,” Computer Engineering and Applications, vol. 47, no. 23, pp. 212-216, 2011.

[37] K. S. Hwang, S. W. Tan, and C. C. Chen, “Cooperative Strategy Based on Adaptive Q-

[30] Y. Takahashi and M. Asada, “Multi-Layered Learning Systems for Vision-based

[1] M. E. Bratman, “Intention, Plans, and Practical Reason,” Harvard University Press, 1987.

[3] B. N. Schilit and M. M. Theimer, “Disseminating Active Map Information to Mobile Hosts, ” IEEE Network, vol. 8, no. 5, pp. 22-32, 1994.