Q-學習法輔助自適應模糊控制在載具跟隨系統之應用

本研究利用三種不同控制方法設計在未知環境下的載具跟隨系統，分別為模糊控制、Q-學習法以及Q-學習輔助模糊控制。在研究中，無人差速輪載具經由實驗空間上方的網路攝影機作為感測器，取得前方引導載具位置資訊及自身的位姿資訊，並使用上述三種演算法進行追蹤引導載具，保持安全距離與貼合引導載具路徑之任務。在使用模糊控制的跟隨系統時，須預先藉由專家經驗得出完整的模糊規則，但當複雜環境改變時，所採用之規則庫可能必須調整，然傳統的模糊控制並未提供調整策略，使其缺乏自適應性；在另一方面，Q-學習法能透過不斷與環境互動進行學習，具有自適應的能力，但因須先進行行為探索，使其應用效率低，且因離散化而產生震盪問題；為解決前兩種控制方法的不足，整合的Q-學習輔助模糊控制則，不但在動態環境下具有學習與適應環境的能力，並可透過模糊規則提高Q-學習法的學習速度。依據模擬和實驗結果，本文所發展之三種控制方法皆能實現任務目標，而經由結果比較可得，Q-學習輔助模糊控制確實能結合兩者優點，在實際導航上具有較高的應用價值。

關鍵字

跟隨系統；模糊控制； Q-學習法； Q-學習輔助模糊控制；自適應性

並列摘要

Three different methods, including fuzzy control, Q-learning, and fuzzy Q-learning, are used to design the vehicle-following system in an unknown environment in this research. A webcam above the experimental space is used as the sensor to obtain the position information of the leader car and the posture information of the follower car. It is desired to accomplish the tasks of tracking the leader car, keeping safe distance, and fitting the path of the leader car. It is seen that the fuzzy control method lacks adaptability, since the fuzzy rules require experts’ knowledge which may not be available in the unknown complex environment. The Q-learning method can improve the performance of the controller by learning through interaction with the environment, so it has the ability of self-tuning. However, the pre-learning process for behavior exploration makes the learning efficiency of this system low. The Q-learning-assisted fuzzy control method can solve the deficiencies of the first two control methods. Not only, the ability to learn and adapt to the dynamic environment, the learning speed can be improved through rule-based adjustments. According to the simulation and experimental results, the three control methods used in this research can all achieve the goal of the task. Through the comparison of the results, it is shown that the Q-learning assisted fuzzy controller on the design of a vehicle-following system can take the advantages of both the fuzzy control method and the Q-learning method, so that it has better application values in navigation and control.

並列關鍵字

vehicle-following system ； fuzzy control ； Q-learning ； Q-learning assisted fuzzy control ； self-tuning

參考文獻

[1] T. H. Cormen, C. E. Leiserson, and R. L. Rivest, Introduction to Algorithms (Third Edition), MIT Press, 2009.

Google Scholar

[2] 王進德, 類神經網路與模糊控制理論入門與應用, 全華圖書股份有限公司, 2007.

Google Scholar

[3] L. A. Zadeh, "Fuzzy sets," Information and Control, pp. 338-353, 1965.

Google Scholar

[4] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (Second Edition), MIT Press, 2014, 2015.

Google Scholar

[5] P. Y. Glorennec and L. Jouffe, "Fuzzy Q-learning," in Proceedings of 6th International Fuzzy Systems Conference, Barcelona, Spain, 1997.

Google Scholar

國際替代計量

Q-學習法輔助自適應模糊控制在載具跟隨系統之應用

全文下載

主題瀏覽