在汽機車混合車流中透過強化學習之自動駕駛控制

An Autonomous Vehicles’ Driving Model in the Mixed Traffic Flow along with Human Driving Vehicles and Motorcycles

謝宗穎(Tsung-Yin Hsieh)

指導教授 ： 陳柏華

國立臺灣大學/工學院/土木工程學研究所/碩士(2021年)

https://doi.org/10.6342/NTU202103826

摘要

由於機車的機動性高，在亞洲部分國家的交通環境中被大量用路人使用。然而其駕駛行為與汽車極為不同，體積小的特點讓機車駕駛常會有較頻繁的變換車道、超車及鑽車縫等行為。因此，這些地區的交通衝突風險比其他國家都來得高。
未來汽車有望於市區實現全自動駕駛，但要從現在幾乎是手動駕駛的車流環境轉變為全自動駕駛車輛（AVs）環境，受限於成本、道路設施及安全法規等勢必非常漫長。因此此段過渡期將成為自動駕駛車輛，完全地行駛與市區道路的阻礙。尤其在混合車流環境下，手動駕駛的機車更可能會頻繁與自動駕駛車輛發生衝突事件。
為了幫助自動駕駛車輛克服混合交通流的過渡期，本研究提出了一個架構，包含在交通模擬軟體中模擬出混合車流環境及結合安全空間模型和強化學習（RL）訓練自駕車行為的方法。本研究首先使用空拍影像資料完成了車輛之間的車流分析、空間互動分析，並透過車流模擬軟體創建特定的道路幾何以模擬出類似台灣真實情況的混合車流。本研究並提出使用安全空間作為車流模擬的驗證指標。
後續本研究透過前述模擬出的車流環境，進行自動駕駛模型的訓練。本研究證實強化學習方法可以讓車輛在複雜的混合車流或純機車車流環境下，學出安全及有效率的駕駛行為模式。此外，本研究還成功利用安全空間模型作為自動駕駛訓練時的安全指標。

關鍵字

混合車流模擬 ； 機車行為 ； 安全空間模型 ； 自動駕駛 ； 強化學習

並列摘要

Motorcycles’ driving behaviors are different from that of cars’ while they are popular on roads in certain regions. The more frequent lane changing and lane splitting behaviors of motorcycles might cause more traffic conflicts and risks.
Vehicles are expected to become fully autonomous in the future, while the transition period from an environment of full human drivers to that of full Autonomous Vehicles (AVs) will become a vital obstacle for AVs’ safe driving in dense traffic conditions. Particularly, manual driving motorcycles might frequently trigger unexpected events to AVs causing conflicts.
To help AVs overcome the situation of the transition period in mixed traffic flow, this research developed a framework including the generation of a mixed traffic flow environment in traffic simulation and training of AVs driving in mixed traffic flows based on the safety space model and reinforcement learning (RL). This research first complete traffic flow analysis and spatial interaction analysis between vehicles from aerial video data, and create specific road geometry to simulate mixed traffic flow similar to the real situation in Taiwan in a traffic simulator. This research also proposes utilizing the Safety Space concept as a calibration index for traffic flow simulation.
In this study, the automated driving model was trained through the aforementioned simulated traffic environment. This research confirms that the reinforcement learning method can allow vehicles to learn safe and efficient driving behavior in mixed traffic or motorcycle-only traffic environment. In addition, this study also successfully uses the Safe Space model as a safety indicator to train an autonomous driving model.

並列關鍵字

Mixed Traffic Flow Simulation ； Motorcycles' behavior ； Safety Space Model ； Autonomous Driving ； Reinforcement Learning

參考文獻

[1] T. Litman, “Autonomous Vehicle Implementation Predictions Implications for Transport Planning,” 2021, Accessed: Aug. 18, 2021. [Online]. Available: www.vtpi.org.

[2] M. Sivak and B. Schoettle, “Road safety with self-driving vehicles: general limitations and road sharing with conventional vehicles,” University of Michigan, Ann Arbor, Transportation Research Institute, 2015. Accessed: Apr. 12, 2021. [Online]. Available: http://www.umich.edu/~umtriswt.

[3] “Deep traffic video analysis - DataFromSky.” https://datafromsky.com/ (accessed Aug. 19, 2021).

[4] N. Saunier and T. Sayed, “A feature-based tracking algorithm for vehicles in intersections,” in Third Canadian Conference on Computer and Robot Vision, CRV 2006, 2006, vol. 2006, p. 59, doi: 10.1109/CRV.2006.3.

[5] N. Saunier and T. Sayed, “Automated analysis of road safety with video data,” Transp. Res. Rec., no. 2019, pp. 57–64, 2007, doi: 10.3141/2019-08.