Applying Deep Learning to MMS Point Cloud Semantic Segmentation and Image Classification for HD Map Generation

應用深度學習進行MMS點雲語意分割及影像分類產製高精地圖

張芷瑀(Chih-Yu Chang) ； 曾義星(Yi-Hsing Tseng)

《航測及遙測學刊》 27卷1期 (2022/03) Pp. 15-28

https://doi.org/10.6574/JPRS.202203_27(1).0002

摘要

The ongoing race toward an autonomous era results in the development of High Definition (HD) Maps. To help extend the vision of self-driving vehicles and guarantee safety, HD maps provide detailed information about on-road environments with precise location and semantic meaning. However, one main challenge when making such a map is that it requires a massive amount of manual annotation, which is time-consuming and laborious. As such, to fulfill automation in extracting information from the sheer amount of data collected by mobile LiDAR scanners and cameras is at most concern. In this study, a workflow for automatically building traffic sign HD maps is proposed. First, traffic islands, traffic signs, signals, and poles are extracted from LiDAR point clouds using PointNet. Then, point clouds of traffic signs are clustered by the DBSCAN algorithm so that the geometric information can be obtained. An evaluation is performed to assess the accuracy of geolocation in the final stage. Next, point clouds in each traffic sign cluster are projected onto corresponding MMS images for classification purposes. The semantic attribute is obtained based on the GoogLeNet classifier and determined by a proposed mechanism, i.e. modified SNR ratio, which ensures the class with the most classified images is significant enough for that cluster to be considered as that specific type. An output text file including precise coordinates of traffic sign center, bottom-left, and top-right of the traffic sign bounding box also their type is generated for further use in HD maps.

關鍵字

HD Maps ； MMS ； Deep Learning ； Point Cloud Semantic Segmentation ； Image Classification

並列摘要

本研究應用深度學習（Deep Learning, DL）的技術提出了一套完整的交通標誌高精地圖產製流程，以期能更自動化地從移動式測繪系統（MMS）所蒐集而來的光達及影像資料中，萃取出建立高精地圖的必要資訊，解決傳統產製方式耗時耗力的問題。首先，使用PointNet從點雲提取交通島、交通標誌、號誌和桿狀物。然後，通過DBSCAN算法對交通標誌點雲進行聚類，以獲得幾何資訊並評估其準確性。接下來，將每個交通標誌簇中的點雲投影到相應影像上進行分類。透過 GoogLeNet及訊噪比確認其語意資訊。最後產出符合高精地圖交通標誌要求之格式，以供自駕車發展之使用。

並列關鍵字

高精地圖 ； 移動式測繪系統 ； 深度學習 ； 點雲語意分割 ； 影像分類

參考文獻

Che, E., Jung, J., and Olsen., M.J., 2019. Object recognition, segmentation, and classification of mobile laser scanning point clouds: A state of the art review, Sensors, 19: 810-10.

Elhousni, M., Lyu, Y., Zhang, Z., and Huang, X., 2020. Automatic building and labeling of HD maps with deep learning, Proceedings of the AAAI Conference on Artificial Intelligence, pp.13255- 13260.

Gargoum, S., and El-Basyouny, K., 2019. Effects of LiDAR point density on extraction of traffic signs: A sensitivity study, Transportation Research Record, 2673(1): 41–51.

Jiao, J., 2018. Machine learning assisted high- definition map creation, Proceedings of the IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC), Tokyo, Japan, pp.367- 373.

Qi, C.R., Su, H., Mo, K., and Guibas, L.J., 2017. Pointnet: Deep learning on point sets for 3D classification and segmentation, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp.652- 660.