以實例切割與夾取點生成卷積類神經網路應用於隨機堆疊物件之分類夾取

Robotic Random Bin Picking and Classification System using Instance Segmentation and Generative Grasping Convolutional Neural Network

李佳蓮(Chia-Lien Li)

指導教授 ： 李志中

國立臺灣大學/工學院/機械工程學研究所/碩士(2020年)

https://doi.org/10.6342/NTU202002128

摘要

本研究針對堆疊物件提出一套模組化的分類夾取流程，使用 RGB-D 相機取得物件堆疊的平面以及深度影像，經過實例切割模型(Mask-RCNN)及夾取點生成卷積類神經網路(Generative Grasping Convolutional Neural Network, GG-CNN)，找出該堆疊中的多個夾取點，最後將所有物件的夾取點彙整至堆疊中，根據深度資訊篩選出不會與鄰物干涉的夾取點，並令機器手臂前往夾取。
在最初的分割步驟中，本研究選擇Mask R-CNN 對堆疊影像進行實例切割(Instance Segmentation)，將物件從堆疊中逐一分離，取得堆疊中物件的位置以及類別資訊，並加入邊緣損失以取得更精確的邊緣輪廓。
第二步驟使用 GG-CNN 對單一物件的深度資訊生成像素級(Pixelwise)的夾取穩定度評分，此模型對於未知物件仍有預測夾取點的能力，因此在增加新的目標物件時，不需再更新此步驟的模型參數。
在第三步驟中透過深度影像，結合第一步驟的物件的位置資訊，以及第二步驟的夾取穩定度評分，剔除可能碰撞夾取點，並依據穩定度排序，即為本流程的最後輸出結果。最後，本研究並以一機器臂系統驗證此一流程之可行性，其夾取成功率可達84.3%。

關鍵字

機械手臂 ； 堆疊夾取 ； 實例切割 ； 深度學習

並列摘要

This thesis presents a robotic grasping and classification system for objects in cluttered environments. The system consists of three main parts: (i)instance segmentation, (ii)grasping candidates generation, and (iii)collision avoidance.
In the first part, the instance segmentation model, Mask R-CNN, isolates each cluttered object from the scene and is improved to obtain an accurate mask edge.
In the second part, Generative Grasping Convolutional Neural Network (GG-CNN) predicts the quality and grasps for every object, which is segmented in the first part. After that, the grasping candidates would be sampled from the pixel-wise prediction of GG-CNN.
In the last part, the algorithm selects collision-free grasps from the grasping candidates based on depth information. Finally, a robotic system is presented to illustrate the effectiveness of the process. It is shown that an 84.3% successful rate of grasp can be achieved.

並列關鍵字

Robotic Arm ； Clutter Grasping ； Instance Segmentation ； Deep Learning

參考文獻

[1] Cornell University. "Cornell grasping dataset." http://pr.cs.cornell.edu/grasping/rect_data/data.php (accessed.

[2] H. Karaoguz and P. Jensfelt, "Object Detection Approach for Robot Grasp Detection," 2019 International Conference on Robotics and Automation (ICRA), pp. 4953-4959, 2019, doi: 10.1109/ICRA.2019.8793751.

[3] J. Ma, W. Shao, H. Ye, L. Wang, H. Wang, Y. Zheng, and X. Xue, "Arbitrary-Oriented Scene Text Detection via Rotation Proposals," IEEE transactions on multimedia., vol. 20, no. 11, pp. 3111-3122, 2018, doi: 10.1109/TMM.2018.2818020.

[4] D. Morrison, P. Corke, and J. Leitner, "Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach," arXiv preprint arXiv:1804.05172, 2018.

[5] H. Liang, X. Ma, S. Li, M. Görner, S. Tang, B. Fang, F. Sun, and J. Zhang, "Pointnetgpd: Detecting grasp configurations from point sets," in 2019 International Conference on Robotics and Automation (ICRA), 2019: IEEE, pp. 3629-3635.