基於目標檢測及實例分割之協作機器人視覺應用

Machine Vision Application of Collaborative Robot Based on Target Detection and Instance Segmentation

林廷軒(Ting-Syuan Lin)

指導教授 ： 劉天倫

中原大學/電機資訊學院/工業與系統工程研究所/碩士(2021年)

https://doi.org/10.6840/cycu202100946

摘要

近年來人工智慧快速發展，機械手臂開始加入影像辨識系統以及深度學習，對於物品的偵測、臉部辨識、手部辨識、人體姿態辨識、分類等等都是重要的發展方向，在某些情況下機械手臂和人類配合才能達到最好的效果，因此協作機器人（Collaborative robot, Cobot）成為目前的趨勢。
本論文將模擬工廠內拆解零件的工作場域，以Intel RealSense D435i作為辨識用的深度攝影機，卷積神經網路使用的是自行蒐集數據後訓練出的Mask R-CNN模型，根據RGB影像以及深度影像的資訊，進行工業用零件的檢測與分類、中心點生成深度資訊、三維座標計算與轉換，最後經由TCP/IP把轉換好的座標傳送給Niryo Ned來對目標物進行夾取，夾取之後送至目的地進行分類，整體過程展示出協作機器人搭配上影像辨識系統後進行零件夾取與分類的情境。
實驗結果顯示，訓練出來的Mask R-CNN模型呈現收斂且辨識的信心程度平均下來都保有9成以上的準確率，座標轉換的世界座標X、Y誤差分別落在5mm、6mm以內，對於本實驗的機械手臂夾取來說是可以接受的範圍。

關鍵字

深度攝影機 ； 座標轉換 ； 協作機器人

並列摘要

In recent years, artificial intelligence has developed rapidly. Robotic arms have begun to add image recognition systems and deep learning. Object detection, face recognition, hand recognition, human posture recognition, classification, etc. are all important development directions. In some cases, the best results can be achieved by the cooperation of mechanical arms and humans. Therefore, collaborative robots (Cobot) have become the current trend.

This research will simulate the working field of disassembled parts in the factory, using Intel RealSense D435i as the depth camera for identification. The convolutional neural network used in the research is Mask R-CNN model, which is trained by the collecting data. According to the RGB image and the depth image information, the detection and classification of industrial parts, the generation of depth information from the center point, the calculation and conversion of three-dimensional coordinates, and finally the converted coordinates are sent to Niryo Ned through TCP/IP to grasp the target object. After that, it is sent to the destination for classification. The overall process shows the situation where the collaborative robot is equipped with the image recognition system to grasp and classify industrial parts.

The experimental results show that the trained Mask R-CNN model is convergent and the recognition confidence is maintained at an accuracy rate of more than 90% on average. The world coordinate X and Y errors of the coordinate conversion are within 5mm and 6mm respectively, which is an acceptable range for the gripping of the robot arm in this experiment.

並列關鍵字

Mask R-CNN

參考文獻

[1] Židek, K., Hosovsky, A., Piteľ, J., Bednár, S. (2019). Recognition of assembly parts by convolutional neural networks. In Advances in Manufacturing Engineering and Materials (pp. 281-289). Springer, Cham.

[2] El Zaatari, S., Marei, M., Li, W., Usman, Z. (2019). Cobot programming for collaborative industrial tasks: An overview. Robotics and Autonomous Systems, 116, 162-180.

[3] Wang, L., Schmidt, B., Nee, A. Y. (2013). Vision-guided active collision avoidance for human-robot collaborations. Manufacturing Letters, 1(1), 5-8.

[4] Wang, Y., Ye, X., Yang, Y., Zhang, W. (2017, November). Collision-free trajectory planning in human-robot interaction through hand movement prediction from vision. In 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids) (pp. 305-310). IEEE.

[5] Matsas, E., Vosniakos, G. C., Batras, D. (2018). Prototyping proactive and adaptive techniques for human-robot collaboration in manufacturing using virtual reality. Robotics and Computer-Integrated Manufacturing, 50, 168-180.