基於影像之微創手術器械姿態估測與追蹤

Image-based Pose Estimation and Tracking of Surgery Instrument in Minimally Invasive Surgery

王郁馨(Yu-Shin Wang)

指導教授 ： 宋開泰

國立交通大學/電機學院/電機與控制工程系所/碩士(2019年)

摘要

本論文提出一種基於視覺之手術器械姿態辨識方法，期望能應用於微創手術訓練。本研究單純利用視覺估算手術器械之姿態，與標準手術動作之手術器械姿態比對，進行訓練效果評量。本論文使用自行安裝之立體攝影機，配合在手術器械關節點上之顏色標記，從影像中擷取手術動作中的器械關節點位置，搭配立體攝影機所產生的深度資訊疊合出器械關節點的三維位置，再透過卡爾曼濾波器(Kalman Filter)得到的器械關節點座標。本論文提出運動學演算法推算出手術器械四個關節轉動角度及機械臂六自由度之位置座標，儲存串列之座標點位置。為了驗證所發展之方法，本論文自製一套微創手術機器人實驗平台，以做為求得手術動作時器械姿態之基準值。經由實驗驗證，本論文視覺系統辨識對六自由度機械臂端點位置的平均絕對誤差值為2.114mm，對安裝在六軸機械臂端點之四自由度手術器械轉角之平均絕對誤差值為0.063rad，另外，實作展示以所估側之手術器械姿態串列值可重現簡單的自動抓取手術動作。本論文之影像姿態估計方法除可應用於微創手術訓練評估以外，未來亦可用於手術自動化的發展。

關鍵字

手術器械辨識 ； 手術器械姿態估測 ； 手術器械追蹤 ； 深度視覺 ； 手術 機器人

並列摘要

This study proposes a method of image-based surgical instrument pose estimation. In minimally invasive surgery (MIS), stereo vision can be used for surgical tool recognition and pose estimation. In this work, color markers are used to indicate the instrument joint positions, which are detected by using a self-built stereo camera to obtain their 3D coordinates. Kalman filter is adopted to obtain the final posture of the instrument. We propose a kinematic algorithm to calculate the surgical instrument's position and rotation angles in Cartesian space. The proposed method has been verified on an experimental robotic platform, which consists of a Hiwin 6-DoF manipulator and a motorized 4-DoF surgical instrument. Experimental results show that the mean absolute error of 2.114 mm at the robot arm endpoint and 0.063 radians for the tool’s angles have been realize. Futher experiments involve the instrument's tracking with a stereo camera in MIS pick-and-place operation, and later to reproduce given operation autonomously based on the recorded instrument poses data. The developed image-based pose estimation system can be used for the surgeon’s training in MIS and can also be applied for autonomous surgical operations in the future.

並列關鍵字

surgical instrument recognition ； surgical instrument pose estimation ； surgical instrument tracking ； stereo vision ； surgical robot

參考文獻

[1] A. Baheti, S. Seshadri, A. Kumar, G. Srimathveeravalli, T. Kesavadas and K. Guru, “Ross:

Virtual reality robotic surgical simulator for the da vinci surgical system,” in Proc. of IEEE

2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems,

Reno, Nevada, USA, 2008, pp. 479–480.

[2] https://technews.tw/2016/11/05/da-vinci-surgical-system-intuitive-surgical/, [Accessed: