內視鏡影像下之三維定位－以多個參考幀加速初始化即時定位與地圖構建系統及點雲註冊達到器官定位

Localization in Minimally Invasive Surgery: Using Multiple Reference Frames in SLAM and Registration

湯梵平(Fan-Ping Tang)

指導教授 ： 施吉昇

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2020年)

https://doi.org/10.6342/NTU202004200

摘要

隨著內視鏡的發展且能夠提供許多傳統手術沒有的優點，外科醫生也必須面臨內視鏡手術下的挑戰，例如視角過小、缺乏深度資訊及觸覺回饋。要如何在狹窄的內視鏡影像中定位出血管、腫瘤、以及各種器官也成為非常重要的問題，因此，近年來越來越多關於內視鏡手術下定位技術的研究，其中多數的研究都需要額外的傳感器，像是超音波、深度、電腦斷層和慣性測量傳感器等，而本篇專注在實務中最常使用的單目內視鏡影像。隨著內視鏡的發展且能夠提供許多傳統手術沒有的優點，然而單一影像所能提供的資訊有限，所以我們採用基於特徵的即時定位與地圖構建和點雲註冊來得到內視鏡與器官組織的相對位置。基於特徵的即時定位與地圖構建演算法需要相機以特定的移動方式來進行初始化且需要一定的時間來找到匹配的前後幀，本篇論文中我們改良了初始化的方法使得成功率提升超過十倍。而在定位的實驗中，器官定位誤差在7.5公釐以下，也證明我們提出的方法可以有效的在內視鏡手術中提供正確的定位。

關鍵字

內視鏡手術 ； 單目內視鏡 ； 即時定位與地圖建構 ； 點雲註冊 ； 定位

並列摘要

Minimally invasive surgery (MIS) is increasingly popular because of advances in robotic and endoscopic technology. While MIS provides significant benefits to patients, surgeons require additional training and practice because of the narrow field of view, lack of depth information and haptic response. One of the most critical challenges for MIS is how to accurately localize the objects such as blood vessels, tumors and organs. Hence, there are growing research interests in helping surgeons. Most of the related works require additional sensors including ultrasound probe, depth sensor, intraoperative CT and IMU. This work focuses on only using monocular endoscope image sequences under surgical scene. Because the information from monocular images is relatively limited, we resort to feature-based real-time SLAM and point cloud registration to provide endoscope location with respect to the internal organs. Most feature-based monocular SLAM initialization methods start with key points matching in nearby frames. However, it can take lots of time to find proper frame pair. In this thesis, we present an effective method to initialize and localize endoscope by registering the map from SLAM to the preoperative 3D model. The accuracy of the location estimation is assessed through simulation surgical video rendered by Blender. Results show that our organ position error is less than 7.5 mm, and the initialization success rate of our proposed method is 10 times higher than that of ORB-SLAM.

並列關鍵字

Minimally Invasive Surgery ； Registration ； SLAM ； Endoscope ； Localization ； Endoscopy

參考文獻

R. Mur-Artal, J. M. M. Montiel, and J. D. Tardós, “Orb-slam: A versatile and accurate monocular slam system,” IEEE Transactions on Robotics, vol. 31, no. 5, pp. 1147–1163, 2015.

L. Chen, W. Tang, N. W. John, T. R. Wan, and J. J. Zhang, “Slambased dense surface reconstruction in monocular minimally invasive surgery and its application to augmented reality,” Computer Methods and Programs in Biomedicine, vol. 158, pp. 135 – 146, 2018. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0169260717301694

G. A. Puerto-Souza, J. A. Cadeddu, and G. Mariottini, “Toward long-term and accurate augmented-reality for monocular endoscopic videos,” IEEE Transactions on Biomedical Engineering, vol. 61, no. 10, pp. 2609–2620, 2014.

B. Münzer, K. Schoeffmann, and L. Böszörmenyi, “Content-based processing and analysis of endoscopic images and videos: A survey,” Multimedia Tools and Applications, vol. 77, no. 1, pp. 1323–1362, Jan 2018. [Online]. Available: https://doi.org/10.1007/s11042-016-4219-z

G. Klein and D. Murray, “Parallel tracking and mapping for small ar workspaces,” in 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, 2007, pp. 225–234.