利用立體相機進行基於粒子濾波器的連續自我校正

Continuous Self-Calibration of Stereo Camera Based on Particle Filter Framework

楊博旭(Bo-Hsu Yang)

指導教授 ： 連豊力

國立臺灣大學/電機資訊學院/電機工程學研究所/碩士(2015年)

https://doi.org/10.6342/NTU.2015.01937

摘要

內視鏡已經成為一種在醫療中廣為使用的感測器。由於操作者只能從內視鏡鏡頭觀察手術進行的過程，如果該內視鏡能提供完整的三維資訊，將會使手術者更能掌握手術內部場景。為了取得三維的資訊，須使用光學三維資訊重建的技術，在可行的三維重建技術中，立體視學是目前在醫療手術中最廣為使用且最成熟的技術。一般而言，使用立體視覺重建環境資訊可分成兩個步驟，首先，先校正左右兩個相機的影像，校正過後，用立體配對法產生深度資訊並投影成三維資料點，完成三維資料點的產生後，使用資料點疊合法將不同視角的資料點作疊合，使得視野因此擴大，有助於對場景的了解。相機參數的精準度影響重建資料點的精準度，然而，相機參數可能會在操作過程中有所變化，因此持續的校正相機參數將會是一個很重要的議題。
本篇論文提出一個連續自我校正基於粒子濾波器的演算法，一開始給定一組起始相機參數，該組參數可能來自於先前的校正結果或是感測器上的規格表。校正過程中，每個時刻會讀入一組影像，從左右影像中擷取出特徵點並匹配成對，用這些配對的特徵點的幾何關係來給定粒子的權重，藉由不斷地更新粒子濾波器的狀態，滿足幾何關係限制的相機參數會被保留下來，因此能不斷地修正更新參數。當相機參數被校正完後，就可產生精準的三維資料點，之後使用資料點疊合法即可將不同時刻的資料點作疊合。我們提出一個兩步驟的資料點疊合法，首先，先用距離轉換將目標資料點轉換成距離地圖，接著使用粒子群優化演算法找尋初始的疊合轉換關係，透過這個轉換關係，可以把離群的資料點移除，移除無關的資料點後，不斷迭代地修正初始的轉換關係得到最後的轉換關係用以疊合。這兩步驟的演算法可以成功得疊合資料點即使一開始沒有給定良好的估測，除此之外，該演算法在有雜訊和離群點的狀況下仍穩健。實驗結果顯示所提出的校正演算法能持續修正參數而產生較好的資料點重建效果，同時，所提出的資料點疊合演算法能疊合不同時刻的三維資料點，進而產生更大的可視範圍。

關鍵字

立體攝影機 ； 連續自動校正 ； 三維環境重建 ； 資料點疊合 ； 形狀疊合 ； 距離轉換 ； 粒子濾波器

並列摘要

Endoscopic camera has become a popular sensor for clinical use. While the operator can only observe the surgery scene via endoscopic camera, it would provide better scene understanding for the endoscopic camera to offer three-dimensional information. To obtain three-dimensional information, optical three-dimensional reconstruction techniques are required. Among existing optical three-dimensional reconstruction techniques, stereoscopy is currently most widely adopted and well developed techniques in clinical practice. In general, the task of three-dimensional reconstruction using stereoscopy can be divided into two steps. First, the image pair obtained at each time step is rectified, and the stereo matching algorithm is performed to generate a three-dimensional point set. After the three-dimensional reconstructed data points are available, the point alignment process is performed to align several point cloud captured from different viewing angles, and the larger field of view is formed for better scene understanding. The accuracy of the camera parameters affect the accuracy of the three-dimensional reconstruction. Considering the camera parameters may change during the operation, it is crucial to constantly track these parameters.
In this thesis, a continuous self-calibration based on particle filter is proposed. Start with an initial set of camera parameters, which is available by previous calibration or specification sheet on the sensors. Our proposed algorithm reads in an image pair at each time step. The feature points are extracted from the images and matched as pairs. These feature matching pairs are used to form epipolar constraints, and the particles are given weights according to these constraints. By constantly update the states of the particle filter, the parameters satisfying the epipolar constraints are maintained, and the camera parameters are thus tracked. After the camera parameters are calibrated, accurate data point cloud can be generated. The point alignment algorithm is then performed to register point clouds captured at different time steps. We proposed a two-step algorithm for conducting point alignment. First, the target point cloud is described by distance map via distance transform. A randomized optimization, Particle Swarm Optimization (PSO), is applied to find initial transformation. With the initial transformation, outliers are removed accordingly, and an iterative process is performed to refine the initial estimation. The two-step point alignment algorithm can align point sets well even if good initial guess is not available. In addition, the algorithm is robust against noise and outliers. The experimental results demonstrate that the proposed algorithm could refine the camera parameter constantly and provide a better reconstruction result, and the proposed point alignment algorithm can align three-dimensional data from different time steps providing larger field of view.

並列關鍵字

Stereo camera ； continuous self-calibration ； 3D environment reconstruction ； point alignment ； shape alignment ； distance transform ； particle filtering

參考文獻

[1: Maier-Hein et al. 2013]

[2: Besl et al. 1992]

[3: Zhengyou Zhang 1999]

Zhengyou Zhang, “Flexible camera calibration by viewing a plane from unknown orientations,” in Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, vol. 1, pp. 666 - 673, Sep., 1999.

[4: Triggs et al. 2000]