應用分水嶺理論於雙張畫面之深度圖產生演算法與硬體實現

Depth Map Generation Based on The Watershed Algorithm and Its Hardware Implementation

游文傑(Wen-Jay Yu)

指導教授 ： 李佩君

國立暨南國際大學/科技學院/電機工程學系/碩士(2012年)

https://doi.org/10.6837/NCNU.2012.00203

摘要

為了能夠讓日常生活中的影像或是圖片擁有3D的視覺效果，許多廠商便發行可以拍攝出3D的設備，目前3D視訊擷取設備中以雙鏡頭取得左右視角後生成虛擬3D視訊，為了提升多人可於多視角觀看3D視訊，則需透過所取得的雙張視訊間的關係，產生深度圖進而生成多視角的 3D 虛擬視訊。由於深度圖的品質攸關最後呈現的3D效果，然而在產生的過程中存在視差(disparity)計算的不準確性，導致深度圖品質下降以及深度圖破碎的結果。為解決此問題，本論文提出利用邊緣資訊搭配分水嶺演算法進行區塊的分割，利用區塊分割使同區塊的物體可被賦予相同之深度值，減少因比對錯誤造成的物體破碎情況發生，再以可調式十字形遮罩方法，對區域上的第一點像素進行遮罩的長成，以至於在比對時能以專屬的遮罩比對，得到每個物件在左右視角兩張畫面間的視差預估(disparity estimation)，進而利用視差(disparity)產生每個物件深度資訊。為加速視訊中深度產生之計算與穩定性，本論文提出利用前後張的動態向量資訊搭配深度圖(I frame)產生每張畫面之深度圖視訊(P frame)。由於分水嶺演算法可以準確的切割出物件，因而提升每一個物件擁有相同的視差值，使產生的深度值有較高的準確性，本論文也設計演算法之電路架構，並以Altera公司的FPGA開發板實現，在實現的電路架構中。
在實驗結果的部分，本論文提出之雙視角深度圖產生方法在執行的結果上，PSNR的表現比傳統的方法上升8.51dB，SSIM上 有0.38的提升。硬體方面根據所合成的電路佔了Altera DE3-150驗證板上95%左右的邏輯使用率，記憶體方面只使用3%內部的記憶體，大大減少記憶體的成本考量。

關鍵字

2D to 3D ； 深度圖 ； DIBR ； Binocular ； Stereo matching ； Segmentation ； H.264

並列摘要

In recent years, the development of stereoscopic display technology has led to 3D related research. The developing 3D video codec (3DVC) standard adopts the multi-view rendering to produces the virtual multi-angle frame by the texture image and its corresponding depth map. Therefore, depth estimation is one of the essential techniques to generate 3D images. Binocular depth generation is one of the depth estimation methods, which is to extract image depth information using two different images that have been captured by right and left views. Among overall processing of stereo vision, a step to find matching points are called stereo matching. The step can be said the core of stereo vision system.
In general, binocular depth generation methods use a fixed mask to compute the smallest matching costs on 2 views stereo matching. This costs value is called to the disparity value which can convert to the depth information. However, this method will lead the matching errors and a lot of the depth discontinuing of an object in depth map. To solve these problems, this thesis proposes a depth map generation algorithm, which gives the same depth value in the same object region for solving the depth discontinuing by using watershed segmentation method. In the stereo matching processing, an adaptive matching window is produced by the cross-based matching algorithm. To enhance 3D expression for viewer and to reduce the depth discounting between the adjacent frames, the proposed algorithm utilizes the motion vectors of texture image to compensate the depth values for the current frame from the previous depth map. In order to improve the processing time for the proposed algorithm, this thesis designs hardware architecture by using Altera company’s FPGA development board
The experimental results show that the depth map quality with the proposed algorithm by comparing the performance of PSNR and SSIM. With proposed method, it increases the PSNR from 1.19dB to 8.51dB and the SSIM from 0.02 to 0.38 compared with the conventional cross-based method and the Belief Propagation method. The circuit synthesis is implemented under the circumstance of Quartus II demonstration software which is developed by Altera Corporation. The gate count usage occupies 95% of the total resource on the Altera DE3-260 demonstration board. Because of the watershed method needs a large number of registers to store the label information which causes the gate count is raise. The internal memory usage only is 3% to store. Greatly reduction of the memory cost considerations.

並列關鍵字

2D to 3D ； depth map ； DIBR ； Binocular depth generation ； Stereo matching ； Segmentation ； H.264

參考文獻

[1] C. Fehn, “A 3D-TV system based on video plus depth information” Conference Record of the Thirty-Seventh Asilomar Conference on Signals, System and Computers, vol. 2,pp. 1529-1533, Jun. 2003.

[2] Y. L. Chang. “Algorithm and Architecture Analysis of the Video Signal Conversion for 2D to 3D Video,” National Taiwan University, Ph.D. dissertation, 2007.

[3] L. Vincent and P. Soille, “Watershed In Digital Spaces: An Efficient Algorithm Based On Immersion Simulations,” IEEE Transaction on Pattern Analysis Machine Intelligenc., vol. 13, no. 6, pp. 583-598, Jun. 1991.

[4] http://mpegsrc.ee.nctu.edu.tw/mpegTalk/MPEGstatus20110826_ALL.pdf, Jul. 2012.

[5] C. C. Cheng, C. T. Li, P. S. Huang, T. K. Lin, Y. M. Tsai and L. G. Chen, “A block-based 2D-to-3D conversion system with bilateral filter,” in Proceedings of International Conference on Consumer Electronics, pp. 1-2, Taipei , Taiwan, Jan. 2009.