運用彩色資訊之深度影像放大技術與視訊應用深度影像繪圖法

Depth Map Enlargement Technology Using Color Information and Depth Image Based Render for Video Application

姚建銓(Chien-Chuan Yao)

指導教授 ： 曹恆偉

共同指導教授 ： 范育成

國立臺灣大學/電機資訊學院/電子工程學研究所/碩士(2009年)

https://doi.org/10.6342/NTU.2009.02372

摘要

顯示技術的發展在平面顯示器上至今已相當成熟，而3D顯示技術將成為未來顯示技術發展的重點之一，由於二維視訊技術的成熟，未來3D顯示系統的發展必頇與二維顯示系統兼容，2002年ATTEST即提出一套2D/3D兼容的視訊系統，可讓使用者同時兼有2D與3D的視覺享受，其製作3D影像原理是利用拍攝一張深度圖與一張彩色圖，透過深度影像繪圖法(DIBR)合成出多視角的影像，然而，現今拍攝立體影像的方式其所消耗的成本相當可觀，不利於商品化，原因是深度攝影機必頇特別製做，價格昂貴，加上深度攝影機所拍攝的影像解析度遠不及一般的彩色攝影機，因此，本論文提出一套方法利用拍攝低解析度的深度圖，搭配高解析度的彩色影像，還原高解析度的深度資訊，再利用所得到的高解析度深度圖與彩色影像做視訊應用DIBR運算，合成多視角的影像，以供3D顯示器播放。
本論文演算法利用人眼JND特性估算出深度圖的邊緣判別門檻，做為區塊分割的依據，深度圖放大之後，透過JND門檻值判別該區是否為平滑區域，接著再透過高解析度的彩色影像對內插點與周圍已知點作區塊歸屬性的分析，最後將區塊的各個歸屬性做不同的分類，各自以其適合的內插法做深度圖未知點的估算，以還原高解析度的深度影像。接著我們提出一套視訊應用DIBR(DIBR for Video)做多視角影像的合成，我們改良了Non-Hole-Filling DIBR演算法，降低了原本演算法所需的運算量，最後以標準的cell base design flow完成DIBR for Video 硬體架構的實現。
關鍵字: 深度影像繪圖法、內插法、區塊分割、立體影像、多視角

關鍵字

內插法 ； 影像放大 ； 多視角 ； 立體影像 ； 深度影像

並列摘要

Development of display technology has been mature, the 3D display technology will be the key of development of display technology. Due to the maturity of 2D video technology, the development of 3D display technology must be compatible with 2D display system.
2002 ATTEST proposed a system that compatible with 2D and 3D system. The principle of production of 3D image is shooting a depth image and a color image, and synthesized by multi-view images through the Depth Image Based Render(DIBR). However, the cost of shooting is very high and resolution of depth camera is lower than typical camera. Hence this thesis proposed a method to shoot low resolution depth map to restore high resolution depth map through high resolution information of color image. Then synthesize by multi-view images for 3D display through DIBR.
This thesis proposed a algorithm that use JND characteristic of human eyes to estimate the threshold of edge detection for region partition. After enlarge depth image, use JND threshold to judge the region smooth or not, then analysis the characteristic of region, finally classify the region and use different interpolation method to estimate the unknown point case by case to restore high resolution depth map.
This thesis proposed a method to synthesize by multi-view image called DIBR for Video. It improves the DIBR to reduce computation. Finally complete the design of hardware architecture with cell base design flow.
Keyword: JND、DIBR、edge detection、multi-view、 threshold

並列關鍵字

interpolation ； image enlargement ； DIBR ； multi-view ； stereo image ； depth image

參考文獻

[4] R. G. Keys, “Cubic Convolution Interpolation for Digital Image Processing,” IEEE Trans. Acoustics, Speech, and Signal Processing, vol. ASSP-29, no. 6, pp.1153-1160, Dec. 1981.

[6] W.K. Pratt, Digital Image Processing, 2nd ed. New York: Wiley, 1991.

[7] M. Unser, A. Aldroubi and M. Eden, “B-Spline Signal Processing: Part II-Efficient Design and Applications,” IEEE Trans. Signal Processing, vol. 41, pp. 834-848, Feb. 1993.

[8 ] M. Unser, A. Aldroubi and M. Eden, “Enlargement or Reduction of Digital Imageswith Minimum Loss of Information,” IEEE Trans. on Image Processing, vol. 4, pp. 247-258, Mar. 1995.

[10] M. Unser, ”Spline: A Perfect Fit for Signal and Image Processing,” IEEE Signal Processing Magazine, pp. 22-38, Nov. 1999.