改善建立虛擬視野的補洞演算法與硬體實現

立體影像的呈現隨著三維電影、三維電視的發展，正頻繁的出現在我們生活周圍，許多國內外知名大學與研究機構也持續的致力於三維影像以及視訊的研究。原本三維影像的產生必須將所有視角的影像全部傳輸過來，這需要大量的頻寬，但由於Depth image based rendering(DIBR)這項技術的提出，解決了有限頻寬的問題，不需要再將所有視角的影像全部傳輸。雖然解決了頻寬，但是這演算法當中也存在著一些問題。首先了解一下Depth image based rendering(DIBR)這項技術，這個技術主要是利用一張已知視角的影像與其相對應的深度圖(depth map)來提供觀賞者另一張不同視角的影像。在DIBR中，利用位移像素(shift pixels)來建立新的視角，但同時有些區域(disoccluded regions)會暴露出來，如果沒有適當的填補這些洞(hole)，新視角的影像品質將會大大的降低。而且隨著三維影像正朝著裸視以及更廣泛的多重視角發展，原本DIBR所面臨到的問題將會更嚴苛。我們提出一些方法，可以用來有效並改善之前填補這些洞(hole-filling)的演算法，主要是利用洞所在周圍的背景當作填補的資訊，但為了做到補洞而不破壞原本背景的結構，我們必須在填補時考慮物體的垂直特徵，以不破壞背景的垂直特徵為前提的情形下，將背景的資訊做為填補的依據。而為了提供更廣泛的視角，區域(disoccluded regions)會暴露的更嚴重，我們所需要填補的洞會越大，在這裡我們研究了DIBR的移位探測器(shift-sensor algorithm) ，貼近人眼對深度變化的真實感受，提出了非線性的視差變化，來處理大視角所需要填補的洞。這方法可以增加背景所留下來的資訊，也可以縮小所需填補的洞的尺寸，而只需要犧牲一點視角的角度。最後，利用軟體模擬提出的演算法的結果，使用硬體描述語言去設計硬體架構，在Xilinx多媒體版實現出來並且驗證。

關鍵字

建立虛擬影像；補洞；多重視角；視差圖

並列摘要

Three dimensional television (3D-TV) is expected to be the next step in the advancement of television. In last few years, the technology of 3D-TV has been presented by advertisement boards, movie theaters, and LCD monitors. Their contents become more and more photographic. But, it is impractical to transmit images of all view angles due to the limitation of bandwidth. Depth image based rendering (DIBR) is a commonly used technique to generate virtual views of a scene from a known view and its corresponding depth map. However, a main problem of DIBR is disocclusion region which causes a terrible visual quality degradation. In multi-view case, the problem is getting even worse. Hence, we proposed a modified parallax map to deal with the problem. The modified parallax map can not only reveal the characteristic of smooth filter, but also prevent the disadvantage of geometric distortion. In the experiment result, the visual quality of generated virtual view can be obviously improved by using our proposed method. Furthermore, in order to develop the multi-view display which contains 9 auto-stereoscopic views, we have to face more serious disocclusion problem caused by multi-view case. In multi-view case, the disocclusion problem can still be solved by modified parallax map because it reserves enough background information from the original image for us to fill the holes. Most important of all, this algorithm is not hard to be implemented by hardware, compared to any smoothing filter. The feasibility of our proposed algorithm can be verified by using Xilinx Virtex II.

並列關鍵字

DIBR ； Hole-filling ； Multi-view ； Parallax map

參考文獻

[1] C.Fehn,“Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3DTV,”in Proc. of SPIE Conference on Stereoscopic Displays and Virtual Reality Systems, vol. 5291, pp. 93-104, San Jose, CA, May 2004.

[4] L.Zhang and W.J.Tam,“Stereoscopic image generation based on depth images for 3D TV,”IEEE Transactions On Broadcast, vol. 51, pp. 191-199, June 2005.

[7] T.C. Lin, H.C. Huang, and Yueh-Min Huang, “Preserving depth resolution of synthesized images using parallax-map-based DIBR for 3D-TV,“ IEEE Trans. on Consumer Electronics, Vol.56, No.2, May. 2010.

[8] K. Gomi, Y. Nishino, K. Tai, M. Yasuda, N. Tetsutani, “Stereoscopic video transmission and presentation system for ISDN,“ IEEE Trans. Consumer Electronics, vol.36, pp. 759-766, Aug 1990.

[9] Y. Takemura, M. Kimura, S. Yamaguchi, J, Hosokawa, Y. Arafune, “Stereoscopic video movie camera using 300K pixel IT-CCD sensors,“ IEEE Trans. Consumer Electronics, vol.37, pp. 759-766, Feb. 1991.

國際替代計量

改善建立虛擬視野的補洞演算法與硬體實現

全文下載

主題瀏覽