立體影像深度基於視覺舒適度最佳化即時處理系統

Real-Time Stereo Video Disparity Optimization for Human Visual Comfort

鄭家豪(Chia-Hao Cheng)

指導教授 ： 陳中平

國立臺灣大學/電機資訊學院/電子工程學研究所/碩士(2014年)

https://doi.org/10.6342/NTU.2014.00915

摘要

立體影像最佳化為立體影像技術發展重要課題之一，目的為讓使用者在觀看立體影片時不會因為影片不適當而導致視覺疲勞之負面感受。隨著現在的影片內容由影像技術發展解析度越來越高，高解析度影像在影像處理上所需要的時間是相當可觀的，因此我們將立體影像最佳化演算法設計為適合硬體架構處理，利用硬體化加速達到可即時的影像處理以解決計算時間過長的問題。
本篇論文中，由於在立體影像中深度資訊對於觀賞者舒適度有很大的影響，於不同影像面板大小有不同的深度舒適區，因此借由調整影像深度可以達到觀測者不會產生視覺疲勞不舒適的情況。另外，觀賞立體影像時需佩戴立體眼鏡的不方便性，可以借由所計算出的深度資訊將雙視角立體影像轉為多視角立體影像，待多視角立體數位影像面板大量產品化，可利用本篇論文方法轉換雙視角影像內容為多視角影像內容。此套方法包含影像重建，視差計算，影像濾波，深度調整等。
本篇論文中使用現場可程式化邏輯陣列版(電子影像套件)包含大容量的同步動態隨機存取記憶體方便實現深度計算演算法，影像界面與深度調整的硬體架構。在架構設計上我們將重點放在於如何利用硬體同步處理大量資訊以達即時化處理。此系統在可程式化邏輯陣列板上，對於輸入訊號為1080逐行掃描的視訊顯示格式可預期達到每秒30張以上畫面處理。視差搜尋範圍為[-30,+30]。

關鍵字

立體三維 ； 雙眼視差 ； 多視角立體數為內容 ； 視差估計 ； 深度圖繪圖法 ； 可程式化邏輯陣列 ； 高解析度多媒體介面

並列摘要

Stereo video optimization has been regarded as one of the most important subject of 3D technology. The main purpose of the research is to let the viewers free from the visual fatigue caused by traditional stereo videos. With the advances in video technology, we have higher video resolution and better image quality; however, computing disparity maps on high-resolution images is time-consuming. To solve this problem, we developed a modified algorithm that is more suitable for hardware implementation, and the system architecture is designed to be capable of processing stereo videos with a large amount of data in real time.
In this thesis, we analyze the causes of visual fatigue from the human factors experiments. The depth information of stereo videos is a significant factor for human visual comfort. Panels of different sizes have different disparity range within the comfort zone, thus, visual fatigue can be relieved by adjusting the disparity range. Furthermore, with the calculated depth information, two-view images can be converted to multi-view images, such that viewers can be free from the inconvenience of wearing special 3D glasses. The methods provided in this thesis can be used as a two-view to multi-view conversion tool for multi-view auto-stereoscopic displays. Our system applies the Depth Image Based Rendering (DIBR) method, disparity map estimation and image filtering.
The on-board collection of peripherals includes high-speed video interface (HDMI) and two 1Gbits SDRAMs. The system on FPGA can achieve 30 frames per second (fps) in 1080p display format, with disparity estimation search range of [-30, +30] in horizontal direction.

並列關鍵字

Stereoscopy ； 3D ； binocular parallax ； multi-view ； disparity estimation ； depth image base rendering (DIBR) ； FPGA ； HDMI

參考文獻

[18] H.S Son, K.R. Bae, Y.H. Lee and B. Moon, "A Real-Time Stereo Matching Hardware Architecture Based on the AD-Census," International Journal of Multimedia and Ubiquitous Engineering, Volume 8, Number 4, July 2013.

[3] N. A. Dodgson, "Autostereoscopic 3D displays." IEEE Computer Magazine, Volume 38, Number 8, Page 31-36, August 2005.

[5] M. T. M. Lambooij, W. A. IJsselsteij, and I. Heynderickx,” Visual Discomfort in Stereoscopic Displays: A Review,” SPIE-IS&T, Volume 6490, Page 1-13, 2007.

[6] N. Qian, "Binocular Disparity and the Perception of Depth," Neuron, Volume 18, Page 359-368, March 1997.

[7] N. A. Dodgson, “Analysis of the Viewing Zone of Multiview Autostereoscopic displays,” Proc. SPIE Stereoscopic Displays and Virtual Reality Systems IX, 254, Volume 4660, Pages 254–265, May 2002.