Title

立體影像深度基於視覺舒適度最佳化即時處理系統

Translated Titles

Real-Time Stereo Video Disparity Optimization for Human Visual Comfort

DOI

10.6342/NTU.2014.00915

Authors

鄭家豪

Key Words

立體三維 ; 雙眼視差 ; 多視角立體數為內容 ; 視差估計 ; 深度圖繪圖法 ; 可程式化邏輯陣列 ; 高解析度多媒體介面 ; Stereoscopy ; 3D ; binocular parallax ; multi-view ; disparity estimation ; depth image base rendering (DIBR) ; FPGA ; HDMI

PublicationName

臺灣大學電子工程學研究所學位論文

Volume or Term/Year and Month of Publication

2014年

Academic Degree Category

碩士

Advisor

陳中平

Content Language

英文

Chinese Abstract

立體影像最佳化為立體影像技術發展重要課題之一,目的為讓使用者在觀看立體影片時不會因為影片不適當而導致視覺疲勞之負面感受。隨著現在的影片內容由影像技術發展解析度越來越高,高解析度影像在影像處理上所需要的時間是相當可觀的,因此我們將立體影像最佳化演算法設計為適合硬體架構處理,利用硬體化加速達到可即時的影像處理以解決計算時間過長的問題。 本篇論文中,由於在立體影像中深度資訊對於觀賞者舒適度有很大的影響,於不同影像面板大小有不同的深度舒適區,因此借由調整影像深度可以達到觀測者不會產生視覺疲勞不舒適的情況。另外,觀賞立體影像時需佩戴立體眼鏡的不方便性,可以借由所計算出的深度資訊將雙視角立體影像轉為多視角立體影像,待多視角立體數位影像面板大量產品化,可利用本篇論文方法轉換雙視角影像內容為多視角影像內容。此套方法包含影像重建,視差計算,影像濾波,深度調整等。 本篇論文中使用現場可程式化邏輯陣列版(電子影像套件)包含大容量的同步動態隨機存取記憶體方便實現深度計算演算法,影像界面與深度調整的硬體架構。在架構設計上我們將重點放在於如何利用硬體同步處理大量資訊以達即時化處理。此系統在可程式化邏輯陣列板上,對於輸入訊號為1080逐行掃描的視訊顯示格式可預期達到每秒30張以上畫面處理。視差搜尋範圍為[-30,+30]。

English Abstract

Stereo video optimization has been regarded as one of the most important subject of 3D technology. The main purpose of the research is to let the viewers free from the visual fatigue caused by traditional stereo videos. With the advances in video technology, we have higher video resolution and better image quality; however, computing disparity maps on high-resolution images is time-consuming. To solve this problem, we developed a modified algorithm that is more suitable for hardware implementation, and the system architecture is designed to be capable of processing stereo videos with a large amount of data in real time. In this thesis, we analyze the causes of visual fatigue from the human factors experiments. The depth information of stereo videos is a significant factor for human visual comfort. Panels of different sizes have different disparity range within the comfort zone, thus, visual fatigue can be relieved by adjusting the disparity range. Furthermore, with the calculated depth information, two-view images can be converted to multi-view images, such that viewers can be free from the inconvenience of wearing special 3D glasses. The methods provided in this thesis can be used as a two-view to multi-view conversion tool for multi-view auto-stereoscopic displays. Our system applies the Depth Image Based Rendering (DIBR) method, disparity map estimation and image filtering. The on-board collection of peripherals includes high-speed video interface (HDMI) and two 1Gbits SDRAMs. The system on FPGA can achieve 30 frames per second (fps) in 1080p display format, with disparity estimation search range of [-30, +30] in horizontal direction.

Topic Category 電機資訊學院 > 電子工程學研究所
工程學 > 電機工程
Reference
  1. [3] N. A. Dodgson, "Autostereoscopic 3D displays." IEEE Computer Magazine, Volume 38, Number 8, Page 31-36, August 2005.
    連結:
  2. [5] M. T. M. Lambooij, W. A. IJsselsteij, and I. Heynderickx,” Visual Discomfort in Stereoscopic Displays: A Review,” SPIE-IS&T, Volume 6490, Page 1-13, 2007.
    連結:
  3. [6] N. Qian, "Binocular Disparity and the Perception of Depth," Neuron, Volume 18, Page 359-368, March 1997.
    連結:
  4. [7] N. A. Dodgson, “Analysis of the Viewing Zone of Multiview Autostereoscopic displays,” Proc. SPIE Stereoscopic Displays and Virtual Reality Systems IX, 254, Volume 4660, Pages 254–265, May 2002.
    連結:
  5. [12] D. M. Hoffman, A. R. Girshick, K. Akeley and M. S. Banks, “Vergence–Accommodation Conflicts Hinder Visual Performance and Cause Visual Fatigue,” Journal of Vision, Volume 8, Number 3, Article 33, March 2008.
    連結:
  6. [13] J. Kim, T. Shibata, D. M. Hoffman and M. S. Banks, “Assessing vergence-accommodation conflict as a source of discomfort in stereo displays,” Journal of Vision, Volume 11, Number 11, Article 324, September 2011.
    連結:
  7. [14] T. Shibata, J. Kim, D. M. Hoffman and M. S. Banks, “The Zone of Comfort: Predicting Visual Discomfort with Stereo Displays,” Journal of Vision, Volume 11, Number 8, Article 11, July 21, 2011.
    連結:
  8. [1] C. Chinnock, "Trends in the 3D TV market," Handbook of visual display technology. Springer Berlin Heidelberg, Page 2599-2606, 2012.
  9. [2] The Nielsen Company, "U.S. consumers show high interest in 3DTV, but cite some concerns," September 2010. http://www.nielsen.com/us/en.html
  10. [4] Wikipedia contributors, "Binocular Vision," Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Binocular_vision
  11. [8] N. A. Dodgson, J. R. Moore, and S. R. Lang, “Multi-view Autostereoscopic 3D Display,” International Broadcasting Convention, Pages 497-502, September 1999.
  12. [9] C. P. Lin, “High Speed FPGA-based Hardware Implementation of 3D Video Optimization for Human Visual Comfort Enhancement,” M.S. thesis, Nation Taiwan University, Taipei, ON, Taiwan, 2012.
  13. [10] T. Y. Wu, “Automation of Disparity Adjustment According to Human Factor for Depth-Image-Based Rendering,” M.S. thesis, Nation Taiwan University, Taipei, ON, Taiwan, 2012.
  14. [11] Wikipedia contributors, "Lenticular_lens," Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Lenticular_lens
  15. [15] H. Lee, “A Visual Comfort Optimizing Two-view to Multi-view Stereo Content Conversion Method based on Human Factor Analysis,” M.S. thesis, Nation Taiwan University, Taipei, ON, Taiwan, 2013.
  16. [16] A. Woods, T. Docherty, and R. Koch, “Image Distortions in Stereoscopic Video Systems,” Proc. SPIE Stereoscopic Displays and Applications, Pages 36-48, February 1993.
  17. [17] R. Gupta and S. Y. Cho, “Real-time Stereo Matching using Adaptive Binary Window,” Proc. 3D Data Processing, Visualization and Transmission, 2010.
  18. [19] S. Jin, J. Cho, X. D. Pham, K. M. Lee, S. K. Park, M. Kim, and J. W. Jeon, " FPGA Design and Implementation of a Real-Time Stereo Vision System," IEEE Trans. Circuits Syst. Video Technol., Volume 20, Number 1, Page 15-26, January 2010.
  19. [20] A. Darabiha, J. Rose, and W. J. Maclean, “Video-rate stereo depth measurement on programmable hardware,” in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognit., Madison, WI, Volume 1, Page 203–210, June 2003.
  20. [21] D. Scharstein and R. Szeliski, Middlebury Stereo Vision Page. http://vision.middlebury.edu/stereo/
  21. [22] Wikipedia contributors, "Binocular Disparity," Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/wiki/Binocular_disparity