考慮光場特性之影像超解析演算法與硬體設計

A super resolution algorithm and its hardware design for light field images

林奕憲(Yi-Hsien Lin)

指導教授 ： 盧奕璋

國立臺灣大學/電機資訊學院/電子工程學研究所/碩士(2014年)

https://doi.org/10.6342/NTU.2014.10821

摘要

光場相機可以記錄來自同一物體不同光線角度的四維光場資料，經由數位變焦處理後，便可後製出可供使用者觀看之二維影像。然而，受到感光器像素的限制，以空間解析度換取足夠的角度解析度記錄光線時，會造成後製的影像空間解析度大幅降低。因此，如何有效的提升空間解析度對於光場相機的發展頗為重要。
　　在本篇論文中，我們提出一套考慮光場特性之影像超解析演算法。藉由推導出的光學成像性質，可以得知物體在各個子影像中重複成像的情況，因為成像結果的大小、位置與物體的深度有關，只要取得物體的深度圖，就可以把這些重複成像都擷取出來，並利用這些低解析度影像還原出一張較高解析度的影像。
　　影像超解析演算法通常耗時且計算量龐大，為了加快運算時間，我們以積體電路實作出適用於光場資料之影像超解析處理器，輸入之四維光場之像素尺寸為27x44x46x46，輸出之二維影像之像素尺寸為2112x1296，以影像長寬放大三倍的情況而言，整個流程可於2.783秒內完成，與軟體相比加速可達14倍，使用的製程為TSMC 90nm、運作頻率為125 MHz、晶片尺寸為 1.377 mm2、消耗功率為 105.4 mW。

關鍵字

光場 ； 針孔陣列遮罩 ； 超解析度 ； 硬體設計

並列摘要

We can use 4D light field cameras to record different directions of light rays from the same object. After digital refocusing, we can get an ordinary 2D image. Because of the limitation of the sensor on the camera, when we trade the spatial resolution for the angular resolution, the resolution of the output images will be small. Therefore, it becomes an important issue to raise the spatial resolution for light field images.
　　In this thesis, we propose an image super resolution method for light field images. We use matrix optics to obtain important light field imaging characteristics. Since the depth of an object is related to the disparity of the repeated light filed sub-images, as long as we can get the depth map of the light field image, we can use the low resolution repeated images to recover a higher resolution image.
　　The image super resolution algorithm is time-consuming. Therefore, we design a processor using TSMC 90nm cell library to speed up calculations. The processor operates at 125 MHz and is capable of processing a 44x27x46x46 light field image to a super resolution result of 2112x1296 pixels within 2.783 s. It is 14 times faster than the software version. Its chip area is 1.377 mm2, and its power consumption is 105.4 mW.

並列關鍵字

light field ； pinhole mask array ； super resolution ； hardware design

參考文獻

[1] C. Zhou and S. K. Nayar, "Computational cameras: convergence of optics and processing," Image Processing, IEEE Transactions on, vol. 20, pp. 3322-3340, 2011.

[2] B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, "High performance imaging using large camera arrays," in ACM Transactions on Graphics (TOG), 2005, pp. 765-776.

[3] S. Wanner and B. Goldluecke, "Globally consistent depth labeling of 4D light fields," in Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, 2012, pp. 41-48.

[4] C. Kim, H. Zimmer, Y. Pritch, A. Sorkine-Hornung, and M. H. Gross, "Scene reconstruction from high spatio-angular resolution light fields," ACM Trans. Graph., vol. 32, p. 73, 2013.

[5] C.-K. Liang, G. Liu, and H. H. Chen, "Light field acquisition using programmable aperture camera," in Image Processing, 2007. ICIP 2007. IEEE International Conference on, 2007, pp. V-233-V-236.