利用光場資料之數位變焦演算法與硬體架構設計

An Algorithm and Architecture Design for Light Field Based Digital Refocusing

陳致傑(Chih-Chieh Chen)

指導教授 ： 盧奕璋

國立臺灣大學/電機資訊學院/電機工程學研究所/碩士(2010年)

https://doi.org/10.6342/NTU.2010.01887

摘要

在本篇論文中，我們建立了一個計算式攝影系統，對四維光場進行取樣。本系統是一個普通的單眼數位相機，利用一個由Kodak LVT技術印製而成的針孔陣列遮罩，將之放於相機中感光器之前，此遮罩擋住了部分的光線，並對其餘光線進行取樣，所取得之影像即為四維光場資料。
利用這些資料，我們提出了數位變焦演算法，經由後處理程序合成出對焦在不同距離之影像，利用此演算法，即可達到一般相機中以鏡片組變焦之功能。並推導數位變焦之數學模型，計算出此系統之變焦範圍，
此外，我們也利用光場資料提出了深度預測之演算法，在第一個方法中，利用所合成的數位變焦影像的傅氏轉換，求得影像中物體之銳度，取得該物體之最加變焦變數再加以換算為物距；第二個方法是利用物體在兩相鄰之分影像中的座標差，來預估此物體之物距。
最後，我們亦提出了數位變焦演算法之硬體架構，藉以提升處理之速度。以TSMC13製程估計，晶片尺寸為2.178mm2，核心尺寸為0.614mm2，運作頻率設計為100MHz，功率消耗為50.28mV。

關鍵字

光場 ； 針孔陣列遮罩 ； 數位變焦 ； 變焦變數 ； 深度預測

並列摘要

In this thesis, a computational photography system is utilized to sample 4D light fields. The system is implemented using a normal DSLR camera with a mask printed using Kodak LVT technique. We reconfigure the camera by inserting a pinhole array mask in front of the sensors. The mask blocks part of light and samples the rest that passes through the pinholes. With these 4D light field data, the refocused images are obtained by rearranging the captured sub-images. Thus, with our algorithm, we can adjust focus distances without a complex lens set..
Also, depth estimation of light field data obtained from the pinhole-masked camera system is proposed. In the first approach, the sharpness of objects in the refocused images is evaluated using FFT method, from which an empirical formula can be obtained to estimate the depth of objects. The second method uses the raw data in sub-images to calculate an index, Average Absolute Difference (AAD), for depth estimation purposes. The advantages and disadvantages of two proposed approaches are discussed and compared.
Finally, to speed up the process for future real-time application, we also implement hardware processor of our algorithm. The chip is implemented with TSMC 130 nm technology. The chip size is 2.178mm2, power consumption is 50.28mV, and the chip operates at 100 MHz when processing a captured image into a 640×480 image.

並列關鍵字

light field ； pinhole array mask ； digital refocusing ； depth estimation

參考文獻

[11] C. K. Liang, G. Liu, and H. H. Chen, “Light field acquisition using programmable aperture camera,” in Proc. ICIP’07, San Antonio, Texas, vol. 5, pp. 233-236. 2007.

[2] P. Moon and D. E. Spencer, The photic field, MIT Press, 1981.

[8] E. H. Adelson and J. Y. A. Wang, “Single lens stereo with a plenoptic camera,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 99-106, Feb, 1992.

[10] C. K. Liang, L. H. Lin, B. Y. Wong, C. Liu, and H. H. Chen, “Programmable aperture photography: multiplexed light field acquisition,” ACM Trans Graphics, vol. 27, no. 3, pp. 55:1-55:10, 2008.

[12] A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin, “Dappled photography: mask enhanced cameras for heterodyned light field and coded aperture refocusing,” ACM Trans. Graphics, ACM Press, New York, NY, USA, vol. 26, issue 3, no. 69, 2007.