用於無透鏡光場相機資料之數位變焦與深度估測演算法

Depth Estimation and Digital Refocusing Algorithms for Lensless Light Field Camera Data

何俊憲(Chun-Hsien Ho)

指導教授 ： 盧奕璋

國立臺灣大學/電機資訊學院/電子工程學研究所/碩士(2014年)

https://doi.org/10.6342/NTU.2014.10772

摘要

現有的手持式光場相機是在透鏡與感光器中間加入微透鏡陣列或是針孔陣列遮罩，記錄四維光場資訊，和一般相機相比，可以進行數位變焦及生成場景深度圖等應用。
在本論文中，我們將相機的鏡頭拆除，將針孔陣列遮罩放置於相機之感光器前，形成一無透鏡光場相機系統，因為少了鏡頭，整個相機的成本、重量與體積都減少了許多，可增加攜帶的便利性。光線經由個別針孔所投影出的子影像，皆可視為一獨立的針孔相機成像。因此，我們只需一次快門便可得到一組針孔相機陣列的光場資料。因為每個子影像視角皆不相同，我們可以綜合運用子影像的資訊進行深度估測，並將個別子影像拼接成一較大影像，然後進行數位變焦，模擬透鏡的對焦效果。
基於深度估測在演算法流程中耗時最長，且以硬體實踐數位變焦演算法的效益不高，在論文的最後，我們以積體電路實作無透鏡光場相機資料深度估測處理器。對於3648×2280之光場資料，可以於2.1秒內完成子影像深度估測，與軟體版本相比加速可達3倍。以TSMC90奈米製程實現，晶片尺寸為1.49×1.49 mm2，核心尺寸為1.029×1.029 mm2，在操作頻率125 MHz下之消耗功率為111.4 mW。

關鍵字

光場 ； 針孔相機 ； 深度估測 ； 數位變焦 ； 硬體設計

並列摘要

Hand-held light field cameras can provide additional applications, such as depth estimation and digital refocusing, which are not available through conventional digital cameras.
In this thesis, we design a lensless light field camera system by removing the camera lens and placing a pinhole array mask in front of the sensor. Without a camera lens, the cost, weight and volume of cameras can be reduced. Each sub-image can be regarded as a single image captured by a small pinhole camera. Therefore, we are able to obtain a set of pinhole camera array data with only one single shot. Due to different view angles between sub-images, we can apply depth estimation technique to sub-image pairs and combine all sub-images into a larger image. Digital refocusing can be achieved using the data from these sub-images as well.
In the last part of the thesis, we implement a depth estimation processor for lensless light field data using TSMC 90 nm technology. This processor is able to process a 3648×2280 light field image and generate a 500×400 depth map within 2.1 s. The chip size is 2.2201 mm2. The power consumption is 111.4 mW when operating at 125 MHz.

並列關鍵字

light field ； pinhole photography ； depth estimation ； digital refocusing ； hardware implementation

參考文獻

[1] M. Levoy and P. Hanrahan, "Light field rendering," in Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, 1996, pp. 31-42.

[2] B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, et al., "High performance imaging using large camera arrays," in ACM Transactions on Graphics (TOG), 2005, pp. 765-776.

[3] C.-K. Liang, G. Liu, and H. H. Chen, "Light Field Acquisition using Programmable Aperture Camera," in Image Processing, 2007. ICIP 2007. IEEE International Conference on, 2007, pp. V - 233-V - 236.

[4] C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, and H. H. Chen, "Programmable aperture photography: multiplexed light field acquisition," in ACM Transactions on Graphics (TOG), 2008, p. 55.

[5] R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, "Light field photography with a hand-held plenoptic camera," Computer Science Technical Report CSTR, vol. 2, 2005.