- 學位論文
適用於低光環境之光場相機深度估測演算法與硬體設計
A Depth Estimation Algorithm and Hardware Design for Light Field Cameras under Low Light Conditions
摘要
在計算攝影學(computational photography)中，光場(light field)是一種描述光的方式，我們在相機的感光器前放置一個針孔陣列遮罩(pin-hole array mask)，使得相機可以取得四維的子影像(sub-image)陣列資料，包含光的位置和行進方向，此四維資料即為光場資料。利用光場資料，便可以分析光的方向，做到物體深度估測(depth estimation)，即取得一般相機無法得知的物體深度資訊。 然而，拍攝時無法避免雜訊的干擾，尤其是低光環境拍攝時，為增加亮度常會提高感光度與曝光時間，如此便會引進更多雜訊。為了降低雜訊對計算深度圖(depth map)的影響，現今的做法主要有兩種，一種是對原圖做完去雜訊後再做深度估測，另一種是直接對深度圖做去雜訊的處理，但前者的問題是事先不知要對原圖做什麼樣的去雜訊處理才合適，後者的問題是如果一開始算出來的深度圖已有太多計算錯誤時，再怎麼處理也無法修正這些錯誤。為避免上述兩個問題，本篇論文提出將去雜訊與深度估測融合的演算法，亦即一邊去雜訊一邊估算深度，兩項工作同時在演算過程中完成。 最後，由於深度估測本身就是一個耗時的演算法，再加上去雜訊的處理，整套演算法的運算量相當大，如果運算速度太慢，將會大幅限制這套演算法的應用範圍，因此我們也提出適用於低光環境之光場相機深度估測演算法的硬體架構，以提升演算法的運算速度。以TSMC90製程估計，晶片尺寸為1.294 mm2，核心尺寸為0.486 mm2，運作頻率設計最高為110 MHz，功率消耗為54.81 mW。
並列摘要
In computational photography, light field is an approach to describe light rays. In this research, we place a pin-hole array mask in front of the camera sensor to obtain positions and directions of lights. With the four-dimensional light field data, we can apply depth estimation technique to get depth information which can’t be obtained from traditional digital cameras. However, it’s difficult to avoid noise impacts when taking photos, especially at night. To increase brightness, people usually increase the ISO and exposure time. Both approaches will induce more noise. Nowadays, there are two kinds of methods to reduce the influences of noise on depth maps. One is to apply denoising techniques to the original image before doing depth estimation. The other one is implementing denoising algorithms directly to depth maps. But there are problems with these methods. For the former method, we don’t know which denoising method is appropriate because we don’t have the specific information of the noise. For the latter method, if there are too many computational errors in the original depth map, it’s difficult for us to correct these errors. In this thesis, we propose an algorithm that combines denoising and depth estimation, which means we conduct denoising and depth estimation procedures at the same time. Therefore, we can solve the problems mentioned above and automatically generate depth maps with less noise impacts. Depth estimation is a time-consuming algorithm. Since we also need to implement a denoising algorithm, the whole complexity is really high. In order to meet the real-time requirement, we propose a hardware processor to improve the computation speed. The chip and core sizes are 1.294 mm2 and 0.486 mm2 respectively. The power consumption is 54.81 mW when running at 110 MHz.