利用影像深度地圖即時產生雙眼立體影像及其硬體設計

Real-Time Stereoscopic Image Generation from Depth Map and Its Hardware Design

鄭芳炫(Fang-Hsuan Cheng) ； 張祐維(Yu-Wei Chang)

《Journal of Information Technology and Applications(資訊科技與應用期刊)》 4卷4期 (2010/12) Pp. 207-217

https://doi.org/10.6302/JITA.201012_4(4).0006

摘要

雙眼觀看一個物體的時候，該物體在左右眼中會有些許的位移，而這個位移就稱為視差。物體越遠視差越小，反之視差越大，因為視差的原因讓我們產生了立體深度的感覺。因此我們可以利用一些方法將一張2D的影像產生出虛擬的雙眼立體影像，例如DIBR（depth-image-based rendering）利用原始2D影像的影像深度圖與虛擬立體攝影機的設置來產生一組有視差的立體影像。但由於我們產生的過程中，變動了影像內物體的位置，造成影像中有空洞的問題，這樣的立體影像是無法觀看的，因此我們必須再利用影像填補（image inpainting）的方法將空洞填補起來。然而影像填補是非常花費時間，為了達到即時（real-time）的應用，我們希望將產生立體影像的系統以硬體來實現，以利於未來的應用。論文中我們設計了一個利用原始單眼影像及其相對應的影像深度地圖來產生雙眼立體影像的硬體架構。我們使用DIBR的演算法來產生出雙眼影像，再使用簡化的影像填補演算法填補產生的空洞。由於硬體設計以及速度的考量，我們簡化了填補演算法，雖然造成填補的效果下降，但其結果都還在一個可接受的範圍內。由於影像結構的關係，常常為了少數幾塊的空洞區域而要重覆掃描整張影像。所以為了提升計算的速度，將整張影像切割分段處理，每次只處理幾行，將計算時間分散，使計算時間較整張處理的方式提升了約50%。實驗中我們以150MHz的時脈效能就可達到即時的運算（320 x 240 @30 fps）。整個設計所使用的邏輯暫存器（logic registers）個數一共有2,849個邏輯暫存器。

關鍵字

立體影像 ； 雙眼視覺 ； 影像填補 ； 空洞填補

並列摘要

When you look at the object, that object will cause some displacement in our eyes. This displacement is called parallax. When the object is farther, the parallax become smaller, and on the other hand, it becomes greater. Because the reason of parallax makes people have feeling to stereoscopic vision. We can utilize some methods to transform the 2D image to virtual stereoscopic image. For example, DIBR (depth-image-based rendering) algorithm utilizes the depth map and virtual stereoscopic camera model of 2D image to produce the stereoscopic image with parallax. In the course which we produce. We change the position of the object in the image, and it results in hole problem in the image. Such stereoscopic image can't be watched. Therefore we should use the image inpainting algorithm to fill the holes. But it spends too much time. In order to achieve the real-time application, we hope that we can transform the whole stereoscopic image generation system to hardware to favor future application. In the paper, we designed depth map that match to 2D image to produce hardware architecture of stereoscopic image generation system. We use the DIBR algorithm to generate the binocular image. Then we use simple image inpainting algorithm to fill the hole. Considering the hardware design and computing time, we simplified the image inpainting algorithm. Although it makes the efficiency getting down, the result is still in an acceptable scope. As a result of image structure, we usually have to repeat scanning the whole image for minority several hole regions. So, in order to increase the computation the speed. We tryed to partition the whole image into segments, and processes several lines each time. Then, we scattered the computing time. And we found that this kind of processing method can improve the efficiency that promote approximately 50% than whole image processing. In the experiment, we achieve the real-time computing in 150MHz clock (320 x 240 @30 fps). There are totally 2,849 logic registers in this design.

並列關鍵字

DIBR(depth-image-based rendering) ； Image inpainting ； Stereoscopic Image ； hole filling ； binocular vision