利用適應性區域加權核心放大影像及深度影像

Image and Depth Image Upsampling Using Adaptive Local Weighted Kernel

周詩涵(Shi-Han Zhou)

指導教授 ： 貝蘇章

國立臺灣大學/電機資訊學院/電信工程學研究所/碩士(2015年)

https://doi.org/10.6342/NTU.2015.02394

摘要

本論文研究兩種影像處理的技術，包含深度圖(depth image)計算以及影像放大(image upsampling)技術，隨著科技日益成長，立體視覺技術廣泛應用於多媒 體及電視影像中，利用立體視覺演算法得到以目標深度為主的資訊，主要藉由兩張影像的特徵點進行比對，找出每個像素點之間的位置差，稱為像差(disparity)，最後以立體視覺成像原理計算出目標深度圖，進而利用深度圖資訊進行新視點圖像的合成(view synthesis)。
一般影像及深度影像放大廣泛應用於電腦視覺處理及3D影像處理中，我們提出一個簡單且有效率的影像放大方法，其中可自動提升影像的解析度並保留影像的重要資訊，透過還原成像程序將影像重組至最接近實物的狀況，快速提升影像至高清規格，有別於傳統的影像放大技術，如最鄰近內插法(nearest interpolation)、雙線性內插法(bilinear interpolation)、雙立方內插法(bicubic interpolation)等，本論文提出適應性區域加權核心(Adaptive Local Weighted Kernel)的影像放大技術可以應用在深度圖以及一般影像上，利用適應性加權核心與低解析度原圖相對應的區域做卷積(convolution)，內插出位在各個原始像素之間的新像素值，進而在快速求出高解析度影像，並針對不同影像來源產生其相對應的加權核心，可以有效提高各個放大影像的訊噪比。另一方面，提出的演算法架構未來可以應用在現今電視2K (High Definition)轉4K (Quad Full High Definition)的影像處理技術中，目的在於將影像以更高效的方式內插出高解析的放大影像。

關鍵字

深度圖 ； 像差 ； 新視點圖像合成 ； 影像放大 ； 最鄰近內插法 ； 雙三次內插 ； 適應性區域加權核心

並列摘要

In this thesis, we study two topics about image processing, including depth image extraction and image upsampling. With the growing of technology, stereoscopic vision widely apply to image processing on multimedia and television. We find depth information of images by using stereo correspondence algorithms. The main idea of algorithms is matching feature points of two images, and then finding the position difference between every pixel, which is called disparity. Finally, the depth value can be obtained by the principle of stereo matching algorithm, and then we use these depth map information to conduct the application of view synthesis.
Images and depth images upsampling are generally used in computer vision and 3D image processing. We propose a simple but effective upsampling method for automatically enhancing the image resolution, while preserving the essential structural information. The main idea is reconstructing images through the procedure of image recovering, that upsample low resolution to high resolution images quickly and get close to ground truth images. Different from the traditional upsampling technique, such as nearest interpolation, bicubic interpolation and etc. We proposed a method, called Adaptive Local Weighted Kernel, which can both apply to depth images and generic images. In undetermined upsampling images, unknown pixels between every original pixel will be interpolated by the convolution of low resolution image and Adaptive Local Weighted Kernel in the corresponding window size, thus this method can upsample images to high resolution rapidly. According to different image sources, the proposed algorithm will generate adaptive weighting to enhance edges, details and image quality of upsampling images respectively. On the other hand, the proposed
algorithm can be applied to the new technique in image processing of television in the future, which can change image resolution from 2K(High Definition) to 4K (Quad Full High Definition) in high speed.

並列關鍵字

depth image ； disparity ； view synthesis ； image upsampling ； nearest interpolation ； bicubic interpolation ； adaptive local weighted kernel

參考文獻

[1] R. Keys, "Cubic convolution interpolation for digital image processing,” IEEE Trans. on Acoustics, Speech, and Signal Processing 29 (6), pp. 1153–1160, 1981.

[2] Freeman, W. T., Jones, T. R., and Pasztor, “Example-based super-resolution,” IEEE Computer Graphics and Applications 22, pp. 56–65, 2002.

[4] H. Hirschmuller and D. Scharstein. “Evaluation of stereo matching costs on images with radiometric differences,” IEEE TPAMI, 31(9):1582–1599, 2009.

[5] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” Int’l J. Computer Vision, 2002.

[7] P. Anandan., “A computational framework and an algorithm for the measurement of visual motion,” IJCV, pp. 283–310, 1989.