將光流法應用於重建式多張影像超解析之可靠方法

A Robust Reconstruction-Based Multi-Frame Super-Resolution Method using Optical Flow

陳智暘(Chih-Yang Chen)

指導教授 ： 莊永裕

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2015年)

https://doi.org/10.6342/NTU.2015.02736

摘要

從一連串觀測到的低解析度影像，合成一張高解析度影像的演算
法，稱之為多張影像超解析度。而重建式多張影像超解析度演算法大
致上可分為兩個步驟: 低解析影像之間的對齊與高解析影像的重建。
在本篇論文中，基於不同張低解析度影像間光強度一致性的假設，
嘗試多種光流法來對齊影像。並且基於來回光流的一致性假設，計算
光流的可信度，將其帶入高解析度影像的重建中，以減少對齊誤差對
重建結果的影響。然而在”分辨力測式卡”這樣的測試資料中，會因
為相機在拍攝過於高頻的圖樣時所產生的錯誤成像，違背光強度一致
性的假設，進而導致光流法對齊失敗。所以我們提出在使用光流法對
齊前，將圖片先進行模糊處理，使得光流法不受此種錯誤成像的影響。
另外，由於現今照片的解析度越來越高，重建高解析度影像需要龐
大的記憶體與時間。本篇論文提出將重建分成多個可平行處理的資料
塊，以減少記憶體用量。並且在硬體方面嘗試使用多執行緒與圖型處
理器加速。重建演算法方面則是提出使用最近鄰居重建法與線性重建
法的結合，進而達到加速的效果。
透過本篇論文提出的方法，能使將光流法運用於多張重建式超解析
度之方法更為可靠。並減少重建的時間與記憶體使用量。

關鍵字

超解析度 ； 加速 ； 平行 ； 光流法 ； 可信度 ； 可靠方法

並列摘要

Method of integrating a high-resolution (HR) image from multiple observed
low-resolution (LR) images is called multi-frame super-resolution (SR).
There are basically two stages of reconstruction-based SR: registration of LR
images and reconstruction of HR image.
In this thesis, we based on the assumption of intensity consistency, and
tried several optical flow methods as registration method. Also, based on another
assumption: ”forward-backward flow consistency”, we calculated the
confidence of a flow, then brought confidence into HR image reconstruction
to reduce the error caused by mis-registration. But in the test sets like ”resolution
chart”, there will be some errors caused by some patterns with frequencies
that is too high. The errors violates the assumption of intensity consistency,
which will cause fail registration of optical flow method. Thus, we proposed
to applying blur before calculating the flow. The method can prevent optical
flow from failing.
Also, due to the resolution of images nowadays becomes higher and higher,
which will make the reconstruction of HR image need enormous amount of
memory usage and time. The thesis proposed to divide the reconstruction
to multiple parallelable data blocks to reduce memory and time usage, and
proposed multi-thread and GPU speed-ups. As for algorithm speed-up, we
proposed combining nearest neighbors (NN) reconstruction and linear reconstruction to achieve acceleration.
With the method proposed by this thesis, we can make using optical flow
in multi-frame reconstruction-based SR more robust, and reduce the reconstruction
time and peak memory usage.

並列關鍵字

super resolution ； acceleration ； parallel ； optical flow ； confidence ； robust method

參考文獻

[1] Berthold K Horn and Brian G Schunck. Determining optical flow. In 1981 Technical

symposium east, pages 319–331. International Society for Optics and Photonics,

[2] Gunnar Farnebäck. Two-frame motion estimation based on polynomial expansion.

realtime tv-l 1 optical flow. In Pattern Recognition, pages 214–223. Springer, 2007.

sublinear optical flow algorithm. In Computer Graphics Forum, volume 31,