應用於快速擬真事後重新對焦之一維視點合成演算法之分析與硬體架構設計

基於新視點內插演算法所做的擬真事後重新對焦需要合成大量的新視點來完成單張重新對焦的照片。為了實踐及時擬真重新對焦，利用此方法所需要內插的新視點數量非常龐大。傳統的新視點內插演算法主要受到高運算量以及不規則記憶體的存取，導致很難用有限的硬體資源來實踐及時擬真重新對焦。基於一個專門為擬真重新對焦所設計的一維視點合成演算法，我們分析此演算法並提出兩個硬體設計上的挑戰。第一個挑戰是處理不同步的視點輸入信號，這會使得暫存器已及平行運算的設計非常困難。第二個挑戰是如何縮短一條電路上含有迭代迴圈(iteration bound)的關鍵路徑(critical path)。我們提出利用一個多畫素先進先出的暫存器來解決第一個挑戰。此暫存器分別設計給左右視點使用，來達到獨立更新資料的效果。此外，存多個畫素的目的是為了達到有效率的平行運算。我們也分析了最好的平行度設計是平行度四倍，原因是平行度四倍的設計比起其他平行度能達到較好的硬體資源利用率。對於第二個挑戰，我們利用超前運算(look-ahead)的電路結構已及重排時間(retiming)的電路技巧來大幅縮短電路上的關鍵路徑。利用這兩個方法時鐘頻率可由125M Hz大幅提升至200MHz。我們實踐了一個支援全高清(Full HD)及時擬真重新對焦的新視點合成引擎。全高清及時擬真重新對焦新視點合成所需要達到的新視點畫素速率是每秒五十億個(5Gpixel/s)。為了達到這個目標，八個一維視點合成器需要同時被使用，也因此消耗許多硬體資源。我們利用一個精準度分析證明可以利用較不精準的除法器在線性內插運算裡，此舉不但合成重新對焦圖片品質幾乎不變，而且能大幅降低所需的硬體資源。基於台積電四零奈米製程所做的電路合成，我們的電路利用六十八點七萬的邏輯閘數來達到每秒三十張的全高清擬真重新對焦的新視點合成引擎。

關鍵字

事後重新對焦；新視點合成；大型積體電路設計

並列摘要

View-interpolation-based refocusing requires a large number of novel views to generate a single refocused image. To realize real-time refocusing, novel pixels needed to be generated are dramatically high. Conventional view interpolation methods usually suffer from high computation complexity and irregular memory access, and therefore they are difficult to achieve real-time refocusing with limited hardware resources. In this thesis, we aim to design and implement a view synthesis engine for real-time realistic refocusing. Based on a center-based 1D line scan view synthesis algorithm, which is specifically designed for refocusing, we analyse the algorithm and find two design challenges for hardware implementation. The first challenge is to deal with asynchronous input access of central and right views, which makes the buffer design and the parallelism design difficult. The second challenge is to reduce a long critical path which consists of an iteration bound. The first challenge is solved by proposed multi-pixel throughput FIFO buffers. The FIFO buffers are applied for central, right and novel views respectively to make data access independent, and the multi-pixel throughput is designed to achieve parallel computation. The parallel designs of the algorithm are analysed and the best parallelism number is four, because it achieves the same output throughput with smaller hardware resources. To reduce the iteration bound, a proposed look-ahead structure and a retiming technique are used to greatly reduce the critical path. As a result, the maximum clock rate can be increased from 125 MHz to over 200 MHz. A view synthesis engine is implemented to support real-time Full HD refocusing. The required novel pixel rate is over 5 Gpixel/s. To achieve this requirement, eight 1D view synthesis engines are needed. Since so many engines consume enormous hardware resources, a precision analysis shows that we can use the less precise dividers for linear interpolation engines to greatly reduce the number of gate count with only a little distortion. Based on logic synthesis results using TSMC 40nm process, the final design consumes 687k gates, and performs 30 fps for Full HD refocusing.

並列關鍵字

refocus ； view synthesis ； VLSI design

參考文獻

[2] Michael Schaffner, Pierre Greisen, Simon Heinzlean, Frank K. Gurkaynak, Hubert Kaeslin, and Aljoscha Smolic, MADmax: A 1080p stereo-to-multiview rendering ASIC in 65 nm CMOS based on image domain warping," in Proc. European Solid-State Circuits Conference, 2013, pp. 61 -64.

[3] M. Harris, Focusing on everything," IEEE Spectrum, vol. 49, no. 5, pp. 44 -50, 2012.

[4] M. Levoy, Light fields and computational imaging," IEEE Computer, vol. 39, no. 8, pp. 46 -55, 2006.

[9] P. Rokita, Generating depth of-field effects in virtual reality applications," IEEE Computer Graphics and Applications, vol. 16, no. 2, 1996.

[10] Yuichi Taguchi, Amit Agrawal, Ashok Veeraraghavan, Srikumar Ramalingam, and Ramesh Raskar, Axial-cones: Modeling spherical catadioptric cameras for wide-angle light field rendering," ACM Trans. Graph., vol. 29, no. 6, pp. 172:1 -172:8, Dec. 2010.

國際替代計量

應用於快速擬真事後重新對焦之一維視點合成演算法之分析與硬體架構設計

全文下載

主題瀏覽