基於視覺顯示之節能與品質提升技術

Visual-sensation-aware Content Processing for Energy-efficient and Quality-enhanced Display

林翰毅(Han-Yi Lin)

指導教授 ： 郭大維

共同指導教授 ： 林彥宇(Yen-Yu Lin)

國立臺灣大學/電機資訊學院/資訊工程學研究所/博士(2020年)

https://doi.org/10.6342/NTU202100749

摘要

近年來由於資訊技術與數位生活的蓬勃發展，我們的生活中充滿了各式各樣的顯示系統，小至智慧型手錶與智慧型手機，大至電腦螢幕與電視螢幕等。隨著材料與製程的進步，視覺效果更清晰的發光材料也被廣泛的應用，並且顯示器的像素密度與解析度也持續提升。然而顯示技術的進步也帶來了許多新的問題，例如新的顯示器使用有機發光二極體(OLED)做為發光元件，但新型態的元件其發光原理與以往的液晶螢幕(LCD)發光原理與功耗模型有很大的不同，這使得傳統的電源管理方法，並無法直接應用在新型態的OLED顯示器上。另一項問題是像素密度不對稱，近年來製程進步讓顯示器的像素密度提升，但有部分錄影設備的解析度還沒相對應的提升，因此需要一個超解析度的方法來將影片提升至對應的解析度。另外隨著顯示器解析度的提升，影片會以較高的解析度呈現在顯示器上，此時若影片中有模糊區域，則模糊將會被放大，因此我們需要一個去模糊的方法來應對。本篇博士論文針對以上顯示器進步所帶來的問題進行探討與研究，提出了對應的方法來解決。針對第一個議題，我們考量人眼視覺特性提出了一個OLED的電源管理機制。接著為了讓影片解析度能夠對應到顯示器的像素密度，我們同時考量空間域與時間域的相依性，提出一個同時增加空間域與時間域解析度的方法。最後我們提出一個自適應新場景的去模糊網路，可以在適應新的場景後再進行去模糊。在本篇論文中所提出的方法，透過實驗驗證都有相當程度的效能提升。

關鍵字

行動裝置節能 ； OLED顯示節能 ； 影像品質提升 ； 電腦視覺 ； 深度學習 ； 超解析度 ； 影格內插 ； 去模糊

並列摘要

In the past few decades, due to the popularity of information technology, our lives have been filled with various display systems, ranging from smartwatches and smartphones to computer screens and TV screens. With the advancement of materials and manufacturing processes, display materials with better imaging quality are widely used, the pixel density and resolution of displays have also increased year by year. However, the progress of display technology has also brought many new issues. For example, the next-generation display uses organic light-emitting diodes (OLED) as the light-emitting elements, but the light-emitting principle of OLED is very different from the LCD. The power model is also different, which makes the traditional power management method cannot be directly applied to the OLED display. Another issue is the problem of asymmetry pixel density. In recent years, advances in manufacturing have increased the pixel density of displays. However, the resolution of many videos has not yet been improved to such a high resolution, so we need a super-resolution approach to up-scale the resolution of videos. In addition, as the resolution of the monitor increases, the video will be displayed on the monitor with a higher resolution. At this time, if there is a blurred area in the video, the blur will be enlarged. Therefore, we need a deblurring method to deal with. This PhD thesis proposes corresponding methods to solve these issues. In response to the first issue, we have proposed a power management mechanism for OLED considering the characteristics of human visual acuity. Then, in order to make the video resolution correspond to the pixel density of the display, we propose a spatiotemporal super-resolution method. Finally, we propose a scene-adaptive deblurring network, which can adapt to novel scenes. The proposed methods are verified through experiments, and have a considerable performance improvement.

並列關鍵字

Energy-efficient mobile display ； Energy-efficient OLED display ； Video quality enhancement ； Computer vision ； Deep learning ； Super-resolution ； Frame interpolation ； Deblurring

參考文獻

K. W. Tan, T. Okoshi, A. Misra, and R. K. Balan. FOCUS: A Usable Effective Approach to OLED Display Power Management. In Proc. of ACM UbiComp, pages 573–582, 2013.

Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, and William T. Freeman. Video enhancement with task­oriented flow. IJCV, 2019.

Simon Baker, Daniel Scharstein, J. P. Lewis, Stefan Roth, Michael J. Black, and Richard Szeliski. A database and evaluation methodology for optical flow. In ICCV, 2007.

Ce Liu and Deqing Sun. A bayesian approach to adaptive video super resolution. In CVPR, 2011.

Jordi Pont­Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbelaez, Alexander Sorkine­Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation. arXiv:1704.00675, 2017.