基於深度學習方法之單張文件影像陰影去除

Deep Learning-based Approach for Single Document Image Shadow Removal

林昀宣(Yun-Hsuan Lin)

指導教授 ： 陳文進

共同指導教授 ： 莊永裕(Yung-Yu Chuang)

國立臺灣大學/電機資訊學院/資訊網路與多媒體研究所/碩士(2019年)

https://doi.org/10.6342/NTU201902033

摘要

在本篇論文之中，我們提出一個深度學習模型BEDSR-Net，專門設計為對一般文件影像進行陰影去除。文件通常具有一個共通的全局背景顏色的資訊，因此我們利用深度學習方式使模型學到如何預測整張文件的全局背景顏色資訊。在模型訓練的過程，模型亦同時掌握了文件影像中陰影和非陰影的位置資訊，透過將模型的中間產物特徵圖視覺化以熱度圖方式呈現，此熱度圖可被定位為表達了文件影像中陰影分布的陰影遮罩。透過全局背景顏色以及陰影位置資訊的協助，我們提出的深度學習架構BEDSR-Net將有效對原圖進行陰影去除，且在大部分的評比之中，我們的效果在各方數據均表現優異，整體來說更優於前人的方法。除此之外，BEDSR-Net僅在合成資料集上進行訓練，應用在實際評比用資料集時表現依舊亮眼，這也反映出我們的模型架構對於表現的穩定度上是有明顯的幫助。在本論文中，對於文件影像陰影去除這個任務，我們收集了兩個資料集，分別為合成影像資料集SDSRD以及實際影像資料集DSRD，前者提供了深度學習在這個領域中足夠的訓練資料，並在文件種類和光線複雜度的這兩個面向中達到了足夠的豐富度；後者更涵蓋了大量複雜文件，可作為一個比較模型表現優劣上更泛用的資料集。

關鍵字

陰影去除 ； 文件影像處理 ； 深度學習 ； 條件生成對抗式網路

並列摘要

In this paper, we propose a novel deep neural network architecture, named BEDSR-Net, which is designed to remove shadow from document images. With our observation that documents usually have single global background color, we utilize deep learning technique to detect the color from a document image. While training process, our model is able to understand the shadow distribution in an image, including intensity and location. We further visualize the knowledge about shadow distribution of our model in the form of heatmap. The heatmap is capable of precisely denoting the shadow location. With the assistance of global background color and the heatmap, our model, BEDSR-Net, achieves state-of-the-art in most evaluation comparison with previous works in the field of document images shadow removal. Also, our model, only trained with a synthetic dataset, still outperforms others in real benchmark datasets, which indeed shows our proposed model's stability and robustness. Besides, we collect two datasets in this task, including a synthetic dataset (SDSRD) and a real dataset (DSRD). The former one enables the training process of deep learning approach in this task while the latter one can be served as a much more general benchmark dataset. Both SDSRD and DSRD are aimed at capturing more diverse scenario.

並列關鍵字

Shadow Removal ； Document Image Processing ； Deep Learning ； Conditional Generative Adversarial Network

參考文獻

[1] S. Bako, S. Darabi, E. Shechtman, J. Wang, K. Sunkavalli, and P. Sen. Removing shadows from images of documents. In Asian Conference on Computer Vision, pages 173–183. Springer, 2016.

[2] C. Clausner, A. Antonacopoulos, and S. Pletschacher. Icdar2017 competition on recognition of documents with complex layouts-rdcl2017. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), volume 1, pages 1404–1410. IEEE, 2017.

[3] X. Huang, G. Hua, J. Tumblin, and L. Williams. What characterizes a shadow boundary under the sun and sky? In 2011 International Conference on Computer Vision, pages 898–905. IEEE, 2011.

[4] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125–1134, 2017.

[5] S. Jung, M. A. Hasan, and C. Kim. Water-filling: An efficient algorithm for digitized document shadow removal. In Asian Conference on Computer Vision, pages 398–414. Springer, 2018.