影像檢索與分類的視覺字典的研究

The Study of Image Retrieval and Categorization using Visual Vocabulary

郭章明(Chang-Ming Kuo)

指導教授 ： 郭忠民

義守大學/電機資訊學院/資訊工程學系/博士(2017年)

摘要

視覺詞彙的方法已經成功應用於許多的多媒體和視覺應用，包括視覺辨識、影像檢索，場景模組建立/分類。這個想法背後所代表的意義是影像可以經由局部特徵的集合為視覺字，藉由各種視覺字更進一步組成語意物件，讓低階特徵提升至高階語意，以改善意涵鴻溝的問題。在這篇論文中，視覺字擁有色彩、結構與紋理特性，將嘗試三種方法建立以視覺字為主的視覺詞彙：(1)以特徵點為基礎的方法建立視覺詞彙，採用尺度不辨特徵轉換(SIFT)擷取特徵點，並且增加色彩資訊，改善傳統SIFT無色彩資訊，讓視覺字的資訊更加豐富與完整。(2)以區塊為主的方法建立視覺詞彙，將以區塊分割的方式來取得影像內容特徵。(3)結合特徵點與區塊為基礎的方法建立視覺詞彙。
本研究考慮到視覺詞彙的同質性，提出一個新穎的視覺字描述效能的研究，引入了巨觀與微觀的想法，將其導入視覺詞彙內，建立影像描述子，進一步的描述影像內容，並且應用在影像檢索。
在影像分類上，依照巨觀與微觀視覺詞彙的基礎，以及改善特徵模組中巨觀與微觀的權重值，根據視覺字組成的各種語意物件，建立影像分類模型，讓每種分類模型擁有獨特性與唯一性，依照機率分類器分類最佳類別，實驗結果證明巨觀與微觀視覺詞彙在影像檢索與分類皆有良好的結果
。

關鍵字

視覺字 ； 巨觀 ； 微觀 ； 檢索 ； 分類

並列摘要

Visual vocabulary representation approach has been successfully applied to many multimedia and vision applications, including visual recognition, image retrieval, and scene modeling/categorization. The idea behind the visual vocabulary representation is that an image can be represented by visual words, a collection of local features of images. In my dissertation, I will develop a new scheme for the construction of visual vocabulary based on the analysis of visual word contents. By considering the content homogeneity of visual words, the developed visual vocabulary contains macro-sense and micro-sense visual words. The two types of visual words are appropriately further combined to describe an image effectively. For micro-sense visual words, we try to investigate the effective from various viewpoints. Firstly, the SIFT is selected for feature points extraction, and then the color features is designed to build new SIFT-based feature descriptor for improving the conventional methods. Secondly, we also consider the block-based visual words as micro-sense descriptor and compares their advantages. Thirdly, discuss the advantages of macro- and micro-sense visual words based on point or block visual word.
In this work, we will try to construct a new visual vocabulary for the applications of image retrieval and categorization, considering the characteristics of visual words. By taking the inhomogeneous and incomplete content of visual words into account, we design a new visual vocabulary that can describe different semantics in images more effectively. The performance evaluation for the two applications indicates that the proposed visual vocabulary systems achieves promising results.

並列關鍵字

Visual words ； Macro-sense ； Micro-sense ； Retrieval ； Categorization

參考文獻

[5] B.S. Manjunath, J.R. Ohm, V. V. Vasudevan and A. Yamada, “Color and texture descriptors,”proc IEEE Transactions on Circuits and Systems for Video Technology Volume 11 Issue 6, pp.703-715, June 2001.

[3] Y. Deng, B.S. Manjunath, C. Kenney, M.S. Moore, H. Shin, “An efficient color representation for image retrieval.” Proc IEEE Transactions on Image Processing Issue, Vol.10, pp.140-147.2001.

[6] A. Mojsilovic, J. Hu and E.Soljanin, “Extraction of perceptually important colors and similarity measurement for image matching, retrieval, and analysis.” Proc IEEE Transactions on Image Processing Volume.11, Issue.11, pp.1238-1248, December 2002.

[7] A. Mojsilović, J. Kovacević, J. Hu, R,J. Safranek and S.K. Ganapathy, “Matching and retrieval based on the vocabulary and grammar of color patterns.” Proc IEEE Transactions on Image Processing, Volume 9 Issue 1, pp38-54, January 2000.

[11] S. Xu, T. Fang, D. Li and S. Wang, “Object classification of aerial images with bag-of-visual words.” Proc IEEE Geoscience and Remote Sensing Letters Volume 7,Issue 2, pp.366-370, April 2010.