視訊資料庫中視訊和音訊的知識結構

4D C-String: A New Audio-visual Knowledge Structure and Similarity Retrieval for Video Database Systems

陳亭諭(Ting-Yu Chen)

指導教授 ： 李瑞庭

國立臺灣大學/管理學院/資訊管理學研究所/碩士(2004年)

https://doi.org/10.6342/NTU.2004.02145

摘要

過去李瑞庭博士提出一種稱作3D C-string的知識結構，將視訊中物件之間的空間與時間關係利用字串的方式記錄下來，以便於搜尋、管理與顯現視訊資料庫中的視訊物件，可是由於視訊除了包含視訊資料以外，也包含了豐富的音訊資料，如果能增加音訊方面的知識結構，將會提高搜尋的準確性。因此我們在本篇論文中以3D C-string 的觀念為基礎提出了一種新的視訊和音訊知識結構來進行視訊資料庫中的搜尋，命名為4D C-string。4D C-string的視覺部分，我們利用視訊物件的投射來表示一個視訊中物件間的空間與時間關係，並且記錄物件移動的軌跡與大小變化，從音訊部分我們萃取三種音訊特徵以形成音訊字串，包括安靜/非安靜、音樂/演講、歌手/演講者辨識。我們也定義了相似度的測量並提出了結合音訊和視訊相似度比對的演算法。最後，我們會藉由一些實驗結果驗證我們所提出演算法的效率。

關鍵字

3D C-string ； 4D C-string ； 視訊資料庫 ； 知識結構

並列摘要

This paper presents a new audio-visual knowledge structure and similarity for video database systems, called 4D C-string. It is based on the 3D C-string, which is a knowledge structure that can express visual characteristic of objects in a video but it does not consider the audio part of videos. So we add audio dimension on it to make the retrieval results more precise. For the visual part, we can generate strings to represent the spatial and temporal relations between the objects in a video and their motions and size changes. For the audio part, we can generate three audio strings. Then we propose the similarity retrieval algorithm based on the visual and audio information to retrieve the similar videos from the database for a given query video. Our proposed method this approach can provide user an easy and efficient way to retrieve, visualize and manipulate video and audio objects in video database systems.

並列關鍵字

Knowledge structure ； Video database ； 3D C-string ； 4D C-string

參考文獻

[5] S.K. Chang, Q.Y. Shi and C.W. Yan, Iconic indexing by 2D strings, IEEE Trans. On Pattern Analysis and Machine Intelligence, vol. 9, 1987, pp. 413-429.

[6] S.K. Chang, E. Jungert and Y. Li, Representation and retrieval of symbolic pictures using generalized 2D strings, Technical Report, University of Pittsburgh, 1988.

[7] P.W. Huang and Y.R. Jean, Using 2D C+-string as spatial knowledge representation for image database systems, Pattern Recognition, vol. 27, 1994, pp. 1249-1257.

[8] P.W. Huang and Y.R. Jean, Spatial reasoning and similarity retrieval for image database systems based on RS-strings, Pattern Recognition, vol. 29, 1996, pp.2103-2114.

[9] S.Y. Lee and F.J. Hsu, 2D C-string: a new spatial knowledge representation for image database system, Pattern Recognition, vol. 23, 1990, pp. 1077-1087.