- 學位論文
基於音頻和視覺特徵的多模式相似影片檢測與定位
Multimodal Video Similarity Detection and Localization Using Audio-Visual Features
摘要
在數位時代，隨著YouTube等平台上的影片內容激增，高效的影片相似性定位（VSL）系統的需求變得至關重要。VSL以其能夠在時間上對齊相似的影片片段的能力，應對了龐大的影片數據量和不斷增加侵權情況所帶來的挑戰。當前的VSL方法通常專注於視覺或聽覺特徵之一，但很少充分發揮兩者之間的協同作用。本研究設計了一個多模式影片相似性定位（MVSL）管道，無縫集成了音頻和視覺特徵，以增強相似性定位任務的性能。本研究引入了一個新型的深度學習管道，並使用一個特別精選的影片相似性定位數據集（VSLD），透過VSLD強調了該方法的效能。MVSL系統從影片預處理、音視頻 特徵提取開始，然後進行相似性映射和時間對齊。其中音頻特徵在VSL中發揮了決定性作用。這項工作為下一代影片內容分析工具奠定了基礎，實現了自動化、可擴展和精確的影片相似性檢測，對於內容創作者和數位媒體平台具有巨大價值。實驗結果與其他研究相比也展示了出色的性能。
並列摘要
In the digital age, with the upsurge in video content on platforms like YouTube, the need for efficient video similarity localization (VSL) systems has become paramount. VSL, with its ability to temporally align similar video segments, addresses challenges posed by the sheer volume of video data and the increasing instances of copyright infringement. Current VSL methods often focus on either visual or auditory features but rarely capitalize on the synergy between the two. This research introduces a Multimodal Video Similarity Localization (MVSL) pipeline that seamlessly integrates audio and visual features to enhance the performance of similarity localization tasks. The study introduces a novel deep learning pipeline, a specially curated Video Similarity Localization Dataset (VSLD), and underscores the efficiency of the approach using the VSLD. The MVSL system begins with video preprocessing, audio-visual feature extraction, and then progresses to similarity mapping and temporal alignment. The results showcase outstanding performance, with auditory features playing a decisive role in VSL. This work lays a foundation for the next generation of video content analysis tools, enabling automated, scalable, and precise video similarity detection that holds immense value for content creators and digital media platforms.