融合社群媒體上文字、影像及圖片屬性的使用者興趣探勘技術

Mining User Interests from Social Media: Fusion of Textual and Visual Features

趙芳譽(Chao, Fang Yu)

指導教授 ： 林嘉文

國立清華大學/電機資訊學院/電機工程學系/碩士(2015年)

摘要

在這篇論文中，我們提出了一種結合使用文字、影像特徵對於用戶生成的社交媒體內容，來找到使用者興趣分佈的方法，該興趣分佈可用於對用戶做個人化的廣告推薦，或是社群內容的推薦。
這篇論文的架構包含了三個步驟，分別是特徵提取、模型訓練以及使用者興趣發現。本研究選取Pinterest 上受歡迎用戶組織良好的板作為訓練和測試資料，總共有Pinterest 推薦的三個受歡迎的主要主題，及十二個較精細的主題。對於每個釘我們提取字詞-文件矩陣作為文字特徵、視覺詞袋模型作為低階視覺特徵、並以圖片屬性作為中階視覺特徵，來減少文字敘述和低階視覺特徵間的語意落差。
在特徵擷取完後，我們進行一個單辭選擇的處理以過濾主題分佈不夠明確的字詞。接著，三類特徵以新的字詞-文件矩陣使用DLDA (discriminative latent Dirichlet allocation) 模型進行主題模型的訓練。最後，我們使用一個選擇代表分佈的方法來決定每個輸入文檔最後的主題分佈。在預測階段，我們使用額外的受歡迎使用者的釘來測試分類精確度，並使用額外的普通用戶的釘來測試推薦系統的實作效果。
實驗結果顯示該方法的成效改善，並且圖片推薦示範核實了該方法應用在真實數據下的可行性。

關鍵字

主題模型 ； 興趣發現 ； 社群網站分析

並列摘要

This thesis proposes a framework that jointly uses textual and visual features of user generated social media data for mining the distribution of user interests. The mined distribution can serve for personalized ads recommendation or social content recommendation. The proposed framework consists of three steps: feature extraction, model training, and user interest mining.We choose boards from popular users on Pinterest to collect training and test data. For each pin we extract the term-document matrices as textual features, bag of visual words (BoVW) as low-level visual features, and attributes
as mid-level visual features to bridge the semantic gap between low-level visual feature and textual descriptions. After feature extraction, a word selection process is applied to filter out words with an ambiguous distribution. The new term-document matrices of three
types of features are then used to train topic models using discriminative latent Dirichlet allocation (DLDA). Finally, a representative distribution selection method is performed to choose the final topic distribution of each input document.
In the prediction phase, pins from other popular user are used to evaluate the classification accuracy and pins from other common users are used to evaluate the
recommendation performance. Our experimental results shows the efficacy of the proposed method. Also, the image recommendation demonstration verifies the feasibility of our method applied on real data.

並列關鍵字

Topic model ； Social media analysis ； Interest mining

參考文獻

[27] Harel, Jonathan, Christof Koch, and Pietro Perona, “Graph-based visual saliency,” Advances in neural information processing systems (2006).

[9] Blei, David M., Andrew Y. Ng, and Michael I. Jordan, “Latent dirichlet allocation,” the Journal of machine Learning research 3 (2003): 993-1022.

[7] Julian J. McAuley, Christopher Targett, Qinfeng Shi, Anton van den Hengel, “Image-based recommendations on styles and substitutes,” SIGIR (2015): 43-52.

[15] Xikui Wang, Yang Liu, Donghui Wang, Fei Wu, “Cross-media topic mining on Wikipedia,” ACM Multimedia (2013): 689-692.

[1] Ting Yao, Tao Mei, Chong-Wah Ngo, Shipeng Li, “Annotation for free: video tagging by mining user search behavior,” ACM Multimedia (2013): 977-986.