照片分享社交網絡中烘焙食品照片推薦

Photo Recommendations of Bakery Products on Photo Sharing Social Networks

薛宜軒(Yi-Hsuan Hsueh)

指導教授 ： 李瑞庭

國立臺灣大學/管理學院/資訊管理學研究所/碩士(2020年)

https://doi.org/10.6342/NTU202000451

摘要

近年來照片分享社群網絡越來越受歡迎。在 Instagram 社群網絡平台上，烘焙食品的照片相當熱門。而一個好的推薦系統可以提升使用者滿意度，並為社群網絡平台帶來商業價值。因此，我們提出一個架構，在照片分享社群網絡中推薦烘焙食品的照片給使用者。我們的架構包含四個階段。首先，我們提出一個 Text-CNN 的模型從文字敘述中擷取出文字特徵，並微調 ResNet-152 的模型從照片中取出視覺特徵。第二階段，我們取出社群特徵，包含每個使用者的活躍度與追蹤關係，以及每張照片的認可度。第三階段，我們提出一個深度學習的方法，結合文字、視覺特徵以及注意力機制來學習使用者對照片的喜好。最後，我們設計一個方法計算每張照片的推薦分數，並將最高分的 k 張照片推薦給使用者。實驗結果顯示，我們的方法有較好的推薦表現且能降低冷啟動的問題。本研究可幫助烘焙業者更有效地接觸潛在客戶，以及擬定更好目標式廣告策略。

關鍵字

照片推薦 ； 照片分享社群網絡 ； 烘焙食品 ； 卷積類神經網路 ； 深度殘差網路-152 ； 注意力機制

並列摘要

Photo sharing social networks have become more and more popular in recent years. Photos of bakery products are very popular on Instagram. A good photo recommender system can increase user satisfaction and bring business value for social network platforms. Therefore, in this study, we propose a framework for recommending photos of bakery products to users on a photo sharing social network. The proposed framework contains four phases. First, we build a model, called Text-CNN, to extract textual features from text descriptions of photos. Also, we fine-tune the ResNet-152 model to extract visual features from photos. Second, we retrieve social features, including activeness and following feature vectors for each user, and recognition feature vector for each photo. Third, we propose a deep learning approach incorporating textual, visual features and attention mechanism to learn users’ preference to photos. Finally, we design a method to compute the recommendation score for each photo, and recommend top-k photos with highest scores to users. The experiment results show that our proposed framework has better recommendation performance and reduces cold start problems. Our study can help bakery manufacturers better reach potential customers and implement effective targeted advertising strategies.

並列關鍵字

photo recommendation ； photo sharing social network ； bakery product ； convolutional neural network ； ResNet-152 ； attention mechanism

參考文獻

[1] J. C. Bezdek, R. Ehrlich and W. Full, FCM: The fuzzy c-means clustering algorithm, Computers & Geosciences, Vol. 10, Issue 2-3, pages 191-203, 1984.

[2] R. Burke, Hybrid recommender systems: survey and experiments, User Modeling and User-Adapted Interaction, Vol. 12, Issue 4, pages 331-370, 2002.

[3] T. Chen, X. He and M. Kan, Context-aware image tweet modeling and recommendation, Proceedings of the 24th ACM International Conference on Multimedia, pages 1018-1027, 2016.

[4] J. Chen, H. Zhang, X. He, L. Nie, W. Liu and T. Chua, Attentive collaborative filtering: Multimedia recommendation with item- and component-level attention, Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 335-344, 2017.

[5] F. Gelli, T. Uricchio, X. He, A. D. Bimbo and T. Chua, Beyond the product: Discovering image posts for brands in social media, Proceedings of the 26th ACM International Conference on Multimedia, pages 465-473, 2018.