大型語言模型對基於文本評論的推薦模型之影響

Impacts of Large Language Models on Text Review-Based Recommendation Models

陳柏言(Po-Yen Chen)

指導教授 ： 曹承礎

國立臺灣大學/管理學院/資訊管理學系/碩士(2024年)

https://doi.org/10.6342/NTU202402016

摘要

本研究以用戶對於餐廳評分之預測任務為例，提出運用大型語言模型（LLM）基於線上評論生成有用的用戶和餐廳文本資訊，探討所生成的文本資訊對於基於評論之推薦模型的預測表現的影響。本研究設計摘要和推薦提示策略，分別使 GPT-3.5 Turbo 基於評論生成用戶和餐廳的描述與推薦資訊。透過預訓練模型，將文本轉換為模型的輸入特徵向量，並以不同方式將 LLM 所生成的文本資訊整合至模型輸入中進行實驗。挑選常見及相關研究的預測模型作為實驗的推薦模型，並針對各模型以使用評論作為輸入資料時的預測表現作為比較基準線。研究結果顯示，相較於使用評論文本，整合 LLM 基於評論所生成的文本資訊，並針對每個模型採用適合的輸入特徵向量生成方法後，能提升各推薦模型的預測表現。此外，對於擁有特定或足夠評論數量的用戶或餐廳，整合 LLM 基於評論所生成的文本資訊可提升各推薦模型的預測表現。另外，整合 LLM 基於評論所生成的文本資訊可提升各推薦模型對低評分餐廳預測的準確性，使預測結果更接近於用戶實際給出的低評分，較可避免推薦用戶不喜歡的餐廳。

關鍵字

大型語言模型 ； 基於評論之推薦 ； 提示策略 ； 餐廳推薦 ； 評分預測

並列摘要

This study proposes leveraging a large language model (LLM) to generate useful textual information about users and restaurants based on online reviews to explore the impacts of the generated textual information on the predictive performance of review-based recommendation models, using the prediction task of users’ ratings for restaurants as an example. Summarization and recommendation prompt strategies were designed to enable GPT-3.5 Turbo to generate descriptions and recommendation information about users and restaurants based on reviews, respectively. Texts were converted into model input feature vectors using pre-trained models. The LLM-generated textual information was integrated into model input in various ways for experiments. Common and relevant prediction models from previous studies were selected as experimental recommendation models. For each model, the predictive performance when using the reviews as input data served as the comparative baseline. The study results demonstrate that integrating the LLM-generated textual information and applying appropriate input feature vector generation methods for each model can improve the predictive performance of each model. Additionally, integrating the LLM-generated textual information can improve the performance of the recommendation models for users or restaurants with a certain or sufficient number of reviews. Moreover, integrating the LLM-generated textual information can improve the performance of predictions for low-rated restaurants, making the predicted ratings closer to the actual low ratings given by users, thereby better avoiding recommending restaurants that users dislike.

並列關鍵字

Large Language Model ； Review-Based Recommendation ； Prompt Strategy ； Restaurant Recommendation ； Rating Prediction

參考文獻

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., ... & McGrew, B. (2023). GPT-4 technical report. arXiv preprint arXiv:2303.08774.

Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford University Press.

Bisong, E. (2019). The multilayer perceptron (MLP). In Bisong, E (Eds.), Building Machine Learning and Deep Learning Models on Google Cloud Platform (pp. 401-405). Apress.

Breiman, L. (2001). Random forests. Machine Learning, 45, 5-32.

Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems, 30(1-7), 107-117.