反事實學習於廣告點擊預測之應用

Counterfactual Learning for Ad Click Prediction

原博文(Bowen Yuan)

指導教授 ： 林智仁

國立臺灣大學/電機資訊學院/資訊工程學研究所/博士(2021年)

https://doi.org/10.6342/NTU202104473

摘要

點擊率預測在廣告系統中扮演著重要的角色。目前，廣泛使用的方法都是將點擊率預測建模為二元分類問題。具體來說，就是在所有已投放的廣告樣本中，把有點擊的廣告樣本和沒有點擊的廣告樣本分別當成在訓練和離線驗證中的正樣本和負樣本。然而，我們指出常見的廣告系統投放廣告的方式使得已投放的廣告樣本中包含一種選擇偏誤。由於這個偏誤的存在，傳統的二元分類模型會給出不準確的點擊率預測從而產生收入的損失。

本文旨在探究如何使用反事實學習方法消除各類廣告系統中的選擇偏誤問題。在本文的第一部分，我們回顧了一些通用的反事實學習方法，但也同時指出使用這些通用方法在廣告點擊率預測的困難。為了克服這些困難，我們為廣告點擊率預測提出了一種新的反事實框架。通過這個框架，我們可以得到比目前其他最先進方法更好的點擊率預測的結果。在本文的第二部分，我們進一步探究了一類更複雜的廣告系統。在這類廣告系統中，由於相同的廣告放在不同的廣告位置將會有不同的點擊率， 位置信息成為了點擊率預測中必不可少的因素。然而，對於這類系統，目前廣泛使用方法依然只考慮從已投放的廣告中學習預測模型，因此選擇偏誤在這類廣告系統中依然存在。並且，由於位置信息參入，這類系統中同時存在著兩種不一樣的選擇偏誤。為了消除掉這兩類的選擇偏誤，我們進一步改造了反事實學習的框架並且通過實驗驗證了這個框架的有效性。究了一類更複雜的廣告系統。在這類廣告系統中，由於相同的廣告放在不同的廣告位置將會有不同的點擊率， 位置信息成為了點擊率預測中必不可少的因素。然而，對於這類系統，目前廣泛使用方法依然只考慮從已投放的廣告中學習預測模型，因此選擇偏誤在這類廣告系統中依然存在。並且，由於位置信息參入，這類系統中同時存在著兩種不一樣的選擇偏誤。為了消除掉這兩類的選擇偏誤，我們進一步改造了反事實學習的框架並且通過實驗驗證了這個框架的有效性。

關鍵字

點擊率預測 ； 推薦系統 ； 選擇偏誤 ； 反事實學習

並列摘要

Click-through rate (CTR) prediction is the core problem of building advertising systems. Most existing state-of-the-art approaches model CTR prediction as binary classification problems, where displayed events with and without click feedbacks are respectively considered as positive and negative instances for training and offline validation.
However, due to the selection mechanism applied in most advertising systems, a selection bias exists in displayed events. Conventional CTR models ignoring the bias may have inaccurate predictions and cause a loss of the revenue.
This thesis aims to address the selection bias issue for different types of advertising systems. The main idea is to conduct counterfactual learning
by considering not only displayed events but also non-displayed events.
In this first part of this thesis, through a review of existing approaches of counterfactual learning, we point out some difficulties for applying these approaches for CTR prediction in a real-world advertising system. To overcome these difficulties, we propose a novel framework for counterfactual CTR prediction. In experiments, we compare our proposed framework against state-of-the-art conventional CTR models and existing counterfactual learning approaches. Experimental results show significant improvements.

In the second part of this thesis, we extend the investigation to position-aware systems, where an ad placed in various positions has different click probabilities, so the position information should be considered in both training and prediction.
For such position-aware systems, existing approaches learn CTR models from clicks/not-clicks on historically displayed events by leveraging the position information in different ways.
We explain that these approaches may give a heavily biased model. We first point out that in position-aware systems, two different types of selection biases coexist in displayed events. Secondly, we explain that some approaches attempting to eliminate the position effect from clicks/not-clicks may possess an additional bias.
Finally, to obtain an unbiased CTR model for position-aware systems, we propose a novel counterfactual learning framework. Experiments confirm both our analysis on selection biases and the effectiveness of our proposed counterfactual learning framework.

並列關鍵字

CTR prediction ； Recommender system ； Selection bias ； Counterfactual Learning

參考文獻

I. Bayer, X. He, B. Kanagal, and S. Rendle. A generic coordinate descent framework for learning from implicit feedback. In Proceedings of the 26th International Conference on World Wide Web, pages 1341–1350, 2017.

M. Blondel, M. Ishihata, A. Fujino, and N. Ueda. Polynomial networks and factorization machines: new insights and efficient training algorithms. In Proceedings of the 33rd International Conference on Machine Learning (ICML), 2016.

S. Bonner and F. Vasile. Causal embeddings for recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems, pages 104–112, 2018.

L. Bottou, J. Peters, J. Quiñonero-Candela, D. X. Charles, D. M. Chickering, E. Portugaly, D. Ray, P. Simard, and E. Snelson. Counterfactual reasoning and learning systems: The example of computational advertising. The Journal of Machine Learning Research, 14: 3207–3260, 2013.

O. Chapelle, E. Manavoglu, and R. Rosales. Simple and scalable response prediction for display advertising. ACM Transactions on Intelligent Systems and Technology, 5(4): 61:1–61:34, 2015.