- 學位論文
增量模型於不均衡實驗組之表現
Uplift Model Performance With Imbalanced Treatment Groups
摘要
有鑑於企業資源有限，辨識出可被轉換的顧客非常重要，且降低過度行銷也能減少顧客的困擾。 增量模型(uplift model) 可將客戶分成四大群：轉換者(Persuadable)、堅決購買者(Sure Thing)、無動於衷者(Lost Causes)和請勿打擾者(Do-Not-Disturb)。增量模型透過隨機實驗（A/B測試）、結合預測模型識別最有可能被轉換的「轉換者」以達到精準行銷(Radcliffe 2007)。研究人員需進行A/B測試、記錄顧客反應，透過預測模型獲得增量分數(uplift score) 並將顧客分成四群。在文獻回顧當中有提到模型的預測表現在很大程度上取決於資料本身的性質和應用。此外，實驗當中需要相同大小的實驗組和對照組(Devriendt et al. 2018)，然而在實務上往往出現不均衡的實驗與控制組，以便給更多的客戶「良好」的刺激物(treatment)、特別是VIP等級的客戶，因此有些論文也在未來研究的部分提到資料分配(distribution)不均衡的問題(Radcliffe 2007; Devriendt et al. 2018; Diemert et al. 2018)。 在本篇研究中，我們研究了增量模型於不均衡實驗組和對照組所引起的問題，特別是當實驗組有更多回應者(responders)的情況下、更貼近實務上的操作。我們將使用真實資料並根據（1）實驗和對照組比率（2）實驗組和對照組中的回應率來操縱它們，並使用增量圖(uplift chart) 和Qini曲線作為評估指標(Radcliffe and Surry 2011)。本研究旨在回答以下問題： (a) 在實驗組中分配更多Ypre = 1是否會惡化/改善模型的表現？ (b) 是否存在其他也會影響模型表現的因素？ (c) 在模型中納入觀測目標於實驗前的回應（pre-treatment outcome, Ypre）是否會提高預測表現？
並列摘要
In applications such as direct (precision) marketing, there are limited resources and therefore it is important to identify the customers most likely to convert if targeted. Such accurate targeting can also reduce the burden on customers, by reducing the amount of irrelevant solicitations. When using a predictive model to contact customers, there are four types of outcomes: Persuadable, Sure Thing, Lost Cause, Do-Not-Disturb. Uplift modeling is used for figuring out the Persuadables, those customers who are most likely to react positively because of targeting (Radcliffe 2007). Uplift modeling uses a randomized experiment (A/B test), and combines it with a predictive model to find out the Persuadables. Researchers need to conduct an A/B test and record the customers’ reactions, by doing so, they can obtain the uplift scores by applying the predictive model and classify customers into the 4 different segments. Previous work have proven the performance of uplift heavily depends on the nature of the data sets and the application. In addition, equal treatment and control group sizes are required (Devriendt et al. 2018), and several papers mention the problem of imbalanced distribution as the future works (Radcliffe 2007; Devriendt et al. 2018; Diemert et al. 2018). However, in practice, companies often prefer unequal treatment and control groups, to give more customers the “good” treatment. Moreover, they want to give their VIP customers more chance to get a “good” treatment. In this research we study the problems arising from imbalanced treatment and control groups when using uplift modeling, and specifically, when the company assigns more responders to the treatment group than to that of control group. We will use real datasets and manipulate them in terms of (1) treatment-control ratio, and (2) the rate of responders in the treatment group and in the control group. We will use uplift chart and Qini curve as the evaluation metrics (Radcliffe and Surry 2011), used in uplift. This research aims to answer the following questions: (a) Does allocating more Ypre= 1 into treatment deteriorate/improve uplift modelling? (b) What factors influence the performance? (c) Does including pre-treatment response (Ypre) in the model improve performance?