使用動態預測決策邊界的集成學習方法解決概念漂移的問題

Predictive Ensemble Learning Based on the Dynamic Predictor for Concept Drift Scenarios

孫念恩(Nien-En Sun)

指導教授 ： 林守德

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2022年)

https://doi.org/10.6342/NTU202202372

摘要

在實務中對資料流進行機器學習模型的訓練與預測時，時常會面臨到資料分布隨著時間而改變的問題，此現象又稱為概念漂移。近年來，深度學習網路已廣泛運用於許多領域，並成為主流。本篇論文首先設計了一個自注意力機制的深度學習網路，並取名為動態預測器。動態預測器透過預測未來資料分布來解決概念漂移問題。基於動態預測器，此篇論文接著提出了兩個集成學習方法DP.FUTURE及DP.ALL來分別解決漸變式的實際概念漂移與真實世界的概念漂移。最後，透過實驗於合成資料集、套用閾值之迴歸資料集及真實世界之資料集，此篇論文所提出的方法比起當前最先進的概念漂移解決方案，達到了更好的預測性能。

關鍵字

概念漂移 ； 深度學習 ； 集成學習 ； 決策邊界 ； 資料流學習

並列摘要

In real-world situations, we often have to handle the problem of the changing data distribution over time, which is also called concept drift. In recent years, neural-network-based methods have become the mainstream in many fields. In this work, we design a self-attention-based network called "dynamic predictor", which can predict the future data distribution to solve concept drift problems. Based on the dynamic predictor, we also propose DP.FUTURE and DP.ALL to handle incremental actual drift and real-world concept drift, respectively. Finally, we conduct experiments on synthetic datasets, regression datasets with thresholds, and real-world datasets. Experiment results show that our proposed methods outperform other SOTAs.

並列關鍵字

Concept Drift ； Deep Learning ； Ensemble Learning ； Decision Boundary ； Data Stream Learning

參考文獻

M. S. Acharya, A. Armaan, and A. S. Antony. A comparison of regression models for prediction of graduate admissions. In 2019 International Conference on Computational Intelligence in Data Science (ICCIDS), pages 1–5, 2019.

Aditya. 100,000 uk used car data set, Jul 2020.

P. Almeida, L. Soares de Oliveira, A. Jr, and R. Sabourin. Adapting the dynamic classifier selection for concept drift scenarios. Expert Systems with Applications, 104:67–85, 08 2018.

K. Bache and M. Lichman. UCI machine learning repository, 2013.

A. Bifet, G. Holmes, R. Kirkby, and B. Pfahringer. Moa: Massive online analysis. J. Mach. Learn. Res., 11:1601–1604, aug 2010.