應用熵與主成份分析法於網路流量異常分析之研究

A Study of Applying Entropy and Principal Componet Analysis for Networking Traffic Anomaly Analysis

卓函穎(CHO-HAN YIN)

指導教授 ： 賴裕昆

中原大學/工學院/電機工程研究所/碩士(2012年)

https://doi.org/10.6840/cycu201200914

摘要

網路系統有許多原因會造成網路異常，網路管理人員可以藉由分析網路流量(Network Flow)的相關資訊，找出造成異常流量的可能原因。諸如，快閃、壅塞、設備毀損、斷線、蠕蟲以及分散式阻斷服務(DDOS)等攻擊。透過分析封包流資訊，可以即時的找出異常，快速的解決那些異常在網路上造成的問題。但現今網路傳輸的速度已有10Gbps、40Gbps，甚至可達到100Gbps的高速傳輸。若要在高速網路下分析數量龐大且高維度的資料，是一項具有挑戰性的任務。

本文探討即時縮減高維度流量資料之川流演算法，並且使用熵(Entropy)與主成份分析(Principal Component Analysis, PCA)，快速的進行流量分析，針對特定標頭資訊分類出的封包流計算熵值。藉由觀察熵值的變化，可反應出封包流出現次數的分散程度，可達成找出流量變異的目的。將熵值進行主成份分析可觀察到異常流量發生時，主成份分數及因子負荷的變化，進而可分辨出異常的類型。論文中探討了三項有關於川流式估計熵的演算法，估計熵的演算法可以對不間斷的網路封包流做一次性的統計處理，不須儲存再重複讀取分析封包資訊，使分析速度更快且節省儲存的空間，達到即時分析的目標。本論文使用估計熵演算法與主成份分析，實現一個即時網路流量分析的方法。我們使用真實含有異常的網路流量檔，做實際的模擬及分析，達到即時的偵測異常並分辨種類，也達到節省空間的目標。

關鍵字

網路流量分析 ； 熵 ； 主成份分析

並列摘要

It's a challenge task to analyze networking traffic and identify anomalies in real-time at wire-speed. In this thesis, we propose a framework with limited memory space to perform network traffic analysis at the core networks. The design is based on sketch algorithm implemented in hardware to digest the highly dimensional traffic information at wire-speed. The compact data structure is then feedback to the system to conduct analysis with Entropy and Principal Component Analysis (PCA) in a streaming fashion. Based on several real-world traffic traces, simulations are performed to reveal the effectiveness of this framework. We also provide in-depth discussions for the system design with tradeoffs on accuracy, time and space.

並列關鍵字

Network traffic analysis ； PCA ； Entropy

參考文獻

[1] A. Lakhina, M. Crovella, and C. Diot. Diagnosing network-wide traffic anomalies. In ACM SIGCOMM Computer Communication Review, volume 34, page 219–230, 2004.

[2] A. Wagner and B. Plattner. Entropy based worm and anomaly detection in fast IP networks. 2005.

[7] D. Brauckhoff, K. Salamatian, and M. May. Applying PCA for traffic anomaly detection: Problems and solutions. In INFOCOM 2009, IEEE, page 2866–2870, 2009.

[9] C. Issariyapat and K. Fukuda. Anomaly detection in IP networks with principal component analysis. In Communications and Information Technology, 2009. ISCIT 2009. 9th International Symposium on, page 1229–1234, 2009.

[12] Y. Liu, L. Zhang, and Y. Guan. Sketch-based streaming PCA algorithm for network-wide traffic anomaly detection. In 2010 International Conference on Distributed Computing Systems, page 807–816, 2010.