以可組合系統建置的虛擬化資料中心之容量規劃與網路傳輸最佳化

Capacity Planning and Goodput Optimization for Virtualized Data Centers with Composable Systems

林安笛(An-Dee Lin)

指導教授 ： 廖婉君

國立臺灣大學/電機資訊學院/電信工程學研究所/博士(2018年)

https://doi.org/10.6342/NTU201803814

摘要

近年來的研究顯示，執行用戶的應用程式所需之運算資源有逐漸和資料中心能提供的硬體資源有不平衡的現象。當運算資源的供給量和需求量沒有匹配時，會導致運算工作無法有效率地映射到傳統資料中心內，固定大小的伺服器來執行。這當中所產生的資源空洞不但會降低硬體利用率，也會妨害資料中心乘載大型運算工作的能力。這個缺點啟發了一種新型態，以機櫃為單位的電腦架構，稱作「可組合系統(composable system)」。這個系統把原本在主機板上，固定大小的運算資源轉換成動態的運算平台。更精確地來說，這個新型態的電腦架構的目標是將各張傳統主機板上的運算資源，例如中央處理器、記憶體、硬碟儲存空間、以及專用指令集處理器等，全部用網路連結再一起。
這個架構形成了一台邏輯上的巨大電腦，並藉由打破各張傳統主機板上的資源界線，使這個系統將能乘載資源需求更多樣化的運算工作。
本項研究可分為三個部分：第一部份我們介紹這個可組合式的架構，並用它來建構雲端資料中心。我們推導運算資源使用量的數學模型，並藉由模擬幾種資料中心內常見的運算工作做驗證。
從模擬的結果可發現，可組合系統比傳統系統能乘載1.6倍的運算工作量，並且它對於運算工作需求的數值分布不敏感。這顯示出可組合是系統確實能支援資料中心的服務。
在接下來的部分，我們將考慮的系統架構從單一資料中心延伸到由多個資料中心所組成的網路。其中每座資料中心都在地理位置上散布在服務區內。一個運算工作有可能為了追求低延遲的傳輸而搬遷到離租用戶最近的資料中心內，並根據用戶的移動方式，這個運算工作在執行時可能被搬移數次。在此我們建構了一個雙層次的模型來推估整體的運算資源用量。我們將用戶的移動模式轉化成對各做資料中心的有效抵達速率。在柏松分布的抵達速率，以及以機率性的移動方式假設下，每座資料中心能利用上一章所推導的模型，平行地計算資源用量。
在最後一部分，我們把目光移到運算工作之間，彼此的通訊管道。一個整合式的服務可能由複數個運算工作聯合組成，每個工作都運作在獨立的虛擬機器內。這些虛擬機器組成的邏輯網路在實際上是散佈在由各個資料中心所組成的實體網路上。然而，傳統實作虛擬網路的協定可能無法直接套用至資料中心內的網路拓樸。最近的研究顯示將網路第二層封裝到第三層的隧道協定是可行的解。但是，我們在實際量測後發現，直接使用隧道協定的效能不佳。更精確地說，伺服器內存在瓶頸。因此，我們提出了能分散中央處理器的機制，這個機制能引導封包到可用的中央處理器上做後續的封包處理，所以能大量的提升網路效能。在以VXLAN所建構的虛擬網路中，我們的方案能在10Gb/s的線路上提升300%的隧道頻寬。

關鍵字

效能分析 ； 可組合式的資料中心 ； 資料中心網路 ； 隧道協議 ； 中央處理器的分攤工作

並列摘要

Recent research trends exhibit a growing imbalance between the demands of tenants’ software applications and the provisioning of hardware resources. Misalignment of demand and supply gradually hinders workloads from being efficiently mapped to fixed-sized server nodes in traditional data centers. The incurred resource holes not only lower infrastructure utilization but also cripple the capability of a data center for hosting large-sized workloads. This deficiency motivates the development of a new rack-wide architecture referred to as the composable system. The composable system transforms traditional server racks of static capacity into a dynamic compute platform. Specifically, this novel architecture aims to link up all compute components that are traditionally distributed on traditional server boards, such as central processing unit (CPU), random access memory (RAM), storage devices, and other application-specific processors. By doing so, a logically giant compute platform is created and this platform is more resistant against the variety of workload demands by breaking the resource boundaries among traditional server boards.
This research is divided into three parts. In the first part, we introduce the concepts of this reconfigurable architecture and design a framework of the composable system for cloud data centers. We then develop mathematical models to describe the resource usage patterns on this platform and enumerate some types of workloads that commonly appear in data centers. From the simulations, we show that the composable system sustains nearly up to 1.6 times stronger workload intensity than that of traditional systems and it is insensitive to the distribution of workload demands. This demonstrates that this composable system is indeed an effective solution to support cloud data center services.
In the next part, we extend the framework from a single data center into a network of data centers, where each of them is geographic distributed in the serving area. A workload may need to migrate to the data center that close to its tenants for lower transmission delay. The migration may happen multiple times during its runtime, conditioned on the mobility of its tenants. We develop a two-tier model to tell the overall resource usage patterns. The mobility patterns of tenants are transformed into the effective arrival rates to each data center. Under the conditions of the Poisson arrivals of incoming workloads and probabilistic mobility patterns, the resource usage patterns of each data center can be calculated in parallel.
In the last part, we turn our viewpoint to the communication links between workloads. An integrated application may consist of multiple workloads which run inside dedicated VMs. These VMs form a logical network which is physically distributed in the network of data centers. However, traditional protocols used in local area networks may not be applicable for data center networks due to the difference in network topology. Recent research suggests that layer-2-in-layer-3 tunneling protocols may be the solution to address the challenges. We find via testbed experiments that directly applying these tunneling protocols toward network virtualization only results in poor performance due to the scalability problems. Specifically, we observe that the bottlenecks actually reside inside the servers. We then propose a CPU offloading mechanism that exploits a packet steering function to balance packet processing among available CPU threads, thus greatly improving network performance. Compared to a virtualized network created based on VXLAN, our scheme improves the bandwidth for up to almost 300 percent on a 10 Gb/s link between a pair of tunnel endpoints.

並列關鍵字

performance analysis ； composable data center ； network of data centers ； tunneling protocol ； CPU offloading

參考文獻

[1] L.A. Barroso, J. Clidaras, and U. Hölzle, 2nd ed., The Data Center as a Computer: An Introduction to the Design of Warehouse-Scale Machines, Morgan & Claypool Publishers, 2013.

[2] C.-S. Li, B. L. Brech, S. Crowder, D.M. Dias, H. Franke, M. Hogstrom, D. Lindquist, G. Pacifici, S. Pappe, B. Rajaraman, J. Rao, R.P. Ratnaparkhi, R.A. Smith, and M.D. Williams, “Software Defined Environments: An Introduction,” IBM J. Res. Dev., vol. 58, no. 2/3, Mar./May 2014, pp. 1-11.

[3] C.-S. Li, H. Franke, C. Parris, B. Abali, M. Kesavan, and V. Chang, “Composable Architecture for Rack Scale Big Data Computing,” Futur. Gener. Comp. Syst., vol. 67, Elsevier, Feb. 2017, pp. 180-193.

[4] J. Hamilton, “Internet-Scale Datacenter Economics: Where the Costs & Opportunities Lie,” Proc. 14th Int’l Workshop on High Performance Transaction Systems (HPTS), 2011.

[5] H. Herodotou, F. Dong, and S. Babu, “No One (Cluster) Size Fits All: Automatic Cluster Sizing for Data-intensive Analytics,” Proc. 2nd ACM Symp. Cloud Comput. (SoCC '11), 2011, article no. 18.