雲端資源於成本及效能要求下之排程技術

Scheduling Techniques on Cloud Resources Under Cost and Performance Requirements

肯莫茲(WendKuuni-Moise Convolbo)

指導教授 ： 鍾葉青

國立清華大學/電機資訊學院/資訊系統與應用研究所/博士(2017年)

摘要

基於運算可以被交付到網際網路並可按現行方式收費這樣的理念，雲端運算已經成為了一個突出的運算典範。
藉由虛擬化技術，雲端給予我們一種錯覺，彷彿在不同組態和成本下也有取之不盡的資源可以使用。正因如此，雲端上的資源管理就成為相當關鍵的議題。
雲端設施的效率相當倚賴虛擬機器(Virtual Machine)如何配置給應用程式，
以及虛擬機器如何映射到實體機器(Physical Machine)上。由於採用不同的策略進行資源管控對於用戶任務的效能、成本以及資源的利用都有極大程度的影響。
很快地，用戶對於有效率的排程要求包含了成本意識以及服務層級協議(Service Level Agreement)的滿足。然而，雲端運算中的成本模型相當複雜，牽涉了資源本身的能力、租賃時間以及資源獲取的模式。
除此之外，近來大數據的出現也促進了大規模數據分析的發展，這些應用通常跨越了地理上分散的數據中心並且具有廣泛地處理要求。在這樣的情形下，雲端用戶經常提起的問題是：如何找到最有效益的方式來使用運算資源並且保證目標函式對於工作負載的執行。有鑑於此，我們考慮的問題是設計一個關於資源排程的技術，這個技術是在效能限制下對於執行成本的最小化。
在本篇論文當中，我們將針對如何有效管理資源以及規劃應用程式的執行， 以期最小化整體計算成本並且能保證滿足效能需求，並且提出新穎的排程技術和演算法。
本篇論文的主要目的是在雲端計算的範疇內提出一具成本意識的排程策略，這樣的策略可以涵蓋多種不同的應用程式，其中包括了高效計算、資料分析以及平行的批次作業處理。
至此，我們的方法是去探索資源的類型，而這包括了利用拍賣資源來降低在特定用戶限制下的執行成本。除此之外，我們也調查了地理分佈式數據中心的相關問題。
我們提出的策略所作出的貢獻可分成三個方面:
清楚地了解雲端資源管理在成本和效能之間的權衡、利用資源租賃模式來使用基於拍賣的雲端資源，最後，我們對比了單一數據中心的排程和最近表現出不同排程機制的地理分佈式要求。

關鍵字

雲端 ； 工作排程 ； 啟發式 ； 虛擬機器 ； 批次處理任務 ； 虛擬叢集 ； 部署 ； 競價型執行個體 ； DAG

並列摘要

Cloud computing has emerged to become a prominent computing paradigm based on the idea
that computation can be delivered over the Internet and be charged at an as-you-go
basis. Through virtualization techniques, the Cloud offers an illusion of limitless
resources with different configurations and costs. As a result, managing cloud resources
has become a critical issue. The efficiency of the whole cloud facilities strongly relies
on how the Virtual Machines (VM) are allocated to the applications and how VMs are mapped
to the Physical Machine (PM). Different resources management strategies can largely affect
the performance of the user's job, the cost, and the resource utilization. Hence,
efficient job scheduling in the user perspective has swift to include cost-awareness and
the satisfaction of the Service Level Agreement (SLA). However, the cost in cloud computing
is a complex model in which involve the resource capacity, the leasing time and the
resource acquisition mode. In addition, the recent advent of Big Data has contributed
to the development of large scale data analytic applications which often span
geographically dispersed data centers and have a wide range of processing requirements.
A problem usually raised by cloud users in this situation, is to find the most cost
effective computing resources to guarantee the objective functions of their workloads
execution. Hence, we consider the problem of designing resource scheduling techniques
to minimize the execution costs under performance constraints.
In this thesis, we present novel scheduling techniques and algorithms to efficiently
manage the resource and plan the execution of application jobs so as to minimize the
overall computation cost and guarantee the performance requirement. The main objective
of this thesis is therefore to provide cost-aware scheduling strategies in cloud
computing for various types of applications including High Performance Computing,
data analytics and Parallel batch jobs. To this end, our approach is to explore the
resource types including auction based resources to leverage the execution cost under
specified user constrains. In addition, we investigate the scheduling problem in
geo-distributed data centers. Contributions in our strategies are three folds:
Ensure a clear understanding of the tradeoff between cost and performance in
Cloud resource management. Exploit the resource leasing model to leverage the
auction-based cloud resources. Finally, we show the contrast with single data center
scheduling with the recent geo-distributed requirement which exhibits different
scheduling mechanisms.

並列關鍵字

cloud ； job scheduling ； heuristic ； virtualmachines ； batchjobs ； virtualcluster ； migration ； spotinstance ； DAG

參考文獻

88. H. Topcuouglu, S. Hariri, and M.-y. Wu, “Performance-effective and lowcomplexity

jobs on cluster of virtual machines,” in 2011 IEEE International Symposium on

efficient cluster scheduling,” in 2011 IEEE International Conference on Cluster

instances,” in 2013 13th IEEE/ACM International Symposium on Cluster, Cloud,

on Networked Systems Design and Implementation (NSDI 16), (Santa Clara, CA),