記憶體式資料庫管理系統之流線化請求處理

記憶體式資料庫管理系統(DBMS)儲存資料於記憶體中，並週期性的將資料轉儲至硬碟，以避免電源錯誤所引起的資料損失。由於資料存取速度取決於記憶體的存取；因此，在記憶體式資料庫管理系統的架構下，效能瓶頸轉移到其他以往認為非關鍵的系統元件，包括：請求或回覆封包處理、日誌、資料持久性維護。此論文描述一個基於網路的記憶體式資料庫管理系統的設計、實作與評測。該系統採用數個技術包括：使用Data Plane Development Kit (DPDK)以及mTCP且令使用者層的應用得以存取TCP封包的低負擔網路封包處理引擎、批次轉錄記憶體式記憶體日誌、以及高效資料持久性維護技術──批次處理循序送出(BOSC)技術。為有效地整合以上技術以實現一完整的系統，我們測試數個軟體架構，最終得出一個能在平行化處理請求與執行緒負擔間取得良好平衡的系統。經過建立一個完整可運作的原型系統，並詳細研究其效能，我們得出：該記憶體式資料庫管理系統，在記憶體被完全佔用且同時進行日誌紀錄的情況下，能夠維持每秒近百萬請求的效能。

關鍵字

記憶體式資料處理；物聯網；高效能TCP處理；遠端機器儲存日誌；順序性硬碟存取

並列摘要

An in-memory database management system (DBMS) holds data in memory, and periodically flushes data to disk to protect them from power failures. Because data access is at main memory speed, the performance bottlenecks of an in-memory DBMS now shift to other system components that were previously considered non-critical, such as request/response packet processing, logging, and persistence. In this thesis, we describe the design, implementation and evaluation of a network-based in-memory DBMS that features a low-overhead network packet processing engine using Data Plane Development Kit (DPDK) and mTCP and allowing user-level applica-tions to manipulate TCP payloads, batched replication of in-memory memory logs, and a highly efficient database persistence mechanism called Batch mOdifications with Se-quential Commit (BOSC). To efficiently integrate these mechanisms into a coherent whole, we have experimented with multiple software architectures and eventually arrived at one that strikes a nice balance between request processing parallelism and threading overhead. A detailed performance study on a fully operational prototype shows that the resulting in-memory DBMS is able to sustain nearly one million requests per second when the memory is fully utilized and logging is turned on.

並列關鍵字

in-memory data processing ； internet of things ； efficient TCP ； remote logging ； sequential disk access

參考文獻

[14] D. Narayanan, E. Thereska, A. Donnelly, S. Elnikety, and A. Row-stron. Migrating server storage to ssds: analysis of tradeoffs. In Proceedings of the 4th ACM European conference on Computer systems, pages 145–158. ACM, 2009.

[23] V. Vasudevan, D. G. Andersen, and M. Kaminsky. The case for vos: The vector operating system. In HotOS, 2011.

[9] Sotiris Ioannidis, Evangelos P. Markatos, and Julia Sevaslidou. On using network memory to improve the performance of transaction-based systems. Technical Report 190, Institute of Computer Science-FO. RTH, 1997.

[7] S. Han, K. Jang, K. Park, and S. Moon. Packetshader: a gpu-accelerated software router. ACM SIGCOMM Computer Communication Review, 41(4):195–206, 2011.

[11] E. Jeong, S. Wood, M. Jamshed, H. Jeong, S. Ihm, D. Han, and K. Park. mtcp: a highly scalable user-level tcp stack for multicore systems. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14), pages 489–502, 2014.

國際替代計量

記憶體式資料庫管理系統之流線化請求處理

全文下載

主題瀏覽