基於MapReduce運算框架之智慧城市感測資訊處理平台實作

Implementation of a MapReduce-based Sensor Data Processing Platform for Intelligent Cities

李佳蓁(Chia-Chen Li)

指導教授 ： 林其誼

淡江大學/工學院/資訊工程學系碩士班/碩士(2013年)

https://doi.org/10.6846/TKU.2013.00252

摘要

鑒於現代城市生活水準的提高帶動人們對環境資訊的需求提升，如何有效地處理、儲存環境中的感測資訊，並適當管理後再提供使用者查詢，已經成為建構智慧型城市的重要基礎。由於人類生活環境中每分每秒所產生的資訊，如早晚氣溫、行車環境、空氣品質等可數值化的資訊過於龐大且繁雜，為了要快速並準確地處理這些數據，我們利用雲端運算技術中的分散式儲存及程式開發模型MapReduce等核心技術，佐以HBase雲端資料庫的管理能力，以達成高效能的運算及儲存機制，並將這些數據提供予使用者或第三方軟體使用，以期實現智慧型城市的建構基礎。

關鍵字

雲端運算 ； Hadoop ； MapReduce

並列摘要

With the rapid development of sensor technologies, in nowadays modern cities are deployed with various types of sensors to gather the environmental data. It is without doubt that how to deal with the huge amount of data collected by various sensors in an efficient way and to transform these data to useful information for the citizens to make use of has become an important research topic in building intelligent cities. The vast amount of data produced around us such as the temperature, the road conditions and the air quality can be numerically analyzed by utilizing the cloud computing technology. In this research, we implemented a MapReduce-based sensor data processing platform for intelligent cities. Specifically, we focus on using the MapReduce framework to process the raw data uploaded from the sensors, and then using HBase, which is a distributed, scalable, big data store, to save the sensor data. Besides, we use the Hadoop Distributed File System (HDFS) to store the street images captured by event data recorders installed in vehicles. In sum, the data processing platform we developed can be an important building block for constructing various useful applications to serve the citizens in intelligent cities.

並列關鍵字

Cloud computing ； Hadoop ； MapReduce

參考文獻

[1] “Prominent users: Yahoo! - Wikipedia，the free encyclopedia,” http://en.wikipedia.org/wiki/Apache_Hadoop#Yahoo.21

[2] Google maps, http://maps.google.com.tw http://en.wikipedia.org/wiki/Line-of-sight_propagation

[3] Apache™ HadoopR project, http://hadoop.apache.org

[4] “Apache Software Foundation - Wikipedia，the free encyclopedia,” http://en.wikipedia.org/wiki/Apache_Software_Foundation

[5] Jeffery Dean and Sanjay Ghemawat, “MapReduce: Simplified Data Processing on Large Clusters,” Google, Inc.