RESEARCH ON INTEGRATION AND SHARING SOLUTIONS FOR BIG STRUCTURED DATA FOR THE MARINE ENVIRONMENT

Wei Wang ； Wenfang Cheng ； Jing Chen ； Ya-Nan Yu ； Jianxun Zhang

《Journal of Marine Science and Technology》 25卷6期 (2017/12) Pp. 791-801

https://doi.org/10.6119/JMST-017-1226-21

摘要

The data-intensive science, a new paradigm in scientific discovery, demands an urgent solution to integrate and share big marine environment data. Through the modeling of marine environmental features (such as Temperature, Salinity, Density et al.), a novel solution is presented in this paper. This solution employs Hadoop and Hbase, based on the Bigtable algorithm first proposed by Google Inc. By using hierarchical technology, scalability of the data integration framework is realized. Additionally, an efficient execution of various queries is realized by using the customized auxiliary index algorithm. In the experimental cluster environment constructed from four ordinary personal computers, experiments involving a huge amount of structured marine environmental data importing, cost-effective storage, and query processing were carried out successfully. The experimental results indicate that this solution to storing and sharing big data for the marine environment has the property of infinite lateral expansion, and is able to execute efficient and complicated query operations.

關鍵字

marine big data ； integration ； sharing