基於高資訊量投影之超高維運算及其在使用者適應學習之應用

Hyperdimensional Computing with Informative Projection and Its Application to User-Adaptation Learning

蕭育任(Yu-Ren Hsiao)

指導教授 ： 吳安宇

國立臺灣大學/電機資訊學院/電子工程學研究所/碩士(2021年)

https://doi.org/10.6342/NTU202104036

摘要

搭載著機器學習模型的物聯網應用絕大部分傾向於將他們的模型在雲端上進行訓練。造成這個現象的主要原因在於邊緣裝置的硬體資源和電池容量有限，難以運作複雜的訓練演算法。因此，一種仿效人腦的運算方式，超高維運算(Hyperdimensional Computing, HDC) 被提出了。藉由對演算法和架構層面的改善，超高維運算擁有高度能源效率以及快速適應性，讓它可以實現高效能的裝置上運算(On-device Computing)。
雖然超高維運算對於硬體資源的要求十分友善，然而它忽略了特徵間潛在的資訊，這使得它無法達到更高的分類準確度。超高維運算遵守兩個假設將數值投影到高維的空間當中，一個是特徵之間不存在關聯性，另一個則是特徵的數值皆呈現均勻分布。然而這兩假設與實際狀況並不相符。因此我們提出了兩個更關注特徵資訊的高維投影方法。第一個方法對資料進行前處理，使其能夠符合超高維運算對欲投影資料所做的假設；另一個則借助神經網路(Neural Networks)擅長特徵萃取(Feature Extraction)的長處，幫助超高維運算的投影學習到特徵間的關聯性，從而使用更少的硬體資源達到更佳的分類表現。
最後，我們將超高維運算應用在使用者適應(User-adaptation Learning)的場景。有鑑於使用以時變訊號作為輸入的延遲敏感(Delay-sensitive)應用之可能性，以及對個體差異所導致的資料分布不一致之敏感性，模型的裝置上適應(On-device Adaptation) 被迫切地需要。超高維運算對於硬體運算資源友善的特性便使其適合用於使用者適應，可以進行高效能的裝置上適應並保持良好的分類準確度。

關鍵字

腦啟發式運算 ； 超高維運算 ； 可學習之投影 ； 使用者適應

並列摘要

The majority of Internet of Things (IoT) applications with the involvement of Machine Learning (ML) models tend to train the models within the cloud. This is mainly because of limited resources and battery capacity of edge devices which make them difficult to run complicated training algorithms. Therefore, a brain-inspired computational paradigm, Hyperdimensional Computing (HDC), is proposed. With major algorithmic and architectural improvements, it achieves the characteristics of high energy efficiency and fast adaptability which enables efficient on-device computing.
Although HDC is hardware-friendly, its lack of feature awareness hinders it from having higher classification accuracy. HDC’s data mapping into the HD domain is based on two impractical assumptions, uncorrelation of features and uniform distribution of feature values. Therefore, we propose two feature-aware HD projection methods. The first one is to design a data pre-processing module that makes transformed data match the assumptions of HDC. Another one is to leverage the advantage of the Neural Networks (NN) in feature extraction to help HDC learning a better projection method. Replacing our proposed feature-aware projection with the original one enables HDC to achieve better classification accuracy with fewer hardware resources.
Finally, we apply HDC with our proposed projection method to the application of user-adaptation learning. Due to the possibility of delay-sensitive applications using time-varying inputs and sensitivity to the misaligned data distribution caused by subject difference, on-device adaptation of models is urgently needed. The hardware-friendly characteristics of HDC make it a suitable choice for user-adaptation learning to perform highly energy-efficient on-device adaptation and keep high classification accuracy.

並列關鍵字

Brain-Inspired Computing ； Hyperdimensional Computing ； Learnable Projection ； User-Adaptation Learning

參考文獻

Gubbi, J., Buyya, R., Marusic, S., Palaniswami, M.: Internet of Things (IoT): A vision, architectural elements, and future directions. Future generation computer systems 29(7), 1645–1660 (2013).

Y. Xiao, Y. Jia, C. Liu, X. Cheng, J. Yu and W. Lv, "Edge Computing Security: State of the Art and Challenges," in Proceedings of the IEEE, vol. 107, no. 8, pp. 1608-1631, Aug. 2019.

Khan, Wazir Zada, et al. "Edge computing: A survey." Future Generation Computer Systems 97 (2019): 219-235.

Abbas, N., Zhang, Y., Taherkordi, A., Skeie, T.: Mobile edge computing: a survey. IEEE Internet Things J. 5(1), 450–465 (2018)

J. Chen and X. Ran, "Deep Learning With Edge Computing: A Review," in Proceedings of the IEEE, vol. 107, no. 8, pp. 1655-1674, Aug. 2019.