基於主動學習的高效率關聯式少樣本學習

Cost-Effective Active Learning for Relational Few-shot Learning

戴長昕(Chang-Sin Dai)

指導教授 ： 廖世偉

國立臺灣大學/電機資訊學院/資訊網路與多媒體研究所/碩士(2018年)

https://doi.org/10.6342/NTU201803026

摘要

近年來，深度學習的發展一日千里，主要原因除了算力增長之外，還包括大量學者投入研究，使得深度學習在如機器視覺、自然語言處理、語音處理等多個領域發光發熱。雖然元學習(Meta learning)和少樣本學習(Few-shot learning)還未能在工業上有效應用，但其理念和創造力走在深度學習的先鋒，發展仍然備受矚目。
在本論文中，我們發現主動學習和少樣本學習有著極為相似的理念：同樣是藉由較少量的訓練樣本，前者從樣本質量出發，而後者從樣本數量出發。我們基於主動學習的理念，設計能為少樣本學習任務挑選合適訓練樣本選擇器來優化模型表現。以模擬人類識別過程的關係網路模型，結合主動學習以降低訓練成本，並讓模型學習到更優質的元知識(meta)來處理未知的任務，最後設計實驗來探討方法中各個部分所造成的影響。透過分析實驗結果，我們希望能夠一步步打開少樣本學習在深度結構模型中的黑箱。

關鍵字

機器學習 ； 深度學習 ； 少樣本學習 ； 元學習 ； 主動學習

並列摘要

In recent years, deep learning is developed in tremendous speed nowsdays. Giving the credits to the increase in computing power, a large number of researchers have invested in research, making deep learning shine in many fields such as machine vision, natural language processing, and speech processing. While meta-learning and few-shot learning have not yet been effectively applied in industry, their ideas and creativity are at the forefront of deep learning, and its development is still attracting attention.
In this thesis, We found that active learning and few-shot learning have very similar ideas. Based on the concept of active learning, we design a suitable training sample selector for small sample learning tasks to optimize model performance.we use a relational network model that simulates the human recognition process, combined with active-learning-like method to reduce training costs and make it efficient, and let the model learn better meta knowledge to deal with unknown tasks. We design experiments to explore the effects of various parts of the method. By analyzing the experimental results, we hope to be able to open the black box in the deep structure model with less sample learning.

並列關鍵字

Machine Learning ； Deep Learning ； Few-shot Learning ； Meta Learning ； Active Learning

參考文獻

[1] Marcin Andrychowicz, Misha Denil, Sergio Gomez Colmenarejo, Matthew W. Hoffman, David Pfau, Tom Schaul, and Nando de Freitas. Learning to learn by gradient descent by gradient descent. CoRR, abs/1606.04474, 2016.

[2] Luca Bertinetto, Jack Valmadre, Jo˜ao F. Henriques, Andrea Vedaldi, and Philip H. S. Torr. Fully-convolutional siamese networks for object tracking. CoRR, abs/1606.09549, 2016.

[3] L´eon Bottou. Large-scale machine learning with stochastic gradient descent. In Yves Lechevallier and Gilbert Saporta, editors, Proceedings of COMPSTAT’ 2010, pages 177–186, Heidelberg, 2010. Physica-Verlag HD.

[4] S. Chakraborty, V. Balasubramanian, Q. Sun, S. Panchanathan, and J. Ye. Active batch selection via convex relaxations with guaranteed solution bounds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(10):1945–1958, Oct 2015.

[5] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable convolutional networks. CoRR, abs/1703.06211, 2017.