用於域適應之域不變性特徵學習

Domain-Invariant Feature Learning for Domain Adaptation

朱家宏(Chia-Hung Chu)

指導教授 ： 林慧珍

淡江大學/工學院/資訊工程學系碩士在職專班/碩士(2022年)

https://doi.org/10.6846/TKU.2022.00588

摘要

非監督式域適應（unsupervised domain adaptation, UDA）主要探討目標域標籤未知的情況下，如何從源域學習到域不變性的特徵（domain-invariant feature）。利用最小化兩域樣本的最大均值差異（maximum mean discrepancy, MMD）已被證明可以有效地拉近兩域樣本在特徵空間中的分布，進而學習到域不變性的特徵；然而MMD所用的參數值卻苦無確定的選擇方法。本篇論文採用一個可以訓練求出最佳參數的具深度核之最大均值差異（MMD with a Deep kernel, MMD-D），提出了一個輪流訓練MMD-D模組和UDA 模組CDA（Cross Domain Adaptation）子網路的機制，使得兩域樣本在特徵空間的分布能被訓練出來最佳的MMD-D有效地拉近，進而學習到更強健的域不變性的特徵。

關鍵字

遷移學習 ； 域適應 ； 特徵學習 ； 最大均值差異

並列摘要

Unsupervised domain adaptation (UDA) mainly explores how to learn domain-invariant features from the source domain when the target domain label is unknown. Minimizing the maximum mean discrepancy (MMD) of samples from two domains has been shown to effectively narrow the distribution of samples from two domains in the feature space, and then learn domain-invariant features; However, there is no sure way to select the parameter values used by MMD. This paper adopts the MMD with a Deep kernel (MMD-D) proposed by　Feng et al. [3] that can be trained to find the best parameters, and proposes to alternatively train the MMD-D module and a CDA (Cross Domain Adaptation) module, so that the distribution of two-domain samples in the feature space can be effectively narrowed by the best MMD-D trained, and then more robust domain-invariant features can be learned.

並列關鍵字

Transfer learning ； Domain adaptation ； Feature learning ； Maximum Mean Discrepancy

參考文獻

[1] Karsten M. Borgwardt, Arthur Gretton, Malte J. Rasch, Hans-Peter Kriegel, Bernhard Sch¨olkopf, and Alexander J. Smola, “Integrating structured biological data by kernel maximum mean discrepancy,” Conference on Intelligent Systems for Molecular Biology, pp. 49-57, 2006.

[2] A. Gretton, K. Borgwardt, M. Rasch, B. Sch¨olkopf, and A. Smola, “A kernel two-sample test,” Journal of Machine Learning Research, 13, pp. 723–773, March 2012.

[3] Feng Liu, Wenkai Xu, Jie Lu, Guangquan Zhang, Arthur Gretton, and DJ Sutherland, “Learning deep kernels for non-parametric two-sample tests,” arXiv preprint arXiv:2002.09116, 2020.

[4] Y. Grandvalet, Y. Bengio, “Semi-supervised learning by entropy minimization,” Conference on Advances in neural information processing systems, pp. 529–536, 2005.

[5] C.-L. Li, W.-C. Chang, Y. Cheng, Y. Yang, and B. Póczos, “MMD GAN: Towards deeper understanding of moment matching network,” Conference on Advances in Neural Information Processing Systems, 2017.