以實例轉換壓縮無監督式領域適配模型

無監督式領域適應 (UDA) 是一種有用的技術，可以從標記良好的來源領域中為未標記的目標領域訓練模型。然而，缺乏目標數據的標籤使得傳統的模型壓縮方法難以同時達到良好的模型精度和壓縮率。在本論文中，我們提出了一種新的UDA通道剪枝方法，稱為TICUDA（以實例轉換壓縮無監督式領域適配模型），它利用變換後的實例來評估通道重要性。轉換後的實例是從來源領域中的數據生成的，以模擬目標領域中的數據。該生成基於兩個數據集的領域差距，由領域對抗訓練神經網路(DANN)測量。我們還提出了一種自適應演算法來動態確定轉換實例的適用性。在ImageCLEF-DA和Office31數據集上使用VGG16和ResNet50框架進行了實驗，以評估提出方法的性能。我們將TICUDA與利用全來源資訊的方法和使用正則化的剪枝方法進行了比較。結果表明，在模型精度、模型大小和計算成本方面，TICUDA 在稱為縮減因子的組合分數方面優於它們。

關鍵字

深度學習模型壓縮；無監督領域適配

並列摘要

Unsupervised domain adaptation (UDA) is a useful technique to train models for an unlabeled target domain from a well-labeled source domain. However, the lack of labels for target data makes conventional model compression methods hard to achieve good model accuracy and compression ratio simultaneously. In this thesis, we propose a new channel pruning method for UDA, called TICUDA (Transformed Instances for Compressing Unsupervised Domain Adaptation), which utilizes the transformed instances to evaluate the channel importance. The transformed instances are generated from the data in the source domain to mimic the data in the target domain. The generation is based on the domain gap for two data-sets, measured by the Domain-Adversarial Training of Neural Networks (DANN). An adaptive algorithm is also proposed to dynamically determine the applicability of transformed instances. Experiments that use VGG16 and ResNet50 frameworks on ImageCLEF-DA and Office31 datasets were conducted to evaluate the performance of the proposed method. We compared TICUDA with the method utilizing full source information and the pruning methods using regularization. The results show that TICUDA outperforms them in terms of a combinative score, called Reduction Factor, for model accuracy, model size, and the computation cost.