雙支架構應用區塊圖片於零次學習

Two-Branch Net for Zero Shot Learning Using Patch Features

許家豪(Chia-Hao Hsu)

指導教授 ： 顏淑惠

淡江大學/工學院/資訊工程學系碩士班/碩士(2020年)

https://doi.org/10.6846/TKU.2020.00708

摘要

本篇論文以VGG19為Backbone基礎模型下，加入Squeeze-and-Excitation net以及加入PatchNet成為雙支網路。其中Squeeze-and-Excitation主要加強channel-wise 的特徵，而PatchNet主要用意為，藉由Patch可以在圖片上各個不同地方都提取到不一樣的特徵，像是取出物件的區塊特徵，可能有耳朵、鼻子、嘴巴、身體...，利用這些特徵在與VGG19取出特徵進行雙線性(Bilinear)的運算，且經過此運算能夠讓這兩個網路取出的特徵相互比較他們的關係性，進而可以提取出更加完整的物件本體，而非只有頭或是身體的特徵。最後本文在AwA2、CUB、SUN資料庫去進行測試，整體上雖然只有AwA2效果能夠與其他論文比較，且稍微的超過它文的結果，但我們主要的想法是提升物件特徵的完整性，也於本文最後使用熱圖(heat map)視覺化本文取出的特徵證實本文提出的方法有效。

關鍵字

零次學習 ； 區塊圖片特徵

並列摘要

We proposed a new patch net structure for zero shot learning (ZSL). In addition to the global features extracted from VGG19, patch net features are intended to catch the overall region-of-interested. These two features are fused via a bilinear operation. The fused image feature is mapped to the semantic space by a fully connected layer. The structure only adopts one simple cross entropy loss function so it is easy to train. According to the experiments, this method can extract more completeness features than those well-known backbones do in some images. In specific dataset, our method is competitive to other state of the art methods.

並列關鍵字

Zero shot learning ； Patch features

參考文獻

[1] J. Song, C. Shen, Y. Yang, Y. Liu and M. Song, "Transductive Unbiased Embedding for Zero-Shot Learning," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp. 1024-1033, doi: 10.1109/CVPR.2018.00113.

[2] A. Paul, N. C. Krishnan and P. Munjal, "Semantically Aligned Bias Reducing Zero Shot Learning," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 7049-7058, doi: 10.1109/CVPR.2019.00722.

[3] H. Huang, C. Wang, P. S. Yu and C. Wang, "Generative Dual Adversarial Network for Generalized Zero-Shot Learning," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 801-810, doi: 10.1109/CVPR.2019.00089.

[4] J. Li, M. Jing, K. Lu, Z. Ding, L. Zhu and Z. Huang, "Leveraging the Invariant Side of Generative Zero-Shot Learning," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 7394-7403, doi: 10.1109/CVPR.2019.00758.

[5] M. B. Sariyildiz and R. G. Cinbis, "Gradient Matching Generative Networks for Zero-Shot Learning," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 2163-2173, doi: 10.1109/CVPR.2019.00227.