Skip to content

Latest commit

 

History

History
 
 

code

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
deep
deep
 
 
distance
distance
 
 
feature_extractor
feature_extractor
 
 
traditional
traditional
 
 
README.md
README.md
 
 

README.md

code_transfer_learning

Some useful transfer learning and domain adaptation codes

It is a waste of time looking for the codes from others. So I collect or reimplement them here in a way that you can easily use. The following are some of the popular transfer learning (domain adaptation) methods in recent years, and I know most of them will be chosen to compare with your own method.

You are welcome to contribute and suggest other methods.

This document contains codes from several aspects: tutorial, theory, traditional methods, and deep methods.

Testing dataset can be found here.

Fine-tune 最简单的深度迁移学习

Deep feature extractor 提取深度网络特征用于传统方法

Deep feature extractor

Basic distance 常用的距离度量

Traditional transfer learning methods 非深度迁移

  • SVM (baseline)
  • TCA (Transfer Component Anaysis, TNN-11) [1]
  • GFK (Geodesic Flow Kernel, CVPR-12) [2]
  • DA-NBNN (Frustratingly Easy NBNN Domain Adaptation, ICCV-13) [39]
  • JDA (Joint Distribution Adaptation, ICCV-13) [3]
  • TJM (Transfer Joint Matching, CVPR-14) [4]
  • CORAL (CORrelation ALignment, AAAI-15) [5]
  • JGSA (Joint Geometrical and Statistical Alignment, CVPR-17) [6]
  • TrAdaBoost (ICML-07)[8]
  • SA (Subspace Alignment, ICCV-13) [11]
  • BDA (Balanced Distribution Adaptation for Transfer Learning, ICDM-17) [15]
  • MTLF (Metric Transfer Learning, TKDE-17) [16]
  • Open Set Domain Adaptation (ICCV-17) [19]
  • TAISL (When Unsupervised Domain Adaptation Meets Tensor Representations, ICCV-17) [21]
  • STL (Stratified Transfer Learning for Cross-domain Activity Recognition, PerCom-18) [22]
  • LSA (Landmarks-based kernelized subspace alignment for unsupervised domain adaptation, CVPR-15) [29]
  • OTL (Online Transfer Learning, ICML-10) [31]
  • RWA (Random Walking, arXiv, simple but powerful) [46]
  • MEDA (Manifold Embedded Distribution Alignment, ACM MM-18) [47]
  • EasyTL (Practically Easy Transfer Learning, ICME-19) [63]

Deep transfer learning methods 深度迁移

  • DaNN (Domain Adaptive Neural Network, PRICAI-14) [41]
  • DeepCORAL (Deep CORAL: Correlation Alignment for Deep Domain Adaptation) [33]
  • DAN/JAN (Deep Adaptation Network/Joint Adaptation Network, ICML-15,17) [9,10]
  • RTN (Unsupervised Domain Adaptation with Residual Transfer Networks, NIPS-16) [12]
  • ADDA (Adversarial Discriminative Domain Adaptation, arXiv-17) [13]
  • DANN/RevGrad (Unsupervised Domain Adaptation by Backpropagation, ICML-15) [14]
  • DANN Domain-Adversarial Training of Neural Networks (JMLR-16)[17]
  • Associative Domain Adaptation (ICCV-17) [18]
  • Deep Hashing Network for Unsupervised Domain (CVPR-17) [20]
  • CCSA (Unified Deep Supervised Domain Adaptation and Generalization, ICCV-17) [23]
  • MRN (Learning Multiple Tasks with Multilinear Relationship Networks, NIPS-17) [24]
  • AutoDIAL (Automatic DomaIn Alignment Layers, ICCV-17) [25]
  • DSN (Domain Separation Networks, NIPS-16) [26]
  • DRCN (Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation, ECCV-16) [27]
  • Multi-task Autoencoders for Domain Generalization (ICCV-15) [28]
  • Encoder based lifelong learning (ICCV-17) [30]
  • MECA (Minimal-Entropy Correlation Alignment, ICLR-18) [32]
  • WAE (Wasserstein Auto-Encoders, ICLR-18) [34]
  • ATDA (Asymmetric Tri-training for Unsupervised Domain Adaptation, ICML-15) [35]
  • PixelDA_GAN (Unsupervised pixel-level domain adaptation with GAN, CVPR-17) [36]
  • ARDA (Adversarial Representation Learning for Domain Adaptation) [37]
  • DiscoGAN (Learning to Discover Cross-Domain Relations with Generative Adversarial Networks) [38]
  • MADA (Multi-Adversarial Domain Adaptation, AAAI-18) [40]
  • MCD (Maximum Classifier Discrepancy, CVPR-18) [42]
  • Adversarial Feature Augmentation for Unsupervised Domain Adaptation (CVPR-18) [43]
  • DML (Deep Mutual Learning, CVPR-18) [44]
  • Self-ensembling for visual domain adaptation (ICLR 2018) [45]
  • PADA (Partial Adversarial Domain Adaptation, ECCV-18) [48]
  • iCAN (Incremental Collaborative and Adversarial Network for Unsupervised Domain Adaptation, CVPR-18) [49]
  • WeightedGAN (Importance Weighted Adversarial Nets for Partial Domain Adaptation, CVPR-18) [50]
  • OpenSet (Open Set Domain Adaptation by Backpropagation) [51]
  • WDGRL (Wasserstein Distance Guided Representation Learning, AAAI-18) [52]
  • JDDA (Joint Domain Alignment and Discriminative Feature Learning) [53]
  • Multi-modal Cycle-consistent Generalized Zero-Shot Learning (ECCV-18) [54]
  • MSTN (Moving Semantic Transfer Network, ICML-18) [55]
  • SAN (Partial Transfer Learning With Selective Adversarial Networks, CVPR-18) [56]
  • M-ADDA (Metric-based Adversarial Discriminative Domain Adaptation, ICML-18 workshop) [57]
  • Openset_DA (Open Set Domain Adaptation by Backpropagation) [58]
  • DIRT-T (A DIRT-T Approach to Unsupervised Domain Adaptation, ICLR-18) [59]
  • CDAN (Conditional Adversarial Domain Adaptation, NeurIPS-18) [60]
  • CMD (Central Moment Discrepancy, ICLR-17 and InfSc-19) [61], [62]
  • OPDA_BP (Open Set Domain Adaptation by Back-propagation, ECCV-18) [64]
  • TCP (Transfer Channel Prunning, IJCNN-19) [65]
  • MTAN (Multi-Task Attention Network, CVPR-19) [66]

Code from HKUST [a bit old]

References

[1] Pan S J, Tsang I W, Kwok J T, et al. Domain adaptation via transfer component analysis[J]TNN, 2011, 22(2): 199-210.

[2] Gong B, Shi Y, Sha F, et al. Geodesic flow kernel for unsupervised domain adaptation[C]//CVPR, 2012: 2066-2073.

[3] Long M, Wang J, Ding G, et al. Transfer feature learning with joint distribution adaptation[C]//ICCV. 2013: 2200-2207.

[4] Long M, Wang J, Ding G, et al. Transfer joint matching for unsupervised domain adaptation[C]//CVPR. 2014: 1410-1417.

[5] Sun B, Feng J, Saenko K. Return of Frustratingly Easy Domain Adaptation[C]//AAAI. 2016, 6(7): 8.

[6] Zhang J, Li W, Ogunbona P. Joint Geometrical and Statistical Alignment for Visual Domain Adaptation[C]//CVPR 2017.

[8] Dai W, Yang Q, Xue G R, et al. Boosting for transfer learning[C]//ICML, 2007: 193-200.

[9] Long M, Cao Y, Wang J, et al. Learning transferable features with deep adaptation networks[C]//ICML. 2015: 97-105.

[10] Long M, Wang J, Jordan M I. Deep transfer learning with joint adaptation networks[J]//ICML 2017.

[11] Fernando B, Habrard A, Sebban M, et al. Unsupervised visual domain adaptation using subspace alignment[C]//ICCV. 2013: 2960-2967.

[12] Long M, Zhu H, Wang J, et al. Unsupervised domain adaptation with residual transfer networks[C]//NIPS. 2016.

[13] Tzeng E, Hoffman J, Saenko K, et al. Adversarial discriminative domain adaptation[J]. arXiv preprint arXiv:1702.05464, 2017.

[14] Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation[C]//International Conference on Machine Learning. 2015: 1180-1189.

[15] Jindong Wang, Yiqiang Chen, Shuji Hao, Wenjie Feng, and Zhiqi Shen. Balanced Distribution Adaptation for Transfer Learning. ICDM 2017.

[16] Y. Xu et al., "A Unified Framework for Metric Transfer Learning," in IEEE Transactions on Knowledge and Data Engineering, vol. 29, no. 6, pp. 1158-1171, June 1 2017. doi: 10.1109/TKDE.2017.2669193

[17] Ganin Y, Ustinova E, Ajakan H, et al. Domain-adversarial training of neural networks[J]. Journal of Machine Learning Research, 2016, 17(59): 1-35.

[18] Haeusser P, Frerix T, Mordvintsev A, et al. Associative Domain Adaptation[C]. ICCV, 2017.

[19] Pau Panareda Busto, Juergen Gall. Open set domain adaptation. ICCV 2017.

[20] Venkateswara H, Eusebio J, Chakraborty S, et al. Deep hashing network for unsupervised domain adaptation[C]. CVPR 2017.

[21] H. Lu, L. Zhang, et al. When Unsupervised Domain Adaptation Meets Tensor Representations. ICCV 2017.

[22] J. Wang, Y. Chen, L. Hu, X. Peng, and P. Yu. Stratified Transfer Learning for Cross-domain Activity Recognition. 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[23] Motiian S, Piccirilli M, Adjeroh D A, et al. Unified deep supervised domain adaptation and generalization[C]//The IEEE International Conference on Computer Vision (ICCV). 2017, 2.

[24] Long M, Cao Z, Wang J, et al. Learning Multiple Tasks with Multilinear Relationship Networks[C]//Advances in Neural Information Processing Systems. 2017: 1593-1602.

[25] Maria Carlucci F, Porzi L, Caputo B, et al. AutoDIAL: Automatic DomaIn Alignment Layers[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 5067-5075.

[26] Bousmalis K, Trigeorgis G, Silberman N, et al. Domain separation networks[C]//Advances in Neural Information Processing Systems. 2016: 343-351.

[27] M. Ghifary, W. B. Kleijn, M. Zhang, D. Balduzzi, and W. Li. "Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation (DRCN)", European Conference on Computer Vision (ECCV), 2016

[28] M. Ghifary, W. B. Kleijn, M. Zhang, D. Balduzzi. Domain Generalization for Object Recognition with Multi-task Autoencoders, accepted in International Conference on Computer Vision (ICCV 2015), Santiago, Chile.

[29] Aljundi R, Emonet R, Muselet D, et al. Landmarks-based kernelized subspace alignment for unsupervised domain adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 56-63.

[30] Rannen A, Aljundi R, Blaschko M B, et al. Encoder based lifelong learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 1320-1328.

[31] Peilin Zhao and Steven C.H. Hoi. OTL: A Framework of Online Transfer Learning. ICML 2010.

[32] Pietro Morerio, Jacopo Cavazza, Vittorio Murino. Minimal-Entropy Correlation Alignment for Unsupervised Deep Domain Adaptation. ICLR 2018.

[33] Sun B, Saenko K. Deep coral: Correlation alignment for deep domain adaptation[C]//European Conference on Computer Vision. Springer, Cham, 2016: 443-450.

[34] Tolstikhin I, Bousquet O, Gelly S, et al. Wasserstein Auto-Encoders[J]. arXiv preprint arXiv:1711.01558, 2017.

[35] Saito K, Ushiku Y, Harada T. Asymmetric tri-training for unsupervised domain adaptation[J]. arXiv preprint arXiv:1702.08400, 2017.

[36] Bousmalis K, Silberman N, Dohan D, et al. Unsupervised pixel-level domain adaptation with generative adversarial networks[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017, 1(2): 7.

[37] Shen J, Qu Y, Zhang W, et al. Adversarial representation learning for domain adaptation[J]. arXiv preprint arXiv:1707.01217, 2017.

[38] Kim T, Cha M, Kim H, et al. Learning to discover cross-domain relations with generative adversarial networks[J]. arXiv preprint arXiv:1703.05192, 2017.

[39] Tommasi T, Caputo B. Frustratingly Easy NBNN Domain Adaptation[C]. international conference on computer vision, 2013: 897-904.

[40] Pei Z, Cao Z, Long M, et al. Multi-Adversarial Domain Adaptation[C] // AAAI 2018.

[41] Ghifary M, Kleijn W B, Zhang M. Domain adaptive neural networks for object recognition[C]//Pacific Rim International Conference on Artificial Intelligence. Springer, Cham, 2014: 898-904.

[42] Saito K, Watanabe K, Ushiku Y, et al. Maximum Classifier Discrepancy for Unsupervised Domain Adaptation[J]. arXiv preprint arXiv:1712.02560, 2017.

[43] Volpi R, Morerio P, Savarese S, et al. Adversarial Feature Augmentation for Unsupervised Domain Adaptation[J]. arXiv preprint arXiv:1711.08561, 2017.

[44] Zhang Y, Xiang T, Hospedales T M, et al. Deep Mutual Learning[C]. CVPR 2018.

[45] French G, Mackiewicz M, Fisher M. Self-ensembling for visual domain adaptation[C]//International Conference on Learning Representations. 2018.

[46] van Laarhoven T, Marchiori E. Unsupervised Domain Adaptation with Random Walks on Target Labelings[J]. arXiv preprint arXiv:1706.05335, 2017.

[47] Jindong Wang, Wenjie Feng, Yiqiang Chen, Han Yu, Meiyu Huang, Philip S. Yu. Visual Domain Adaptation with Manifold Embedded Distribution Alignment. ACM Multimedia conference 2018.

[48] Zhangjie Cao, Mingsheng Long, et al. Partial Adversarial Domain Adaptation. ECCV 2018.

[49] Zhang W, Ouyang W, Li W, et al. Collaborative and Adversarial Network for Unsupervised domain adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 3801-3809.

[50] Zhang J, Ding Z, Li W, et al. Importance Weighted Adversarial Nets for Partial Domain Adaptation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 8156-8164.

[51] Saito K, Yamamoto S, Ushiku Y, et al. Open Set Domain Adaptation by Backpropagation[J]. arXiv preprint arXiv:1804.10427, 2018.

[52] Shen J, Qu Y, Zhang W, et al. Wasserstein Distance Guided Representation Learning for Domain Adaptation[C]//AAAI. 2018.

[53] Chen C, Chen Z, Jiang B, et al. Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation[J]. arXiv preprint arXiv:1808.09347, 2018.

[54] Felix R, Vijay Kumar B G, Reid I, et al. Multi-modal Cycle-consistent Generalized Zero-Shot Learning. ECCV 2018.

[55] Xie S, Zheng Z, Chen L, et al. Learning Semantic Representations for Unsupervised Domain Adaptation[C]//International Conference on Machine Learning. 2018: 5419-5428.

[56] Cao Z, Long M, Wang J, et al. Partial transfer learning with selective adversarial networks. CVPR 2018.

[57] Issam Laradji, Reza Babanezhad. M-ADDA: Unsupervised Domain Adaptation with Deep Metric Learning. ICML 2018 workshop.

[58] Saito K, Yamamoto S, Ushiku Y, et al. Open Set Domain Adaptation by Backpropagation[J]. arXiv preprint arXiv:1804.10427, 2018.

[59] Shu R, Bui H H, Narui H, et al. A DIRT-T Approach to Unsupervised Domain Adaptation[J]. arXiv preprint arXiv:1802.08735, 2018.

[60] Mingsheng Long, et al. Conditional Adversarial Domain Adaptation. NeurIPS 2018.

[61] W.Zellinger, T. Grubinger, E. Lughofer, T. Natschlaeger, and Susanne Saminger-Platz, "Central moment discrepancy (cmd) for domain-invariant representation learning," ICLR 2017.

[62] W. Zellinger, B.A. Moser, T. Grubinger, E. Lughofer, T. Natschlaeger, and S. Saminger-Platz, "Robust unsupervised domain adaptation for neural networks via moment alignment," Information Sciences (in press), 2019, https://doi.org/10.1016/j.ins.2019.01.025, arXiv preprint arxiv:1711.06114

[63] Jindong Wang, Yiqiang Chen, Han Yu, Meiyu Huang, Qiang Yang. Easy Transfer Learning By Exploiting Intra-domain Structures. IEEE International Conference on Multimedia & Expo (ICME) 2019.

[64] Saito K, Yamamoto S, Ushiku Y, et al. Open set domain adaptation by backpropagation[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 153-168.

[65] Chaohui Yu, Jindong Wang, Yiqiang Chen, Zijing Wu. Accelerating Deep Unsupervised Domain Adaptation with Transfer Channel Pruning. IJCNN 2019.

[66] Shikun Liu, Edward Johns, and Andrew Davison. End-to-End Multi-Task Learning with Attention. CVPR 2019.