Transfer Learning

Papers

Discriminative Transfer Learning with Tree-based Priors

• intro: NIPS 2013
• paper: http://deeplearning.net/wp-content/uploads/2013/03/icml13_workshop.pdf
• paper: http://www.cs.toronto.edu/~nitish/treebasedpriors.pdf

How transferable are features in deep neural networks?

• intro: NIPS 2014
• arxiv: http://arxiv.org/abs/1411.1792
• paper: http://papers.nips.cc/paper/5347-how-transferable-are-features-in-deep-neural-networks.pdf
• github: https://github.com/yosinski/convnet_transfer

Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks

• paper: http://research.microsoft.com/pubs/214307/paper.pdf

Learning Transferable Features with Deep Adaptation Networks

• intro: ICML 2015
• arxiv: https://arxiv.org/abs/1502.02791
• gihtub: https://github.com/caoyue10/icml-caffe

Transferring Knowledge from a RNN to a DNN

• intro: CMU
• arxiv: https://arxiv.org/abs/1504.01483

Simultaneous Deep Transfer Across Domains and Tasks

• intro: ICCV 2015
• arxiv: http://arxiv.org/abs/1510.02192

Net2Net: Accelerating Learning via Knowledge Transfer

• arxiv: http://arxiv.org/abs/1511.05641
• github: https://github.com/soumith/net2net.torch
• notes(by Hugo Larochelle): https://www.evernote.com/shard/s189/sh/46414718-9663-440e-bbb7-65126b247b42/19688c438709251d8275d843b8158b03

Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping

• arxiv: http://arxiv.org/abs/1510.00098

A theoretical framework for deep transfer learning

• key words: transfer learning, PAC learning, PAC-Bayesian, deep learning
• homepage: http://imaiai.oxfordjournals.org/content/early/2016/04/28/imaiai.iaw008
• paper: http://imaiai.oxfordjournals.org/content/early/2016/04/28/imaiai.iaw008.full.pdf

Transfer learning using neon

• blog: http://www.nervanasys.com/transfer-learning-using-neon/

Hyperparameter Transfer Learning through Surrogate Alignment for Efficient Deep Neural Network Training

• arxiv: http://arxiv.org/abs/1608.00218

What makes ImageNet good for transfer learning?

• project page: http://minyounghuh.com/papers/analysis/
• arxiv: http://arxiv.org/abs/1608.08614

Fine-tuning a Keras model using Theano trained Neural Network & Introduction to Transfer Learning

• github: https://www.analyticsvidhya.com/blog/2016/11/fine-tuning-a-keras-model-using-theano-trained-neural-network-introduction-to-transfer-learning/

Multi-source Transfer Learning with Convolutional Neural Networks for Lung Pattern Analysis

• arxiv: https://arxiv.org/abs/1612.02589

Borrowing Treasures from the Wealthy: Deep Transfer Learning through Selective Joint Fine-tuning

• intro: CVPR 2017. The University of Hong Kong
• arxiv: https://arxiv.org/abs/1702.08690

Optimal Transport for Deep Joint Transfer Learning

https://arxiv.org/abs/1709.02995

CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise

• intro: CVPR 2018. Microsoft AI and Research i& JD AI Research & Facebook
• keywords: Food-101N
• project page: https://kuanghuei.github.io/CleanNetProject/
• arxiv: https://arxiv.org/abs/1711.07131
• github(Tensorflow): https://github.com/kuanghuei/clean-net
• blog: https://www.microsoft.com/en-us/research/blog/using-transfer-learning-to-address-label-noise-for-large-scale-image-classification/

Transfer Learning with Binary Neural Networks

• intro: Machine Learning on the Phone and other Consumer Devices, NIPS2017 Workshop
• arxiv: https://arxiv.org/abs/1711.10761

Gradual Tuning: a better way of Fine Tuning the parameters of a Deep Neural Network

• intro: Université Paris Descartes, Paris
• arxiv: https://arxiv.org/abs/1711.10177

Born Again Neural Networks

• intro: University of Southern California & CMU & Amazon AI
• paper: http://metalearning.ml/papers/metalearn17_furlanello.pdf

Taskonomy: Disentangling Task Transfer Learning

• intro: CVPR 2018 (Oral). CVPR 2018 Best paper award. Stanford University & UC Berkeley
• project page: http://taskonomy.stanford.edu/
• arxiv: https://arxiv.org/abs/1804.08328

Do Better ImageNet Models Transfer Better?

• intro: Google Brain
• arxiv: https://arxiv.org/abs/1805.08974

SOSELETO: A Unified Approach to Transfer Learning and Training with Noisy Labels

• keywords: SOSELETO (SOurce SELEction for Target Optimization)
• arxiv: https://arxiv.org/abs/1805.09622

GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations

• intro: Carnegie Mellon University & New York University & Facebook AI Research
• arxiv: https://arxiv.org/abs/1806.05662

Taskonomy: Disentangling Task Transfer Learning

• intro: CVPR 2018 oral
• project page: http://taskonomy.stanford.edu/
• arxiv: https://arxiv.org/abs/1804.08328
• github: https://github.com/StanfordVL/taskonomy/tree/master/taskbank

One Shot Learning

One-shot Learning with Memory-Augmented Neural Networks

• intro: Google DeepMind
• arxiv: https://arxiv.org/abs/1605.06065
• github(Tensorflow): https://github.com/hmishra2250/NTM-One-Shot-TF
• note: http://rylanschaeffer.github.io/content/research/one_shot_learning_with_memory_augmented_nn/main.html

Matching Networks for One Shot Learning

• intro: Google DeepMind
• arxiv: https://arxiv.org/abs/1606.04080
• notes: https://blog.acolyer.org/2017/01/03/matching-networks-for-one-shot-learning/

Learning feed-forward one-shot learners [NIPS 2016] [VALSE seminar]

• youtube: https://www.youtube.com/watch?v=BnLN3uoXMRY
• mirror: https://pan.baidu.com/s/1mhAITmS

Generative Adversarial Residual Pairwise Networks for One Shot Learning

• intro: Indian Institute of Science
• arxiv: https://arxiv.org/abs/1703.08033

Few-Shot Learning

Optimization as a Model for Few-Shot Learning

• intro: Twitter
• paper: https://openreview.net/pdf?id=rJY0-Kcll
• github: https://github.com/twitter/meta-learning-lstm

Learning to Compare: Relation Network for Few-Shot Learning

• intro: Queen Mary University of London & The University of Edinburgh
• arxiv: https://arxiv.org/abs/1711.06025

Unleashing the Potential of CNNs for Interpretable Few-Shot Learning

• intro: Beihang University & Johns Hopkins University
• arxiv: https://arxiv.org/abs/1711.08277

Low-Shot Learning from Imaginary Data

• intro: Facebook AI Research (FAIR) & CMU & Cornell University
• arxiv: https://arxiv.org/abs/1801.05401

Semantic Feature Augmentation in Few-shot Learning

• keywords: TriNet
• arxiv: https://arxiv.org/abs/1804.05298
• github: https://github.com/tankche1/Semantic-Feature-Augmentation-in-Few-shot-Learning

Transductive Propagation Network for Few-shot Learning

• intro: achieved the state-of-the-art results on miniImagenet
• arxiv: https://arxiv.org/abs/1805.10002

TADAM: Task dependent adaptive metric for improved few-shot learning

• intro: Element AI
• arxiv: https://arxiv.org/abs/1805.10123