Papers

Learning Convolutional Neural Networks for Graphs

• intro: ICML 2016
• arxiv: http://arxiv.org/abs/1605.05273

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

• arxiv: https://arxiv.org/abs/1606.09375
• github: https://github.com/mdeff/cnn_graph
• github: https://github.com/pfnet-research/chainer-graph-cnn

Semi-Supervised Classification with Graph Convolutional Networks

• arxiv: http://arxiv.org/abs/1609.02907
• github: https://github.com/tkipf/gcn
• blog: http://tkipf.github.io/graph-convolutional-networks/

Graph Based Convolutional Neural Network

• intro: BMVC 2016
• arxiv: http://arxiv.org/abs/1609.08965

How powerful are Graph Convolutions? (review of Kipf & Welling, 2016)

http://www.inference.vc/how-powerful-are-graph-convolutions-review-of-kipf-welling-2016-2/

Graph Convolutional Networks

• blog: http://tkipf.github.io/graph-convolutional-networks/

DeepGraph: Graph Structure Predicts Network Growth

• arxiv: https://arxiv.org/abs/1610.06251

Deep Learning with Sets and Point Clouds

• intro: CMU
• arxiv: https://arxiv.org/abs/1611.04500

Deep Learning on Graphs

• lecture: https://figshare.com/articles/Deep_Learning_on_Graphs/4491686

Robust Spatial Filtering with Graph Convolutional Neural Networks

https://arxiv.org/abs/1703.00792

Modeling Relational Data with Graph Convolutional Networks

https://arxiv.org/abs/1703.06103

Distance Metric Learning using Graph Convolutional Networks: Application to Functional Brain Networks

• intro: Imperial College London
• arxiv: https://arxiv.org/abs/1703.02161

Deep Learning on Graphs with Graph Convolutional Networks

• slides: http://tkipf.github.io/misc/GCNSlides.pdf

Deep Learning on Graphs with Keras

• intro:; Keras implementation of Graph Convolutional Networks
• github: https://github.com/tkipf/keras-gcn

Learning Graph While Training: An Evolving Graph Convolutional Neural Network

https://arxiv.org/abs/1708.04675

Graph Attention Networks

• intro: ICLR 2018
• intro: University of Cambridge & Centre de Visio per Computador, UAB & Montreal Institute for Learning Algorithms
• project page: http://petar-v.com/GAT/
• arxiv: https://arxiv.org/abs/1710.10903
• github: https://github.com/PetarV-/GAT

Residual Gated Graph ConvNets

https://arxiv.org/abs/1711.07553

Probabilistic and Regularized Graph Convolutional Networks

• intro: CMU
• arxiv: https://arxiv.org/abs/1803.04489

Videos as Space-Time Region Graphs

https://arxiv.org/abs/1806.01810

Relational inductive biases, deep learning, and graph networks

• intro: DeepMind & Google Brain & MIT & University of Edinburgh
• arxiv: https://arxiv.org/abs/1806.01261

Can GCNs Go as Deep as CNNs?

• project: https://sites.google.com/view/deep-gcns
• arxiv: https://arxiv.org/abs/1904.03751
• slides: https://docs.google.com/presentation/d/1L82wWymMnHyYJk3xUKvteEWD5fX0jVRbCbI65Cxxku0/edit#slide=id.p
• github(official, TensorFlow): https://github.com/lightaime/deep_gcns

GMNN: Graph Markov Neural Networks

• intro: ICML 2019
• ariv: https://arxiv.org/abs/1905.06214
• github: https://github.com/DeepGraphLearning/GMNN

DeepGCNs: Making GCNs Go as Deep as CNNs

• intro: ICCV 2019 Oral
• arxiv: https://arxiv.org/abs/1910.06849
• github: https://github.com/lightaime/deep_gcns_torch
• github: https://github.com/lightaime/deep_gcns

Rethinking pooling in graph neural networks

• intro: NeurIPS 2020
• arxiv: https://arxiv.org/abs/2010.11418

Generative Adversarial Networks

Generative Adversarial Nets

• arxiv: http://arxiv.org/abs/1406.2661
• paper: https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf
• github: https://github.com/goodfeli/adversarial
• github: https://github.com/aleju/cat-generator

Adversarial Feature Learning

• intro: ICLR 2017
• arxiv: https://arxiv.org/abs/1605.09782
• github: https://github.com/jeffdonahue/bigan

Generative Adversarial Networks

• intro: by Ian Goodfellow, NIPS 2016 tutorial
• arxiv: https://arxiv.org/abs/1701.00160
• slides: http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf
• mirror: https://pan.baidu.com/s/1gfBNYW7

Adversarial Examples and Adversarial Training

• intro: NIPS 2016, Ian Goodfellow OpenAI
• slides: http://www.iangoodfellow.com/slides/2016-12-9-AT.pdf

How to Train a GAN? Tips and tricks to make GANs work

• github: https://github.com/soumith/ganhacks

Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks

• intro: CatGAN
• arxiv: http://arxiv.org/abs/1511.06390

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

• intro: DCGAN
• arxiv: http://arxiv.org/abs/1511.06434
• github: https://github.com/jazzsaxmafia/dcgan_tensorflow
• github: https://github.com/Newmu/dcgan_code
• github: https://github.com/mattya/chainer-DCGAN
• github: https://github.com/soumith/dcgan.torch
• github: https://github.com/carpedm20/DCGAN-tensorflow

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

• arxiv: https://arxiv.org/abs/1606.03657
• github: https://github.com/openai/InfoGAN
• github(Tensorflow): https://github.com/buriburisuri/supervised_infogan

Learning Interpretable Latent Representations with InfoGAN: A tutorial on implementing InfoGAN in Tensorflow

• blog: https://medium.com/@awjuliani/learning-interpretable-latent-representations-with-infogan-dd710852db46#.r0kur3aum
• github: https://gist.github.com/awjuliani/c9ecd8b37d33d6855cd4ed9aa16ce89f#file-infogan-tutorial-ipynb

Coupled Generative Adversarial Networks

• arxiv: https://arxiv.org/abs/1606.07536

Energy-based Generative Adversarial Network

• intro: EBGAN
• author: Junbo Zhao, Michael Mathieu, Yann LeCun
• arxiv: http://arxiv.org/abs/1609.03126
• github(Tensorflow): https://github.com/buriburisuri/ebgan

SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient

• github: https://github.com/LantaoYu/SeqGAN

Connecting Generative Adversarial Networks and Actor-Critic Methods

• arxiv: https://arxiv.org/abs/1610.01945

Generative Adversarial Nets from a Density Ratio Estimation Perspective

• arxiv: https://arxiv.org/abs/1610.02920

Unrolled Generative Adversarial Networks

• paper: http://openreview.net/pdf?id=BydrOIcle
• github: https://github.com/bstriner/keras-adversarial

Generative Adversarial Networks as Variational Training of Energy Based Models

• arxiv: https://arxiv.org/abs/1611.01799
• github: https://github.com/Shuangfei/vgan

Multi-class Generative Adversarial Networks with the L2 Loss Function

Least Squares Generative Adversarial Networks

• arxiv: https://arxiv.org/abs/1611.04076

Inverting The Generator Of A Generative Adversarial Networ

• intro: NIPS 2016 Workshop on Adversarial Training
• arxiv: https://arxiv.org/abs/1611.05644

ml4a-invisible-cities

• project page: https://opendot.github.io/ml4a-invisible-cities/
• arxiv: https://github.com/opendot/ml4a-invisible-cities

Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks

• arxiv: https://arxiv.org/abs/1611.06430

Associative Adversarial Networks

• intro: NIPS 2016 Workshop on Adversarial Training
• arxiv: https://arxiv.org/abs/1611.06953

Temporal Generative Adversarial Nets

• arxiv: https://arxiv.org/abs/1611.06624

Handwriting Profiling using Generative Adversarial Networks

• intro: Accepted at The Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17 Student Abstract and Poster Program)
• arxiv: https://arxiv.org/abs/1611.08789

C-RNN-GAN: Continuous recurrent neural networks with adversarial training

• intro: Constructive Machine Learning Workshop (CML) at NIPS 2016
• project page: http://mogren.one/publications/2016/c-rnn-gan/
• arxiv: https://arxiv.org/abs/1611.09904
• github: https://github.com/olofmogren/c-rnn-gan

Ensembles of Generative Adversarial Networks

• intro: NIPS 2016 Workshop on Adversarial Training
• arxiv: https://arxiv.org/abs/1612.00991

Improved generator objectives for GANs

• intro: NIPS 2016 Workshop on Adversarial Training
• arxiv: https://arxiv.org/abs/1612.02780

Stacked Generative Adversarial Networks

• intro: SGAN
• arxiv: https://arxiv.org/abs/1612.04357
• github: https://github.com/xunhuang1995/SGAN

Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks

• intro: Google Brain & Google Research
• arxiv: https://arxiv.org/abs/1612.05424

AdaGAN: Boosting Generative Models

• intro: Max Planck Institute for Intelligent Systems & Google Brain
• arxiv: https://arxiv.org/abs/1701.02386

Towards Principled Methods for Training Generative Adversarial Networks

• intro: Courant Institute of Mathematical Sciences & Facebook AI Research
• arxiv: https://arxiv.org/abs/1701.04862

Wasserstein GAN

• intro: Courant Institute of Mathematical Sciences & Facebook AI Research
• arxiv: https://arxiv.org/abs/1701.07875
• github: https://github.com/martinarjovsky/WassersteinGAN
• github: https://github.com/Zardinality/WGAN-tensorflow
• github(Tensorflow/Keras): https://github.com/kuleshov/tf-wgan
• github: https://github.com/shekkizh/WassersteinGAN.tensorflow
• gist: https://gist.github.com/soumith/71995cecc5b99cda38106ad64503cee3
• reddit: https://www.reddit.com/r/MachineLearning/comments/5qxoaz/r_170107875_wasserstein_gan/

Improved Training of Wasserstein GANs

• intro: NIPS 2017
• arxiv: https://arxiv.org/abs/1704.00028
• github(TensorFlow): https://github.com/igul222/improved_wgan_training
• github: https://github.com/jalola/improved-wgan-pytorch

On the effect of Batch Normalization and Weight Normalization in Generative Adversarial Networks

On the Effects of Batch and Weight Normalization in Generative Adversarial Networks

• arxiv: https://arxiv.org/abs/1704.03971
• github: https://github.com/stormraiser/GAN-weight-norm

Controllable Generative Adversarial Network

• intro: Korea University
• arxiv: https://arxiv.org/abs/1708.00598

Generative Adversarial Networks: An Overview

• intro: Imperial College London & Victoria University of Wellington & University of Montreal & Cortexica Vision Systems Ltd
• intro: IEEE Signal Processing Magazine Special Issue on Deep Learning for Visual Understanding
• arxiv: https://arxiv.org/abs/1710.07035

CyCADA: Cycle-Consistent Adversarial Domain Adaptation

https://arxiv.org/abs/1711.03213

Spectral Normalization for Generative Adversarial Networks

https://openreview.net/forum?id=B1QRgziT-

Are GANs Created Equal? A Large-Scale Study

• intro: Google Brain
• arxiv: https://arxiv.org/abs/1711.10337
• reddit: https://www.reddit.com/r/MachineLearning/comments/7gwip3/d_googles_large_scale_gantuning_paper_unfairly/

GAGAN: Geometry-Aware Generative Adverserial Networks

https://arxiv.org/abs/1712.00684

CycleGAN: a Master of Steganography

• intro: NIPS 2017, workshop on Machine Deception
• arxiv: https://arxiv.org/abs/1712.02950

PacGAN: The power of two samples in generative adversarial networks

• intro: CMU & University of Illinois at Urbana-Champaign
• arxiv: https://arxiv.org/abs/1712.04086

ComboGAN: Unrestrained Scalability for Image Domain Translation

• arxiv: https://arxiv.org/abs/1712.06909
• github: https://github.com/AAnoosheh/ComboGAN

Decoupled Learning for Conditional Adversarial Networks

https://arxiv.org/abs/1801.06790

No Modes left behind: Capturing the data distribution effectively using GANs

• intro: AAAI 2018
• arxiv: https://arxiv.org/abs/1802.00771

Improving GAN Training via Binarized Representation Entropy (BRE) Regularization

• intro: ICLR 2018
• arxiv: https://arxiv.org/abs/1805.03644
• github: https://github.com/BorealisAI/bre-gan

On GANs and GMMs

https://arxiv.org/abs/1805.12462

The Unusual Effectiveness of Averaging in GAN Training

https://arxiv.org/abs/1806.04498

Understanding the Effectiveness of Lipschitz Constraint in Training of GANs via Gradient Analysis

https://arxiv.org/abs/1807.00751

The GAN Landscape: Losses, Architectures, Regularization, and Normalization

• intro: Google Brain
• arxiv: https://arxiv.org/abs/1807.04720
• github: https://github.com/google/compare_gan

Which Training Methods for GANs do actually Converge?

• intro: ICML 2018. MPI Tübingen & Microsoft Research
• project page: https://avg.is.tuebingen.mpg.de/publications/meschedericml2018
• paper: https://avg.is.tuebingen.mpg.de/uploads_file/attachment/attachment/424/Mescheder2018ICML.pdf
• github: https://github.com/LMescheder/GAN_stability

Convergence Problems with Generative Adversarial Networks (GANs)

• intro: University of Oxford
• arxiv: https://arxiv.org/abs/1806.11382

Bayesian CycleGAN via Marginalizing Latent Sampling

https://arxiv.org/abs/1811.07465

GAN Dissection: Visualizing and Understanding Generative Adversarial Networks

https://arxiv.org/abs/1811.10597

Do GAN Loss Functions Really Matter?

https://arxiv.org/abs/1811.09567

Image-to-Image Translation

Pix2Pix

Image-to-Image Translation with Conditional Adversarial Networks

• intro: CVPR 2017
• project page: https://phillipi.github.io/pix2pix/
• arxiv: https://arxiv.org/abs/1611.07004
• github: https://github.com/phillipi/pix2pix
• github(TensorFlow): https://github.com/yenchenlin/pix2pix-tensorflow
• github(Chainer): https://github.com/mattya/chainer-pix2pix
• github(PyTorch): https://github.com/mrzhu-cool/pix2pix-pytorch
• github(Chainer): https://github.com/wuhuikai/chainer-pix2pix

Remastering Classic Films in Tensorflow with Pix2Pix

• blog: https://hackernoon.com/remastering-classic-films-in-tensorflow-with-pix2pix-f4d551fa0503#.6dmahnt8n
• github: https://github.com/awjuliani/Pix2Pix-Film
• model: https://drive.google.com/file/d/0B8x0IeJAaBccNFVQMkQ0QW15TjQ/view

Image-to-Image Translation in Tensorflow

• blog: http://affinelayer.com/pix2pix/index.html
• github: https://github.com/affinelayer/pix2pix-tensorflow

webcam pix2pix

https://github.com/memo/webcam-pix2pix-tensorflow

Unsupervised Image-to-Image Translation with Generative Adversarial Networks

• intro: Imperial College London & Indian Institute of Technology
• arxiv: https://arxiv.org/abs/1701.02676

Unsupervised Image-to-Image Translation Networks

• intro: NIPS 2017 Spotlight
• intro: unsupervised/unpaired image-to-image translation using coupled GANs
• project page: http://research.nvidia.com/publication/2017-12_Unsupervised-Image-to-Image-Translation
• arxiv: https://arxiv.org/abs/1703.00848
• github: https://github.com/mingyuliutw/UNIT

Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

• intro: UC Berkeley
• project page: https://junyanz.github.io/CycleGAN/
• arxiv: https://arxiv.org/abs/1703.10593
• github(official, Torch): https://github.com/junyanz/CycleGAN
• github(official, PyTorch): https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix
• github(PyTorch): https://github.com/eveningglow/semi-supervised-CycleGAN
• github(Chainer): https://github.com/Aixile/chainer-cyclegan

CycleGAN and pix2pix in PyTorch

• intro: Image-to-image translation in PyTorch (e.g. horse2zebra, edges2cats, and more)
• github: https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix

Perceptual Adversarial Networks for Image-to-Image Transformation

https://arxiv.org/abs/1706.09138

XGAN: Unsupervised Image-to-Image Translation for many-to-many Mappings

• intro: IST Austria & Google Brain & Google Research
• arxiv: https://arxiv.org/abs/1711.05139

In2I : Unsupervised Multi-Image-to-Image Translation Using Generative Adversarial Networks

https://arxiv.org/abs/1711.09334

StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation

• intro: Korea University & Clova AI Research
• arxiv: https://arxiv.org/abs/1711.09020
• github: https://github.com//yunjey/StarGAN

Discriminative Region Proposal Adversarial Networks for High-Quality Image-to-Image Translation

https://arxiv.org/abs/1711.09554

Toward Multimodal Image-to-Image Translation

• intro: NIPS 2017. BicycleGAN
• project page: https://junyanz.github.io/BicycleGAN/
• arxiv: https://arxiv.org/abs/1711.11586
• github(official, PyTorch): https://github.com//junyanz/BicycleGAN
• github: https://github.com/gitlimlab/BicycleGAN-Tensorflow
• github: https://github.com/kvmanohar22/img2imgGAN
• github: https://github.com/eveningglow/BicycleGAN-pytorch

Face Translation between Images and Videos using Identity-aware CycleGAN

https://arxiv.org/abs/1712.00971

Unsupervised Multi-Domain Image Translation with Domain-Specific Encoders/Decoders

https://arxiv.org/abs/1712.02050

High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs

• intro: NVIDIA Corporation, UC Berkeley
• project page: https://tcwang0509.github.io/pix2pixHD/
• arxiv: https://arxiv.org/abs/1711.11585
• github: https://github.com/NVIDIA/pix2pixHD
• youtube: https://www.youtube.com/watch?v=3AIpPlzM_qs&feature=youtu.be

On the Effectiveness of Least Squares Generative Adversarial Networks

https://arxiv.org/abs/1712.06391

GANs for Limited Labeled Data

• intro: Ian Goodfellow
• slides: http://www.iangoodfellow.com/slides/2017-12-09-label.pdf

Defending Against Adversarial Examples

• intro: Ian Goodfellow
• slides: http://www.iangoodfellow.com/slides/2017-12-08-defending.pdf

Conditional Image-to-Image Translation

• intro: CVPR 2018
• arxiv: https://arxiv.org/abs/1805.00251

XOGAN: One-to-Many Unsupervised Image-to-Image Translation

https://arxiv.org/abs/1805.07277

Unsupervised Attention-guided Image to Image Translation

https://arxiv.org/abs/1806.02311

Exemplar Guided Unsupervised Image-to-Image Translation

https://arxiv.org/abs/1805.11145

Improving Shape Deformation in Unsupervised Image-to-Image Translation

https://arxiv.org/abs/1808.04325

Video-to-Video Synthesis

• arxiv: https://arxiv.org/abs/1808.06601
• github: https://github.com/NVIDIA/vid2vid

Segmentation Guided Image-to-Image Translation with Adversarial Networks

https://arxiv.org/abs/1901.01569

ForkGAN: Seeing into the rainy night

• intro: ECCV 2020 oral
• intro: UISEE Technology & Kyoto University & University of Pennsylvania
• paper: https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123480154.pdf
• github: https://github.com/zhengziqiang/ForkGAN
• presentation: https://www.youtube.com/watch?v=O2nxRsSwkzs&t=1s

Projects

Generative Adversarial Networks with Keras

• github: https://github.com/phreeza/keras-GAN

Generative Adversarial Network Demo for Fresh Machine Learning #2

• youtube: https://www.youtube.com/watch?v=deyOX6Mt_As&feature=em-uploademail
• github: https://github.com/llSourcell/Generative-Adversarial-Network-Demo
• demo: http://cs.stanford.edu/people/karpathy/gan/

TextGAN: A generative adversarial network for text generation, written in TensorFlow.

• github: https://github.com/AustinStoneProjects/TextGAN

cleverhans v0.1: an adversarial machine learning library

• arxiv: https://arxiv.org/abs/1610.00768
• github: https://github.com/openai/cleverhans

Deep Convolutional Variational Autoencoder w/ Adversarial Network

• intro: An implementation of the deep convolutional generative adversarial network, combined with a varational autoencoder
• github: https://github.com/staturecrane/dcgan_vae_torch

A versatile GAN(generative adversarial network) implementation. Focused on scalability and ease-of-use.

• github: https://github.com/255BITS/HyperGAN

AdaGAN: Boosting Generative Models

• intro: AdaGAN: greedy iterative procedure to train mixtures of GANs
• intro: Max Planck Institute for Intelligent Systems & Google Brain
• arxiv: https://arxiv.org/abs/1701.02386
• github: https://github.com/tolstikhin/adagan

TensorFlow-GAN (TFGAN)

• intro: TFGAN: A Lightweight Library for Generative Adversarial Networks
• github: https://github.com//tensorflow/tensorflow/tree/master/tensorflow/contrib/gan
• blog: https://research.googleblog.com/2017/12/tfgan-lightweight-library-for.html

Blogs

Generative Adversial Networks Explained

• blog: http://kvfrans.com/generative-adversial-networks-explained/

Generative Adversarial Autoencoders in Theano

• blog: https://swarbrickjones.wordpress.com/2016/01/24/generative-adversarial-autoencoders-in-theano/
• github: https://github.com/mikesj-public/dcgan-autoencoder

An introduction to Generative Adversarial Networks (with code in TensorFlow)

• blog: http://blog.aylien.com/introduction-generative-adversarial-networks-code-tensorflow/
• github: https://github.com/AYLIEN/gan-intro

Difficulties training a Generative Adversarial Network

• github: https://github.com/shekkizh/neuralnetworks.thought-experiments/blob/master/Generative%20Models/GAN/Readme.md

Are Energy-Based GANs any more energy-based than normal GANs?

http://www.inference.vc/are-energy-based-gans-actually-energy-based/

Generative Adversarial Networks Explained with a Classic Spongebob Squarepants Episode: Plus a Tensorflow tutorial for implementing your own GAN

• blog: https://medium.com/@awjuliani/generative-adversarial-networks-explained-with-a-classic-spongebob-squarepants-episode-54deab2fce39#.rpiunhdjh
• gist: https://gist.github.com/awjuliani/8ebf356d03ffee139659807be7fa2611

Deep Learning Research Review Week 1: Generative Adversarial Nets

• blog: https://adeshpande3.github.io/adeshpande3.github.io/Deep-Learning-Research-Review-Week-1-Generative-Adversarial-Nets

Stability of Generative Adversarial Networks

• blog: http://www.araya.org/archives/1183

Instance Noise: A trick for stabilising GAN training

• blog: http://www.inference.vc/instance-noise-a-trick-for-stabilising-gan-training/

Generating Fine Art in 300 Lines of Code

• intro: DCGAN
• blog: https://medium.com/@richardherbert/generating-fine-art-in-300-lines-of-code-4d37218216a6#.63qm8ef9g

Talks / Videos

Generative Adversarial Network visualization

• youtube: https://www.youtube.com/watch?v=mObnwR-u8pc

Resources

The GAN Zoo

• intro: A list of all named GANs!
• github: https://github.com/hindupuravinash/the-gan-zoo

AdversarialNetsPapers: The classical Papers about adversial nets

• github: https://github.com/zhangqianhui/AdversarialNetsPapers

GAN Timeline

• intro: A timeline showing the development of Generative Adversarial Networks (GAN)
• github: https://github.com//dongb5/GAN-Timeline

Painting

Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting

• intro: ICML 2012
• arxiv: https://arxiv.org/abs/1206.4634

Emoji

Brewing EmojiNet

• blog: http://engineering.curalate.com/2016/01/20/emojinet.html
• website: https://emojini.curalate.com/

Image2Emoji: Zero-shot Emoji Prediction for Visual Media

• paper: http://isis-data.science.uva.nl/cgmsnoek/pub/cappallo-image2emoji-mm2015.pdf

Teaching Robots to Feel: Emoji & Deep Learning 👾 💭 💕

• blog: http://getdango.com/emoji-and-deep-learning.html
• app: https://play.google.com/store/apps/details?id=co.dango.emoji.gif

Text input with relevant emoji sorted with deeplearning

• homepage: http://codepen.io/Idlework/pen/xOgGqM

Sketch

Sketch-a-Net that Beats Humans

• project page: http://www.eecs.qmul.ac.uk/~tmh/downloads.html
• arxiv: http://arxiv.org/abs/1501.07873
• paper: http://www.eecs.qmul.ac.uk/~tmh/papers/yu2015sketchanet.pdf
• code: http://www.eecs.qmul.ac.uk/~tmh/downloads/SketchANet_Code.zip

How Do Humans Sketch Objects?

• project page: http://cybertron.cg.tu-berlin.de/eitz/projects/classifysketch/
• paper: http://cybertron.cg.tu-berlin.de/eitz/pdf/2012_siggraph_classifysketch.pdf
• github: https://github.com/Zebreu/SketchingAI
• gitxiv: http://gitxiv.com/posts/ZBCxEc9g3Fg5xCQ6n/sketchingai

Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup (SIGGRAPH 2016)

• homepage: http://hi.cs.waseda.ac.jp/~esimo/en/research/sketch/
• paper: http://hi.cs.waseda.ac.jp/~esimo/publications/SimoSerraSIGGRAPH2016.pdf

Convolutional Sketch Inversion

• arxiv: http://arxiv.org/abs/1606.03073
• review: https://www.technologyreview.com/s/601684/machine-vision-algorithm-learns-to-transform-hand-drawn-sketches-into-photorealistic-images/
• review: https://techcrunch.com/2016/07/24/researchers-use-neural-networks-to-turn-face-sketches-into-photos/

Sketch Me That Shoe (CVPR 2016)

• project page: http://www.eecs.qmul.ac.uk/~qian/Project_cvpr16.html
• paper: http://www.eecs.qmul.ac.uk/~qian/SketchMeThatShoe.pdf
• github: https://github.com/seuliufeng/DeepSBIR

Mastering Sketching: Adversarial Augmentation for Structured Prediction

• keywords: Sketch Simplification
• project page: http://hi.cs.waseda.ac.jp/~esimo/en/research/sketch_master/
• arxiv: https://arxiv.org/abs/1703.08966
• github: https://github.com/bobbens/sketch_simplification

SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis

• intro: Georgia Institute of Technology
• arxiv: https://arxiv.org/abs/1801.02753

Image Colorization

Deep Colorization

• paper: http://www.cs.cityu.edu.hk/~qiyang/publications/iccv-15.pdf

Learning Large-Scale Automatic Image Colorization

• paper: http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Deshpande_Learning_Large-Scale_Automatic_ICCV_2015_paper.pdf

Learning Representations for Automatic Colorization

• homepage: http://people.cs.uchicago.edu/~larsson/colorization/
• arxiv: http://arxiv.org/abs/1603.06668
• github: https://github.com/gustavla/autocolorize

Colorful Image Colorization

• intro: ECCV 2016
• project page: http://richzhang.github.io/colorization/
• arxiv: http://arxiv.org/abs/1603.08511
• github: https://github.com/richzhang/colorization
• demo: http://demos.algorithmia.com/colorize-photos/
• github: https://github.com/tdeboissiere/DeepLearningImplementations/tree/master/Colorful github(Tensorflow): https://github.com/nilboy/colorization-tf

Colorising Black & White Photos using Deep Learning

https://hackernoon.com/colorising-black-white-photos-using-deep-learning-4da22a05f531

Automatic Colorization (Tensorflow + VGG)

• blog: http://tinyclouds.org/colorize/

colornet: Neural Network to colorize grayscale images

https://github.com/pavelgonchar/colornet

Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification (SIGGRAPH 2016)

• homepage: http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/
• paper: http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/data/colorization_sig2016.pdf
• github(Torch7): https://github.com/satoshiiizuka/siggraph2016_colorization

Convolutional autoencoder to colorize greyscale images

• github: https://github.com/YerevaNN/neural-colorizer

Image-Color: A deep learning approach to colorizing images

• github: https://github.com/cameronfabbri/Colorful-Image-Colorization

Creating an artificial artist: Color your photos using Neural Networks

• blog: https://www.analyticsvidhya.com/blog/2016/11/creating-an-artificial-artist-color-your-photos-using-neural-networks/

Paints Chainer: line drawing colorization using chainer

• github: https://github.com/pfnet/PaintsChainer
• demo: http://paintschainer.preferred.tech/

Unsupervised Diverse Colorization via Generative Adversarial Networks

• arxiv: https://arxiv.org/abs/1702.06674

(DE)^2 CO: Deep Depth Colorization

https://arxiv.org/abs/1703.10881

A Neural Representation of Sketch Drawings

• intro: Google Brain
• arxiv: https://arxiv.org/abs/1704.03477

Real-Time User-Guided Image Colorization with Learned Deep Priors

• intro: SIGGRAPH 2017
• project page: https://richzhang.github.io/ideepcolor/
• arxiv: https://arxiv.org/abs/1705.02999
• github(official, Caffe): https://github.com/junyanz/interactive-deep-colorization

PixColor: Pixel Recursive Colorization

• intro: Google Research
• arxiv: https://arxiv.org/abs/1705.07208

cGAN-based Manga Colorization Using a Single Training Image

• intro: University of Tokyo
• arxiv: https://arxiv.org/abs/1706.06918

Interactive Deep Colorization With Simultaneous Global and Local Inputs

https://arxiv.org/abs/1801.09083

Image Colorization with Generative Adversarial Networks

https://arxiv.org/abs/1803.05400

Learning to Color from Language

• intro: Allen Institute of Artificial Intelligence & University of Massachusetts
• arxiv: https://arxiv.org/abs/1804.06026

Deep Exemplar-based Colorization

• intro: Siggraph 2018
• arxiv: https://arxiv.org/abs/1807.06587

Pixel-level Semantics Guided Image Colorization

https://arxiv.org/abs/1808.01597

User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks

• intro: 2018 ACM Multimedia Conference (MM ‘18)
• arxiv: https://arxiv.org/abs/1808.03240

Pixelated Semantic Colorization

https://arxiv.org/abs/1901.10889

Colorization Transformer

• intro: ICLR 2021
• intro: Google Research, Brain Team
• arxiv: https://arxiv.org/abs/2102.04432
• openreview: https://openreview.net/forum?id=5NA1PinlGFu
• github: https://github.com/google-research/google-research/tree/master/coltran

Sounds

Visually Indicated Sounds

• project page: http://vis.csail.mit.edu/
• arxiv: http://arxiv.org/abs/1512.08512

Music

GRUV: Algorithmic Music Generation using Recurrent Neural Networks

• github: https://github.com/MattVitelli/GRUV

DeepHear - Composing and harmonizing music with neural networks

• website: http://web.mit.edu/felixsun/www/neural-music.html
• github: https://github.com/fephsun/neuralnetmusic

Using AutoHarp and a Character-Based RNN to Create MIDI Drum Loops

• website: http://www.inquisitivists.com/2015/09/16/using-autoharp-and-a-character-based-rnn-to-create-midi-drum-loops

Musical Audio Synthesis Using Autoencoding Neural Nets

• paper: http://www.cs.dartmouth.edu/~sarroff/papers/sarroff2014a.pdf
• github: https://github.com/woodshop/deepAutoController/tree/icmc_smc_2014
• video: https://vimeo.com/121827215

sound-rnn: Generating sound using recurrent neural networks

• github: https://github.com/johnglover/sound-rnn
• blog: http://www.johnglover.net/blog/generating-sound-with-rnns.html

Using LSTM Recurrent Neural Networks for Music Generation (Project for AI Prac Fall 2015 at Cornell)

• youtube: https://www.youtube.com/watch?v=aSr8_QQYpYM
• video: http://video.weibo.com/show?fid=1034:4be01d679bb1a68a634fe0f589caa779

Visually Indicated Sounds (MIT. 2015)

• arxiv: http://arxiv.org/abs/1512.08512

Training a Recurrent Neural Network to Compose Music

• blog: https://maraoz.com/2016/02/02/abc-rnn/

LSTM Realbook

• blog: https://keunwoochoi.wordpress.com/2016/02/19/lstm-realbook/
• github: https://github.com/keunwoochoi/lstm_real_book

LSTMetallica: Generation drum tracks by learning the drum tracks of 60 Metallica songs

• blog: https://keunwoochoi.wordpress.com/2016/02/23/lstmetallica/

deepjazz: Deep learning driven jazz generation using Keras & Theano!

• homepage: https://jisungk.github.io/deepjazz/
• github：https://github.com/jisungk/deepjazz

Magenta: Music and Art Generation with Machine Intelligence

• homepage: http://magenta.tensorflow.org/
• github: https://github.com/tensorflow/magenta

Music Transcription with Convolutional Neural Networks

• blog: https://www.lunaverus.com/cnn
• download: https://www.lunaverus.com/download

Long Short-Term Memory Recurrent Neural Network Architectures for Generating Music and Japanese Lyrics

• paper: http://cslab1.bc.edu/~csacademics/pdf/16Mikami.pdf
• github: https://github.com/mikamia

BachBot: Use deep learning to generate and harmonize music in the style of Bach

• intro: BachBot is a research project utilizing long short term memory (LSTMs) to generate Bach compositions
• homepage: http://bachbot.com/
• github: https://github.com/feynmanliang/bachbot

Generate Music in TensorFlow

• youtube: https://www.youtube.com/watch?v=ZE7qWXX05T0
• github: https://github.com/llSourcell/Music_Generator_Demo

Generate new lyrics in the style of any artist using LSTMs and TensorFlow

• github: https://github.com/dyelax/encore.ai

sound-GAN: Generative Adversial Network for music composition

• github: https://github.com/jacotar/sound-GAN

Analyzing Six Deep Learning Tools for Music Generation

• intro: Magenta / DeepJazz / BachBot / FlowMachines / WaveNet / GRUV
• blog: http://www.asimovinstitute.org/notes-vs-waves/

WIMP2: Creating Music with AI: Highlights of Current Research

• youtube: https://www.youtube.com/watch?v=5K4hn6cBUPU

Song From PI: A Musically Plausible Network for Pop Music Generation

• paper: http://openreview.net/pdf?id=ByBwSPcex
• project page: http://www.cs.toronto.edu/songfrompi/

Grammar Argumented LSTM Neural Networks with Note-Level Encoding for Music Composition

• arxiv: https://arxiv.org/abs/1611.05416

用TensorFlow生成周杰伦歌词

• blog: http://leix.me/2016/11/28/tensorflow-lyrics-generation/
• github: https://github.com/leido/char-rnn-cn

Hip-Hop - Generating lyrics with RNNs

• blog: http://affinelayer.com/hiphop/index.html

Metis Final Project: Music Composition with LSTMs

http://blog.naoya.io/metis-final-project-music-composition-with-lstms/

Neural Translation of Musical Style

• blog: http://imanmalik.com/cs/2017/06/05/neural-style.html
• github: https://github.com/imalikshake/StyleNet

Poetry

NeuralSnap: Generates poetry from images using convolutional and recurrent neural networks

• github: https://github.com/rossgoodwin/neuralsnap

Generating Chinese Classical Poems with RNN Encoder-Decoder

• arxiv: http://arxiv.org/abs/1604.01537

Chinese Poetry Generation with Planning based Neural Network

• intro: COLING 2016. University of Science and Technology of China & Baidu
• arxiv: https://arxiv.org/abs/1610.09889
• blog: http://freecoder.me/archives/213.html

Weiqi (Go)

Teaching Deep Convolutional Neural Networks to Play Go

• arxiv: http://arxiv.org/abs/1412.3409
• demo: https://chrisc36.github.io/deep-go/

Move Evaluation in Go Using Deep Convolutional Neural Networks(Google DeepMind, Google Brain)

• arxiv: http://arxiv.org/abs/1412.6564

Training Deep Convolutional Neural Networks to Play Go

• paper: http://jmlr.org/proceedings/papers/v37/clark15.pdf

Computer Go Research - The Challenges Ahead (Martin Müller. IEEE CIG 2015)

• homepage: https://webdocs.cs.ualberta.ca/~mmueller/talks.html

GoCNN: Using CNN for Go (Weiqi/Baduk) board evaluation with tensorflow

• github: https://github.com/jmgilmer/GoCNN

DarkGo: Go in Darknet

• homepage: http://pjreddie.com/darknet/darkgo-go-in-darknet/

BetaGo: Go bots for the people

• homepage: http://maxpumperla.github.io/betago/
• github: https://github.com/maxpumperla/betago

Deep Learning and the Game of Go

• book: https://www.manning.com/books/deep-learning-and-the-game-of-go
• github: https://github.com//maxpumperla/deep_learning_and_the_game_of_go

DarkForest

Better Computer Go Player with Neural Network and Long-term Prediction (Facebook AI Research)

• arxiv: http://arxiv.org/abs/1511.06410
• github: https://github.com/facebookresearch/darkforestGo
• MIT tech review: http://www.technologyreview.com/view/544181/how-facebooks-ai-researchers-built-a-game-changing-go-engine/

AlphaGo

Mastering the game of Go with deep neural networks and tree search

• intro: AlphaGo. Google DeepMind
• homepage: http://www.deepmind.com/alpha-go.html
• paper: https://storage.googleapis.com/deepmind-data/assets/papers/deepmind-mastering-go.pdf
• naturep page: http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html
• paper: https://gogameguru.com/i/2016/03/deepmind-mastering-go.pdf
• slides: http://www.bioinfo.org.cn/~casp/temp/alphago_slides.pdf
• blog: http://www.furidamu.org/blog/2016/01/26/mastering-the-game-of-go-with-deep-neural-networks-and-tree-search/
• blog(“AlphaGo: From Intuitive Learning to Holistic Knowledge”): https://caminao.wordpress.com/2016/02/01/alphago/
• github: https://github.com/Rochester-NRT/AlphaGo

AlphaGo Teach

• intro: Let the AlphaGo Teaching Tool help you find new and creative ways of playing Go
• homepage: https://alphagoteach.deepmind.com/

AlphaGo的分析

• intro: by 田渊栋
• blog: http://zhuanlan.zhihu.com/yuandong/20607684

How Alphago Works

• slides: http://www.slideshare.net/ShaneSeungwhanMoon/how-alphago-works
• slides: http://pan.baidu.com/s/1qXwagGW

AlphaGo in Depth

• intro: by Mark Chang
• slides: http://www.slideshare.net/ckmarkohchang/alphago-in-depth?qid=283ab3bc-7d04-4e14-a205-b0b671ca4099
• mirror: https://pan.baidu.com/s/1i5JNeRj

Leela

• intro: Leela is a strong Go playing program combining advances in Go programming and further original research into a small, easy to use graphical interface.
• homepage: https://sjeng.org/leela.html

Mastering the game of Go without human knowledge

• nature page: http://www.nature.com/nature/journal/v550/n7676/full/nature24270.html
• paper: https://deepmind.com/documents/119/agz_unformatted_nature.pdf
• notes: https://blog.acolyer.org/2017/11/17/mastering-the-game-of-go-without-human-knowledge/

Computer Go & AlphaGo Zero

• youtube: https://www.youtube.com/watch?v=6fKG4wJ7uBk
• mirror: https://www.bilibili.com/video/av16428694/
• slides: https://drive.google.com/file/d/1rmUyIitEmAtMUKdKEnlHRfmXtpyoxxey/view

AlphaZero: Mastering Games without Human Knowledge - NIPS 2017

• intro: Keynote by David Silver on AlphaGo, AlphaGo Zero and AlphaZero, at the 2017 NIPS Deep Reinforcement Learning Symposium, 6 Dec, Long Beach, CA
• youtube: https://www.youtube.com/watch?v=A3ekFcZ3KNw
• mirror: https://www.bilibili.com/video/av17210816/

PhoenixGo

• intro: Go AI program which implement the AlphaGo Zero paper
• github: https://github.com/Tencent/PhoenixGo

The future is here – AlphaZero learns chess

https://en.chessbase.com/post/the-future-is-here-alphazero-learns-chess

AlphaGo Zero Cheat Sheet

https://applied-data.science/static/main/res/alpha_go_zero_cheat_sheet.png

Chess

Giraffe: Using Deep Reinforcement Learning to Play Chess

• intro: MSc thesis
• arxiv: http://arxiv.org/abs/1509.01549

Spawkfish: neural network based chess engine

• homepage: http://spawk.fish/

Chess position evaluation with convolutional neural network in Julia

• blog: http://int8.io/chess-position-evaluation-with-convolutional-neural-networks-in-julia/

Deep Learning for … Chess

• blog: http://blog.yhat.com/posts/deep-learning-chess.html
• github: https://github.com/erikbern/deep-pink

DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess

• intro: Winner of Best Paper Award in ICANN 2016
• arxiv: https://arxiv.org/abs/1711.09667
• paper: http://www.cs.tau.ac.il/~wolf/papers/deepchess.pdf
• github: https://github.com/mr-press/DeepChess

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

• intro: DeepMind
• arxiv: https://arxiv.org/abs/1712.01815

Game

Learning Game of Life with a Convolutional Neural Network

• blog: http://danielrapp.github.io/cnn-gol/
• github: https://github.com/DanielRapp/cnn-gol

Reinforcement Learning using Tensor Flow: A deep Q learning demonstration using Google Tensorflow

• github: https://github.com/nivwusquorum/tensorflow-deepq

Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games Using Convolutional Networks

• arxiv: http://arxiv.org/abs/1509.06731
• paper: http://colinraffel.com/publications/aaai2016poker.pdf
• github: https://github.com/moscow25/deep_draw
• slides: https://drive.google.com/file/d/0B5eOIUHA0khiMjN1YnEtZHMwams/view
• slides: http://pan.baidu.com/s/1nu5zpZ7

TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games

• intro: Connecting Torch to StarCraft
• arxiv: https://arxiv.org/abs/1611.00625
• github: https://github.com/TorchCraft/TorchCraft

BlizzCon 2016 DeepMind and StarCraft II Deep Learning Panel Transcript

• part 1: http://starcraft.blizzplanet.com/blog/comments/blizzcon-2016-deepmind-and-starcraft-ii-deep-learning-panel-transcript
• part 2: http://starcraft.blizzplanet.com/blog/comments/blizzcon-2016-deepmind-and-starcraft-ii-deep-learning-panel-transcript/2

DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

• arxiv: https://arxiv.org/abs/1701.01724
• github: https://github.com/lifrordi/DeepStack-Leduc

Gym StarCraft: StarCraft environment for OpenAI Gym, based on Facebook’s TorchCraft

• intro: Gym StarCraft is an environment bundle for OpenAI Gym. It is based on Facebook’s TorchCraft, which is a bridge between Torch and StarCraft for AI research.
• github: https://github.com/deepcraft/gym-starcraft

Multiagent Bidirectionally-Coordinated Nets for Learning to Play StarCraft Combat Games

https://arxiv.org/abs/1703.10069

Learning Macromanagement in StarCraft from Replays using Deep Learning

• intro: CIG 2017. IT University of Copenhagen
• arxiv: https://arxiv.org/abs/1707.03743

Multi-platform Version of StarCraft: Brood War in a Docker Container: Technical Report

• intro: Czech Technical University in Prague
• arxiv: https://arxiv.org/abs/1801.02193
• gihtub: https://github.com/Games-and-Simulations/sc-docker

Macro action selection with deep reinforcement learning in StarCraft

• intro: Bilibili & Nanjing University
• arxiv: https://arxiv.org/abs/1812.00336
• github: https://github.com/Bilibili/LastOrder

DeepLeague

DeepLeague: leveraging computer vision and deep learning on the League of Legends mini map + giving away a dataset of over 100,000 labeled images to further esports analytics research

• blog: https://medium.com/@farzatv/deepleague-leveraging-computer-vision-and-deep-learning-on-the-league-of-legends-mini-map-giving-d275fd17c4e0

DeepLeague (Part 2): The Technical Details

• blog: https://medium.com/@farzatv/deepleague-part-2-the-technical-details-374439e7e09a
• github: https://github.com/farzaa/DeepLeague

Courses

Learning Machines

http://www.patrickhebron.com/learning-machines/

Learning Bit by Bit

https://itp.nyu.edu/varwiki/Syllabus/LearningBitbyBitS10

MACHINE LEARNING FOR MUSICIANS AND ARTISTS (Course opens January 2016)

https://www.kadenze.com/courses/machine-learning-for-musicians-and-artists/info

Machine learning for artists @ ITP-NYU, Spring 2016

• videos/lectures/course notes: http://ml4a.github.io/classes/itp-S16/
• index: http://ml4a.github.io/index/
• github: https://github.com/ml4a/ml4a.github.io
• notes: http://www.kdnuggets.com/2016/04/machine-learning-artists-video-lectures-notes.html
• blog: https://medium.com/@genekogan/machine-learning-for-artists-e93d20fdb097#.25w95beqb

Machine Learning for Artists @ OpenDot, November 2016

• homepage: http://ml4a.github.io/classes/opendot/

The Neural Aesthetic @ SchoolOfMa, Summer 2016

http://ml4a.github.io/classes/neural-aesthetic/

Blogs

Review of machine / deep learning in an artistic context

https://medium.com/@memoakten/machine-deep-learning-in-an-artistic-context-441f28774bcc#.gegpq99ag

Apprentice Work

https://www.technologyreview.com/s/600762/apprentice-work/

Exploring the Intersection of Art and Machine Intelligence

http://googleresearch.blogspot.jp/2016/02/exploring-intersection-of-art-and.html

Using machine learning to generate music

http://www.datasciencecentral.com/profiles/blogs/using-machine-learning-to-generate-music

art in the age of machine intelligence

https://medium.com/artists-and-machine-intelligence/what-is-ami-ccd936394a83#.hyt4ei9a9

Understanding Aesthetics with Deep Learning

https://devblogs.nvidia.com/parallelforall/understanding-aesthetics-deep-learning/

Go, Marvin Minsky, and the Chasm that AI Hasn’t Yet Crossed

blog: https://medium.com/backchannel/has-deepmind-really-passed-go-adc85e256bec#.inx8nfid0

A Return to Machine Learning

• intro: This post is aimed at artists and other creative people who are interested in a survey of recent developments in machine learning research that intersect with art and culture.
• blog: https://medium.com/@kcimc/a-return-to-machine-learning-2de3728558eb#.bp2b1ax2x

Resources

Music, Art and Machine Intelligence Workshop 2016

• youtube: https://www.youtube.com/playlist?list=PL8h7Hk91ScJ2NKFS_i8y4DYk4Zg1LHn0a

Papers

Bayesian

Tutorials

Deep learning

• intro: From Wikipedia, the free encyclopedia
• blog: https://www.wikiwand.com/en/Deep_learning

Toward Theoretical Understanding of Deep Learning

• intro: ICML 2018 Tutorial. by Sanjeev Arora, Princeton University
• slides: https://www.dropbox.com/s/qonozmne0x4x2r3/deepsurveyICML18final.pptx?dl=0
• mirror: https://pan.baidu.com/s/1r_lz6rMoSIinvfovMFFbug

VGG Convolutional Neural Networks Practical

• homepage: http://www.robots.ox.ac.uk/~vgg/practicals/cnn/index.html
• github: https://github.com/vedaldi/practical-cnn

Hacker’s guide to Neural Networks

http://karpathy.github.io/neuralnets/

Deep Learning Tutorials

• website: http://deeplearning.net/tutorial/
• code: https://github.com/lisa-lab/DeepLearningTutorials

Deep Learning in a Nutshell: Core Concepts

http://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/

Deep Learning in a Nutshell: History and Training

http://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-history-training/

A Deep Learning Tutorial: From Perceptrons to Deep Networks

http://www.toptal.com/machine-learning/an-introduction-to-deep-learning-from-perceptrons-to-deep-networks

Deep Neural Networks (with Python code)

• paper: http://scholarbank.nus.edu.sg/bitstream/handle/10635/120564/DeepNeuralNetworks.pdf?sequence=1

Three Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey

• paper: http://research.microsoft.com/pubs/192937/Transactions-APSIPA.pdf

Stanford Unsupervised Feature Learning and Deep Learning Tutorial: UFLDL Tutorial

• homepage: http://ufldl.stanford.edu/tutorial/
• programming exercises: https://github.com/amaas/stanford_dl_ex

The Unreasonable Effectiveness of Deep Learning (LeCun)

• slides: http://www.ee.ucl.ac.uk/sahd2014/resources/LeCun.pdf

Deep learning from the bottom up

• blog: https://www.metacademy.org/roadmaps/rgrosse/deep_learning

Introduction to Deep Learning with Python (By Alec Radford. Theano)

• youtube: https://www.youtube.com/watch?v=S75EdAcXHKk&hd=1

New to deep learning? Here are 4 easy lessons from Google

• blog: https://gigaom.com/2015/01/29/new-to-deep-learning-here-are-4-easy-lessons-from-google/

Deep Learning 101

• blog: http://markus.com/deep-learning-101/

Neural Networks Demystified

• Part 1: Data and Architecture: https://www.youtube.com/watch?v=bxe2T-V8XRs
• Part 2: Forward Propagation: https://www.youtube.com/watch?v=UJwK6jAStmg
• Part 3: Gradient Descent: https://www.youtube.com/watch?v=5u0jaA3qAGk
• Part 4: Backpropagation: https://www.youtube.com/watch?v=GlcnxUlrtek
• Part 5: Numerical Gradient Checking: https://www.youtube.com/watch?v=pHMzNW8Agq4
• Part 6: Training: https://www.youtube.com/watch?v=9KM9Td6RVgQ

• Part 7: Overfitting, Testing, and Regularization: https://www.youtube.com/watch?v=S4ZUwgesjS8

• all-pack: http://pan.baidu.com/s/1dDq5oNB

Deep Learning SIMPLIFIED

• playlist: https://www.youtube.com/playlist?list=PLjJh1vlSEYgvGod9wWiydumYl8hOXixNu

A ‘Brief’ History of Neural Nets and Deep Learning

• part 1: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning/
• part 2: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning-part-2/
• part 3: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning-part-3/
• part 4: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning-part-4/

Deep Neural Networks — An Overview

https://medium.com/@asjad/deep-neural-networks-an-overview-480112b12a13#.i7apzmnso

A Tutorial on Deep Neural Networks for Intelligent Systems

• arxiv: http://arxiv.org/abs/1603.07249

Deep Learning for Computer Vision – Introduction to Convolution Neural Networks

• blog: http://www.analyticsvidhya.com/blog/2016/04/deep-learning-computer-vision-introduction-convolution-neural-networks/

BI Lab Deep Learning Tutorial

• github: https://github.com/bi-lab/deeplearning_tutorial

Deep Learning Tutorials

• github: https://github.com/sjchoi86/Deep-Learning-101

Neural Network Architectures

• blog: http://culurciello.github.io/tech/2016/06/04/nets.html

A Practical Introduction to Deep Learning with Caffe and Python

• blog: http://adilmoujahid.com/posts/2016/06/introduction-deep-learning-python-caffe/

Notes on Convolutional Neural Networks

• homepage: http://cogprints.org/5869/
• paper: http://cogprints.org/5869/1/cnn_tutorial.pdf

Feed Forward and Backward Run in Deep Convolution Neural Network

• intro: 20th International Conference on Computer Vision and Image Processing
• arxiv: https://arxiv.org/abs/1711.03278

Convolutional Networks

http://deeplearning4j.org/convolutionalnets.html

Exploring convolutional neural networks with DL4J

• blog: http://brooksandrew.github.io/simpleblog/articles/convolutional-neural-network-training-with-dl4j/

Understanding Convolutional Neural Networks

• arxiv: http://arxiv.org/abs/1605.09081

Laws, Sausages and ConvNets

• blog: http://www.trivialorwrong.com/2016/06/01/laws-sausages-and-convnets.html

Convolutional Neural Networks (CNNs): An Illustrated Explanation

• blog: http://xrds.acm.org/blog/2016/06/convolutional-neural-networks-cnns-illustrated-explanation/

intro_deep: Introduction tutorials to deep learning with Theano and OpenDeep

• slides: https://docs.google.com/presentation/d/1cg9Tn2wWwqJmaSSDnlBDBEETD5SyV6TJSD8qiDJFgEM
• mirror: http://pan.baidu.com/s/1hqIR0yC
• youtube: https://www.youtube.com/watch?v=afUvcD3tEoQ
• mirror: http://pan.baidu.com/s/1qWHp7xa
• github: https://github.com/mbeissinger/intro_deep

Deep Learning on Java by Breandan Considine

• video: https://speakerdeck.com/breandan/deep-learning-on-java

Using Convolutional Neural Networks and TensorFlow for Image Classification (NYC TensorFlow meetup)

• blog: http://blog.altoros.com/using-convolutional-neural-networks-and-tensorflow-for-image-classification-and-search.html

Neural networks with Theano and Lasagne

• youtube: https://www.youtube.com/watch?v=dtGhSE1PFh0
• mirror: http://pan.baidu.com/s/1kUl3PvL
• github: https://github.com/ebenolson/pydata2015

Introduction to Deep Learning

• github: https://github.com/rouseguy/intro2deeplearning
• slides: https://speakerdeck.com/bargava/introduction-to-deep-learning

Introduction to Deep Learning for Image Recognition - SciPy US 2016

• github: https://github.com/rouseguy/scipyUS2016_dl-image
• slides: https://speakerdeck.com/bargava/introduction-to-deep-learning-for-image-processing

Deep learning tutorials (2nd ed.)

• github: https://github.com/sjchoi86/dl_tutorials

A Beginner’s Guide To Understanding Convolutional Neural Networks

• part 1: https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner’s-Guide-To-Understanding-Convolutional-Neural-Networks/

A Beginner’s Guide To Understanding Convolutional Neural Networks Part 2

• part 2: https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner’s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/

The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)

• part 3: https://adeshpande3.github.io/adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html

Deep Learning Part 1: Comparison of Symbolic Deep Learning Frameworks

• blog: http://blog.revolutionanalytics.com/2016/08/deep-learning-part-1.html

Deep Learning Part 2: Transfer Learning and Fine-tuning Deep Convolutional Neural Networks

• blog: http://blog.revolutionanalytics.com/2016/08/deep-learning-part-2.html

Deep Learning Part 3: Combining Deep Convolutional Neural Network with Recurrent Neural Network

• blog: http://blog.revolutionanalytics.com/2016/09/deep-learning-part-3.html

Introduction to Deep Learning for Image Processing

• slides: https://speakerdeck.com/bargava/introduction-to-deep-learning-for-image-processing

The best explanation of Convolutional Neural Networks on the Internet!

• blog: https://medium.com/technologymadeeasy/the-best-explanation-of-convolutional-neural-networks-on-the-internet-fbb8b1ad5df8

The Evolution and Core Concepts of Deep Learning & Neural Networks

• blog: https://www.analyticsvidhya.com/blog/2016/08/evolution-core-concepts-deep-learning-neural-networks/

An Intuitive Explanation of Convolutional Neural Networks

• blog: https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/

How Convolutional Neural Networks Work

• blog: http://brohrer.github.io/how_convolutional_neural_networks_work.html

Preliminary Note on the Complexity of a Neural Network

• blog: http://r2rt.com/preliminary-note-on-the-complexity-of-a-neural-network.html

Deep Learning Tutorial

• intro: Hung-yi Lee. 李宏毅
• slides: http://www.slideshare.net/tw_dsconf/ss-62245351?qid=c0f0f97a-6ca8-4df0-97e2-984452215ee7&v=&b=&from_search=1
• mirror: https://pan.baidu.com/s/1mhMhuFQ

Jupyter notebooks and code for Intro to DL talk at Genesys

• blog: http://sujitpal.blogspot.com/2016/08/kerasjupyter-notebooks-for-my.html
• github: https://github.com/sujitpal/intro-dl-talk-code

Learn Deep Learning the Hard Way

• blog: https://medium.com/artifacia/learn-deep-learning-the-hard-way-e5d844f9fbc1#.yitf25xg5

A Complete Guide on Getting Started with Deep Learning in Python

• blog: https://www.analyticsvidhya.com/blog/2016/08/deep-learning-path/

Deep learning for complete beginners: Recognising handwritten digits

• blog: http://online.cambridgecoding.com/notebooks/cca_admin/deep-learning-for-complete-beginners-recognising-handwritten-digits

Deep learning for complete beginners: Using convolutional nets to recognise images

• blog: http://online.cambridgecoding.com/notebooks/cca_admin/convolutional-neural-networks-with-keras

Deep learning for complete beginners: neural network fine-tuning techniques

• blog: http://online.cambridgecoding.com/notebooks/cca_admin/neural-networks-tuning-techniques

How do Convolutional Neural Networks work?

http://brohrer.github.io/how_convolutional_neural_networks_work.html

Creating a Neural Network That Can Tell if a Name Is Male or Female, in JavaScript

• blog: https://medium.com/@hathibel/creating-a-neural-network-that-can-tell-if-a-name-is-male-or-female-in-javascript-3061029be396#.89l3znc2l

Softmax Classifiers Explained

• blog: http://www.pyimagesearch.com/2016/09/12/softmax-classifiers-explained/

The Softmax function and its derivative

• blog: http://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative/

How an algorithm behind Deep Learning works

• video: http://blog.revolutionanalytics.com/2016/09/how-the-algorithm-behind-deep-learning-works.html
• slides: https://github.com/brohrer/public-hosting/raw/master/How_CNNs_work.pdf
• blog: http://www.kdnuggets.com/2016/08/brohrer-convolutional-neural-networks-explanation.html
• mirror: http://v.youku.com/v_show/id_XMTcyNTgwNDQyOA==.html

The Neural Network Zoo

http://www.asimovinstitute.org/neural-network-zoo/

Recognising Beer with TensorFlow

• blog: https://medium.com/@chrishawkins/recognising-beer-with-tensorflow-9dedfee3c3c0#.pn5gm3fgc
• gist: https://gist.github.com/chrishawkins/177e37756c833768a21d446cc4921c6e

Deep learning architecture diagrams

• intro: LSTM diagrams
• blog: http://fastml.com/deep-learning-architecture-diagrams/

Getting Started with Deep Learning and Python

• blog: http://www.pyimagesearch.com/2014/09/22/getting-started-deep-learning-python

Deep Learning Practicals

• intro: Video playlist of Torch Video Tutorials
• youtube: https://www.youtube.com/playlist?list=PLLHTzKZzVU9ebuL6DCclzI54MrPNFGqbW
• mirror: https://pan.baidu.com/s/1skMFGkt

A simple workflow for deep learning

• blog: https://cartesianfaith.com/2016/09/29/a-simple-workflow-for-deep-learning/
• github: https://github.com/zatonovo/deep_learning_ex

A primer on universal function approximation with deep learning (in Torch and R)

• blog: https://cartesianfaith.com/2016/09/23/a-primer-on-universal-function-approximation-with-deep-learning-in-torch-and-r/

An Introduction to Implementing Neural Networks using TensorFlow

https://www.analyticsvidhya.com/blog/2016/10/an-introduction-to-implementing-neural-networks-using-tensorflow/

A Gentle Introduction to Convolutional Neural Networks

• blog: https://miguelgfierro.com/blog/2016/a-gentle-introduction-to-convolutional-neural-networks/

Beginning Machine Learning with Keras and TensorFlow

• blog: http://blog.thoughtram.io/machine-learning/2016/09/23/beginning-ml-with-keras-and-tensorflow.html

Shortest Way to Deep Learning

• blog: https://blog.asadmemon.com/shortest-way-to-deep-learning-41e704d65ef#.19iowho8t

Deep learning with Matlab

• intro: Covered topics of the presentation: Machine learning workflow, Extracting feaures from images (colours, edges, corners, etc.)
• youtube: https://www.youtube.com/watch?v=r4D3NxQ0Xhg

Convolutional neural networks for computer vision with Matlab

• youtube: https://www.youtube.com/watch?v=MwUic9FCBJI

Neural Net Computing Explodes

• blog: http://semiengineering.com/neural-net-computing-explodes/

Tutorial: Optimizing Neural Networks using Keras (with Image recognition case study)

https://www.analyticsvidhya.com/blog/2016/10/tutorial-optimizing-neural-networks-using-keras-with-image-recognition-case-study/

15 Deep Learning Tutorials

• blog: http://www.datasciencecentral.com/profiles/blogs/15-deep-learning-tutorials

Deep Learning Episode 1: Optimizing DeepMind’s A3C on Torch

http://www.allinea.com/blog/201607/deep-learning-episode-1-optimizing-deepminds-a3c-torch

Deep Learning Episode 2: Scaling TensorFlow over multiple EC2 GPU nodes

http://www.allinea.com/blog/201608/deep-learning-episode-2-scaling-tensorflow-over-multiple-ec2-gpu-nodes

Deep Learning Episode 3: Supercomputer vs Pong

http://www.allinea.com/blog/201610/deep-learning-episode-3-supercomputer-vs-pong

Deep Learning Episode 4: Supercomputer vs Pong II

http://www.allinea.com/blog/201610/deep-learning-episode-4-supercomputer-vs-pong-ii

Nuts and Bolts of Applying Deep Learning — Summary

• blog: https://medium.com/@aniketvartak/nuts-and-bolts-of-applying-deep-learning-summary-84b8a8e873d5#.y7xidlhb4

Intro to Deep Learning for Computer Vision

http://chaosmail.github.io/deeplearning/2016/10/22/intro-to-deep-learning-for-computer-vision/

If I Can Learn to Play Atari, I Can Learn TensorFlow

• intro: Here is a summary of new deep learning libraries, tools, and updates to existing frameworks.
• blog: https://dzone.com/articles/deep-learning-resources

TensorFlow workshop materials

• github: https://github.com/amygdala/tensorflow-workshop

Some theorems on deep learning

• intro: Tomaso Poggio [MIT]
• youtube: https://www.youtube.com/watch?v=YVjvRvvVn4w
• mirror: https://pan.baidu.com/s/1o8o7LjW

Pokemon, Colors, and Deep Learning

• blog: https://juandes.com/pokemon-colors-and-deep-learning-95fb715be46
• github: https://github.com/juandes/PokemonTypesDeepLearning

Why Deep Learning is Radically Different from Machine Learning

• blog: https://medium.com/intuitionmachine/why-deep-learning-is-radically-different-from-machine-learning-945a4a65da4d#.yuqxqxpsz

Deep Learning: The Unreasonable Effectiveness of Randomness

• blog: https://medium.com/intuitionmachine/deep-learning-the-unreasonable-effectiveness-of-randomness-14d5aef13f87#.1n775yl4i

Deep Meta-Learning : Machines now Bootstrap Themselves

• blog: https://medium.com/intuitionmachine/deep-learning-can-now-create-itself-92e7ff0d59a7#.ml0dy8m9a

Are Deep Neural Networks Creative?

• blog: http://www.kdnuggets.com/2016/05/deep-neural-networks-creative-deep-learning-art.html

Are Deep Neural Networks Creative? v2

• blog: http://approximatelycorrect.com/2016/11/11/are-deep-neural-networks-creative/

Develop/Train A Convolutional Neural Netwok For MNIST Dataset

• github: https://github.com/mirjalil/DataScience/blob/master/notebooks/deeplearning/tensorflow_03_CNN.ipynb

Rethinking Generalization in Deep Learning

https://medium.com/intuitionmachine/rethinking-generalization-in-deep-learning-ec66ed684ace#.tcnsqik5w

The hard thing about deep learning

• blog: https://www.oreilly.com/ideas/the-hard-thing-about-deep-learning

The hard thing about deep learning

• blog: https://www.oreilly.com/ideas/the-hard-thing-about-deep-learning

Introduction to Autoencoders

• blog: https://pgaleone.eu/neural-networks/2016/11/18/introduction-to-autoencoders/

Two Days to a Demo

• blog: https://developer.nvidia.com/embedded/twodaystoademo

Deep Learning Tutorials for 10 Weeks

• github: https://github.com/sjchoi86/dl_tutorials_4th

Deep Learning in Clojure With Cortex

• blog: http://gigasquidsoftware.com/blog/2016/12/27/deep-learning-in-clojure-with-cortex/

A Guide to Deep Learning by YerevaNN

• blog: http://yerevann.com/a-guide-to-deep-learning/

Learning to Learn, to Program, to Explore and to Seek Knowledge

• intro: Nando de Freitas, NIPS 2016
• youtube: https://www.youtube.com/watch?v=tPWGGwmgwG0
• mirror: https://pan.baidu.com/s/1b2VZsE

Have Fun with Machine Learning: A Guide for Beginners

• intro: An absolute beginner’s guide to Machine Learning and Image Classification with Neural Networks
• github: https://github.com/humphd/have-fun-with-machine-learning

Deep Learning Cheat Sheet

• blog: https://hackernoon.com/deep-learning-cheat-sheet-25421411e460#.e0jwzxpkm

How to train your Deep Neural Network

http://rishy.github.io//ml/2017/01/05/how-to-train-your-dnn/

A deep learning traffic light detector using dlib and a few images from Google street view

• blog: https://sagivtech.com/2016/11/10/post-2/

Recognizing Traffic Lights With Deep Learning

• blog: https://medium.com/@davidbrai/recognizing-traffic-lights-with-deep-learning-23dae23287cc#.k22tnf37a
• github: https://github.com/davidbrai/deep-learning-traffic-lights

Tutorials for deep learning

• github: https://github.com/oduerr/dl_tutorial

The Holographic Principle: Why Deep Learning Works

https://medium.com/intuitionmachine/the-holographic-principle-and-deep-learning-52c2d6da8d9

Deep Neural Networks - A Brief History

• arxiv: https://arxiv.org/abs/1701.05549

Fundamental Deep Learning code in TFLearn, Keras, Theano and TensorFlow

• blog: https://insights.untapt.com/fundamental-deep-learning-code-in-tflearn-keras-theano-and-tensorflow-66be10a03227#.hoaw8fp9p
• slides: https://static1.squarespace.com/static/5362fa11e4b035b5651b7f7e/t/588fb378cd0f687201a2e317/1485812622873/Jon_Krohn_NYHackR_Deep_Learning_2017_01_30.pdf

Deep Neural Network from scratch

https://matrices.io/deep-neural-network-from-scratch/

Convolutional Neural Networks

https://github.com/Alfredvc/cnn_workshop

Exploring Optimizers

https://github.com//KeremTurgutlu/deeplearning/blob/master/Exploring%20Optimizers.ipynb

A Gentle Introduction to Exploding Gradients in Neural Networks

https://machinelearningmastery.com/exploding-gradients-in-neural-networks/

Only Numpy: (Why I do Manual Back Propagation) Implementing Multi Channel/Layer Convolution Neural Network on Numpy with Interactive Code

https://medium.com/swlh/only-numpy-why-i-do-manual-back-propagation-implementing-multi-channel-layer-convolution-neural-7d83242fcc24

92.45% on CIFAR-10 in Torch

• intro: Dropout after Convolution
• blog: http://torch.ch/blog/2015/07/30/cifar.html

Convolution

Understanding Convolutions

• blog: http://colah.github.io/posts/2014-07-Understanding-Convolutions/

Note on the implementation of a convolutional neural networks

• intro: CS231n, Convolutional layer, Pooling layer, Forward pass, Backward pass
• blog: http://cthorey.github.io./backprop_conv/
• github: https://github.com/cthorey/CS231

Convolution in Caffe: a memo

• blog: https://github.com/Yangqing/caffe/wiki/Convolution-in-Caffe:-a-memo

我对卷积的理解

• blog: http://mengqi92.github.io/2015/10/06/convolution/
• blog: https://segmentfault.com/a/1190000004706582

An Analysis of Convolution for Inference

http://www.slideshare.net/nervanasys/an-analysis-of-convolution-for-inference

Understanding Convolution in Deep Learning

• blog: http://timdettmers.com/2015/03/26/convolution-deep-learning/

A guide to convolution arithmetic for deep learning

• arxiv: http://arxiv.org/abs/1603.07285
• github: https://github.com/vdumoulin/conv_arithmetic

Going beyond full utilization: The inside scoop on Nervana’s Winograd kernels

• blog: https://www.nervanasys.com/winograd-2/

Playing with convolutions in TensorFlow: From a short introduction to convolution to a complete model

• blog: http://mourafiq.com/2016/08/10/playing-with-convolutions-in-tensorflow.html
• github: https://github.com/mouradmourafiq/tensorflow-convolution-models

How convolutional neural networks see the world: An exploration of convnet filters with Keras

• blog: https://blog.keras.io/how-convolutional-neural-networks-see-the-world.html

One by One [ 1 x 1 ] Convolution - counter-intuitively useful

http://iamaaditya.github.io/2016/03/one-by-one-convolution/

Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize

• intro: Twitter
• arxiv: https://arxiv.org/abs/1707.02937

Receptive Field

A guide to receptive field arithmetic for Convolutional Neural Networks

https://medium.com/@nikasa1889/a-guide-to-receptive-field-arithmetic-for-convolutional-neural-networks-e0f514068807

Momentum

Why Momentum Really Works

• blog: http://distill.pub/2017/momentum/
• github: https://github.com/distillpub/post–momentum

maxDNN

maxDNN: An Efficient Convolution Kernel for Deep Learning with Maxwell GPUs

• arxiv: http://arxiv.org/abs/1501.06633
• github: https://github.com/eBay/maxDNN

GEMM (General Matrix Matrix Multiply)

Why GEMM is at the heart of deep learning

• blog: http://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/

A full walk through of the SGEMM implementation

• github-wiki: https://github.com/NervanaSystems/maxas/wiki/SGEMM

Backpropagation

Learning representations by back-propagating errors

Learning Internal Representations by Error Propagating

• author: David E. Rumelhart, Geoffrey E. Hinton & Ronald J. Williams. 1986
• paper: http://www.nature.com/nature/journal/v323/n6088/pdf/323533a0.pdf
• mirror: http://pan.baidu.com/s/1bo30gHp
• mirror: http://pan.baidu.com/s/1kVfJ4of

Calculus on Computational Graphs: Backpropagation

• blog: http://colah.github.io/posts/2015-08-Backprop/

Styles of Truncated Backpropagation

• blog: http://r2rt.com/styles-of-truncated-backpropagation.html

Is BackPropagation Necessary?

• blog: http://deliprao.com/archives/191

Backpropagation In Convolutional LSTMs

https://www.doc.ic.ac.uk/~ahanda/ConvLSTMs.pdf

Backward Pass on Conv Layer

Convolutional Neural Networks backpropagation: from intuition to derivation

• blog: https://grzegorzgwardys.wordpress.com/2016/04/22/8/

Backpropagation In Convolutional Neural Networks

• blog: http://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/

Why do we rotate weights when computing the gradients in a convolution layer of a convolution network?

http://soumith.ch/ex/pages/2014/08/07/why-rotate-weights-convolution-gradient/

Note on the implementation of a convolutional neural networks

http://cthorey.github.io./backprop_conv/

Attention

Attention in a Convolutional Neural Net

• blog: http://www.danvatterott.com/blog/2016/09/20/attention-in-a-convolutional-neural-net/

Attention-based Networks

• intro: M. Malinowski. Max Planck Institut Informatik
• slides: http://download.mpi-inf.mpg.de/d2/mmalinow-slides/attention_networks.pdf

Attention in Neural Networks and How to Use It

http://akosiorek.github.io/ml/2017/10/14/visual-attention.html

Softmax

Hierarchical softmax and negative sampling: short notes worth telling

https://towardsdatascience.com/hierarchical-softmax-and-negative-sampling-short-notes-worth-telling-2672010dbe08

Caffe

DIY Deep Learning for Vision: a Hands-On Tutorial with Caffe

• homepage: http://tutorial.caffe.berkeleyvision.org/
• slides: https://docs.google.com/presentation/d/1UeKXVgRvvxg9OUdh_UiC5G71UMscNPlvArsWER41PsU/edit#slide=id.gc2fcdcce7_216_0

Deep learning tutorial on Caffe technology : basic commands, Python and C++ code

http://christopher5106.github.io/deep/learning/2015/09/04/Deep-learning-tutorial-on-Caffe-Technology.html

Using Caffe with your own dataset

https://medium.com/@alexrachnog/using-caffe-with-your-own-dataset-b0ade5d71233

OpenCV 3.0.0-dev: Load Caffe framework models

http://docs.opencv.org/master/d5/de7/tutorial_dnn_googlenet.html#gsc.tab=0

Chainer

Chainer Info

https://github.com/hidetomasuoka/chainer-info

Keras

Keras tutorial

• intro: Tutorial teaching the basics of Keras and some deep learning concepts
• github: https://github.com/jfsantos/keras-tutorial

Keras Tutorial: The Ultimate Beginner’s Guide to Deep Learning in Python

https://elitedatascience.com/keras-tutorial-deep-learning-in-python

Deep Learning with Keras: Tutorial @ EuroScipy 2016

• github: https://github.com/leriomaggio/deep-learning-keras-tensorflow

Transfer Learning and Fine Tuning for Cross Domain Image Classification with Keras

• slides: http://www.slideshare.net/sujitpal/transfer-learning-and-fine-tuning-for-cross-domain-image-classification-with-keras
• mirror: https://pan.baidu.com/s/1gfn1xuj
• github: https://github.com/sujitpal/fttl-with-keras

MXNet

10 Deep Learning projects based on Apache MXNet

https://medium.com/@julsimon/10-deep-learning-projects-based-on-apache-mxnet-8231109f3f64

Awesome MXNet(Beta)

https://github.com/chinakook/Awesome-MXNet

TVM

Optimize Deep Learning GPU Operators with TVM: A Depthwise Convolution Example

http://tvmlang.org/2017/08/22/Optimize-Deep-Learning-GPU-Operators-with-TVM-A-Depthwise-Convolution-Example.html

Theano

Theano Tutorial @ LTI, Carnegie Mellon University

• github: https://github.com/k-kawakami/Theano_Tutorial/blob/master/Theano_Tutorial.ipynb

An Introduction to MXNet/Gluon

• intro: @李沐
• github: https://github.com/mli/cvpr17

TensorFlow

LearningTensorFlow.com: A beginners guide to a powerful framework.

• homepge: http://learningtensorflow.com/index.html

TensorFlow Examples: TensorFlow tutorials and code examples for beginners

• github: https://github.com/aymericdamien/TensorFlow-Examples

Awesome TensorFlow: A curated list of awesome TensorFlow experiments, libraries, and projects

• github: https://github.com/jtoy/awesome-tensorflow/

The Good, Bad, & Ugly of TensorFlow: A survey of six months rapid evolution (+ tips/hacks and code to fix the ugly stuff)

• blog: https://indico.io/blog/the-good-bad-ugly-of-tensorflow/

Tensorflow Tutorials using Jupyter Notebook

• github: https://github.com/sjchoi86/Tensorflow-101

TensorFlow Tutorial

• homepage: http://terryum.io/ml_practice/2016/05/28/TFIntroSlides/
• slides: https://s3.amazonaws.com/www.terryum.io/images/TensorFlow_Intro_160529.pptx
• mirror: http://pan.baidu.com/s/1c5cICY
• github: https://github.com/terryum/TensorFlow_Exercises

FIRST CONTACT WITH TENSORFLOW

• ebook: http://www.jorditorres.org/first-contact-with-tensorflow/

Introduction to TensorFlow

• github: https://github.com/nihit/TensorFlow101

TensorFlow-Tutorials: Simple tutorials using Google’s TensorFlow Framework

• github: https://github.com/nlintz/TensorFlow-Tutorials

Neural Network Toolbox on TensorFlow

• github: https://github.com/ppwwyyxx/tensorpack

Awesome Tensorflow Implementations

• github: https://github.com/TensorFlowKR/awesome_tensorflow_implementations

The Ultimate List of TensorFlow Resources: Books, Tutorials & More

• blog: https://hackerlists.com/tensorflow-resources/

Install TensorFlow: Slides and code from our TensorFlow Workshop

• github: https://github.com/random-forests/tensorflow-workshop

A Tour of TensorFlow

• arxiv: https://arxiv.org/abs/1610.01178

TensorFlow Tutorials

• youtube: https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ
• github: https://github.com/Hvass-Labs/TensorFlow-Tutorials

Shapes and dynamic dimensions in TensorFlow

• blog: https://blog.metaflow.fr/shapes-and-dynamic-dimensions-in-tensorflow-7b1fe79be363#.1293uf94t

TensorFlow saving/restoring and mixing multiple models

https://blog.metaflow.fr/tensorflow-saving-restoring-and-mixing-multiple-models-c4c94d5d7125#.242xy4d46

Getting to Know TensorFlow

• blog: https://hackernoon.com/machine-learning-with-tensorflow-8873fdee2b68#.90j2jz5av

Image Classification and Segmentation with Tensorflow and TF-Slim http://warmspringwinds.github.io/tensorflow/tf-slim/2016/10/30/image-classification-and-segmentation-using-tensorflow-and-tf-slim/

Not another MNIST tutorial with TensorFlow

• blog: https://www.oreilly.com/learning/not-another-mnist-tutorial-with-tensorflow

Dive Into TensorFlow

• Part I: Getting Started with TensorFlow: http://textminingonline.com/dive-into-tensorflow-part-i-getting-started-with-tensorflow
• Part II: Basic Concepts: http://textminingonline.com/dive-into-tensorflow-part-ii-basic-concepts
• Part III: GTX 1080+Ubuntu16.04+CUDA8.0+cuDNN5.0+TensorFlow: http://textminingonline.com/dive-into-tensorflow-part-iii-gtx-1080-ubuntu16-04-cuda8-0-cudnn5-0-tensorflow
• Part IV: Hello MNIST: http://textminingonline.com/dive-into-tensorflow-part-iv-hello-mnist
• Part V: Deep MNIST: http://textminingonline.com/dive-into-tensorflow-part-v-deep-mnist
• Part VI: Beyond Deep Learning: http://textminingonline.com/dive-into-tensorflow-part-vi-beyond-deep-learning

TensorFlow Exercises - focusing on the comparison with NumPy.

• github: https://github.com/Kyubyong/tensorflow-exercises

A Gentle Guide to Using Batch Normalization in Tensorflow

• blog: http://ruishu.io/2016/12/27/batchnorm/

Using TensorFlow in Windows with a GPU

• blog: http://www.heatonresearch.com/2017/01/01/tensorflow-windows-gpu.html

Tensorflow and deep learning - without a PhD

• youtube: https://www.youtube.com/watch?v=vq2nnJ4g6N0
• mirror: https://pan.baidu.com/s/1o8HF9R8
• blog: https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/#0

4 Steps To Learn TensorFlow When You Already Know scikit-learn https://medium.com/@Zelros/4-steps-to-learn-tensorflow-when-you-already-know-scikit-learn-3cd0340456b5#.q206au7u9

Gentlest Introduction to Tensorflow

• part 1: https://medium.com/all-of-us-are-belong-to-machines/the-gentlest-introduction-to-tensorflow-248dc871a224#.fxyclr1ui
• part 2: https://medium.com/all-of-us-are-belong-to-machines/gentlest-introduction-to-tensorflow-part-2-ed2a0a7a624f#.vf7p9upg2
• part 3: https://medium.com/all-of-us-are-belong-to-machines/gentlest-intro-to-tensorflow-part-3-matrices-multi-feature-linear-regression-30a81ebaaa6c#.bvjru1f88
• part 4: https://medium.com/all-of-us-are-belong-to-machines/gentlest-intro-to-tensorflow-4-logistic-regression-2afd0cabc54#.seh1fbr24

learn code with tensorflow

• github: https://github.com/burness/tensorflow-101

TensorFlow Machine Learning Cookbook

• book: https://www.packtpub.com/big-data-and-business-intelligence/tensorflow-machine-learning-cookbook
• github: https://github.com/nfmcclure/tensorflow_cookbook

TensorFlow Image Recognition on a Raspberry Pi

http://svds.com/tensorflow-image-recognition-raspberry-pi/

TensorFlow For Machine Intelligence

• book: https://bleedingedgepress.com/tensor-flow-for-machine-intelligence/
• github: https://github.com/backstopmedia/tensorflowbook

Installing TensorFlow on Raspberry Pi 3 (and probably 2 as well)

• intro: TensorFlow for Raspberry Pi
• github: https://github.com/samjabrahams/tensorflow-on-raspberry-pi

CodinGame: Deep Learning - TensorFlow

• homepage: https://www.codingame.com/games/machine-learning?utm_source=CodinGame&utm_medium=Email&utm_campaign=tensorflow

A Practical Guide for Debugging Tensorflow Codes

Debugging Tips on TensorFlow

• slides: https://wookayin.github.io/TensorflowKR-2016-talk-debugging
• github: https://github.com/wookayin/TensorflowKR-2016-talk-debugging

Tensorflow Projects: Deep learning using tensorflow

• intro: A repo of everything deep and neurally related. Implementations and ideas are largely based on papers from arxiv and implementations, tutorials from the internet.
• github: https://github.com/shekkizh/TensorflowProjects

Machine Learning with TensorFlow

• homepage: http://www.tensorflowbook.com/
• github: https://github.com/BinRoot/TensorFlow-Book
• blog: https://www.manning.com/books/machine-learning-with-tensorflow

Convolutional Networks: from TensorFlow to iOS BNNS

• blog: https://paiv.github.io/blog/2016/09/25/tensorflow-to-bnns.html
• github: https://github.com/paiv/mnist-bnns

Android TensorFlow Machine Learning Example

• blog: https://blog.mindorks.com/android-tensorflow-machine-learning-example-ff0e9b2654cc#.ysg0ss9r2
• github: https://github.com/MindorksOpenSource/AndroidTensorFlowMachineLearningExample

TensorFlow and Deep Learning Tutorials

https://github.com/wagamamaz/tensorflow-tutorial

Finetuning AlexNet with TensorFlow

• blog: https://kratzert.github.io/kratzert.github.io/2017/02/24/finetuning-alexnet-with-tensorflow.html
• github: https://github.com/kratzert/finetune_alexnet_with_tensorflow

Deep Learning examples using Tensorflow

https://github.com/aditya101993/Deep-Learning

How To Write Your Own Tensorflow in C++

https://oneraynyday.github.io/ml/2017/10/20/Tensorflow-C++/

Tensorflow on Android

A Guide to Running Tensorflow Models on Android

• youtube: https://www.youtube.com/watch?v=kFWKdLOxykE
• mirror: http://www.bilibili.com/video/av9806881/index_10.html
• github: https://github.com/llSourcell/A_Guide_to_Running_Tensorflow_Models_on_Android

TensorFlow Android stand-alone demo

• intro: Android demo source files extracted from original TensorFlow source. (TensorFlow r0.10)
• github: https://github.com/miyosuda/TensorFlowAndroidDemo

Torch

Torch Developer Guide

• github: https://github.com/Atcold/Torch-Developer-Guide

PyTorch

Practical PyTorch tutorials

• github: https://github.com/spro/practical-pytorch

The Incredible PyTorch

• github: https://github.com/ritchieng/the-incredible-pytorch

PyTorch quick start: Classifying an image

• blog: http://blog.outcome.io/pytorch-quick-start-classifying-an-image/
• ipn: https://gist.github.com/jbencook/9918217f866c1aa9967391ba62d123b5

tutorial for researchers to learn deep learning with pytorch.

https://github.com/yunjey/pytorch-tutorial

Building a System for Fun!

Facial Recognition On A Jetson TX1 In Tensorflow

• blog: http://www.mattkrzus.com/face.html

Build an AI Cat Chaser with Jetson TX1 and Caffe

• blog: https://devblogs.nvidia.com/parallelforall/ai-cat-chaser-jetson-tx1-caffe/

Deep Learning in Aerial Systems Using Jetson

• blog: https://devblogs.nvidia.com/parallelforall/deep-learning-in-aerial-systems-jetson/
• github: https://github.com/amitibo/auvsi-targets

Cherry Autonomous Racecar (CAR): NCAT ECE Senior Design Project

• intro: Implementation of the CNN from End to End Learning for Self-Driving Cars on a Nvidia Jetson TX1 using Tensorflow and ROS
• github: https://github.com/DJTobias/Cherry-Autonomous-Racecar

Traffic Signs Classification

Traffic signs classification with Deep Learning.

• blog: https://hackernoon.com/traffic-signs-classification-with-deep-learning-b0cb03e23efb#.n0fjehwo6
• github: https://github.com/MehdiSv/TrafficSignsRecognition/

Traffic Sign Recognition with TensorFlow

• blog: https://medium.com/@waleedka/traffic-sign-recognition-with-tensorflow-629dffc391a6#.2khbv6a9a

Traffic signs classification with a convolutional network

http://navoshta.com/traffic-signs-classification/

Convolutional Neural Network for Traffic Sign Classification — CarND

• blog: https://medium.com/@gruby/convolutional-neural-network-for-traffic-sign-classification-carnd-e46e95453899#.lfutfs21v

Talks

A Tour of Deep Learning With C++

• intro: CppCon 2017, Peter Goldsborough
• youtube: https://www.youtube.com/watch?v=9-1lcss0NMg
• bilibili: https://www.bilibili.com/video/av20675156/

Papers

Practical recommendations for gradient-based training of deep architectures

• author: Yoshua Bengio
• arxiv: http://arxiv.org/abs/1206.5533

Bag of Tricks for Image Classification with Convolutional Neural Networks

• intro: Amazon Web Services
• arxiv: https://arxiv.org/abs/1812.01187

Blogs

Efficient BackProp

• intro: Neural Networks: Tricks of the Trade, 2nd
• blog: http://blog.csdn.net/zouxy09/article/details/45288129

Deep Learning for Vision: Tricks of the Trade

• intro: CVPR. Marc’Aurelio Ranzato
• slides: http://bavm2013.splashthat.com/img/events/46439/assets/34a7.ranzato.pdf

Optimizing RNN performance

• intro: Silicon Valley AI Lab
• keywords: Optimize GEMM, parallel GPU, GRU and LSTM…
• blog: http://svail.github.io/

Must Know Tips/Tricks in Deep Neural Networks

• intro: by Xiu-Shen Wei, NJU LAMDA
• blog: http://lamda.nju.edu.cn/weixs/project/CNNTricks/CNNTricks.html
• slides: http://lamda.nju.edu.cn/weixs/slide/CNNTricks_slide.pdf

Training Tricks from Deeplearning4j

http://deeplearning4j.org/trainingtricks.html

Suggestions for DL from Llya Sutskeve

• intro: data, preprocessing, mini-batch, gradient normalization, learning rate, weight initialization, data augmentation, dropout and ensemble
• blog: http://yyue.blogspot.com/2015/01/a-brief-overview-of-deep-learning.html

Efficient Training Strategies for Deep Neural Network Language Models

• intro: batch-size, initial learning rate, network initialization
• blog: https://fb56552f-a-62cb3a1a-s-sites.googlegroups.com/site/deeplearningworkshopnips2014/71.pdf?attachauth=ANoY7cp_eDwTXPm6iWHdBRhlIsgPASEAwkW-exLSOsz467mge7zLCkBMWznOu_G90vGVtqNvXOusc4z6cC6hEnHk6YzHtuEr_kyU0fyme7asaECN0zvoNwDk5258CueoB6fY3WtLvbJzYok1xiIeWSFYtk5mKXCXFDMI6djwhjCX1xi0GEEv_x7uMQwTdQlDItZ3kgLnZ2RjctQmIXDCu58fS3Wby4vWX3CkhMIf_EpCXx7jDn_M2SM%3D&attredirects=0

Neural Networks Best Practice

• intro: Uber
• paper: http://www.kentran.net/2013/04/neural-network-best-practices.html

Dark Knowledge from Hinton

• youtube: https://www.youtube.com/watch?v=EK61htlw8hY
• slides: http://www.ttic.edu/dl/dark14.pdf
• notes: http://deepdish.io/2014/10/28/hintons-dark-knowledge/
• notes: http://fastml.com/geoff-hintons-dark-knowledge/

Stochastic Gradient Descent Tricks(Leon Bottou)

http://leon.bottou.org/publications/pdf/tricks-2012.pdf

Advice for applying Machine Learning

https://jmetzen.github.io/2015-01-29/ml_advice.html

How to Debug Learning Algorithm for Regression Model

http://vitalflux.com/machine-learning-debug-learning-algorithm-regression-model/

Large-scale L-BFGS using MapReduce

• intro: NIPS 2014
• paper: http://papers.nips.cc/paper/5333-large-scale-l-bfgs-using-mapreduce.pdf

Selecting good features

– Part I: univariate selection: http://blog.datadive.net/selecting-good-features-part-i-univariate-selection/ – Part II: linear models and regularization: http://blog.datadive.net/selecting-good-features-part-ii-linear-models-and-regularization/ – Part III: random forests: http://blog.datadive.net/selecting-good-features-part-iii-random-forests/ – Part IV: stability selection, RFE and everything side by side: http://blog.datadive.net/selecting-good-features-part-iv-stability-selection-rfe-and-everything-side-by-side/

机器学习代码心得之有监督学习的模块

http://www.weibo.com/p/1001603795687165852957

Stochastic Gradient Boosting: Choosing the Best Number of Iterations

• intro: Kaggle winner YANIR SEROUSSI
• blog: http://yanirseroussi.com/2014/12/29/stochastic-gradient-boosting-choosing-the-best-number-of-iterations/

Large-Scale High-Precision Topic Modeling on Twitter

• intro: Twitter senior researcher. KDD 2014
• paper: http://www.eeshyang.com/papers/KDD14Jubjub.pdf

H2O World - Top 10 Deep Learning Tips & Tricks - Arno Candel

http://www.slideshare.net/0xdata/h2o-world-top-10-deep-learning-tips-tricks-arno-candel

How To Improve Deep Learning Performance: 20 Tips, Tricks and Techniques That You Can Use To Fight Overfitting and Get Better Generalization

http://machinelearningmastery.com/improve-deep-learning-performance/

Neural Network Training Speed Trick

https://medium.com/machine-learning-at-petiteprogrammer/neural-network-training-speed-trick-92d6b22a7754#.4v6qukpn7

The Black Magic of Deep Learning - Tips and Tricks for the practitioner

http://nmarkou.blogspot.ru/2017/02/the-black-magic-of-deep-learning-tips.html