Deep Learning Tutorials

Tutorials

Deep learning

• intro: From Wikipedia, the free encyclopedia
• blog: https://www.wikiwand.com/en/Deep_learning

Toward Theoretical Understanding of Deep Learning

• intro: ICML 2018 Tutorial. by Sanjeev Arora, Princeton University
• slides: https://www.dropbox.com/s/qonozmne0x4x2r3/deepsurveyICML18final.pptx?dl=0
• mirror: https://pan.baidu.com/s/1r_lz6rMoSIinvfovMFFbug

VGG Convolutional Neural Networks Practical

• homepage: http://www.robots.ox.ac.uk/~vgg/practicals/cnn/index.html
• github: https://github.com/vedaldi/practical-cnn

Hacker’s guide to Neural Networks

http://karpathy.github.io/neuralnets/

Deep Learning Tutorials

• website: http://deeplearning.net/tutorial/
• code: https://github.com/lisa-lab/DeepLearningTutorials

Deep Learning in a Nutshell: Core Concepts

http://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/

Deep Learning in a Nutshell: History and Training

http://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-history-training/

A Deep Learning Tutorial: From Perceptrons to Deep Networks

http://www.toptal.com/machine-learning/an-introduction-to-deep-learning-from-perceptrons-to-deep-networks

Deep Neural Networks (with Python code)

• paper: http://scholarbank.nus.edu.sg/bitstream/handle/10635/120564/DeepNeuralNetworks.pdf?sequence=1

Three Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey

• paper: http://research.microsoft.com/pubs/192937/Transactions-APSIPA.pdf

Stanford Unsupervised Feature Learning and Deep Learning Tutorial: UFLDL Tutorial

• homepage: http://ufldl.stanford.edu/tutorial/
• programming exercises: https://github.com/amaas/stanford_dl_ex

The Unreasonable Effectiveness of Deep Learning (LeCun)

• slides: http://www.ee.ucl.ac.uk/sahd2014/resources/LeCun.pdf

Deep learning from the bottom up

• blog: https://www.metacademy.org/roadmaps/rgrosse/deep_learning

Introduction to Deep Learning with Python (By Alec Radford. Theano)

• youtube: https://www.youtube.com/watch?v=S75EdAcXHKk&hd=1

New to deep learning? Here are 4 easy lessons from Google

• blog: https://gigaom.com/2015/01/29/new-to-deep-learning-here-are-4-easy-lessons-from-google/

Deep Learning 101

• blog: http://markus.com/deep-learning-101/

Neural Networks Demystified

• Part 1: Data and Architecture: https://www.youtube.com/watch?v=bxe2T-V8XRs
• Part 2: Forward Propagation: https://www.youtube.com/watch?v=UJwK6jAStmg
• Part 3: Gradient Descent: https://www.youtube.com/watch?v=5u0jaA3qAGk
• Part 4: Backpropagation: https://www.youtube.com/watch?v=GlcnxUlrtek
• Part 5: Numerical Gradient Checking: https://www.youtube.com/watch?v=pHMzNW8Agq4
• Part 6: Training: https://www.youtube.com/watch?v=9KM9Td6RVgQ

• Part 7: Overfitting, Testing, and Regularization: https://www.youtube.com/watch?v=S4ZUwgesjS8

• all-pack: http://pan.baidu.com/s/1dDq5oNB

Deep Learning SIMPLIFIED

• playlist: https://www.youtube.com/playlist?list=PLjJh1vlSEYgvGod9wWiydumYl8hOXixNu

A ‘Brief’ History of Neural Nets and Deep Learning

• part 1: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning/
• part 2: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning-part-2/
• part 3: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning-part-3/
• part 4: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning-part-4/

Deep Neural Networks — An Overview

https://medium.com/@asjad/deep-neural-networks-an-overview-480112b12a13#.i7apzmnso

A Tutorial on Deep Neural Networks for Intelligent Systems

• arxiv: http://arxiv.org/abs/1603.07249

Deep Learning for Computer Vision – Introduction to Convolution Neural Networks

• blog: http://www.analyticsvidhya.com/blog/2016/04/deep-learning-computer-vision-introduction-convolution-neural-networks/

BI Lab Deep Learning Tutorial

• github: https://github.com/bi-lab/deeplearning_tutorial

Deep Learning Tutorials

• github: https://github.com/sjchoi86/Deep-Learning-101

Neural Network Architectures

• blog: http://culurciello.github.io/tech/2016/06/04/nets.html

A Practical Introduction to Deep Learning with Caffe and Python

• blog: http://adilmoujahid.com/posts/2016/06/introduction-deep-learning-python-caffe/

Notes on Convolutional Neural Networks

• homepage: http://cogprints.org/5869/
• paper: http://cogprints.org/5869/1/cnn_tutorial.pdf

Feed Forward and Backward Run in Deep Convolution Neural Network

• intro: 20th International Conference on Computer Vision and Image Processing
• arxiv: https://arxiv.org/abs/1711.03278

Convolutional Networks

http://deeplearning4j.org/convolutionalnets.html

Exploring convolutional neural networks with DL4J

• blog: http://brooksandrew.github.io/simpleblog/articles/convolutional-neural-network-training-with-dl4j/

Understanding Convolutional Neural Networks

• arxiv: http://arxiv.org/abs/1605.09081

Laws, Sausages and ConvNets

• blog: http://www.trivialorwrong.com/2016/06/01/laws-sausages-and-convnets.html

Convolutional Neural Networks (CNNs): An Illustrated Explanation

• blog: http://xrds.acm.org/blog/2016/06/convolutional-neural-networks-cnns-illustrated-explanation/

intro_deep: Introduction tutorials to deep learning with Theano and OpenDeep

• slides: https://docs.google.com/presentation/d/1cg9Tn2wWwqJmaSSDnlBDBEETD5SyV6TJSD8qiDJFgEM
• mirror: http://pan.baidu.com/s/1hqIR0yC
• youtube: https://www.youtube.com/watch?v=afUvcD3tEoQ
• mirror: http://pan.baidu.com/s/1qWHp7xa
• github: https://github.com/mbeissinger/intro_deep

Deep Learning on Java by Breandan Considine

• video: https://speakerdeck.com/breandan/deep-learning-on-java

Using Convolutional Neural Networks and TensorFlow for Image Classification (NYC TensorFlow meetup)

• blog: http://blog.altoros.com/using-convolutional-neural-networks-and-tensorflow-for-image-classification-and-search.html

Neural networks with Theano and Lasagne

• youtube: https://www.youtube.com/watch?v=dtGhSE1PFh0
• mirror: http://pan.baidu.com/s/1kUl3PvL
• github: https://github.com/ebenolson/pydata2015

Introduction to Deep Learning

• github: https://github.com/rouseguy/intro2deeplearning
• slides: https://speakerdeck.com/bargava/introduction-to-deep-learning

Introduction to Deep Learning for Image Recognition - SciPy US 2016

• github: https://github.com/rouseguy/scipyUS2016_dl-image
• slides: https://speakerdeck.com/bargava/introduction-to-deep-learning-for-image-processing

Deep learning tutorials (2nd ed.)

• github: https://github.com/sjchoi86/dl_tutorials

---

A Beginner’s Guide To Understanding Convolutional Neural Networks

• part 1: https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner’s-Guide-To-Understanding-Convolutional-Neural-Networks/

A Beginner’s Guide To Understanding Convolutional Neural Networks Part 2

• part 2: https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner’s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/

The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)

• part 3: https://adeshpande3.github.io/adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html

---

Deep Learning Part 1: Comparison of Symbolic Deep Learning Frameworks

• blog: http://blog.revolutionanalytics.com/2016/08/deep-learning-part-1.html

Deep Learning Part 2: Transfer Learning and Fine-tuning Deep Convolutional Neural Networks

• blog: http://blog.revolutionanalytics.com/2016/08/deep-learning-part-2.html

Deep Learning Part 3: Combining Deep Convolutional Neural Network with Recurrent Neural Network

• blog: http://blog.revolutionanalytics.com/2016/09/deep-learning-part-3.html

---

Introduction to Deep Learning for Image Processing

• slides: https://speakerdeck.com/bargava/introduction-to-deep-learning-for-image-processing

The best explanation of Convolutional Neural Networks on the Internet!

• blog: https://medium.com/technologymadeeasy/the-best-explanation-of-convolutional-neural-networks-on-the-internet-fbb8b1ad5df8

The Evolution and Core Concepts of Deep Learning & Neural Networks

• blog: https://www.analyticsvidhya.com/blog/2016/08/evolution-core-concepts-deep-learning-neural-networks/

An Intuitive Explanation of Convolutional Neural Networks

• blog: https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/

How Convolutional Neural Networks Work

• blog: http://brohrer.github.io/how_convolutional_neural_networks_work.html

Preliminary Note on the Complexity of a Neural Network

• blog: http://r2rt.com/preliminary-note-on-the-complexity-of-a-neural-network.html

Deep Learning Tutorial

• intro: Hung-yi Lee. 李宏毅
• slides: http://www.slideshare.net/tw_dsconf/ss-62245351?qid=c0f0f97a-6ca8-4df0-97e2-984452215ee7&v=&b=&from_search=1
• mirror: https://pan.baidu.com/s/1mhMhuFQ

Jupyter notebooks and code for Intro to DL talk at Genesys

• blog: http://sujitpal.blogspot.com/2016/08/kerasjupyter-notebooks-for-my.html
• github: https://github.com/sujitpal/intro-dl-talk-code

Learn Deep Learning the Hard Way

• blog: https://medium.com/artifacia/learn-deep-learning-the-hard-way-e5d844f9fbc1#.yitf25xg5

A Complete Guide on Getting Started with Deep Learning in Python

• blog: https://www.analyticsvidhya.com/blog/2016/08/deep-learning-path/

Deep learning for complete beginners: Recognising handwritten digits

• blog: http://online.cambridgecoding.com/notebooks/cca_admin/deep-learning-for-complete-beginners-recognising-handwritten-digits

Deep learning for complete beginners: Using convolutional nets to recognise images

• blog: http://online.cambridgecoding.com/notebooks/cca_admin/convolutional-neural-networks-with-keras

Deep learning for complete beginners: neural network fine-tuning techniques

• blog: http://online.cambridgecoding.com/notebooks/cca_admin/neural-networks-tuning-techniques

How do Convolutional Neural Networks work?

http://brohrer.github.io/how_convolutional_neural_networks_work.html

Creating a Neural Network That Can Tell if a Name Is Male or Female, in JavaScript

• blog: https://medium.com/@hathibel/creating-a-neural-network-that-can-tell-if-a-name-is-male-or-female-in-javascript-3061029be396#.89l3znc2l

Softmax Classifiers Explained

• blog: http://www.pyimagesearch.com/2016/09/12/softmax-classifiers-explained/

The Softmax function and its derivative

• blog: http://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative/

How an algorithm behind Deep Learning works

• video: http://blog.revolutionanalytics.com/2016/09/how-the-algorithm-behind-deep-learning-works.html
• slides: https://github.com/brohrer/public-hosting/raw/master/How_CNNs_work.pdf
• blog: http://www.kdnuggets.com/2016/08/brohrer-convolutional-neural-networks-explanation.html
• mirror: http://v.youku.com/v_show/id_XMTcyNTgwNDQyOA==.html

The Neural Network Zoo

http://www.asimovinstitute.org/neural-network-zoo/

Recognising Beer with TensorFlow

• blog: https://medium.com/@chrishawkins/recognising-beer-with-tensorflow-9dedfee3c3c0#.pn5gm3fgc
• gist: https://gist.github.com/chrishawkins/177e37756c833768a21d446cc4921c6e

Deep learning architecture diagrams

• intro: LSTM diagrams
• blog: http://fastml.com/deep-learning-architecture-diagrams/

Getting Started with Deep Learning and Python

• blog: http://www.pyimagesearch.com/2014/09/22/getting-started-deep-learning-python

Deep Learning Practicals

• intro: Video playlist of Torch Video Tutorials
• youtube: https://www.youtube.com/playlist?list=PLLHTzKZzVU9ebuL6DCclzI54MrPNFGqbW
• mirror: https://pan.baidu.com/s/1skMFGkt

A simple workflow for deep learning

• blog: https://cartesianfaith.com/2016/09/29/a-simple-workflow-for-deep-learning/
• github: https://github.com/zatonovo/deep_learning_ex

A primer on universal function approximation with deep learning (in Torch and R)

• blog: https://cartesianfaith.com/2016/09/23/a-primer-on-universal-function-approximation-with-deep-learning-in-torch-and-r/

An Introduction to Implementing Neural Networks using TensorFlow

https://www.analyticsvidhya.com/blog/2016/10/an-introduction-to-implementing-neural-networks-using-tensorflow/

A Gentle Introduction to Convolutional Neural Networks

• blog: https://miguelgfierro.com/blog/2016/a-gentle-introduction-to-convolutional-neural-networks/

Beginning Machine Learning with Keras and TensorFlow

• blog: http://blog.thoughtram.io/machine-learning/2016/09/23/beginning-ml-with-keras-and-tensorflow.html

Shortest Way to Deep Learning

• blog: https://blog.asadmemon.com/shortest-way-to-deep-learning-41e704d65ef#.19iowho8t

Deep learning with Matlab

• intro: Covered topics of the presentation: Machine learning workflow, Extracting feaures from images (colours, edges, corners, etc.)
• youtube: https://www.youtube.com/watch?v=r4D3NxQ0Xhg

Convolutional neural networks for computer vision with Matlab

• youtube: https://www.youtube.com/watch?v=MwUic9FCBJI

Neural Net Computing Explodes

• blog: http://semiengineering.com/neural-net-computing-explodes/

Tutorial: Optimizing Neural Networks using Keras (with Image recognition case study)

https://www.analyticsvidhya.com/blog/2016/10/tutorial-optimizing-neural-networks-using-keras-with-image-recognition-case-study/

15 Deep Learning Tutorials

• blog: http://www.datasciencecentral.com/profiles/blogs/15-deep-learning-tutorials

Deep Learning Episode 1: Optimizing DeepMind’s A3C on Torch

http://www.allinea.com/blog/201607/deep-learning-episode-1-optimizing-deepminds-a3c-torch

Deep Learning Episode 2: Scaling TensorFlow over multiple EC2 GPU nodes

http://www.allinea.com/blog/201608/deep-learning-episode-2-scaling-tensorflow-over-multiple-ec2-gpu-nodes

Deep Learning Episode 3: Supercomputer vs Pong

http://www.allinea.com/blog/201610/deep-learning-episode-3-supercomputer-vs-pong

Deep Learning Episode 4: Supercomputer vs Pong II

http://www.allinea.com/blog/201610/deep-learning-episode-4-supercomputer-vs-pong-ii

Nuts and Bolts of Applying Deep Learning — Summary

• blog: https://medium.com/@aniketvartak/nuts-and-bolts-of-applying-deep-learning-summary-84b8a8e873d5#.y7xidlhb4

Intro to Deep Learning for Computer Vision

http://chaosmail.github.io/deeplearning/2016/10/22/intro-to-deep-learning-for-computer-vision/

If I Can Learn to Play Atari, I Can Learn TensorFlow

• intro: Here is a summary of new deep learning libraries, tools, and updates to existing frameworks.
• blog: https://dzone.com/articles/deep-learning-resources

TensorFlow workshop materials

• github: https://github.com/amygdala/tensorflow-workshop

Some theorems on deep learning

• intro: Tomaso Poggio [MIT]
• youtube: https://www.youtube.com/watch?v=YVjvRvvVn4w
• mirror: https://pan.baidu.com/s/1o8o7LjW

Pokemon, Colors, and Deep Learning

• blog: https://juandes.com/pokemon-colors-and-deep-learning-95fb715be46
• github: https://github.com/juandes/PokemonTypesDeepLearning

Why Deep Learning is Radically Different from Machine Learning

• blog: https://medium.com/intuitionmachine/why-deep-learning-is-radically-different-from-machine-learning-945a4a65da4d#.yuqxqxpsz

Deep Learning: The Unreasonable Effectiveness of Randomness

• blog: https://medium.com/intuitionmachine/deep-learning-the-unreasonable-effectiveness-of-randomness-14d5aef13f87#.1n775yl4i

Deep Meta-Learning : Machines now Bootstrap Themselves

• blog: https://medium.com/intuitionmachine/deep-learning-can-now-create-itself-92e7ff0d59a7#.ml0dy8m9a

Are Deep Neural Networks Creative?

• blog: http://www.kdnuggets.com/2016/05/deep-neural-networks-creative-deep-learning-art.html

Are Deep Neural Networks Creative? v2

• blog: http://approximatelycorrect.com/2016/11/11/are-deep-neural-networks-creative/

Develop/Train A Convolutional Neural Netwok For MNIST Dataset

• github: https://github.com/mirjalil/DataScience/blob/master/notebooks/deeplearning/tensorflow_03_CNN.ipynb

Rethinking Generalization in Deep Learning

https://medium.com/intuitionmachine/rethinking-generalization-in-deep-learning-ec66ed684ace#.tcnsqik5w

The hard thing about deep learning

• blog: https://www.oreilly.com/ideas/the-hard-thing-about-deep-learning

The hard thing about deep learning

• blog: https://www.oreilly.com/ideas/the-hard-thing-about-deep-learning

Introduction to Autoencoders

• blog: https://pgaleone.eu/neural-networks/2016/11/18/introduction-to-autoencoders/

Two Days to a Demo

• blog: https://developer.nvidia.com/embedded/twodaystoademo

Deep Learning Tutorials for 10 Weeks

• github: https://github.com/sjchoi86/dl_tutorials_4th

Deep Learning in Clojure With Cortex

• blog: http://gigasquidsoftware.com/blog/2016/12/27/deep-learning-in-clojure-with-cortex/

A Guide to Deep Learning by YerevaNN

• blog: http://yerevann.com/a-guide-to-deep-learning/

Learning to Learn, to Program, to Explore and to Seek Knowledge

• intro: Nando de Freitas, NIPS 2016
• youtube: https://www.youtube.com/watch?v=tPWGGwmgwG0
• mirror: https://pan.baidu.com/s/1b2VZsE

Have Fun with Machine Learning: A Guide for Beginners

• intro: An absolute beginner’s guide to Machine Learning and Image Classification with Neural Networks
• github: https://github.com/humphd/have-fun-with-machine-learning

Deep Learning Cheat Sheet

• blog: https://hackernoon.com/deep-learning-cheat-sheet-25421411e460#.e0jwzxpkm

How to train your Deep Neural Network

http://rishy.github.io//ml/2017/01/05/how-to-train-your-dnn/

A deep learning traffic light detector using dlib and a few images from Google street view

• blog: https://sagivtech.com/2016/11/10/post-2/

Recognizing Traffic Lights With Deep Learning

• blog: https://medium.com/@davidbrai/recognizing-traffic-lights-with-deep-learning-23dae23287cc#.k22tnf37a
• github: https://github.com/davidbrai/deep-learning-traffic-lights

Tutorials for deep learning

• github: https://github.com/oduerr/dl_tutorial

The Holographic Principle: Why Deep Learning Works

https://medium.com/intuitionmachine/the-holographic-principle-and-deep-learning-52c2d6da8d9

Deep Neural Networks - A Brief History

• arxiv: https://arxiv.org/abs/1701.05549

Fundamental Deep Learning code in TFLearn, Keras, Theano and TensorFlow

• blog: https://insights.untapt.com/fundamental-deep-learning-code-in-tflearn-keras-theano-and-tensorflow-66be10a03227#.hoaw8fp9p
• slides: https://static1.squarespace.com/static/5362fa11e4b035b5651b7f7e/t/588fb378cd0f687201a2e317/1485812622873/Jon_Krohn_NYHackR_Deep_Learning_2017_01_30.pdf

Deep Neural Network from scratch

https://matrices.io/deep-neural-network-from-scratch/

Convolutional Neural Networks

https://github.com/Alfredvc/cnn_workshop

Exploring Optimizers

https://github.com//KeremTurgutlu/deeplearning/blob/master/Exploring%20Optimizers.ipynb

A Gentle Introduction to Exploding Gradients in Neural Networks

https://machinelearningmastery.com/exploding-gradients-in-neural-networks/

Only Numpy: (Why I do Manual Back Propagation) Implementing Multi Channel/Layer Convolution Neural Network on Numpy with Interactive Code

https://medium.com/swlh/only-numpy-why-i-do-manual-back-propagation-implementing-multi-channel-layer-convolution-neural-7d83242fcc24

92.45% on CIFAR-10 in Torch

• intro: Dropout after Convolution
• blog: http://torch.ch/blog/2015/07/30/cifar.html

Convolution

Understanding Convolutions

• blog: http://colah.github.io/posts/2014-07-Understanding-Convolutions/

Note on the implementation of a convolutional neural networks

• intro: CS231n, Convolutional layer, Pooling layer, Forward pass, Backward pass
• blog: http://cthorey.github.io./backprop_conv/
• github: https://github.com/cthorey/CS231

Convolution in Caffe: a memo

• blog: https://github.com/Yangqing/caffe/wiki/Convolution-in-Caffe:-a-memo

我对卷积的理解

• blog: http://mengqi92.github.io/2015/10/06/convolution/
• blog: https://segmentfault.com/a/1190000004706582

An Analysis of Convolution for Inference

http://www.slideshare.net/nervanasys/an-analysis-of-convolution-for-inference

Understanding Convolution in Deep Learning

• blog: http://timdettmers.com/2015/03/26/convolution-deep-learning/

A guide to convolution arithmetic for deep learning

• arxiv: http://arxiv.org/abs/1603.07285
• github: https://github.com/vdumoulin/conv_arithmetic

Going beyond full utilization: The inside scoop on Nervana’s Winograd kernels

• blog: https://www.nervanasys.com/winograd-2/

Playing with convolutions in TensorFlow: From a short introduction to convolution to a complete model

• blog: http://mourafiq.com/2016/08/10/playing-with-convolutions-in-tensorflow.html
• github: https://github.com/mouradmourafiq/tensorflow-convolution-models

How convolutional neural networks see the world: An exploration of convnet filters with Keras

• blog: https://blog.keras.io/how-convolutional-neural-networks-see-the-world.html

One by One [ 1 x 1 ] Convolution - counter-intuitively useful

http://iamaaditya.github.io/2016/03/one-by-one-convolution/

Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize

• intro: Twitter
• arxiv: https://arxiv.org/abs/1707.02937

Receptive Field

A guide to receptive field arithmetic for Convolutional Neural Networks

https://medium.com/@nikasa1889/a-guide-to-receptive-field-arithmetic-for-convolutional-neural-networks-e0f514068807

Momentum

Why Momentum Really Works

• blog: http://distill.pub/2017/momentum/
• github: https://github.com/distillpub/post–momentum

maxDNN

maxDNN: An Efficient Convolution Kernel for Deep Learning with Maxwell GPUs

• arxiv: http://arxiv.org/abs/1501.06633
• github: https://github.com/eBay/maxDNN

GEMM (General Matrix Matrix Multiply)

Why GEMM is at the heart of deep learning

• blog: http://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/

A full walk through of the SGEMM implementation

• github-wiki: https://github.com/NervanaSystems/maxas/wiki/SGEMM

Backpropagation

Learning representations by back-propagating errors

Learning Internal Representations by Error Propagating

• author: David E. Rumelhart, Geoffrey E. Hinton & Ronald J. Williams. 1986
• paper: http://www.nature.com/nature/journal/v323/n6088/pdf/323533a0.pdf
• mirror: http://pan.baidu.com/s/1bo30gHp
• mirror: http://pan.baidu.com/s/1kVfJ4of

Calculus on Computational Graphs: Backpropagation

• blog: http://colah.github.io/posts/2015-08-Backprop/

Styles of Truncated Backpropagation

• blog: http://r2rt.com/styles-of-truncated-backpropagation.html

Is BackPropagation Necessary?

• blog: http://deliprao.com/archives/191

Backpropagation In Convolutional LSTMs

https://www.doc.ic.ac.uk/~ahanda/ConvLSTMs.pdf

Backward Pass on Conv Layer

Convolutional Neural Networks backpropagation: from intuition to derivation

• blog: https://grzegorzgwardys.wordpress.com/2016/04/22/8/

Backpropagation In Convolutional Neural Networks

• blog: http://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/

Why do we rotate weights when computing the gradients in a convolution layer of a convolution network?

http://soumith.ch/ex/pages/2014/08/07/why-rotate-weights-convolution-gradient/

Note on the implementation of a convolutional neural networks

http://cthorey.github.io./backprop_conv/

Attention

Attention in a Convolutional Neural Net

• blog: http://www.danvatterott.com/blog/2016/09/20/attention-in-a-convolutional-neural-net/

Attention-based Networks

• intro: M. Malinowski. Max Planck Institut Informatik
• slides: http://download.mpi-inf.mpg.de/d2/mmalinow-slides/attention_networks.pdf

Attention in Neural Networks and How to Use It

http://akosiorek.github.io/ml/2017/10/14/visual-attention.html

Softmax

Hierarchical softmax and negative sampling: short notes worth telling

https://towardsdatascience.com/hierarchical-softmax-and-negative-sampling-short-notes-worth-telling-2672010dbe08

Caffe

DIY Deep Learning for Vision: a Hands-On Tutorial with Caffe

• homepage: http://tutorial.caffe.berkeleyvision.org/
• slides: https://docs.google.com/presentation/d/1UeKXVgRvvxg9OUdh_UiC5G71UMscNPlvArsWER41PsU/edit#slide=id.gc2fcdcce7_216_0

Deep learning tutorial on Caffe technology : basic commands, Python and C++ code

http://christopher5106.github.io/deep/learning/2015/09/04/Deep-learning-tutorial-on-Caffe-Technology.html

Using Caffe with your own dataset

https://medium.com/@alexrachnog/using-caffe-with-your-own-dataset-b0ade5d71233

OpenCV 3.0.0-dev: Load Caffe framework models

http://docs.opencv.org/master/d5/de7/tutorial_dnn_googlenet.html#gsc.tab=0

Chainer

Chainer Info

https://github.com/hidetomasuoka/chainer-info

Keras

Keras tutorial

• intro: Tutorial teaching the basics of Keras and some deep learning concepts
• github: https://github.com/jfsantos/keras-tutorial

Keras Tutorial: The Ultimate Beginner’s Guide to Deep Learning in Python

https://elitedatascience.com/keras-tutorial-deep-learning-in-python

Deep Learning with Keras: Tutorial @ EuroScipy 2016

• github: https://github.com/leriomaggio/deep-learning-keras-tensorflow

Transfer Learning and Fine Tuning for Cross Domain Image Classification with Keras

• slides: http://www.slideshare.net/sujitpal/transfer-learning-and-fine-tuning-for-cross-domain-image-classification-with-keras
• mirror: https://pan.baidu.com/s/1gfn1xuj
• github: https://github.com/sujitpal/fttl-with-keras

MXNet

10 Deep Learning projects based on Apache MXNet

https://medium.com/@julsimon/10-deep-learning-projects-based-on-apache-mxnet-8231109f3f64

Awesome MXNet(Beta)

https://github.com/chinakook/Awesome-MXNet

TVM

Optimize Deep Learning GPU Operators with TVM: A Depthwise Convolution Example

http://tvmlang.org/2017/08/22/Optimize-Deep-Learning-GPU-Operators-with-TVM-A-Depthwise-Convolution-Example.html

Theano

Theano Tutorial @ LTI, Carnegie Mellon University

• github: https://github.com/k-kawakami/Theano_Tutorial/blob/master/Theano_Tutorial.ipynb

An Introduction to MXNet/Gluon

• intro: @李沐
• github: https://github.com/mli/cvpr17

TensorFlow

LearningTensorFlow.com: A beginners guide to a powerful framework.

• homepge: http://learningtensorflow.com/index.html

TensorFlow Examples: TensorFlow tutorials and code examples for beginners

• github: https://github.com/aymericdamien/TensorFlow-Examples

Awesome TensorFlow: A curated list of awesome TensorFlow experiments, libraries, and projects

• github: https://github.com/jtoy/awesome-tensorflow/

The Good, Bad, & Ugly of TensorFlow: A survey of six months rapid evolution (+ tips/hacks and code to fix the ugly stuff)

• blog: https://indico.io/blog/the-good-bad-ugly-of-tensorflow/

Tensorflow Tutorials using Jupyter Notebook

• github: https://github.com/sjchoi86/Tensorflow-101

TensorFlow Tutorial

• homepage: http://terryum.io/ml_practice/2016/05/28/TFIntroSlides/
• slides: https://s3.amazonaws.com/www.terryum.io/images/TensorFlow_Intro_160529.pptx
• mirror: http://pan.baidu.com/s/1c5cICY
• github: https://github.com/terryum/TensorFlow_Exercises

FIRST CONTACT WITH TENSORFLOW

• ebook: http://www.jorditorres.org/first-contact-with-tensorflow/

Introduction to TensorFlow

• github: https://github.com/nihit/TensorFlow101

TensorFlow-Tutorials: Simple tutorials using Google’s TensorFlow Framework

• github: https://github.com/nlintz/TensorFlow-Tutorials

Neural Network Toolbox on TensorFlow

• github: https://github.com/ppwwyyxx/tensorpack

Awesome Tensorflow Implementations

• github: https://github.com/TensorFlowKR/awesome_tensorflow_implementations

The Ultimate List of TensorFlow Resources: Books, Tutorials & More

• blog: https://hackerlists.com/tensorflow-resources/

Install TensorFlow: Slides and code from our TensorFlow Workshop

• github: https://github.com/random-forests/tensorflow-workshop

A Tour of TensorFlow

• arxiv: https://arxiv.org/abs/1610.01178

TensorFlow Tutorials

• youtube: https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ
• github: https://github.com/Hvass-Labs/TensorFlow-Tutorials

Shapes and dynamic dimensions in TensorFlow

• blog: https://blog.metaflow.fr/shapes-and-dynamic-dimensions-in-tensorflow-7b1fe79be363#.1293uf94t

TensorFlow saving/restoring and mixing multiple models

https://blog.metaflow.fr/tensorflow-saving-restoring-and-mixing-multiple-models-c4c94d5d7125#.242xy4d46

Getting to Know TensorFlow

• blog: https://hackernoon.com/machine-learning-with-tensorflow-8873fdee2b68#.90j2jz5av

Image Classification and Segmentation with Tensorflow and TF-Slim http://warmspringwinds.github.io/tensorflow/tf-slim/2016/10/30/image-classification-and-segmentation-using-tensorflow-and-tf-slim/

Not another MNIST tutorial with TensorFlow

• blog: https://www.oreilly.com/learning/not-another-mnist-tutorial-with-tensorflow

Dive Into TensorFlow

• Part I: Getting Started with TensorFlow: http://textminingonline.com/dive-into-tensorflow-part-i-getting-started-with-tensorflow
• Part II: Basic Concepts: http://textminingonline.com/dive-into-tensorflow-part-ii-basic-concepts
• Part III: GTX 1080+Ubuntu16.04+CUDA8.0+cuDNN5.0+TensorFlow: http://textminingonline.com/dive-into-tensorflow-part-iii-gtx-1080-ubuntu16-04-cuda8-0-cudnn5-0-tensorflow
• Part IV: Hello MNIST: http://textminingonline.com/dive-into-tensorflow-part-iv-hello-mnist
• Part V: Deep MNIST: http://textminingonline.com/dive-into-tensorflow-part-v-deep-mnist
• Part VI: Beyond Deep Learning: http://textminingonline.com/dive-into-tensorflow-part-vi-beyond-deep-learning

TensorFlow Exercises - focusing on the comparison with NumPy.

• github: https://github.com/Kyubyong/tensorflow-exercises

A Gentle Guide to Using Batch Normalization in Tensorflow

• blog: http://ruishu.io/2016/12/27/batchnorm/

Using TensorFlow in Windows with a GPU

• blog: http://www.heatonresearch.com/2017/01/01/tensorflow-windows-gpu.html

Tensorflow and deep learning - without a PhD

• youtube: https://www.youtube.com/watch?v=vq2nnJ4g6N0
• mirror: https://pan.baidu.com/s/1o8HF9R8
• blog: https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/#0

4 Steps To Learn TensorFlow When You Already Know scikit-learn https://medium.com/@Zelros/4-steps-to-learn-tensorflow-when-you-already-know-scikit-learn-3cd0340456b5#.q206au7u9

Gentlest Introduction to Tensorflow

• part 1: https://medium.com/all-of-us-are-belong-to-machines/the-gentlest-introduction-to-tensorflow-248dc871a224#.fxyclr1ui
• part 2: https://medium.com/all-of-us-are-belong-to-machines/gentlest-introduction-to-tensorflow-part-2-ed2a0a7a624f#.vf7p9upg2
• part 3: https://medium.com/all-of-us-are-belong-to-machines/gentlest-intro-to-tensorflow-part-3-matrices-multi-feature-linear-regression-30a81ebaaa6c#.bvjru1f88
• part 4: https://medium.com/all-of-us-are-belong-to-machines/gentlest-intro-to-tensorflow-4-logistic-regression-2afd0cabc54#.seh1fbr24

learn code with tensorflow

• github: https://github.com/burness/tensorflow-101

TensorFlow Machine Learning Cookbook

• book: https://www.packtpub.com/big-data-and-business-intelligence/tensorflow-machine-learning-cookbook
• github: https://github.com/nfmcclure/tensorflow_cookbook

TensorFlow Image Recognition on a Raspberry Pi

http://svds.com/tensorflow-image-recognition-raspberry-pi/

TensorFlow For Machine Intelligence

• book: https://bleedingedgepress.com/tensor-flow-for-machine-intelligence/
• github: https://github.com/backstopmedia/tensorflowbook

Installing TensorFlow on Raspberry Pi 3 (and probably 2 as well)

• intro: TensorFlow for Raspberry Pi
• github: https://github.com/samjabrahams/tensorflow-on-raspberry-pi

CodinGame: Deep Learning - TensorFlow

• homepage: https://www.codingame.com/games/machine-learning?utm_source=CodinGame&utm_medium=Email&utm_campaign=tensorflow

A Practical Guide for Debugging Tensorflow Codes

Debugging Tips on TensorFlow

• slides: https://wookayin.github.io/TensorflowKR-2016-talk-debugging
• github: https://github.com/wookayin/TensorflowKR-2016-talk-debugging

Tensorflow Projects: Deep learning using tensorflow

• intro: A repo of everything deep and neurally related. Implementations and ideas are largely based on papers from arxiv and implementations, tutorials from the internet.
• github: https://github.com/shekkizh/TensorflowProjects

Machine Learning with TensorFlow

• homepage: http://www.tensorflowbook.com/
• github: https://github.com/BinRoot/TensorFlow-Book
• blog: https://www.manning.com/books/machine-learning-with-tensorflow

Convolutional Networks: from TensorFlow to iOS BNNS

• blog: https://paiv.github.io/blog/2016/09/25/tensorflow-to-bnns.html
• github: https://github.com/paiv/mnist-bnns

Android TensorFlow Machine Learning Example

• blog: https://blog.mindorks.com/android-tensorflow-machine-learning-example-ff0e9b2654cc#.ysg0ss9r2
• github: https://github.com/MindorksOpenSource/AndroidTensorFlowMachineLearningExample

TensorFlow and Deep Learning Tutorials

https://github.com/wagamamaz/tensorflow-tutorial

Finetuning AlexNet with TensorFlow

• blog: https://kratzert.github.io/kratzert.github.io/2017/02/24/finetuning-alexnet-with-tensorflow.html
• github: https://github.com/kratzert/finetune_alexnet_with_tensorflow

Deep Learning examples using Tensorflow

https://github.com/aditya101993/Deep-Learning

How To Write Your Own Tensorflow in C++

https://oneraynyday.github.io/ml/2017/10/20/Tensorflow-C++/

Tensorflow on Android

A Guide to Running Tensorflow Models on Android

• youtube: https://www.youtube.com/watch?v=kFWKdLOxykE
• mirror: http://www.bilibili.com/video/av9806881/index_10.html
• github: https://github.com/llSourcell/A_Guide_to_Running_Tensorflow_Models_on_Android

TensorFlow Android stand-alone demo

• intro: Android demo source files extracted from original TensorFlow source. (TensorFlow r0.10)
• github: https://github.com/miyosuda/TensorFlowAndroidDemo

Torch

Torch Developer Guide

• github: https://github.com/Atcold/Torch-Developer-Guide

PyTorch

Practical PyTorch tutorials

• github: https://github.com/spro/practical-pytorch

The Incredible PyTorch

• github: https://github.com/ritchieng/the-incredible-pytorch

PyTorch quick start: Classifying an image

• blog: http://blog.outcome.io/pytorch-quick-start-classifying-an-image/
• ipn: https://gist.github.com/jbencook/9918217f866c1aa9967391ba62d123b5

tutorial for researchers to learn deep learning with pytorch.

https://github.com/yunjey/pytorch-tutorial

Building a System for Fun!

Facial Recognition On A Jetson TX1 In Tensorflow

• blog: http://www.mattkrzus.com/face.html

Build an AI Cat Chaser with Jetson TX1 and Caffe

• blog: https://devblogs.nvidia.com/parallelforall/ai-cat-chaser-jetson-tx1-caffe/

Deep Learning in Aerial Systems Using Jetson

• blog: https://devblogs.nvidia.com/parallelforall/deep-learning-in-aerial-systems-jetson/
• github: https://github.com/amitibo/auvsi-targets

Cherry Autonomous Racecar (CAR): NCAT ECE Senior Design Project

• intro: Implementation of the CNN from End to End Learning for Self-Driving Cars on a Nvidia Jetson TX1 using Tensorflow and ROS
• github: https://github.com/DJTobias/Cherry-Autonomous-Racecar

Traffic Signs Classification

Traffic signs classification with Deep Learning.

• blog: https://hackernoon.com/traffic-signs-classification-with-deep-learning-b0cb03e23efb#.n0fjehwo6
• github: https://github.com/MehdiSv/TrafficSignsRecognition/

Traffic Sign Recognition with TensorFlow

• blog: https://medium.com/@waleedka/traffic-sign-recognition-with-tensorflow-629dffc391a6#.2khbv6a9a

Traffic signs classification with a convolutional network

http://navoshta.com/traffic-signs-classification/

Convolutional Neural Network for Traffic Sign Classification — CarND

• blog: https://medium.com/@gruby/convolutional-neural-network-for-traffic-sign-classification-carnd-e46e95453899#.lfutfs21v

Talks

A Tour of Deep Learning With C++

• intro: CppCon 2017, Peter Goldsborough
• youtube: https://www.youtube.com/watch?v=9-1lcss0NMg
• bilibili: https://www.bilibili.com/video/av20675156/