Video Applications

Papers

Published: 09 Oct 2015

Unsupervised Learning

Restricted Boltzmann Machine (RBM)

Published: 09 Oct 2015

Transfer Learning

Papers

Published: 09 Oct 2015

Training Deep Neural Networks

Tutorials

Published: 09 Oct 2015

Tracking

Learning A Deep Compact Image Representation for Visual Tracking

Hierarchical Convolutional Features for Visual Tracking

Robust Visual Tracking via Convolutional Networks

Transferring Rich Feature Hierarchies for Robust Visual Tracking

Learning Multi-Domain Convolutional Neural Networks for Visual Tracking

RATM: Recurrent Attentive Tracking Model

Understanding and Diagnosing Visual Tracking Systems

Recurrently Target-Attending Tracking

Visual Tracking with Fully Convolutional Networks

Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks

Learning to Track at 100 FPS with Deep Regression Networks

Learning by tracking: Siamese CNN for robust target association

Fully-Convolutional Siamese Networks for Object Tracking

Hedged Deep Tracking

Spatially Supervised Recurrent Convolutional Neural Networks for Visual Object Tracking

Visual Tracking via Shallow and Deep Collaborative Model

Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking

Unsupervised Learning from Continuous Video in a Scalable Predictive Recurrent Network

Modeling and Propagating CNNs in a Tree Structure for Visual Tracking

Robust Scale Adaptive Kernel Correlation Filter Tracker With Hierarchical Convolutional Features

Deep Tracking on the Move: Learning to Track the World from a Moving Vehicle using Recurrent Neural Networks

OTB Results: visual tracker benchmark results

Convolutional Regression for Visual Tracking

Semantic tracking: Single-target tracking with inter-supervised convolutional networks

SANet: Structure-Aware Network for Visual Tracking

ECO: Efficient Convolution Operators for Tracking

Dual Deep Network for Visual Tracking

Deep Motion Features for Visual Tracking

Globally Optimal Object Tracking with Fully Convolutional Networks

Robust and Real-time Deep Tracking Via Multi-Scale Domain Adaptation

Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies

Large Margin Object Tracking with Circulant Feature Maps

DCFNet: Discriminant Correlation Filters Network for Visual Tracking

End-to-end representation learning for Correlation Filter based tracking

Context-Aware Correlation Filter Tracking

Robust Multi-view Pedestrian Tracking Using Neural Networks

https://arxiv.org/abs/1704.06370

Re3 : Real-Time Recurrent Regression Networks for Object Tracking

Robust Tracking Using Region Proposal Networks

https://arxiv.org/abs/1705.10447

Hierarchical Attentive Recurrent Tracking

Siamese Learning Visual Tracking: A Survey

https://arxiv.org/abs/1707.00569

Robust Visual Tracking via Hierarchical Convolutional Features

CREST: Convolutional Residual Learning for Visual Tracking

Learning Policies for Adaptive Tracking with Deep Feature Cascades

Recurrent Filter Learning for Visual Tracking

Correlation Filters with Weighted Convolution Responses

Semantic Texture for Robust Dense Tracking

https://arxiv.org/abs/1708.08844

Learning Multi-frame Visual Representation for Joint Detection and Tracking of Small Objects

https://arxiv.org/abs/1709.04666

Tracking Persons-of-Interest via Unsupervised Representation Adaptation

End-to-end Flow Correlation Tracking with Spatial-temporal Attention

https://arxiv.org/abs/1711.01124

UCT: Learning Unified Convolutional Networks for Real-time Visual Tracking

Pixel-wise object tracking

https://arxiv.org/abs/1711.07377

MAVOT: Memory-Augmented Video Object Tracking

https://arxiv.org/abs/1711.09414

Learning Hierarchical Features for Visual Object Tracking with Recursive Neural Networks

https://arxiv.org/abs/1801.02021

Parallel Tracking and Verifying

https://arxiv.org/abs/1801.10496

Saliency-Enhanced Robust Visual Tracking

https://arxiv.org/abs/1802.02783

A Twofold Siamese Network for Real-Time Object Tracking

Learning Dynamic Memory Networks for Object Tracking

https://arxiv.org/abs/1803.07268

Context-aware Deep Feature Compression for High-speed Visual Tracking

VITAL: VIsual Tracking via Adversarial Learning

Unveiling the Power of Deep Tracking

https://arxiv.org/abs/1804.06833

A Novel Low-cost FPGA-based Real-time Object Tracking System

MV-YOLO: Motion Vector-aided Tracking by Semantic Object Detection

https://arxiv.org/abs/1805.00107

Information-Maximizing Sampling to Promote Tracking-by-Detection

https://arxiv.org/abs/1806.02523

Instance Segmentation and Tracking with Cosine Embeddings and Recurrent Hourglass Networks

Face Tracking

Mobile Face Tracking: A Survey and Benchmark

https://arxiv.org/abs/1805.09749

Multi-Object Tracking (MOT)

Virtual Worlds as Proxy for Multi-Object Tracking Analysis

Multi-Class Multi-Object Tracking using Changing Point Detection

POI: Multiple Object Tracking with High Performance Detection and Appearance Feature

Simple Online and Realtime Tracking with a Deep Association Metric

Deep Network Flow for Multi-Object Tracking

Online Multi-Object Tracking Using CNN-based Single Object Tracker with Spatial-Temporal Attention Mechanism

https://arxiv.org/abs/1708.02843

Recurrent Autoregressive Networks for Online Multi-Object Tracking

https://arxiv.org/abs/1711.02741

SOT for MOT

Multi-Target, Multi-Camera Tracking by Hierarchical Clustering: Recent Progress on DukeMTMC Project

https://arxiv.org/abs/1712.09531

Multiple Target Tracking by Learning Feature Representation and Distance Metric Jointly

https://arxiv.org/abs/1802.03252

Tracking Noisy Targets: A Review of Recent Object Tracking Approaches

https://arxiv.org/abs/1802.03098

Machine Learning Methods for Solving Assignment Problems in Multi-Target Tracking

Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World

Features for Multi-Target Multi-Camera Tracking and Re-Identification

Trajectory Factory: Tracklet Cleaving and Re-connection by Deep Siamese Bi-GRU for Multiple Object Tracking

Automatic Adaptation of Person Association for Multiview Tracking in Group Activities

Improving Online Multiple Object tracking with Deep Metric Learning

https://arxiv.org/abs/1806.07592

Multiple People Tracking

Multi-Person Tracking by Multicut and Deep Matching

Joint Flow: Temporal Flow Fields for Multi Person Tracking

Multiple People Tracking by Lifted Multicut and Person Re-identification

Tracking by Prediction: A Deep Generative Model for Mutli-Person localisation and Tracking

Tracking with Reinforcement Learning

Deep Reinforcement Learning for Visual Object Tracking in Videos

Visual Tracking by Reinforced Decision Making

End-to-end Active Object Tracking via Reinforcement Learning

https://arxiv.org/abs/1705.10561

Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning

Tracking as Online Decision-Making: Learning a Policy from Streaming Videos with Reinforcement Learning

https://arxiv.org/abs/1707.04991

Detect to Track and Track to Detect

Projects

Tensorflow_Object_Tracking_Video

Published: 09 Oct 2015

Super-Resolution

Papers

Published: 09 Oct 2015

Segmentation

Papers

Published: 09 Oct 2015

RNN and LSTM

Types of RNN

1) Plain Tanh Recurrent Nerual Networks

2) Gated Recurrent Neural Networks (GRU)

3) Long Short-Term Memory (LSTM)

Tutorials

The Unreasonable Effectiveness of Recurrent Neural Networks

Understanding LSTM Networks

A Beginner’s Guide to Recurrent Networks and LSTMs

http://deeplearning4j.org/lstm.html

A Deep Dive into Recurrent Neural Nets

http://nikhilbuduma.com/2015/01/11/a-deep-dive-into-recurrent-neural-networks/

Exploring LSTMs

http://blog.echen.me/2017/05/30/exploring-lstms/

A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the “echo state network” approach

Long Short-Term Memory: Tutorial on LSTM Recurrent Networks

http://people.idsia.ch/~juergen/lstm/index.htm

LSTM implementation explained

http://apaszke.github.io/lstm-explained.html

Recurrent Neural Networks Tutorial

Recurrent Neural Networks in DL4J

http://deeplearning4j.org/usingrnns.html

Learning RNN Hierarchies

Element-Research Torch RNN Tutorial for recurrent neural nets : let’s predict time series with a laptop GPU

RNNs in Tensorflow, a Practical Guide and Undocumented Features

Learning about LSTMs using Torch

Build a Neural Network (LIVE)

Deriving LSTM Gradient for Backpropagation

http://wiseodd.github.io/techblog/2016/08/12/lstm-backprop/

TensorFlow RNN Tutorial

https://svds.com/tensorflow-rnn-tutorial/

RNN Training Tips and Tricks

https://github.com/karpathy/char-rnn#tips-and-tricks

Tips for Training Recurrent Neural Networks

http://danijar.com/tips-for-training-recurrent-neural-networks/

A Tour of Recurrent Neural Network Algorithms for Deep Learning

http://machinelearningmastery.com/recurrent-neural-network-algorithms-for-deep-learning/

Fundamentals of Deep Learning – Introduction to Recurrent Neural Networks

https://www.analyticsvidhya.com/blog/2017/12/introduction-to-recurrent-neural-networks/

Essentials of Deep Learning : Introduction to Long Short Term Memory

https://www.analyticsvidhya.com/blog/2017/12/fundamentals-of-deep-learning-introduction-to-lstm/

How to build a Recurrent Neural Network in TensorFlow

How to build a Recurrent Neural Network in TensorFlow (1/7)

https://medium.com/@erikhallstrm/hello-world-rnn-83cd7105b767#.2vozogqf7

Using the RNN API in TensorFlow (2/7)

https://medium.com/@erikhallstrm/tensorflow-rnn-api-2bb31821b185#.h0ycrjuo3

Using the LSTM API in TensorFlow (3/7)

https://medium.com/@erikhallstrm/using-the-tensorflow-lstm-api-3-7-5f2b97ca6b73#.k7aciqaxn

Using the Multilayered LSTM API in TensorFlow (4/7)

https://medium.com/@erikhallstrm/using-the-tensorflow-multilayered-lstm-api-f6e7da7bbe40#.dj7dy92m5

Using the DynamicRNN API in TensorFlow (5/7)

https://medium.com/@erikhallstrm/using-the-dynamicrnn-api-in-tensorflow-7237aba7f7ea#.49qw259ks

Using the Dropout API in TensorFlow (6/7)

https://medium.com/@erikhallstrm/using-the-dropout-api-in-tensorflow-2b2e6561dfeb#.a7mc3o9aq

Unfolding RNNs

Unfolding RNNs: RNN : Concepts and Architectures

Unfolding RNNs II: Vanilla, GRU, LSTM RNNs from scratch in Tensorflow

Train RNN

On the difficulty of training Recurrent Neural Networks

A Simple Way to Initialize Recurrent Networks of Rectified Linear Units

Batch Normalized Recurrent Neural Networks

Sequence Level Training with Recurrent Neural Networks

Training Recurrent Neural Networks (PhD thesis)

Deep learning for control using augmented Hessian-free optimization


Hierarchical Conflict Propagation: Sequence Learning in a Recurrent Deep Neural Network

Recurrent Batch Normalization

Batch normalized LSTM for Tensorflow

Optimizing Performance of Recurrent Neural Networks on GPUs

Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations

Explaining and illustrating orthogonal initialization for recurrent neural networks

Professor Forcing: A New Algorithm for Training Recurrent Networks

Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences

Tuning Recurrent Neural Networks with Reinforcement Learning (RL Tuner)

Capacity and Trainability in Recurrent Neural Networks

Learn To Execute Programs

Learning to Execute

Neural Programmer-Interpreters

A Programmer-Interpreter Neural Network Architecture for Prefrontal Cognitive Control

Convolutional RNN: an Enhanced Model for Extracting Features from Sequential Data

Neural Random-Access Machines

Attention Models

Recurrent Models of Visual Attention

Recurrent Model of Visual Attention

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

A Neural Attention Model for Abstractive Sentence Summarization

Effective Approaches to Attention-based Neural Machine Translation

Generating Images from Captions with Attention

Attention and Memory in Deep Learning and NLP

Survey on the attention based RNN model and its applications in computer vision

Attention in Long Short-Term Memory Recurrent Neural Networks

How to Visualize Your Recurrent Neural Network with Attention in Keras

Papers

Generating Sequences With Recurrent Neural Networks

A Clockwork RNN

Unsupervised Learning of Video Representations using LSTMs

An Empirical Exploration of Recurrent Network Architectures

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

LSTM: A Search Space Odyssey

Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets

A Critical Review of Recurrent Neural Networks for Sequence Learning

Visualizing and Understanding Recurrent Networks

Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks

Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

Grid Long Short-Term Memory

Depth-Gated LSTM

Deep Knowledge Tracing

Top-down Tree Long Short-Term Memory Networks

Improving performance of recurrent neural network with relu nonlinearity

Alternative structures for character-level RNNs

Long Short-Term Memory-Networks for Machine Reading

Lipreading with Long Short-Term Memory

Associative Long Short-Term Memory

Representation of linguistic form and function in recurrent neural networks

Architectural Complexity Measures of Recurrent Neural Networks

Easy-First Dependency Parsing with Hierarchical Tree LSTMs

Training Input-Output Recurrent Neural Networks through Spectral Methods

Sequential Neural Models with Stochastic Layers

Neural networks with differentiable structure

What You Get Is What You See: A Visual Markup Decompiler

Hybrid computing using a neural network with dynamic external memory

Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks

Dilated Recurrent Neural Networks

Excitation Backprop for RNNs

https://arxiv.org/abs/1711.06778

Recurrent Relational Networks for Complex Relational Reasoning

Learning Compact Recurrent Neural Networks with Block-Term Tensor Decomposition

  • intro: University of Electronic Science and Technology of China & Brown University & University of Utah & XJERA LABS PTE.LTD
  • arxiv: https://arxiv.org/abs/1712.05134

LSTMVis

Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks

Recurrent Memory Array Structures

Recurrent Highway Networks

DeepSoft: A vision for a deep model of software

Recurrent Neural Networks With Limited Numerical Precision

Hierarchical Multiscale Recurrent Neural Networks

LightRNN

LightRNN: Memory and Computation-Efficient Recurrent Neural Networks

Full-Capacity Unitary Recurrent Neural Networks

DeepCoder: Learning to Write Programs

shuttleNet: A biologically-inspired RNN with loop connection and parameter sharing

Tracking the World State with Recurrent Entity Networks

Robust LSTM-Autoencoders for Face De-Occlusion in the Wild

Simplified Gating in Long Short-term Memory (LSTM) Recurrent Neural Networks

The Statistical Recurrent Unit

Factorization tricks for LSTM networks

Bayesian Recurrent Neural Networks

Fast-Slow Recurrent Neural Networks

Visualizing LSTM decisions

https://arxiv.org/abs/1705.08153

Recurrent Additive Networks

Recent Advances in Recurrent Neural Networks

Grow and Prune Compact, Fast, and Accurate LSTMs

https://arxiv.org/abs/1805.11797

Projects

NeuralTalk (Deprecated): a Python+numpy project for learning Multimodal Recurrent Neural Networks that describe images with sentences

NeuralTalk2: Efficient Image Captioning code in Torch, runs on GPU

char-rnn in Blocks

Project: pycaffe-recurrent

Using neural networks for password cracking

torch-rnn: Efficient, reusable RNNs and LSTMs for torch

Deploying a model trained with GPU in Torch into JavaScript, for everyone to use

LSTM implementation on Caffe

JNN: Java Neural Network Library

LSTM-Autoencoder: Seq2Seq LSTM Autoencoder

RNN Language Model Variations

keras-extra: Extra Layers for Keras to connect CNN with RNN

Dynamic Vanilla RNN, GRU, LSTM,2layer Stacked LSTM with Tensorflow Higher Order Ops

PRNN: A fast implementation of recurrent neural network layers in CUDA

min-char-rnn: Minimal character-level language model with a Vanilla Recurrent Neural Network, in Python/numpy

rnn: Recurrent Neural Network library for Torch7’s nn

word-rnn-tensorflow: Multi-layer Recurrent Neural Networks (LSTM, RNN) for word-level language models in Python using TensorFlow

tf-char-rnn: Tensorflow implementation of char-rnn

translit-rnn: Automatic transliteration with LSTM

tf_lstm.py: Simple implementation of LSTM in Tensorflow in 50 lines (+ 130 lines of data generation and comments)

Handwriting generating with RNN

RecNet - Recurrent Neural Network Framework

Blogs

Survey on Attention-based Models Applied in NLP

http://yanran.li/peppypapers/2015/10/07/survey-attention-model-1.html

Survey on Advanced Attention-based Models

http://yanran.li/peppypapers/2015/10/07/survey-attention-model-2.html

Online Representation Learning in Recurrent Neural Language Models

http://www.marekrei.com/blog/online-representation-learning-in-recurrent-neural-language-models/

Fun with Recurrent Neural Nets: One More Dive into CNTK and TensorFlow

http://esciencegroup.com/2016/03/04/fun-with-recurrent-neural-nets-one-more-dive-into-cntk-and-tensorflow/

Materials to understand LSTM

https://medium.com/@shiyan/materials-to-understand-lstm-34387d6454c1#.4mt3bzoau

Understanding LSTM and its diagrams

:star::star::star::star::star:

Persistent RNNs: 30 times faster RNN layers at small mini-batch sizes

Persistent RNNs: Stashing Recurrent Weights On-Chip

All of Recurrent Neural Networks

https://medium.com/@jianqiangma/all-about-recurrent-neural-networks-9e5ae2936f6e#.q4s02elqg

Rolling and Unrolling RNNs

https://shapeofdata.wordpress.com/2016/04/27/rolling-and-unrolling-rnns/

Sequence prediction using recurrent neural networks(LSTM) with TensorFlow: LSTM regression using TensorFlow

LSTMs

Machines and Magic: Teaching Computers to Write Harry Potter

Crash Course in Recurrent Neural Networks for Deep Learning

http://machinelearningmastery.com/crash-course-recurrent-neural-networks-deep-learning/

Understanding Stateful LSTM Recurrent Neural Networks in Python with Keras

http://machinelearningmastery.com/understanding-stateful-lstm-recurrent-neural-networks-python-keras/

Recurrent Neural Networks in Tensorflow

Written Memories: Understanding, Deriving and Extending the LSTM

http://r2rt.com/written-memories-understanding-deriving-and-extending-the-lstm.html

Attention and Augmented Recurrent Neural Networks

Interpreting and Visualizing Neural Networks for Text Processing

https://civisanalytics.com/blog/data-science/2016/09/22/neural-network-visualization/

A simple design pattern for recurrent deep learning in TensorFlow

RNN Spelling Correction: To crack a nut with a sledgehammer

Recurrent Neural Network Gradients, and Lessons Learned Therein

A noob’s guide to implementing RNN-LSTM using Tensorflow

http://monik.in/a-noobs-guide-to-implementing-rnn-lstm-using-tensorflow/

Non-Zero Initial States for Recurrent Neural Networks

Interpreting neurons in an LSTM network

http://yerevann.github.io/2017/06/27/interpreting-neurons-in-an-LSTM-network/

Optimizing RNN (Baidu Silicon Valley AI Lab)

Optimizing RNN performance

Optimizing RNNs with Differentiable Graphs

Resources

Awesome Recurrent Neural Networks - A curated list of resources dedicated to RNN

Jürgen Schmidhuber’s page on Recurrent Neural Networks

http://people.idsia.ch/~juergen/rnn.html

Reading and Questions

Are there any Recurrent convolutional neural network network implementations out there ?

Published: 09 Oct 2015