Machine Learning Vocabulary

The Machine Learning and Deep Learning vocabulary. If you are new to machine learning, researching each of the below terms will help you get started in this field. I put this list together myself after a year-long period of trial and error, false starts, and finally some level of understanding of this realm. My pain is your gain πŸ™‚

This is by no means exhaustive, but if you browse through this list and don’t recognize what any of it is, I would recommend just googling each term and spending 10 minutes or so getting to know each idea. Unlike typical application programming, in ML there is a big giant wall of vocabulary that makes it hard to get in to. So this is counter-intuitive to just go through the vocabulary, but I do believe it’s the best way to get started.

Common Python libraries
numpy # use numbers matrices
pandas # Work with data as tables
marplotlib.pyplot # display graphs
tensorflow # wrapper on TensorFlow

Cutting Edge Stuff
Dynamic computation graphs
Synthetic Gradients
Memory Networks
Serial-to-Serial Networks

Commonly used variables
X, W, b, Y

Deep Learning Vocab
MLP (Multi-Layer Perceptron)
Hidden Layers
Activation Function or Operator
Dense Layer
Convolutional Layer
Sparse Data
One-hot encoding

Note:

Lots of these terms can be found by reviewing functions in the keras code, like this for activations for example:
https://github.com/fchollet/keras/blob/master/keras/activations.py

More Vocab
Unsupervised Learning
Supervised Learning
Reinforcement Learning
Q-Learning
Deep Learning
Linear Regression
Neural Networks

Libraries, Platforms, and SDKs for ML
Keras
Torch
Caffe
DSSTNE
TensorFlow
Theano
Lasagne

Types of networks
LSTM (Long Short Term Memory)
RNN (Recurrent Neural Network)
CNN (Convolution Neural Network)

Gradient
Gradient Descent
Cost Function
RMS

Optimizers
Adam Optimizer
SGD
RMSProp
Adadelta
Adagrad
Adamax
Nadam

Activations
http://tflearn.org/activations/
ELU
RELU
Tanh
Sigmoid
Hard Sigmoid
Linear
softmax
softplus
softsign

Data Sets / Repositories
MNIST
UCI Machine Learning Repository (The motherload)
notMNIST
MIAS
IRIS

More Stuff
AlexNet
SciPy
CUDA
cuDNN
Autoencoder
One-shot Learning