Restricted Boltzmann Machines (RBM)

RBMs are an Unsupervised Learning algorithm with shallow, two-layer neural networks without any intra-layer connections that constitute the building blocks of Deep Belief Networks (DBN). They are similar to Boltzmann Machines (BM) with restriction in connecting groups of neurons to another group(instead of Hopfield Networks (HN) like full connection) where their neurons must form a bipartite graph. This restriction makes them more usable.

RBMs can be trained like Feed-Forward Neural Networks (FFNN) however instead of passing data forward and then back-propagating, you forward pass the data and then backward pass the data (back to the first layer). After that you train with forward-and-back-propagation.

Notes:

• It's an Unsupervised Learning algorithm. However building RBMs involves both unsupervised pre-training and supervised fine-tuning.
• Restricted means that there are no connections within a layer.
• Composition: visible/input layer and the hidden layer without any intra-layer connections
• They are less used, with Generative Adversarial Networks (GAN) and Variational Autoencoders (VAE).
• DBNs are less computationally expensive and less vulnerable to the Vanishing Gradient problem than Feed-Forward Neural Networks (FFNN). However they become linearly more computationally expensive for each added layer.

Applications:

• Recommender Systems: Collaborative Filtering
• Pattern Recognition
• Classification
• Regression
• Dimensionality Reduction
• Feature Engineering
• Topic Modeling
• Computer Vision including:
  • Image Classification
  • Video sequencing
  • Motion capture