Artificial Neural Networks (ANN)

Artificial Neural Networks are Machine Learning algorithm based on the concept of a human neuron. They receive data as inputs, then the neurons consolidate all the information, then the output is generated.

Training Process:

1. First Phase: The first phase consists of applying a nonlinear transformation of the input and create a statistical model as output.
2. Second Phase: The second phase aims at improving the model with a mathematical method known as derivative.
3. This process is repeated in a number of epochs(iterations) until it has reached a tolerable level of accuracy.

Components:

• Neural Network Layers:
  • Input Layer: The first layer of a neural network that receives the data as input.
  • Hidden Layer: This layer is responsible for the calculations performed on features(giveb as input data).
  • Output Layer: Last layer of a neural network that provides the final output.
• Perceptron
• Weights
• Activation Functions
• Bias

Architectures:

• Radial Basis Function (RBF)
• Restricted Boltzmann Machines (RBM)
• Recurrent Neural Networks (RNN)
• Long-Short Term Memory (LSTM)
• Convolutional Neural Networks (CNN)
• Deep Reinforcement Learning (Deep RL)
• Generative Adversarial Networks (GAN)

Classification of Neural Networks:

• Shallow neural network: The Shallow neural network has only one hidden layer between the input and output.
• Deep Neural Networks: Deep Learning

Regularization techniques for ANNs:

• Dropout: This method is ideal for ANNs.
• L1 Regularization
• L2 regularization
• Data Augmentation: It requires fake data to be created as a part of training set.

Optimization Techniques:

• Adam: Currently the best option as it's faster and more efficient. However Stochastic Gradient Descent(SGD) can converge to more optimal solutions
• Gradient Descent
• Stochastic Gradient Descent(SGD)
• Mini-batch Gradient Descent
• Nesterov Accelerated Gradient
• Momentum
• Adagrad
• AdaDelta

Limitations of ANNs:

• Data labeling
• Training dataset’s size
• Models are not explainable: see eXplainable Artificial Intelligence (XAI)

Goals of ANN Optimization:

• Faster performance
• Reduced computational requirements
• Optimized space usage