```
tags:
- AI/Algorithms/ANN
- AI/ML
aliases:
- Artificial Neural Networks
- ANN
- Multi-layer Perceptrons
```

Artificial Neural Networks are Machine Learning algorithm based on the concept of a human neuron. They receive data as inputs, then the neurons consolidate all the information, then the output is generated.

Training Process:

- First Phase: The first phase consists of applying a nonlinear transformation of the input and create a statistical model as output.
- Second Phase: The second phase aims at improving the model with a mathematical method known as derivative.
- This process is repeated in a number of epochs(iterations) until it has reached a tolerable level of accuracy.

Info

First a neural network is trained with a given dataset and optimized through testing and evaluation. Then the model and it's algorithm is used as a software function to perform task.

Components:

- Neural Network Layers:
- Input Layer: The first layer of a neural network that receives the data as input.
- Hidden Layer: This layer is responsible for the calculations performed on features(giveb as input data).
- Output Layer: Last layer of a neural network that provides the final output.

- Perceptron
- Weights
- Activation Functions
- Bias

Architectures:

Classification of Neural Networks:

- Shallow neural network: The Shallow neural network has only one hidden layer between the input and output.
- Deep Neural Networks: Deep Learning

Regularization techniques for ANNs:

- Dropout: This method is ideal for ANNs.
- L1 Regularization
- L2 Regularization
- Data Augmentation: It requires fake data to be created as a part of training set.

Optimization Techniques:

- Adam: Currently the best option as it's faster and more efficient. However Stochastic Gradient Descent(SGD) can converge to more optimal solutions
- Gradient Descent
- Stochastic Gradient Descent(SGD)
- Mini-batch Gradient Descent
- Nesterov Accelerated Gradient
- Momentum
- Adagrad
- AdaDelta

Limitations of ANNs:

- Data labeling
- Training dataset’s size
- Models are not explainable: see eXplainable Artificial Intelligence (XAI)

Goals of ANN Optimization:

- Faster performance
- Reduced computational requirements
- Optimized space usage

Interactive Graph

Table Of Contents