Pruning

Pruning is the task of determining importance of components of Artificial Neural Networks (ANN) models and pruning the unimportant parts. Pruning methods include weight pruning, channel(synapse) pruning, and neuron pruning.


Advantages:

  • Smaller model
  • Faster model
  • Reduce computational resource requirements.
  • Reduce computational resources required for training.

Challenges:

  • Avoiding Over-pruning
  • Balancing compression and accuracy

Methods:

  • Weight Pruning
    • Magnitude pruning: pruning based on near zero weights.
    • Iterative Pruning: pruning over several epochs
  • Channel(synapse) Pruning
  • Neuron Pruning
  • Layer Pruning
  • Filter Pruning
  • Feature map pruning in Convolutional Neural Networks (CNN)
  • Activation Pruning
  • Redundancy Pruning