Low-rank factorization

Low-Rank Factorization technique is used to identifies redundant parameters in Artificial Neural Networks (ANN) using matrix and Tensor decomposition on weight matrices into smaller “low-rank” matrices.


Notes:

  • The low-rank factorization of the dense layer matrices improves model size for storage.
  • The factorization of convolutional layers makes the inference process faster.