Model Resource Management

• High Dimensionality: Artificial Neural Networks (ANN) mostly ignore irrelevant features. but adding them to model increases resource costs such as computation demands and storage, adds unnecessary complexity and lowers interpretability, and increase the risk of over-fitting.
  • Dimensionality Reduction Techniques: We want to keep as much predictive information as possible with as few features as possible.
    • Manual dimensionality reduction: involves understanding the data and business context and leveraging domain knowledge to perform feature engineering and selection.
    • algorithmic approaches for dimensionality reduction: Dimensionality Reduction
• Quantization: Quantization is a technique for optimizing ML models, where it transforms a model into an equivalent representation using lower-precision parameters and computations. Although quantization may reduce model accuracy, it improves execution performance and efficiency by shrinking neural network size, reducing computational resources, and decreasing latency.
• Best Model Selection: Selecting smallest model with acceptable performance.
• Pruning: Pruning is an optimization technique that improves model efficiency by removing parts that did not contribute substantially to producing accurate results.

Related:

• Model Compression Techniques