Dimensionality Reduction

Dimensionality Reduction Technique(Feature Reduction) is the process of reducing the number of features in a resource heavy computation without losing important information. it reduces model complexity and Overfitting at the cost of accuracy.

Dimensionality Reduction is part of two processes:

  • Feature Selection: select a subset of features using Feature Selection Techniques. It perform dimensionality reduction by disregarding less valuable features.
  • Feature Extraction: extracting or deriving information from the original features. It perform dimensionality reduction by disregarding many features in favor of fewer, more valuable features.

Dimensionality Reduction is a response to the problems caused by the Curse of Dimensionality, such as increased complexity of models, increased errors, and need for more data.