Wrapper Method

The wrapper methods is a search strategy that create several models which are having different subsets of input feature variables. Later the selected features which result in the best performing model in accordance with the performance metric.


Wrapper method techniques:

  • Forward Selection
  • Backward Elimination
  • Bi-directional elimination
  • Exhaustive Search

Notes:

  • Wrapper method is computationally very expensive.
  • It can accept all variable types.
  • Wrapper method is prone to high Overfitting.
  • Stopping Criteria: It determines when to stop the search.
    • Performance increase
    • Performance decrease
    • Predefined number of features is reached