Ensemble Methods

Ensemble models are a machine learning approach to combine multiple other weaker models(often decision trees) to produce one optimal predictive model. These models are referred to as base estimators.

Challenges they tackle:

  • High variance: The model is very sensitive to the provided inputs for the learned features.
  • Low accuracy: One model (or one algorithm) to fit the entire training data might not provide you with the nuance your project requires.
  • Features noise and bias: The model relies heavily on too few features while making a prediction.

Objectives for aggregating the output from each model(Voting on final result):

  • Reducing the model error
  • Maintaining the model’s generalization

Types of Ensemble Modeling Techniques: