Ensemble Model

Ensemble Models are a machine learning approaches to combine multiple weaker models(Weak Learners) to produce one optimal predictive model(Strong Learner).


Notes:

  • Weak Learners are an essential part of Ensemble Models.
  • Aggregation method should be suitable for base modes. For high-bias/low-variance models, we should choose models that reduce bias, for low-bias/high-variance models, we should choose models that reduce variance.

Challenges they tackle:

  • High variance: The model is very sensitive to the provided inputs for the learned features.
  • Low accuracy: One model (or one algorithm) to fit the entire training data might not provide you with the nuance your project requires.
  • Features noise and bias: The model relies heavily on too few features while making a prediction.

Objectives for aggregating the output from each model(Voting on final result):

  • Reducing the model error
  • Maintaining the model’s generalization

Types of Ensemble Modeling Techniques: