Bagging

Bootstrap Aggregation (Bagging) is an Ensemble Method(meta-algorithm) used in parallel learning.
It's used in training multiple Weak Learners on random subsets of the training data with each weak learner trained on a different subset of the data. This Weak Learners are then aggregated to make a final model in a way that it's predictions are averaged.

Training
Training
Training
Training
Prediction
Prediction
Prediction
Prediction
Vote/Agregate
Input Data
Bootstrap Sample
Bootstrap Sample
...
Bootstrap Sample
Weak Learner
Weak Learner
...
Weak Learner
Aggrigator
Final Model

Goals:

  • avoid Overfitting of data
  • reduce the variance in the predictions

Algorithm:

  1. Select random subset
  2. Bootstrap Sampling
  3. Bootstrapping
  4. Independent Model Training
  5. Majority Voting
  6. Aggregation

Notes:

  • Models used in bagging are often the same type.
  • For Regression usually the average is taken for output.
  • Bagging focuses on creating a stable model by averaging out individual errors.

Bagging is used for:


Algorithms using bagging: