Regression Models

Regression Models investigate and discover the relationship between dependent(target, ) and independent(predictor, ) variables. This models perform Regression Tasks and use Regression Mathematics.

Linear Assumption

Regression Models assume a linear dependency between the dependend and independent variables. If such a Linear Relationship doens't exist, then regression is not a good solution and machine learning model should be used that don't follow Linear Assumption.

Attributes of regression models:

  • Number of independent variables
    • Univariate: 1 feature or independent variables
    • Bivariate: 2 features or independent variables
    • Multivariate: more than 2 feature or independent variables
  • Shape of the regression
  • Type of dependent variables



  • Regression models are based on mathematical Regression Mathematics topic.
  • Random Forest can be utilized in regression tasks. Then the mean or average prediction of the individual trees is returned.
  • Regression models are highly interpretable and fast to train.
  • Based on degree of independent variable they are divided into:
  • MSE and MAE Loss Functions are used for regression tasks.
  • Regression models assume Linearity, Multivariate normality, Homoscedasticity, and no Multicollinearity.