Traditional algorithms: All instructions are explicit and only input parameters effects the function.

Machine Learning: Machine Learning functions use a data model generated during training process. Model is trained from dataset and is effected by hyperparameters evaluation process.

Types of Machine Learning models:

• Generative Models: They model distribution of the classes within a dataset to find joint probability(Where given input feature coexists with the desired output) to generate new data such as images, text, code, and audio(music, or speech). They use joint probability distribution: .
• Discriminative Models: They model boundary between classes within a dataset to find the decision boundary(Where boundary of classes are distinguished or the output label of a classifier is ambiguous) and discriminate based on it's training data to make predictions or classifications. They use Conditional Probability: .
• Parametric Models
• Non-Parametric Models
• Tree-Based Models
• Non-Tree-Based Models

Differences between Deep Learning and traditional Machine Learning:

• In Deep Learning the task of Feature Extraction and preparing model for specific patterns is performed by learning model, instead of a Data Scientist or AI Developer.
• Deep Learning model is more complex and can perform general tasks much better. in contrast traditional Machine Learning is domain specific.

• Supervised Learning
  • Classification Algorithms:
    • Decision Trees : (if/then/else) non-contagious data
    • Random Forest
    • SVM Classification: Maximum margin classifier
    • k-Nearest Neighbors (KNN): Group membership in classes based on proximity
    • Discriminant Analysis
    • Naive Bayes Classifier: Updates knowledge based on new evidence(prior knowledge of conditions that might be related to the event)
      • Gaussian Naive Bayes: When the predictors take up a continuous value and are not discrete, we assume that these values are sampled from a Gaussian distribution.
      • Multi-nominal Naive Bayes: The method is used mostly for document classification.
      • Bernoulli Naive Bayes
  • Regression
    • Linear Regression Used for performing predictions on numerical data. it makes predictions on continuous dependent variables with the use of independent variables.
    • Lasso Regression
    • Ridge Regression
    • Elastic-Net
    • Logistic Regression
    • Neural Network Regression
    • Support Vector Machine (SVM)
    • Decision Tree Regression (DTR)
    • Polynomial Regression
    • Assembly Methods
    • Stepwise Regression
• Unsupervised Learning
  • Clustering
    • K-Means, K-mediods, Fuzzy C-Means: Cluster similar data into groups based on centroids
    • Hidden Markov Model (HMM)
    • Hierarchical Clustering
    • DBSCAN
    • Gaussian Mixture
    • Bayesian Models
    • Markov networks; Belief propagation/message-passing algorithms
    • Apriori Algorithm Algorithm for Rule-Based System
  • Self Organizing Maps(SOM)
  • Boltzmann Machines (BM)
  • Autoencoders (AE)
  • eXtreme Gradient Boosting (XGBoost)
  • Outlier Detection Algorithms
  • Dimensionality Reduction
  • Singular Value Decomposition (SVD)
• Semi-supervised Learning
  • Support Vector Machine (SVM)
  • Generative Models
  • Low-density separation
  • Laplacian regularization
  • Heuristic approaches
• Reinforcement Learning
  • Q Learning
  • Deep Q Network (DQN)
  • R Learning
  • Temporal difference Learning
  • SARSA (State-Action-Reward-State-Action) algorithm
• Artificial Neural Networks (ANN): Used in classification, regression, and clustering
  • Basic neural networks and multi-layer perceptron
  • Deep Learning
    • Convolutional Neural Networks (CNN)
    • Recurrent Neural Networks (RNN)
      • Long-Short Term Memory (LSTM)
    • Deep Belief Networks (DBN)
    • Generative Adversarial Networks (GAN)
    • Boltzmann Machines (BM)
    • Self-Organized Maps
    • Generative Models:
      • Generative Adversarial Networks (GAN): a latent variable model
      • Autoencoders (AE)
  • Back Propagation Neural Networks(BPNN)

Learning Material:

• Google: What is Machine Learning?
• Kaggle: Intro to Machine Learning
• Microsoft: Machine Learning for Beginners
• ML Glossary: Machine Learning Glossary
• DAIR.AI: ML Course Notes

Resources:

• Machine learning Python libraries
• Made With ML
• Homemade Machine Learning: Python examples of popular machine learning algorithms with interactive Jupyter demos and math being explained

References:

• Reinforcement Learning: An Introduction (2nd Edition)