Embedding

Embedding is the process of converting tokens(often words) into numbers(often vectors) which are easier for a computers to process.

Notes:

  • Often embedding changes each token into a vector (a list of numbers), but other types of embeddings exist as well.
  • If two Tokens are similar, then the numbers in their corresponding vectors are similar to each other.
  • Embedding can be imagined as a geometric shape such as a vector of two numbers, I.e. . they are are not limited to an easy to imagine dimension such as 2D or 3d, rather they may have thousands of dimensions.