Positional Encoding

Positional encoding is a form of Word Embedding where starting position in sentence is added to each word vector to keep track of their position in sentence.

The reason for Position Encoding is that vector operations are not effected by order of words, ignoring the structure of sentences. so sentences such as "I'm healthy, he's ill." and "I'm ill, he's healthy." which have similar vector calculation results could be differentiated.


Positional encoding is a representation of the order in which input words occur.