Token

A token is the base unit in natural language processing and can be thought of as a grouping of one or more characters. A token might be at the character level, meaning a character represents a token, or at the word level, meaning a word represents a token.