What's is Tokens Related: Specification of Tokens - Lexical Analysis,...
In general, a token is an object that represents something else, such as another object (either physical or virtual), or an abstract concept as, for example, a gift is sometimes referred to as a token of the giver's esteem for the recipient. In computers, there are a number of types of tokens.
1) In a token ring network, the presence of a token (which is simply a particular bit setting) in a continually circulating transmission stream allows a device to change the bit setting (thus taking the token) and put a message in its place. The receiver of the message elsewhere in the token ring network removes the message and resets the bit setting (thus putting the token back) so that someone else in the ring of devices will be able to have a turn at using that message space.
2) A programming token is the basic component of source code. Characters are categorized as one of five classes of tokens that describe their functions (constants, identifiers, operators, reserved words, and separators) in accordance with the rules of the programming language.
3) A security token is a physical device, such as a special smart card, that together with something that a user knows, such as a PIN, will enable authorized access to a computer system or network.