Page 1 Lexical Analysis Page 2 Lexical Analysis The Scanner ? The job of the scanner is to read characters from the source code and form them into tokens. ? Token classes include: – Identifiers – Literals (numeric, string, character) – Keywords and reserved words – Operators – Others Page 3 Lexical Analysis The Scanner ? The job of the scanner is to read characters from the source code and form them into tokens. ? Token classes include: – Identifiers – Literals (numeric, string, character) – Keywords and reserved words – Operators – Others Representing Tokens ? using the #define macro (C/C++): #define IF 256 #define THEN 257 #define ELSE 258 ... Page 4 Lexical Analysis The Scanner ? The job of the scanner is to read characters from the source code and form them into tokens. ? Token classes include: – Identifiers – Literals (numeric, string, character) – Keywords and reserved words – Operators – Others Representing Tokens ? using the #define macro (C/C++): #define IF 256 #define THEN 257 #define ELSE 258 ... Representing Tokens ? Using named constants (Java): public static final IF = 256; public static final THEN = 257; public static final ELSE = 258; ... Page 5 Lexical Analysis The Scanner ? The job of the scanner is to read characters from the source code and form them into tokens. ? Token classes include: – Identifiers – Literals (numeric, string, character) – Keywords and reserved words – Operators – Others Representing Tokens ? using the #define macro (C/C++): #define IF 256 #define THEN 257 #define ELSE 258 ... Representing Tokens ? Using named constants (Java): public static final IF = 256; public static final THEN = 257; public static final ELSE = 258; ... Representing Tokens ? Token classes may have attributes – Identifiers have a string name – Numeric literals have a value – Relational operators have 1 of 6 values ? We need more than a single scalar object to represent tokens. – struct – classRead More