Lexical Analysis - Campbell, Bill; Iyer, Swami; Akbal-Delibas, Bahar (Introduction to Compiler Construction in a Java World, 2013)
The first step in compiling a program is to break it into tokens. For example, given the j-- programwe want to produce the sequence of tokens package, pass, ;, import, java, ., lang, ., System, ;, public, class, Factorial, {, public, static, int, factorial, (, int, n,), {, if, (, n, <=, 0, ), }, return, 1, ;, else, return, n, *, factorial, (, n, -, 1, ), }, ;, }, public, static, void, main, (, String, [, ], args, ), }, {, int, x, =, n, ;, System, ., out, ., println, (, x, +, "!=", +, factorial, (, x, ), ), }, ;, }, static, int, n, =, 5, ;, and }.