The words generated by the linear analysis may be of different kinds:
keyword (if, while, ...),
multi-character operator (:=, ->, ...).
Such a kind is called a TOKEN and an element of a kind
is called a LEXEME.
A word is recognized to be a lexeme for a certain token by PATTERN MATCHING.
For instance letter followed by letters and digits is a pattern
that matches a word like x or y with the token id (= identifier).
x y n0
letter followed by letters and digits
any numeric constant
any string of characters (except ``) between `` and ``