Rework YANG lexer/parser
The definitions in the parser and lexer are quite arcane and are
actually wrong, not allowing for a number of edge cases, which are
completely valid.
Furthermore the definition of IDENTIFIER is wrong, as it allows /
and : to appear in it -- effectively ruining the tokenization in
parser.
Refactor the lexer to perform correct tokenization in every situation.
This makes it more complicated, but also much more obvious as to
what is going on -- especially with regard as to what decisions
end up being parser's responsibility.
Refactor the parser so it recognizes YANG tokenization constructs,
notably quoted and unquoted strings, and assemble them from lexer
tokens.
JIRA: YANGTOOLS-1089
Change-Id: I34472bf0a7e262d4f633ce271952bbcd7639ef2f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>