Add a specialized token factory 00/92200/1
authorRobert Varga <robert.varga@pantheon.tech>
Mon, 10 Aug 2020 12:07:00 +0000 (14:07 +0200)
committerRobert Varga <robert.varga@pantheon.tech>
Thu, 20 Aug 2020 08:20:17 +0000 (10:20 +0200)
commit5aab023a02ae13bc731d6fc19cfb389379b7d214
treec068941e2aa2f10809daef5a71a6cce2c3495984
parent0f4f821f9b6b59a2246470fdbfbc11a8b07156b7
Add a specialized token factory

Our baseline parsing footprint is rather large. We can improve it
by looking at token component sizes and using distinct token classes.

Aside from CommonToken, which are using as fallbacks for text
handling, we introduce 4 specialized classes, three of them seeing
typical use.

Since these classes use smaller fields to hold lazy state, as well
as eliminate typical invariants, we end up saving around 39MiB (12%)
of AST size in a typical case from the field.

JIRA: YANGTOOLS-1128
Change-Id: I600ae6cbe755212e2d1a15ae3e6b369ed0b3b962
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit f65682b440f3a2767a7852c9b1b3f0d0ea25f95d)
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/AbstractSourceToken.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/AbstractToken.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/CompactTokenFactory.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/CompactYangStatementLexer.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/ExplicitTextToken.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/Token12122.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/Token12144.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/Token44444.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/repo/YangStatementStreamSource.java