Add a specialized token factory 07/92007/16
authorRobert Varga <robert.varga@pantheon.tech>
Mon, 10 Aug 2020 12:07:00 +0000 (14:07 +0200)
committerRobert Varga <robert.varga@pantheon.tech>
Thu, 20 Aug 2020 07:42:40 +0000 (09:42 +0200)
commitf65682b440f3a2767a7852c9b1b3f0d0ea25f95d
tree305b73a2ee07231a196e41c27498b89cb86b4c97
parent66f1d0a87f3f4f4eaa0aacef0977506866bc74ee
Add a specialized token factory

Our baseline parsing footprint is rather large. We can improve it
by looking at token component sizes and using distinct token classes.

Aside from CommonToken, which are using as fallbacks for text
handling, we introduce 4 specialized classes, three of them seeing
typical use.

Since these classes use smaller fields to hold lazy state, as well
as eliminate typical invariants, we end up saving around 39MiB (12%)
of AST size in a typical case from the field.

JIRA: YANGTOOLS-1128
Change-Id: I600ae6cbe755212e2d1a15ae3e6b369ed0b3b962
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/AbstractSourceToken.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/AbstractToken.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/CompactTokenFactory.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/CompactYangStatementLexer.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/ExplicitTextToken.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/Token12122.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/Token12144.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/Token44444.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/repo/YangStatementStreamSource.java