Add a specialized token factory 02/92202/1
authorRobert Varga <robert.varga@pantheon.tech>
Mon, 10 Aug 2020 12:07:00 +0000 (14:07 +0200)
committerRobert Varga <robert.varga@pantheon.tech>
Thu, 20 Aug 2020 08:20:33 +0000 (10:20 +0200)
commite66e724b7eece352212ddb274d417ffc1c9b49f0
tree046484bc4ed8df580fb52d44de5b5b0c279bc0fc
parentd96ffa0b908d37e2e5ccd2025ce6a0327b10f972
Add a specialized token factory

Our baseline parsing footprint is rather large. We can improve it
by looking at token component sizes and using distinct token classes.

Aside from CommonToken, which are using as fallbacks for text
handling, we introduce 4 specialized classes, three of them seeing
typical use.

Since these classes use smaller fields to hold lazy state, as well
as eliminate typical invariants, we end up saving around 39MiB (12%)
of AST size in a typical case from the field.

JIRA: YANGTOOLS-1128
Change-Id: I600ae6cbe755212e2d1a15ae3e6b369ed0b3b962
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit f65682b440f3a2767a7852c9b1b3f0d0ea25f95d)
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/AbstractSourceToken.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/AbstractToken.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/CompactTokenFactory.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/CompactYangStatementLexer.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/ExplicitTextToken.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/Token12122.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/Token12144.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/antlr/Token44444.java [new file with mode: 0644]
yang/yang-parser-rfc7950/src/main/java/org/opendaylight/yangtools/yang/parser/rfc7950/repo/YangStatementStreamSource.java