2 // Copyright (c) 2015 Cisco Systems, Inc. and others. All rights reserved.
4 // This program and the accompanying materials are made available under the
5 // terms of the Eclipse Public License v1.0 which accompanies this distribution,
6 // and is available at http://www.eclipse.org/legal/epl-v10.html
8 parser grammar YangStatementParser;
10 // This ANTLR4 grammar serves as the lexer for the actual YANG parser, which
11 // is built on top of it in Java code. The reason for this split is that we
12 // need to perform string interpretation in a way which takes into account
13 // yang-version, since RFC7950 (YANG 1.1) is stricter (and saner) when it comes
14 // to escaping and similar.
16 tokenVocab = YangStatementLexer;
19 // NOTE: we need to use SEP*/SEP+ because comments end up breaking whitespace
20 // sequences into two.
21 file : SEP* statement SEP* EOF;
22 statement : keyword (SEP+ argument)? SEP* (SEMICOLON | LEFT_BRACE SEP* (statement SEP*)* RIGHT_BRACE);
23 keyword : IDENTIFIER (COLON IDENTIFIER)?;
25 // Alright, so what constitutes a string is rather funky. We need to deal with
26 // the flaky definitions of RFC6020, which allow for insane quoting as well as
27 // exclusion of comments. We also need to allow for stitching back tokens like
28 // PLUS/COLON, which may end up being valid identifiers.
30 // Note on optimization: we are allowing a single IDENTIFIER, although it
31 // is already part of unquotedString. This is strictly superfluous, but a
32 // single IDENTIFIER arguments are very common and this eliminates an
33 // indirection costing us at least two objects. This is not quite a case
34 // of premature optimization, but rather IDENTIFIER is really so very
35 // special and deserving of this treatment.
38 // Quoted string and concatenations thereof. We are sacrificing brewity
39 // here to eliminate the need for another parser construct. Quoted strings
40 // account for about 50% of all arguments encountered -- hence the added
41 // parse tree indirection is very visible in terms of memory usage.
42 (DQUOT_STRING? DQUOT_END | SQUOT_STRING? SQUOT_END)
43 (SEP* PLUS SEP* (DQUOT_STRING? DQUOT_END | SQUOT_STRING? SQUOT_END))*
52 // Alright this is written in a non-trivial manner due to us wanting to
53 // keep the number of parser objects (and hence memory pressure) down.
55 // Our aim is to forbid '//', '/*' and '*/' from being accepted as a
56 // valid unquoted string. Normally we would write this as a recursive
57 // parser rule for concatenating on '*' and '/' and let ANTLR figure it
58 // out. Unfortunately that results in a deep parse tree, essentially
59 // having one level for each such concatenation. For a test case imagine
60 // how "a*b/c*d*e**f" would get parsed with a recursive grammar.
62 // Now we cannot do much about tokenization, but we can statically express
63 // the shape we are looking for:
65 // so an unquoted string may optionally start with a single SLASH or any
66 // number of STARs ...
69 // ... but that needs to be followed by at least one span of other
70 // content, which is what we are really aiming for. This ensures
71 // any leading SLASH/STAR is followed by a non-(SLASH|STAR) ...
72 (COLON | PLUS | IDENTIFIER | UQUOT_STRING)+
74 // ... and based on that knowledge, we allow another SLASH or run of
75 // STARs to follow, but it has to be again followed by a run of
76 // of other tokens -- and rinse&repeat that any number of times.
77 // We still have ensured that the span matched does not end with
78 // a SLASH or a STAR ...
79 // ways retaining the 'does not end with SLASH or STAR' invariant
80 ((SLASH | STAR+) (COLON | PLUS | IDENTIFIER | UQUOT_STRING)+)*
82 // ... and therefore it is always safe to have such a span end with