Skip to content

Grammar Rule Set

A declarative artifact — instantiates Grammar-Guided Structure Recovery

The declarative set of production rules that defines well-formed composition — the reference grammar every parser consults to license or reject a structure.

The Grammar Rule Set is the declarative artifact at the center of the archetype: a finite set of production rules stating which tokens may combine into which larger constituents, and — through precedence and associativity — how an otherwise-ambiguous constituent should attach. It is not a parser and recovers nothing on its own; it is the shared reference a parser consults to decide whether a proposed structure is licensed. Its defining move is to separate the definition of well-formedness from the act of recovering it: change a rule and every parser that reads the set changes behavior, with no new parsing code. Everything else in the archetype — chart, disambiguation, validation — is downstream of what this set declares legal.

Example

A team writing a small calculator language needs to pin down what a valid expression is before any parser can be built. They write the rules:

expr   := expr "+" term  |  term
term   := term "*" factor  |  factor
factor := NUMBER  |  "(" expr ")"

Nothing here executes. But the layering already settles the classic question: because term is reachable only inside expr, multiplication binds tighter than addition, so 2 + 3 * 4 is licensed only as 2 + (3 * 4), never (2 + 3) * 4. Left-recursion in expr and term fixes associativity so 10 - 3 - 2 groups left. The same three lines both permit nested parentheses to arbitrary depth and forbid 2 + + 3. When a reader later disagrees with the calculator, the argument is about these rules — not about the parser's code.

How it works

The set is built from nonterminals (categories like expr) and productions (a nonterminal rewritten as a sequence of terminals and nonterminals). Recursion in the productions is what admits unbounded nesting. Attachment — the part that decides how an ambiguous constituent groups — is encoded not as separate logic but inside the rule shape: precedence falls out of which nonterminal nests inside which, and associativity falls out of whether a rule recurses on its left or its right. A start symbol and a terminal alphabet (supplied by the tokenizer) close the set. The result is consulted declaratively, never run as a procedure.

Tuning parameters

  • Grammar class — regular vs. context-free vs. PEG vs. context-sensitive. More expressive power captures more constraints but costs parseability and can cost decidability; pick the weakest class that still describes the language.
  • Ambiguity stance — allow an ambiguous grammar (a token string may have several trees) or constrain to an unambiguous one. Ambiguity you permit here is work you push onto the disambiguation step.
  • Precedence & associativity — the attachment dials. Tightening them removes candidate parses at the source rather than resolving them later.
  • Nonterminal granularity — fine categories encode more constraints but bloat the set and slow every consumer.
  • Left-recursion & factoring — shaping the rules to suit the parser family that will read them (some engines cannot consume left-recursion directly).

When it helps, and when it misleads

Its strength is that a precise, shared grammar makes well-formedness checkable and portable: one artifact governs every parser, generator, and editor that reads it, and evolving the language is a rule edit rather than a code rewrite. Its failure modes are all about the gap between the written rules and the real language. An over-permissive grammar admits garbage; an ambiguous grammar quietly multiplies parses that something downstream must now resolve; and a grammar drifts from the input it is supposed to model — the spec says one thing while real-world inputs (the canonical case being HTML in the wild) say another. The classic misuse is treating the grammar as ground truth and rejecting meaningful inputs that merely violate the written form — or, run backwards, loosening rules after the fact to bless whatever a parser already emitted. The discipline that guards against this is to test the grammar against a real corpus and version its evolution deliberately rather than reactively.[1]

How it implements the components

The Grammar Rule Set realizes the rule-defining core of the archetype — the components that state what structure is legal, not the ones that recover or check it:

  • composition_rule_grammar — the productions themselves are this component: the enumerated ways constituents may compose.
  • structural_attachment_rule — precedence and associativity, encoded in rule shape, are the attachment rules that decide how an ambiguous constituent groups.

It does not tokenize the input (that is Lexical Scanning and Tokenization), does not generate candidate parses (that is Chart Parsing and the other parsing engines), and does not check conformance of a recovered structure (that is Linting or Validation Rule and Semantic Schema Validation).

  • Instantiates: Grammar-Guided Structure Recovery — the Grammar Rule Set is the reference every other mechanism in the archetype is defined against.
  • Sibling mechanisms: Chart Parsing · Controlled Disambiguation Test · Ambiguity Register · Interpretation Walkthrough · Linting or Validation Rule · Invalid Combination Linter · Semantic Schema Validation · Recursive-Descent Parsing · Shift-Reduce Parsing · Probabilistic Grammar Parsing · Lexical Scanning and Tokenization

Notes

The grammar governs syntax — what is well-formed — and says nothing about meaning. A structure can be perfectly grammatical yet semantically invalid, and that second judgment belongs to Semantic Schema Validation. Keeping the syntactic grammar and the semantic schema as separate artifacts is what lets either evolve without dragging the other with it.

References

[1] A context-free grammar written in Backus–Naur Form (BNF) is the standard vehicle for a rule set of this kind: productions of the form nonterminal → sequence, with no left context required to apply a rule. The class is expressive enough for most nesting yet still admits efficient parsers, which is why it, rather than a more powerful grammar, is the usual default. ↩