Metamodel-guided decoding improves domain-specific language generation consistency
Metamodel-Guided LLM Decoding for Consistent DSL Generation
Software EngineeringArtificial Intelligence
Summary
Generating code or instructions in special computer languages can be tricky because they must follow precise rules. The authors introduce MeMoDecode, a method that helps large language models produce code that not only looks right but also follows deep structural and meaning rules. It checks the code as it’s being written and corrects mistakes to keep everything consistent. Tests show this approach improves accuracy compared to older methods without much extra time needed.
What this means in practice
- •For software engineering teams: Generate more consistent and valid domain-specific code automatically during model-based software development.
- •For programming tool developers: Build language-aware code generation tools that enforce deep structural correctness beyond grammar rules.
Authors
Boqi Chen, Aren A. Babikian, José Antonio Hernández López
Abstract
Large language models (LLMs) are increasingly used to generate software engineering artifacts written in domain-specific languages (DSLs). Correct DSL instances usually must comply with not only the concrete syntax but also the abstract syntax defined by a metamodel augmented with semantic well-formedness constraints. Existing constrained decoding approaches typically guarantee compliance only with concrete syntax or with a limited set of semantic constraints expressed over the generated text, making metamodel-level constraint consistency difficult to enforce during generation. We present MeMoDecode: a decoding framework that enforces a DSL's concrete syntax, abstract syntax, and well-formedness constraints jointly. Specifically, whenever the LLM generates a complete metamodel concept, MeMoDecode incrementally maps the current output to a partial model and verifies whether it can still be completed into a consistent instance. We combine MeMoDecode with a simple retry mechanism that resumes decoding from the most recent completable snapshot whenever an inconsistency is detected. Under a sound backend verifier, MeMoDecode never performs worse than grammar-constrained decoding on alignment metrics satisfying natural assumptions. Across four DSLs and six open-weight LLMs ranging from 0.6B to 70B parameters, MeMoDecode achieves the highest consistency, improving consistency by 23% on average over unconstrained decoding and by 8% on average over grammar-constrained decoding. It also improves benchmark performance while incurring only modest runtime overhead. These results show that metamodel-guided decoding is an effective approach for enforcing DSL consistency during decoding across diverse DSLs and LLM scales.