neogeny / TatSu

.. Copyright (c) 2017-2026 Juancarlo Añez (apalala@gmail.com) .. SPDX-License-Identifier: BSD-4-Clause .. |dragon| unicode:: 0x7ADC .. unicode dragon .. |nbsp| unicode:: 0xA0 .. non breakable space .. |TatSu| replace:: |dragon|\ |nbsp|\ **TatSu** .. |TatSu-LTS| replace:: |dragon|\ |nbsp|\ **TatSu-LTS** .. _RELEASES: https://github.com/neogeny/TatSu/releases | |license| ​ |pyversions| | |fury| ​ |actions| ​ |docs| ​ |installs| ​ |codspeed| | |sponsor| | |TatSu| ======= *At least for the people who send me mail about a new language that they're designing, the general advice is: do it to learn about how to write a compiler. Don't have any expectations that anyone will use it, unless you hook up with some sort of organization in a position to push it hard. It's a lottery, and some can buy a lot of the tickets. There are plenty of beautiful languages (more beautiful than C) that didn't catch on. But someone does win the lottery, and doing a language at least teaches you something.* _ (1941-2011) Creator of the C_ programming language and of Unix_ |TatSu| is a tool that takes grammars in extended _ as input, and outputs _ ( _) _ parsers in _. The classic variations of EBNF_ (Tomassetti, EasyExtend, Wirth) and _ are supported as input grammar formats. Why use a _ parser generator? ---------------------------------- Regular expressions are *“memory-less”*—they excel at finding flat patterns like email addresses or phone numbers, but they fail once data becomes hierarchical. Regular expressions cannot *"count"* or balance demarcations (a regex cannot reliably validate whether opening and closing parenthesis are matched in a nested math equation). Parsing is the essential step up when you need to understand the **logic and structure** of information rather than just its appearance. Parsing constructs an **Abstract Syntax Tree** (AST_) of the input, a hierarchical map that represents how different parts of a sequence relate to one another. • **Recursive Structures:** Whenever a piece of data can contain a version of itself (like a folder inside a folder, or a conditional if statement inside another if ), you need a parser to track the depth and scope. • **Translating Formats:** When converting one format into another, a parser ensures that the *meaning* of the original structure is preserved, preventing the *"data soup"* that occurs when using simple find-and-replace tools. • **Ambiguity Resolution:** In complex sequences, the same sub-sequence might mean different things depending on where it sits in the tree. A parser uses the surrounding context to decide how to treat that sequence, whereas a regex treats every match in isolation. • **Domain-Specific Languages (DSL):** Parsing allows the creation of specialized *"mini-languages"* tailored to a specific field, such as hardware description, music notation, or complex business rules. • **Executable Logic:** While a regex can tell you if a string *looks* like a command, a parser turns that string into an object that a computer can actually execute, ensuring the order of operations and dependencies are strictly followed. |TatSu| can compile a grammar stored in a string into a Grammar object that can be used to parse any given input (much like the _ module does with regular expressions). |TatSu| can also generate a Python_ module that implements the parser. |TatSu| supports _ rules in PEG_ grammars using the algorithm_ by *Laurent* and *Mens*. The generated AST_ has the expected left associativity. Compatibility ------------- |TatSu| expects a maintained_ version of Python (>=3.13), but currently all tests run in versions of Python down to Python 3.12. |TatSu| is also compatible with the current pre-release version of Python 3.15. *For older versions of Python, you may consider* _, *a friendly fork of* |TatSu| *aimed at compatibility*. .. _algorithm: http://norswap.com/pubs/sle2016.pdf .. _TatSu-LTS: https://pypi.org/project/TatSu-LTS/ Installation ------------ .. code-block:: bash $ pip install TatSu Using the Tool -------------- |TatSu| can be used as a library, much like _'s re , by embedding grammars as strings and generating grammar models instead of generating Python_ code. This compiles the grammar and generates an in-memory *parser* that can subsequently be used for parsing input with: .. code-block:: python parser = tatsu.compile(grammar) Compiles the grammar and parses the given input producing an AST_ as result: .. code-block:: python ast = tatsu.parse(grammar, input) The result is equivalent to calling: .. code-block:: python parser = compile(grammar) ast = parser.parse(input) Compiled grammars are cached for efficiency. This compiles the grammar to the _ source code that implements the parser: .. code-block:: python parser_source = tatsu.to_python_sourcecode(grammar) This is an example of how to use |TatSu| as a library: .. code-block:: python GRAMMAR = ''' start: expression $ expression: | expression '+' term | expression '-' term | term term: | term '*' factor | term '/' factor | factor factor: | '(' expression ')' | number number: /\d+/ ''' if __name__ == '__main__': import json from tatsu import parse from tatsu.util import asjson ast = parse(GRAMMAR, '3 + 5 * ( 10 - 20 )') print(ast.asjsons()) .. |TatSu| will use the first rule defined in the grammar as the *start* rule. This is the output: .. code-block:: console [ "3", "+", [ "5", "*", [ "10", "-", "20" ] ] ] Documentation ------------- For a detailed explanation of what |TatSu| is capable of, please see the documentation_. .. _documentation: http://tatsu.readthedocs.io/ Questions? ---------- Please use the _ tag on _ for general Q&A, and limit GitHub issues to bugs, enhancement proposals, and feature requests. .. _[tatsu]: https://stackoverflow.com/tags/tatsu/info Changes ------- See the _ for details. License ------- You may use |TatSu| under the terms of the _-…