TextX Project Setup

We give a guide to create a simple command line tool to load and validate a model and to generate some artifact.

In this example, we propose different models for the meta model structure (grammar), the scoping, and the artifact generation. Moreover, a main and a setup script are added to control the software.

For this example we use Python 3. You need to install

TextX (see references).
Arpeggio (see TextX in references)
pytest (for unittests)

Using pip you can:

pip3 install --upgrade textx arpeggio pytest

File structure

Although everything can be packed within one file (see "TextX Intro"), the code is better structured into different modules with individual responsibilities into different files.

├── setup.py
├── simple_dsl
│   ├── codegen.py
│   ├── console
│   │   ├── __init__.py
│   │   └── validate.py
│   ├── __init__.py
│   ├── metamodel.py
│   └── validation.py
└── tests
    ├── models
    │   ├── model_not_ok.dsl
    │   └── model_ok.dsl
    └── test_validation.py

We have chosen a distribution as follows:

setup.py is the standard python project configuration
simple_dsl contain all dsl related logic:
- codegen.py: code generation.
- metamodel.py: the meta model (grammar, scoping and validation config; the user classes are also stored here but could be moved elsewhere for more complex projects.
- validation.py: validation logic.
- console/*.py: console programs (configured in setup.py).
- __init__.py represent module entry points.
tests contains unittests.

File: metamodel.py

Here, we define the grammar. We allocate the scope providers to individual elements, and register validation code.

from textx import metamodel_from_str
from textx.scoping.providers import RelativeName, FQN
import simple_dsl.validation as validation

def get_metamodel():
    # GRAMMAR
    # (you also use metamodel_from_file with a *.tx file)
    meta_model = metamodel_from_str('''
        Model: aspects+=Aspect scenarios+=Scenario testcases+=Testcase;
        Scenario: 'SCENARIO' name=ID 'BEGIN' 
            configs+=Config
        'END';
        Config: 'CONFIG' name=ID 'HAS' '(' haves*=[Aspect] ')';
        Aspect: 'ASPECT' name=ID;
        Testcase: 'TESTCASE' name=ID 'BEGIN'
            'USES' scenario=[Scenario] 'WITH' config=[Config]
            'NEEDS' '(' needs*=[Aspect] ')'
        'END';
        Comment: /\/\/.*/;
    ''')

    # SCOPING
    meta_model.register_scope_providers({
        '*.*': FQN(),
        'Testcase.config': RelativeName('scenario.configs')
    })

    # ADD VALIDATION
    meta_model.register_obj_processors({
            'Testcase': validation.check_testcase
    })

    return meta_model

File: validation.py

This file contains validation functions registered in the meta model.

from textx.exceptions import TextXError
from textx.scoping.tools import get_location

def check_testcase(testcase):
    """
    checks that the config used by the testcase fulfills its needs
    """
    for need in testcase.needs:
        if need not in testcase.config.haves:
            raise (TextXError("{}: {} not found in {}.{}".format(
                    testcase.name,
                    need.name, 
                    testcase.scenario.name,
                    testcase.config.name
                    ),
                **get_location(testcase) # unpack location info
            ))

File: tests/test_validation.py

This file is a unittest using the metamodel (exposed via __init__.py) and checks the correct functionality of the validation code.

from pytest import raises
from simple_dsl import get_metamodel
from os.path import dirname, join
from textx import get_children_of_type

def test_validation_ok():
    mm = get_metamodel()
    m = mm.model_from_file(join(dirname(__file__),
                            'models',
                            'model_ok.dsl'))
    assert 2==len(get_children_of_type('Aspect',m))

def test_validation_not_ok():
    mm = get_metamodel()
    with raises(Exception,
                match=r'NetworkTraffic.*not found.*S001.*NoNetworkTraffic'):
        _ = mm.model_from_file(join(dirname(__file__),
                                'models',
                                'model_not_ok.dsl'))

The two model files used in this tests are shown in the following subsections.

Model: tests/models/model_ok.dsl

ASPECT NetworkTraffic
ASPECT FileAccess
SCENARIO S001 BEGIN
    CONFIG HeavyNetworkTraffic HAS (NetworkTraffic)
    CONFIG NoNetworkTraffic HAS ()
END
SCENARIO S002 BEGIN
    CONFIG WithFileAccess HAS (NetworkTraffic FileAccess)
    CONFIG NoFileAccess HAS (NetworkTraffic)
END
TESTCASE T001 BEGIN
    USES S001 WITH HeavyNetworkTraffic
    NEEDS (NetworkTraffic)
END
TESTCASE T002 BEGIN
    USES S001 WITH NoNetworkTraffic // Error
    //USES S002 WITH NoFileAccess
    NEEDS (NetworkTraffic)
END

Model: tests/models/model_not_ok.dsl

ASPECT NetworkTraffic
ASPECT FileAccess
SCENARIO S001 BEGIN
    CONFIG HeavyNetworkTraffic HAS (NetworkTraffic)
    CONFIG NoNetworkTraffic HAS ()
END
SCENARIO S002 BEGIN
    CONFIG WithFileAccess HAS (NetworkTraffic FileAccess)
    CONFIG NoFileAccess HAS (NetworkTraffic)
END
TESTCASE T001 BEGIN
    USES S001 WITH HeavyNetworkTraffic
    NEEDS (NetworkTraffic)
END
TESTCASE T002 BEGIN
    //USES S001 WITH NoNetworkTraffic // Error
    USES S002 WITH NoFileAccess
    NEEDS (NetworkTraffic)
END

Edit, Run and Test

Use an appropriate IDE (e.g., PyCharm) to run the tests and, thus, test and debug your new language.

Install/Uninstall the Language

After all tests passed you can try to install your language. Do not forget to adapt setup.py:

from setuptools import setup,find_packages

setup(name='simple_dsl',
      version='0.1',
      description='a simple model validator and artifact compiler',
      url='',
      author='YOUR NAME',
      author_email='YOUR.NAME@ADDRESS',
      license='TODO',
      packages=find_packages(),
      package_data={'': ['*.tx', '*.template', 'support_*_code/**/*']},
      install_requires=["textx","arpeggio"],
      tests_require=[
          'pytest',
      ],
      keywords="parser meta-language meta-model language DSL",
      entry_points={
          'console_scripts': [
              'simple_dsl_validate=simple_dsl.console.validate:validate',
          ]
      },
      )


# to play around without installing: do "export PYTHONPATH=."

The registered console command (validate.py) contains:

import argparse
from simple_dsl import get_metamodel

def validate():
    mm = get_metamodel()
    parser = argparse.ArgumentParser(description='validate simple_dsl files.')
    parser.add_argument('model_files', metavar='model_files', type=str,
                        nargs='+',
                        help='model filenames')
    args = parser.parse_args()

    for filename in args.model_files:
        try:
            print('validating {}'.format(filename))
            _ = mm.model_from_file(filename)
        except BaseException as e:
            print('  WARNING/ERROR: {}'.format(e))


if __name__=='__main__':
    validate()

Installation

Install the software permanently for all users (change directory to the folder with the setup.py file):

sudo -H pip3 install --upgrade .

You can now start the new commands defined in the setup.py:

simple_dsl_validate --help

Uninstallation

Uninstall the software:

sudo -H pip3 uninstall simple_dsl