Commit 1c945a6e authored by Sebastian Ebert's avatar Sebastian Ebert
Browse files

initial merge state

parents d740a665 d79785b9
......@@ -6,3 +6,4 @@
- build
- test
# - publish
#- ragdoc_build
#- ragdoc_view
#- publish
- export GRADLE_USER_HOME=`pwd`/.gradle
......@@ -21,38 +24,84 @@ build:
- ./gradlew --console=plain --no-daemon assemble jar
- "/builds/jastadd/ragconnect/ragconnect.base/build/libs/ragconnect-*.jar"
- "ragconnect.base/build/libs/ragconnect-*.jar"
- "ragconnect.base/src/gen"
expire_in: 1 week
image: openjdk:11
stage: test
- name: "eclipse-mosquitto:1.6.9"
- name: "eclipse-mosquitto:1.6"
alias: "mqtt"
- build
- ./gradlew --console=plain --no-daemon allTests
when: always
junit: "/builds/jastadd/ragconnect/ragconnect.tests/build/test-results/test/TEST-*.xml"
junit: "ragconnect.tests/build/test-results/**/TEST-*.xml"
expire_in: 1 week
# image: openjdk:11
# stage: publish
# needs:
# - test
# script:
# - "./gradlew publish"
# only:
# - dev
# - master
# image: python:3.7-alpine
# stage: publish
# image:
# name: ""
# entrypoint: [""]
# stage: ragdoc_build
# needs:
# - build
# script:
# - pip install -U sphinx sphinx-rtd-theme recommonmark sphinxemoji sphinx-markdown-tables
# - sphinx-build -b html pages/ public
# - JAVA_FILES=$(find ragconnect.base/src/ -name '*.java')
# - /ragdoc-builder/ -excludeGenerated -d data/ $JAVA_FILES
# artifacts:
# paths:
# - public
# - "data/"
# image:
# name: ""
# entrypoint: [""]
# stage: ragdoc_view
# needs:
# - ragdoc_build
# script:
# - DATA_DIR=$(pwd -P)/data
# - mkdir -p pages/docs/ragdoc
# - OUTPUT_DIR=$(pwd -P)/pages/docs/ragdoc
# - cd /ragdoc-view/src/ && rm -rf data && ln -s $DATA_DIR
# - /ragdoc-view/ --output-path=$OUTPUT_DIR
# only:
# - master
# - dev
# - master
# artifacts:
# paths:
# - "pages/docs/ragdoc"
# image: python:3.8-buster
# stage: publish
# needs:
# - ragdoc_view
# - test
# before_script:
# - pip install -U mkdocs mkdocs-macros-plugin mkdocs-git-revision-date-localized-plugin
# script:
# - cd pages && mkdocs build
# - dev
# - master
# paths:
# - public
[submodule "relast-preprocessor"]
path = relast-preprocessor
url = ../relast-preprocessor.git
branch = develop
[submodule "ragconnect.base/src/main/jastadd/mustache"]
path = ragconnect.base/src/main/jastadd/mustache
url = ../mustache
url =
# RagConnect
[RagConnect]( is a preprocessor to enable easy connection to/from models based on [Reference Attribute Grammars]( and [Relational Reference Attribute Grammars]( built with [JastAdd](
Documentation can be found at <> including an API documentation.
The most recent version is listed at the [package registry](
_static/poster-presentation.mp4 filter=lfs diff=lfs merge=lfs -text
docs/img/poster-presentation.mp4 filter=lfs diff=lfs merge=lfs -text
docs/img/moving-robot.mp4 filter=lfs diff=lfs merge=lfs -text
# Minimal makefile for Sphinx documentation
# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXBUILD ?= sphinx-build
BUILDDIR = public
# Put it first so that "make" without argument is like "make help".
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
# Configuration file for the Sphinx documentation builder.
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
import sphinx_rtd_theme
# -- Project information -----------------------------------------------------
project = 'RagConnect'
copyright = '2021, René Schöne, Johannes Mey'
author = 'René Schöne, Johannes Mey'
# The full version, including alpha/beta/rc tags
ragconnectVersionFileName = '../ragconnect.base/src/main/resources/'
with open(ragconnectVersionFileName) as ragconnectVersionFile:
versionFileContent =
release = version = versionFileContent[versionFileContent.rindex('version=') + 8:].strip()
print('Version: ' + version)
# -- General configuration ---------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = []
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'sphinx_rtd_theme'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
sphinxemoji_style = 'twemoji'
{% block footer %}
<p>{% if config.copyright %}
<small>{{ config.copyright }}<br></small>
{% endif %}
Built with <a href="">MkDocs</a> using a <a href="">theme</a> provided by <a href="">Read the Docs</a>.
{% if page and page.meta and page.meta.git_revision_date_localized %}
<small><br><i>Last updated {{ page.meta.git_revision_date_localized }}</i></small>
{% endif %}
{% endblock %}
# Changelog
## 0.3.1
- Full support for incremental dependency tracking
- Full support for subtree endpoint definitions ([#9](
- Internal: Use updated gradle plugin for tests ([#18](
- Bugfix [#22]( Correct handling of malformed URIs passed when connecting an endpoint
- Bugfix [#23]( Correct handling of OptComponents as endpoints
- Bugfix [#27]( Correctly handle whitespaces in grammars
## 0.3.0
- Added [API documentation](ragdoc/index.html) to documentation
- Add methods to `disconnect` an endpoint
- Internal: PoC for incremental dependency tracking and subtree endpoint definitions ([#14](
- Bugfix [#17]( Added missing support for `boolean`
## 0.2.2
- Allow normal tokens to be used in send definitions
## 0.2.1
- New communication protocol: REST
- Selection of protocol when `connect` methods are called, by scheme of given URI
- Development changes:
- Supported printing out YAML data used for mustache templates
- Moved string constants to `MRagConnect` structure
## 0.2.0
- Version submitted in paper "A Connection from ROS to RAG-Based Models" (2020)
- Supported communication protocols: MQTT
# Extending `RagConnect`
To add a new communication protocol, the following locations have to be changed (replace `ABC` and `abc` with the name of the protocol):
Within `ragconnect.base/src/main/resources`:
{% raw %}
- Add a new handler `ABCHandler`, if appropriate, similar to the existing handlers
- If further methods are needed for handler initialization, add a new template `abc.mustache` containing those procedures. Add `{{#usesABC}}{{> abc}}{{/usesABC}}` at the top of `ragconnect.mustache` to use this template
- In `receiveDefinition.mustache` and `sendDefinition.mustache`: add a new case in the switch statement defining the logic to happen for both definitions. If the new protocol is close to a PUSH semantic, follow `mqtt`. If it is closer to PULL semantic, follow `rest`.
{% endraw %}
Within `ragconnect.base/src/main/jastadd`:
- In `backend/Configuration`:
- Add a new static boolean flag `usesABC` to indicate whether the protocol is used
- In `backend/Generation`:
- Add new attributes for type `MRagConnect` for handler attribute and handler field, if needed
- Add attributes for newly introduced references in changed mustache templates, if any
- Add a newly constructed handler within the definition of `RagConnect.toMustache` with the needed fields (class name, construction snippet, handler attribute, handler field, the boolean flag you just added to Configuration)
- In `backend/MustacheNodesToYAML`:
- Add key-value-pair for `usesABC` (and handler, if any)
- Add key-value-pairs for newly introduced referemces in changed mustache templates, if any
In `ragconnect.base/src/main/java/org/jastadd/ragconnect/compiler/`:
- Add a new choice for `--protocols` similar to the existing ones
- Set the flag `usesABC` if the choice is given.
- Add code to add the handler to the list `handlers` if the choice is given, i.e., if `ASTNode.usesABC`
Furthermore, new test cases are appreciated, see [below](#writing-tests).
## Writing Tests
To add new tests, have a look at the module `ragconnect.tests`.
It has three parts:
1) In `src/test/01-input/*` are the [specifications](#specifications) that are going to be compiled (in principle using the steps described in [the guide to add RagConnect](adding)).
2) In `src/test/java`, the jUnit 5 [test classes](#test-classes) are implemented. They mostly correspond 1-to-1 to a directory of the first part.
3) In `build.gradle` the [instructions how to compile](#buildgradle) the specifications using the gradle plugin [PreprocessorPlugin][preprocessor-plugin] (`org.jastadd.preprocessor:testing`).
### Specifications
Every specification must have at least a `` to describe the purpose of the test, a grammar `Test.relast`, and a RagConnect specification `Test.connect`.
Usually an aspect file `Test.jadd` is included.
### Test Classes
Based on jUnit 5, the test classes testing some behaviour. If sending and/or receiving functionality is used, consider extending `AbstractMqttTest` in order to avoid duplicate code. In case of extending this class, please order the methods according to their lifecycle, i.e.:
- createModel
- setupReceiverAndConnect
- communicateSendInitialValue
- communicateOnlyUpdatedValue
- closeConnections
Within `AbstractMqttTest`, an `MqttHandler` named `publisher` is available to publish content.
Some convenience methods are provided in `TestUtils`, e.g., the `DefaultMappings`, and `mqttUri` to prepend `"mqtt://"` and the correct host for the mqtt broker (`localhost` or a CI-specific host).
All tests are required to run both locally, and within the CI.
### build.gradle
Use the [PreprocessorPlugin][preprocessor-plugin], the build process can be written concisely in three parts per task:
task compileTreeAllowedTokens(type: RagConnectTest) {
ragconnect {
outputDir = file('src/test/02-after-ragconnect/treeAllowedTokens')
inputFiles = [file('src/test/01-input/treeAllowedTokens/Test.relast'),
rootNode = 'Root'
relast {
useJastAddNames = true
grammarName = 'src/test/03-after-relast/treeAllowedTokens/treeAllowedTokens'
serializer = 'jackson'
jastadd {
jastAddList = 'JastAddList'
packageName = 'treeAllowedTokens.ast'
inputFiles = [file('src/test/01-input/treeAllowedTokens/Test.jadd')]
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment