Add pages.

.gitlab-ci.yml

@@ -3,6 +3,7 @@ variables:
 
 stages:
 - build
+- deploy
 
 before_script:
   - export GRADLE_USER_HOME=`pwd`/.gradle
@@ -20,3 +21,15 @@ build:
   artifacts:
     paths:
       - "/builds/jastadd/ragconnect/build/libs/ragconnect-*.jar"
+
+pages:
+  image: python:3.7-alpine
+  stage: deploy
+  script:
+  - pip install -U sphinx sphinx-rtd-theme recommonmark sphinxemoji sphinx-markdown-tables
+  - sphinx-build -b html pages/ public
+  artifacts:
+    paths:
+    - public
+  only:
+  - master

pages/.gitignore

+/public/

pages/Makefile

+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = public
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

pages/adding.md

+# Adding `ragconnect` to your project
+
+If you want to use `ragconnect`, the currently suggested way is to first build the jar from this repository:
+
+```bash
+cd ragconnect
+./gradlew jar
+ls ragconnect.base/build/libs/
+```
+
+This `ragconnect-<version>.jar` can then be copied to your project. Please note, that you can safely use `ragconnect.jar` as filename, because the version can always be printed using `java -jar path/to/ragconnect.jar --version`.
+
+```bash
+cp ragconnect.base/build/libs/ragconnect-<version>.jar ../your-project/libs/ragconnect.jar
+cd ../your-project/
+```
+
+Finally, this jar has to be integrated into your build process. In case, [Gradle](https://gradle.org/) is used, a task could look like the following (example taken from the [ros2rag usecase](https://git-st.inf.tu-dresden.de/jastadd/ros2rag)). The path to the jar files may need to be changed according to your project structure.
+
+```groovy
+task ragConnect(type: JavaExec) {
+    group = 'Build'
+    main = '-jar'
+
+    args([
+            '../libs/ragconnect.jar',
+            '--verbose',
+            '--o=src/gen/jastadd',
+            'src/main/jastadd/GoalModel.relast',
+            'src/main/jastadd/GoalModel.connect',
+            '--rootNode=GoalModel'
+    ])
+}
+```
+
+This is typically accompanied with a task to invoke the [RelAst compiler](http://relational-rags.eu/) and the [JastAdd gradle plugin](https://plugins.gradle.org/plugin/org.jastadd). The additional arguments "--useJastAddNames", "--listClass", "--jastAddList" and "--resolverHelper" to relast are not required. Please see the user manual of the RelAst compiler for more information.
+
+```groovy
+task relastToJastAdd(type: JavaExec) {
+    group = 'Build'
+    main = "-jar"
+
+    args(["../libs/relast.jar",
+            "--grammarName=./src/gen/jastadd/model",
+            "--useJastAddNames",
+            "--listClass=java.util.ArrayList",
+            "--jastAddList=JastAddList",
+            "--resolverHelper",
+            "--file",
+            "src/gen/jastadd/GoalModel.relast",
+            "src/gen/jastadd/RagConnect.relast"])
+}
+
+jastadd {
+...
+}
+```
+
+One also has to specifiy the dependencies to get correct ordering of tasks.
+
+```groovy
+generateAst.dependsOn relastToJastAdd
+relastToJastAdd.dependsOn ragConnect
+```
+
+## Introduced dependencies
+
+Ragconnect itself does not introduce dependencies.
+However, depending on the selected protocols (see [compiler options](/using#compiler-options)), additional dependencies are required.
+
+| Protocol | Dependency (Gradle format) | Remarks |
+|-|-|-|
+| `mqtt` | `group: 'org.fusesource.mqtt-client', name: 'mqtt-client', version: '1.15'` | Mqtt is selected by default, so this dependency therefore is required "by default". Might work with other versions as well. |
+| `rest` | `group: 'com.sparkjava', name: 'spark-core', version: '2.9.2'` | Might work with other versions as well. For debugging, it is beneficial to include an implementation for [SLF4J](http://www.slf4j.org/). |
+

pages/conf.py

+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
+import sphinx_rtd_theme
+
+
+# -- Project information -----------------------------------------------------
+
+project = 'RagConnect'
+copyright = '2020, René Schöne, Johannes Mey'
+author = 'René Schöne, Johannes Mey'
+
+# The full version, including alpha/beta/rc tags
+release = version = '0.2.2'
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx_rtd_theme',
+    'recommonmark',
+    'sphinxemoji.sphinxemoji',
+    'sphinx_markdown_tables'
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = []
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'sphinx_rtd_theme'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+sphinxemoji_style = 'twemoji'

pages/extending.md

+# Extending `ragconnect`
+
+To add a new communication protocol `ABC`, the following locations have to be changed:
+
+- within `ragconnect.base/src/main/resources`
+  - add a new handler `ABCHandler`, if appropriate, similar to the existing handlers
+  - add a new template `abc.mustache` containing an aspect `ABC` initializing the handler (if any) similar to the existing protocols
+  - in `ragconnect.mustache`: add `{{#usesABC}}{{> abc}}{{/usesABC}}` and `{{#usesABC}}{{abcHandlerField}}.close();{{/usesABC}}`, respectively if appropriate
+  - in `receiveDefinition.mustache` and `sendDefinition.mustache`: add a new case in the switch statement defining the logic to happen for both definitions. If the new protocol is close to a PUSH semantic, follow `mqtt`. If it is closer to PULL semantic, follow `rest`.
+- within `ragconnect.base/src/main/jastadd`
+  - in `backend/Configuration`: add a new static boolean flag `usesABC` to indicate whether the protocol is used
+  - in `backend/Generation`:
+    - add new attributes for type `MRagConnect` for handler-attribute and handler-field, if needed
+    - add attributes for newly introduced references in changed mustache templates, if any
+  - in `backend/MustacheNodesToYAML`:
+    - add key-value-pair for `usesABC` (and handler, if any)
+    - add key-value-pairs for newly introduced referemces in changed mustache templates, if any
+- in `ragconnect.base/src/main/java/org/jastadd/ragconnect/compiler/Compiler.java`
+  - add a new choice for `--protocols` similar to the existing ones
+  - set the flag `usesABC` if the choice is given.
+  - add code to add the handler to the list `handlers` if the choice is given, i.e., if `ASTNode.usesABC`
+
+Furthermore, new test cases are appreciated. They can be added in the [ragconnect.rests repository](https://git-st.inf.tu-dresden.de/jastadd/ragconnect-tests)

pages/index.rst

+.. RagConnect documentation master file, created by
+   sphinx-quickstart on Fri Aug 28 10:16:26 2020.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+RagConnect Documentation
+========================
+
+RagConnect is a preprocessor to enable easy connection to/from models based on `Reference Attribute Grammars <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.108.8792&rep=rep1&type=pdf>`_ and `Relation Reference Attribute Gramamrs <https://doi.org/10.1016/j.cola.2019.100940>`_ built with `JastAdd <http://jastadd.org/>`_.
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   use_cases.md
+   adding.md
+   inner-workings.md
+   using.md
+   extending.md
+
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`search`

pages/inner-workings.md

+# Inner workings of `ragconnect`
+
+![ros2rag-process](material/ros2rag-process.png)
+
+- `ragconnect` uses the [relast-preprocessor](https://git-st.inf.tu-dresden.de/jastadd/relast-preprocessor) to parse `.relast` grammar files. This results in an ASTNode of type `Program`.
+- it further uses a dedicated parser for `.connect` files containing endpoint-, mapping-, and dependency-definitions. This results in an ASTNode of type `RagConnect`.
+- the goal is to generate an aspect file containing setters and getters of tokens referred to by endpoint-definitions
+- we use [mustache](https://mustache.github.io/) (currently its [Java version](https://github.com/spullara/mustache.java)) making use of partials resulting in a set of `.mustache` files located in `ragconnect.base/src/main/resources`
+- the generation process uses an intermediate NTA of type `MRagConnect` defined in `MustacheNodes.relast` to separate this generation concern from the content of the DSL
+- there are aspect files for `Navigation` (mainly isX/asX attributes), `Analysis` (lookup attributes), `Printing`, `backend/Mappings` (default mappings)
+- one of the main aspects is `backend/Generation` containing attributes to construct the `MRagConnect` NTA and all necessary attributes used within the mustache templates
+- the other main aspect (which is currently not really used) is `backend/MustacheNodesToYAML.jrag` containing the transformation from a `MRagConnect` subtree to a `Document` subtree defined by `YAML.relast`. This is used to generate a YAML file containing the data used by mustache. It can be used by the default mustache implementation together with the templates.

pages/make.bat

+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=.
+set BUILDDIR=public
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.http://sphinx-doc.org/
+	exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

pages/use_cases.md

+# Use cases with `ragconnect`
+
+## 1⃣ MPM4CPS Paper - Codename 'Ros2Rag'
+
+In the publication "Connecting conceptual models using Relational Reference Attribute Grammars" (⚠ DOI pending), a use case involving a simulated robot arm and two different models conencted to it was shown.
+One model was used to ensure a low speed of the robot when within a safety zone (purple boxes in the picture below), and the other model executes a workflow to control the robot.
+
+![](material/robo3d.png)
+
+The repository with the used source code can be found at: <https://git-st.inf.tu-dresden.de/ceti/ros/mpm4cps2020>
+The usage is dockerized, so starting the application involves the commands listed below.
+As ROS takes some time to start up, it is best to use separate terminals for the three applications (ROS, Safety-Model, Goal-Model):
+
+```bash
+# Preparation (only need once)
+./prepare-docker-compose.sh
+docker-compose up -d mosquitto  # Starts the MQTT broker
+
+# Terminal 1
+docker-compose up --build ros
+# Wait until you see no more new logging output
+
+# Terminal 2
+docker-compose up rag_app
+
+# Terminal 3
+docker-compose up rag_goal
+```

pages/using.md

+# Using `ragconnect` -- an example
+
+## Preparation
+
+The following examples are inspired by the real test case [read1write2](https://git-st.inf.tu-dresden.de/jastadd/ragconnect-tests/-/tree/master/ragconnect.tests/src/test/01-input/read1write2)
+The idea is to have two nonterminals, where input information is received on one of them, and - after transformation - is sent out by both.
+
+Let the following grammar be used:
+
+```bnf
+A ::= <Input:String> /<OutputOnA:String>/ B* ;
+B ::= /<OutputOnB:String>/ ;
+```
+
+To declare receiving and sending tokens, a dedicated DSL is used:
+
+```java
+// endpoint definitions
+receive A.Input ;
+send A.OutputOnA ;
+send B.OutputOnB using Transformation ;
+
+// mapping definitions
+Transformation maps String s to String {:
+  return s + "postfix";
+:}
+```
+
+This defines `A.Input` to receive updates, and the other two tokens to send their value, whenever it changes.
+Additionally, a transformation will be applied on `B.OutputOnB` before sending out its value.
+
+Such mapping definitions can be defined for receiving tokens as well.
+In this case, they are applied before the value is set.
+If no mapping definition is given, or if the required type (depending on the communication protocol, see later) does not match, a "default mapping definition" is used to avoid boilerplate code converting from or to primitive types.
+
+Furthermore, let the following attribute definitions be given:
+
+```java
+syn String A.getOutputOnA() = "a" + getInput();
+
+syn String B.getOutputOnB() = "b" + input();
+inh String B.input();
+eq A.getB().input() = getInput();
+```
+
+In other words, `OutputOnA` depends on `Input` of the same node, and `OutputOnB` depends on `Input` of its parent node.
+Currently, those dependencies have to be explicitely written down. It is expected, that in future version, this won't be necessary anymore.
+This happens also in the DSL (dependencies have to be named to uniquely identify them):
+
+```java
+// dependency definitions
+A.OutputOnA canDependOn A.Input as dependencyA ;
+B.OutputOnB canDependOn A.Input as dependencyB ;
+```
+
+## Using generated code
+
+After specifying everything, code will be generated if [setup properly](adding).
+Let's create an AST in some driver code:
+
+```java
+A a = new A();
+// set some default value for input
+a.setInput("");
+B b1 = new B();
+B b2 = new B();
+a.addB(b1);
+a.addB(b2);
+```
+
+Now, we have to set the dependencies as described earlier.
+
+```java
+// a.OutputOnA -> a.Input
+a.addDependencyA(a);
+// b1.OutputOnB -> a.Input
+b1.addDependencyB(a);
+// b2.OutputOnB -> a.Input
+b2.addDependencyB(a);
+```
+
+Finally, we can actually _connect_ the tokens.
+Depending on the enabled protocols, [different URI schemes are allowed](#communication-protocol-characteristics).
+In this example, we use the default protocol: MQTT.
+
+```java
+a.connectInput("mqtt://localhost/topic/for/input");
+a.connectOutputOnA("mqtt://localhost/a/out", true);
+b1.connectOutputOnB("mqtt://localhost/b1/out", true);
+b2.connectOutputOnB("mqtt://localhost/b2/out", false);
+```
+
+The first parameter of those connect-methods is always an URI-like String, to identify the protocol to use, the server operating the protocol, and a path to identify the concrete token.
+In case of MQTT, the server is the host running an MQTT broker, and the path is equal to the topic to publish or subscribe to.
+Please note, that the first leading slash (`/`) is removed for MQTT topics, e.g., for `A.Input` the topic is actually `topic/for/input`.
+
+For sending endpoints, there is a second boolean parameter to specify whether the current value shall be send immediately after connecting.
+
+## Communication protocol characteristics
+
+| Protocol | URI scheme | Default port | Type for mapping definitions | Remarks |
+|-|-|-|-|-|
+| `mqtt` | `mqtt://<broker-host>[:port]/<topic>` | 1883 | `byte[]` | First leading slash not included in topic. |
+| `rest` | `rest://localhost[:port]/<path>` | 4567 | `String` | Host is always `localhost`. |
+
+## Compiler options
+
+The compiler is JastAdd-compliant, i.e., it accepts all flags available for JastAdd, though there is no process how to chain pre-processors _yet_.
+Additional options are as follows.
+
+| Name | Required (Default) | Description |
+|-|-|-|
+| `--rootNode` | Yes | Root node in the base grammar. |
+| `--protocols` | No (`mqtt`) | Protocols to enable, currently available: `mqtt, rest`. |
+| `--printYaml` | No (false) | Print out YAML instead of generating files. |
+| `--verbose` | No (false) | Print more messages while compiling. |
+| `--logReads` | No (false) | Enable logging for every received message. |
+| `--logWrites` | No (false) | Enable logging for every sent message. |
+| `--version` | No (false) | Print version info and exit (reused JastAdd option) |
+| `--o` | No (`.`) | Output directory (reused JastAdd option) |
+
+All files to be process have to be passed as arguments. Their type is decided by the file extension (`ast` and `relast` for input grammars, `connect` and `ragconnect` for ragconnect definitions file).
+
+## Remarks
+
+When constructing the AST and connecting it, one should always set dependencies before connecting, especially if updates already arriving for receiving endpoints.
+Otherwise, updates might not be propagated after setting dependencies, if values are equal after applying transformations of mapping definitions.
+
+As an example, when using the following grammar and ragconnect-definitions ...
+
+```bnf
+A ::= <Input:int> /<Output:String>/ ;
+```
+
+```java
+receive A.Input using Round ;
+send A.Output ;
+
+A.Output canDependOn A.Input as dependency1 ;
+
+Round maps float f to int {:
+  return Math.round(f);
+:}
+```
+
+... connecting first could mean to store the first rounded value and not propagating this update, since no dependencies are set, and not propagating further updates leading to the same rounded value even after setting the dependencies.