Finalize tests for initial JavaHandler

- moved methods "registerConsumer" and "push" up to root node (prefixed)
- mark Java handler as experimental
- update documentation
- begin with changelog for 1.0.0

pages/docs/changelog.md

 # Changelog
 
+## 1.0.0 (dev)
+
+### Changes
+
+- Allow connection endpoints for relations ([#37](https://git-st.inf.tu-dresden.de/jastadd/ragconnect/-/issues/37)) and attributes ([#38](https://git-st.inf.tu-dresden.de/jastadd/ragconnect/-/issues/38))
+- Allow send connection endpoints non-NTA nonterminals ([#36](https://git-st.inf.tu-dresden.de/jastadd/ragconnect/-/issues/36))
+- Allow context-free context endpoints ([#34](https://git-st.inf.tu-dresden.de/jastadd/ragconnect/-/issues/34))
+- Experimental support for Java handler ([#52](https://git-st.inf.tu-dresden.de/jastadd/ragconnect/-/issues/52))
+- Make specification language more concise ([#33](https://git-st.inf.tu-dresden.de/jastadd/ragconnect/-/issues/33))
+
+### Development Changes
+
+- Make grammar(s) more concise  ([#40](https://git-st.inf.tu-dresden.de/jastadd/ragconnect/-/issues/40))
+- Enhance documentation, adding a DSL description
+
 ## 0.3.2
 
 - Allow connection endpoints for list nonterminals ([#21](https://git-st.inf.tu-dresden.de/jastadd/ragconnect/-/issues/21))

pages/docs/compiler.md

@@ -6,16 +6,16 @@ Additional options are as follows.
 | Name | Required (Default) | Description |
 |---|---|---|
 | `--rootNode` | Yes | Root node in the base grammar. |
-| `--protocols` | No (`mqtt`) | Protocols to enable, currently available: `mqtt, rest`. |
+| `--protocols` | No (`mqtt`) | Protocols to enable, currently available: `java` (experimental), `mqtt`, `rest`. |
 | `--printYaml` | No (false) | Print out YAML instead of generating files. |
 | `--verbose` | No (false) | Print more messages while compiling. |
 | `--logReads` | No (false) | Enable logging for every received message. |
 | `--logWrites` | No (false) | Enable logging for every sent message. |
 | `--logIncremental` | No (false) | Enable logging for observer in incremental dependency tracking. |
 | `--logTarget` | No (`console`) | Logging target to use, currently available: `console, slf4j`. |
-| `--experimental-jastadd-329` | No (false) | Use trace events `INC_FLUSH_START` and `INC_FLUSH_END` ([JastAdd issue #329][jastadd-issue-329]), see [section about automatic dependency tracking](/using#dependency-tracking-automatically-derived). |
-| `--incremental` | No (false) | Enables incremental dependency tracking (if `trace` is also set appropriately). |
-| `--trace[=flush]` | No (false) | Enables incremental dependency tracking (if `incremental` is also set appropriately). |
+| `--experimental-jastadd-329` | No (false) | Use tracing events `INC_FLUSH_START` and `INC_FLUSH_END` ([JastAdd issue #329][jastadd-issue-329]), see [section about automatic dependency tracking](/using#dependency-tracking-automatically-derived). |
+| `--incremental` | No (false) | Enables incremental dependency tracking (if `tracing` is also set appropriately). |
+| `--tracing[=flush]` | No (false) | Enables incremental dependency tracking (if `incremental` is also set appropriately). |
 | `--version` | No (false) | Print version info and exit (reused JastAdd option) |
 | `--o` | No (`.`) | Output directory (reused JastAdd option) |
 
@@ -29,6 +29,16 @@ However, depending on the selected protocols and/or used features, additional de
 
 ## Communication protocol characteristics
 
+### Java
+
+- Protocol identifier: `java`
+- URI scheme: `java://<ignored-host>[:ignored-port]/<topic>`
+    - the value for host and port are always ignored, but are necessary to form a legal URI
+- No required runtime dependencies
+- Additional remarks:
+    - First leading slash not included in topic.
+    - Currently, the default mappings are applied, which requires a consumer to expect `byte[]` (instead of a more intuitive token or node value). This might change in future versions.
+
 ### MQTT
 
 - Protocol identifier: `mqtt`
@@ -59,15 +69,15 @@ However, depending on the selected protocols and/or used features, additional de
 
 ### Automatic dependency tracking
 
-- Condition: When passing `--incremental` and `--trace=flush` to RagConnect
+- Condition: When passing `--incremental` and `--tracing=flush` to RagConnect
 - Required runtime dependencies: _none_
 - Required options for RelAST compiler: _none_
 - Required options for JastAdd:
     - `--incremental`
-    - `--trace=flush`
+    - `--tracing=flush`
 - Remarks:
     - Other (additional) values passed to those two options must be equal (e.g., `--incremental=param` passed to RagConnect must be also passed to JastAdd)
-    - Other values besides `flush` can be added to `--trace`
+    - Other values besides `flush` can be added to `--tracing`
     - [Feature description](/using#dependency-tracking-automatically-derived)
 
 ### (Safer) Automatic dependency tracking

pages/docs/dsl.md

@@ -49,7 +49,7 @@ Specifying such an endpoint has several consequences:
 
 **Example**:
 
-```java
+```
 // grammar
 Root ::= A SingleA:A [OptA:A] ListA:A* ;
 A ::= <Value> ;
@@ -60,13 +60,15 @@ receive Root.SingleA using MyMapping; // specialized endpoint
 ```
 
 Implied, additional connect specifications:
-```java
+
+```
 receive Root.A;
 receive Root.OptA;
 receive indexed Root.ListA;
 ```
 
 Application code:
+
 ```java
 A a = root.getOptA();
 // new method on A:

pages/docs/using.md

@@ -47,7 +47,7 @@ Otherwise, the deprecated manual dependencies must be used.
 
 ### Dependency tracking: Automatically derived
 
-To automatically track dependencies, the two additional parameters `--incremental` and `--trace=flush` have to be provided to both RagConnect and (in the later stage) JastAdd.
+To automatically track dependencies, the two additional parameters `--incremental` and `--tracing=flush` have to be provided to both RagConnect and (in the later stage) JastAdd.
 This will generate a different implementation of RagConnect relying on enabled incremental evaluation of JastAdd.
 The value for `incremental` has only been tested for `incremental=param`.
 The value for `trace` can include other values besides `flush`.

ragconnect.base/src/main/jastadd/Intermediate.jadd

@@ -126,6 +126,10 @@ aspect MustacheHandler {
 
   syn String Handler.fieldName() = ragconnect().internalRagConnectPrefix() + getUniqueName() + "Handler";
 
+  syn String Handler.pushMethodName() = "ragconnect" + capitalize(getUniqueName()) + "Push";
+
+  syn String Handler.registerConsumerMethodName() = "ragconnect" + capitalize(getUniqueName()) + "RegisterConsumer";
+
   syn String Handler.setupWaitUntilReadyMethodName() = "ragconnectSetup" + capitalize(getUniqueName()) + "WaitUntilReady";
 }
 

ragconnect.base/src/main/java/org/jastadd/ragconnect/compiler/Compiler.java

@@ -181,7 +181,7 @@ public class Compiler extends AbstractCompiler {
         new ValueOption("protocols", "Protocols to enable")
             .acceptMultipleValues(true)
             .addDefaultValue(OPTION_PROTOCOL_MQTT, "Enable MQTT")
-            .addAcceptedValue(OPTION_PROTOCOL_JAVA, "Enable Java")
+            .addAcceptedValue(OPTION_PROTOCOL_JAVA, "Enable Java (experimental)")
             .addAcceptedValue(OPTION_PROTOCOL_REST, "Enable REST")
     );
     optionPrintYaml = addOption(
@@ -314,7 +314,7 @@ public class Compiler extends AbstractCompiler {
     ragConnect.getConfiguration().setExperimentalJastAdd329(optionExperimentalJastAdd329.value());
     ragConnect.getConfiguration().setEvaluationCounter(optionEvaluationCounter.value());
 
-    // reuse "--incremental" and "--trace=flush" options of JastAdd
+    // reuse "--incremental" and "--tracing=flush" options of JastAdd
     boolean incrementalOptionActive = this.getConfiguration().incremental() && this.getConfiguration().traceFlush();
     ragConnect.getConfiguration().setIncrementalOptionActive(incrementalOptionActive);
     if (isVerbose()) {

ragconnect.base/src/main/resources/JavaHandler.mustache

@@ -23,14 +23,13 @@ public class JavaHandler {
     return path;
   }
 
-  public RagConnectToken registerConsumer(String path, java.util.function.Consumer<byte[]> consumer) {
+  RagConnectToken registerConsumer(String path, java.util.function.Consumer<byte[]> consumer) {
     RagConnectToken token = new RagConnectToken(java.net.URI.create("internal://host/" + path), null);
     registerCallback(token, (s, bytes) -> consumer.accept(bytes));
     return token;
   }
 
-
-  public boolean registerCallback(RagConnectToken connectToken, java.util.function.BiConsumer<String, byte[]> callback) {
+  boolean registerCallback(RagConnectToken connectToken, java.util.function.BiConsumer<String, byte[]> callback) {
     String path = extractPath(connectToken.uri);
     {{logInfo}}("[JAVA_HANDLER] Registering new callback for {{log_}}.", path);
 
@@ -46,7 +45,7 @@ public class JavaHandler {
     return true;
   }
 
-  public boolean unregisterCallback(RagConnectToken connectToken) {
+  boolean unregisterCallback(RagConnectToken connectToken) {
     String path = extractPath(connectToken.uri);
     java.util.function.BiConsumer<String, byte[]> callback = tokensForRemoval.get(connectToken);
     {{logInfo}}("[JAVA_HANDLER] Unregistering callback with uuid: on path: {{log_}}", path);
@@ -54,10 +53,10 @@ public class JavaHandler {
     return callbackList.get(path).remove(callback);
   }
 
-  public void close() {
+  void close() {
   }
 
-  public boolean push(String uriString, byte[] data) {
+  boolean push(String uriString, byte[] data) {
     String path = extractPath(java.net.URI.create(uriString));
     {{logDebug}}("[JAVA_HANDLER] Pushing a message for {{log_}}.", path);
     if (data == null) {

ragconnect.base/src/main/resources/handler.mustache

@@ -21,6 +21,12 @@ aspect RagConnectHandler {
 
 {{#javaHandler}}
   {{#InUse}}
+  public RagConnectToken {{rootNodeName}}.{{registerConsumerMethodName}}(String path, java.util.function.Consumer<byte[]> consumer) {
+    return {{fieldName}}.registerConsumer(path, consumer);
+  }
+  public boolean {{rootNodeName}}.{{pushMethodName}}(String uriString, byte[] data) {
+    return {{fieldName}}.push(uriString, data);
+  }
   {{> JavaHandler}}
   {{/InUse}}
 {{/javaHandler}}

ragconnect.tests/build.gradle

@@ -673,9 +673,6 @@ task compileJavaIncremental(type: RagConnectTest) {
         inputFiles = [file('src/test/01-input/java/Test.relast'),
                       file('src/test/01-input/java/Test.connect')]
         rootNode = 'Root'
-        logReads = true
-        logWrites = true
-        logIncremental = true
         protocols = ['java']
         extraOptions = defaultRagConnectOptionsAnd(['--experimental-jastadd-329', '--evaluationCounter'])
     }
@@ -694,14 +691,6 @@ task compileJavaIncremental(type: RagConnectTest) {
 
 // --- Task order ---
 classes.dependsOn(':ragconnect.base:jar')
-// TODO remove special handling of compileJavaIncremental once finished
-compileJavaIncremental {
-    doFirst {
-        fileTree(dir: 'src/test/java-gen/javaInc/ast/', exclude: '.gitignore')
-    }
-}
-//compileJavaIncremental.outputs.upToDateWhen { false }
-compileJavaIncremental.dependsOn(':ragconnect.base:assemble')
 
 // --- Misc ---
 static ArrayList<String> defaultRagConnectOptionsAnd(ArrayList<String> options = []) {

ragconnect.tests/src/test/01-input/java/Test.connect

@@ -5,9 +5,9 @@ send SenderRoot.SendNTA ;
 
 AddSuffix maps A a to A {:
   A result = new A();
-  String changedValue = a.getValue() + "post";
+  String changedValue = a.getValue() + "-post";
   result.setValue(changedValue);
-  result.setInner(new Inner("inner" + a.getInner().getInnerValue()));
+  result.setInner(new Inner(a.getInner().getInnerValue() + "-post"));
   return result;
 :}
 

ragconnect.tests/src/test/java/org/jastadd/ragconnect/tests/JavaTest.java

@@ -45,7 +45,6 @@ public class JavaTest {
   private JastAddList<A> lastValueMany;
   private A lastValueNTA;
   private TestChecker checker;
-  private JavaHandler handler;
 
   void createModel() {
     model = new Root();
@@ -67,16 +66,24 @@ public class JavaTest {
             .setCheckForTuple(TOPIC_SEND_MANY, (name, expected) -> checkAList(expected, lastValueMany, name))
             .setCheckForString(TOPIC_SEND_NTA, (name, expected) -> checkA(expected, lastValueNTA, name))
             .setActualString(TOPIC_RECEIVE_TOKEN, () -> receiverRoot.getSomeToken())
+            .setCheckForString(TOPIC_RECEIVE_NODE_PLAIN, (name, expected) ->
+                    checkA(expected, receiverRoot.getSomeNode(), name))
+            .setCheckForString(TOPIC_RECEIVE_NODE_MAPPED, (name, expected) ->
+                    checkA(expected, receiverRoot.getSomeNodeWithMapping(), name))
+            .setCheckForTuple(TOPIC_RECEIVE_MANY, (name, expected) ->
+                    checkAList(expected, receiverRoot.getManyNodeList(), name))
+            .setCheckForString(TOPIC_RECEIVE_NTA, (name, expected) ->
+                    checkA(expected, receiverRoot.getNTA(), name))
             .put(TOPIC_RECEIVE_TOKEN, "")
+            .put(TOPIC_RECEIVE_MANY, tuple())
             .setActualNumberOfValues(() -> 0)
             .disableManualWait();
 
     // callbacks
-    handler = model._ragconnect_javaHandler();
-    handler.registerConsumer(TOPIC_SEND_TOKEN, bytes -> lastValueToken = new String(bytes));
-    handler.registerConsumer(TOPIC_SEND_NODE, bytes -> lastValueNode = ExposingASTNode.INSTANCE.bytesToA(bytes));
-    handler.registerConsumer(TOPIC_SEND_MANY, bytes -> lastValueMany = ExposingASTNode.INSTANCE.bytesToList(bytes));
-    handler.registerConsumer(TOPIC_SEND_NTA, bytes -> lastValueNTA = ExposingASTNode.INSTANCE.bytesToA(bytes));
+    model.ragconnectJavaRegisterConsumer(TOPIC_SEND_TOKEN, bytes -> lastValueToken = new String(bytes));
+    model.ragconnectJavaRegisterConsumer(TOPIC_SEND_NODE, bytes -> lastValueNode = ExposingASTNode.INSTANCE.bytesToA(bytes));
+    model.ragconnectJavaRegisterConsumer(TOPIC_SEND_MANY, bytes -> lastValueMany = ExposingASTNode.INSTANCE.bytesToList(bytes));
+    model.ragconnectJavaRegisterConsumer(TOPIC_SEND_NTA, bytes -> lastValueNTA = ExposingASTNode.INSTANCE.bytesToA(bytes));
 
     // receive
     receiverRoot.connectSomeToken(javaUri(TOPIC_RECEIVE_TOKEN));
@@ -99,7 +106,7 @@ public class JavaTest {
     checker.put(TOPIC_SEND_TOKEN, "")
             .put(TOPIC_SEND_NODE, "1")
             .put(TOPIC_SEND_MANY, tuple())
-            .put(TOPIC_SEND_NTA, "1")
+            .put(TOPIC_SEND_NTA, "1|1")
     ;
 
     communicateBoth();
@@ -123,7 +130,7 @@ public class JavaTest {
     checker.check();
 
     senderRoot.setInput("2");
-    checker.put(TOPIC_SEND_NTA, "2").check();
+    checker.put(TOPIC_SEND_NTA, "2|1").check();
 
     senderRoot.getSendNode().setValue("3");
     checker.put(TOPIC_SEND_NODE, "3").check();
@@ -134,10 +141,20 @@ public class JavaTest {
     senderRoot.addSendManyNode(createA("5"));
     checker.put(TOPIC_SEND_MANY, tuple("5")).check();
 
-    handler.push(TOPIC_RECEIVE_TOKEN, ExposingASTNode.INSTANCE.stringToBytes("7"));
+    model.ragconnectJavaPush(TOPIC_RECEIVE_TOKEN, ExposingASTNode.INSTANCE.stringToBytes("7"));
     checker.put(TOPIC_RECEIVE_TOKEN, "7").check();
 
-    // TODO check other receive ports
+    model.ragconnectJavaPush(TOPIC_RECEIVE_NODE_PLAIN, ExposingASTNode.INSTANCE.aToBytes(createA("8")));
+    checker.put(TOPIC_RECEIVE_NODE_PLAIN, "8").check();
+
+    model.ragconnectJavaPush(TOPIC_RECEIVE_NODE_MAPPED, ExposingASTNode.INSTANCE.aToBytes(createA("9")));
+    checker.put(TOPIC_RECEIVE_NODE_MAPPED, "9-post|inner-post").check();
+
+    model.ragconnectJavaPush(TOPIC_RECEIVE_MANY, ExposingASTNode.INSTANCE.listToBytes(new JastAddList<>(createA("10"), createA("11"))));
+    checker.put(TOPIC_RECEIVE_MANY, tuple("10", "11")).check();
+
+    model.ragconnectJavaPush(TOPIC_RECEIVE_NTA, ExposingASTNode.INSTANCE.aToBytes(createA("12")));
+    checker.put(TOPIC_RECEIVE_NTA, "12").check();
 
     System.out.println(model.ragconnectEvaluationCounterSummary());
   }
@@ -146,8 +163,19 @@ public class JavaTest {
     if (expectedValue == null) {
       assertNull(actual, alias);
     } else {
+      final String expectedValueInA, expectedInnerValue;
+      if (expectedValue.contains("|")) {
+        String[] tokens = expectedValue.split("\\|");
+        expectedValueInA = tokens[0];
+        expectedInnerValue = tokens[1];
+      } else {
+        expectedValueInA = expectedValue;
+        expectedInnerValue = "inner";
+      }
       assertNotNull(actual, alias);
-      assertEquals(expectedValue, actual.getValue(), alias);
+      assertEquals(expectedValueInA, actual.getValue(), alias);
+      assertNotNull(actual.getInner(), alias + ".inner");
+      assertEquals(expectedInnerValue, actual.getInner().getInnerValue(), alias + ".inner");
     }
   }
 

ragconnect.tests/src/test/java/org/jastadd/ragconnect/tests/TestUtils.java

@@ -217,12 +217,41 @@ public class TestUtils {
         awaitMqtt().alias(name).until(actual, Predicate.isEqual(expected));
       }
     }
-    private final static String NUMBER_OF_VALUES = "numberOfValues";
-    private final Map<String, ActualAndExpected<String>> stringValues = new HashMap<>();
-    private final Map<String, ActualAndExpected<Tuple>> tupleValues = new HashMap<>();
-    private final Map<String, ActualAndExpected<Integer>> intValues = new HashMap<>();
-    private boolean needManualWait = true;
-    private boolean useManualWait = true;
+    static class ValuesToCompare<T> {
+      protected final Map<String, ActualAndExpected<T>> values = new HashMap<>();
+      protected final TestChecker parent;
+
+      ValuesToCompare(TestChecker parent) {
+        this.parent = parent;
+      }
+
+      public TestChecker setActual(String name, Callable<T> actual) {
+        values.computeIfAbsent(name, ActualAndExpected::new).actual = actual;
+        return parent;
+      }
+
+      public TestChecker setCheck(String name, BiConsumer<String, T> check) {
+        values.computeIfAbsent(name, ActualAndExpected::new).customCheck = check;
+        return parent;
+      }
+
+      public TestChecker put(String name, T expected) {
+        values.computeIfAbsent(name, ActualAndExpected::new).expected = expected;
+        return parent;
+      }
+
+      ActualAndExpected<T> get(String name) {
+        return values.get(name);
+      }
+
+      void forEach(BiConsumer<? super String, ? super ActualAndExpected<T>> action) {
+        values.forEach(action);
+      }
+    }
+    static class IntegerValuesToCompare extends ValuesToCompare<Integer> {
+      IntegerValuesToCompare(TestChecker parent) {
+        super(parent);
+      }
 
       public TestChecker incNumberOfValues() {
         return addToNumberOfValues(1);
@@ -230,60 +259,70 @@ public class TestUtils {
 
       public TestChecker addToNumberOfValues(int increment) {
         // if there is at least one call to this, we do not need to manually wait in the next check()
-      needManualWait = false;
-      Integer currentExpected = intValues.computeIfAbsent(NUMBER_OF_VALUES, ActualAndExpected::new).expected;
+        parent.needManualWait = false;
+        Integer currentExpected = values.computeIfAbsent(NUMBER_OF_VALUES, ActualAndExpected::new).expected;
         return put(NUMBER_OF_VALUES, currentExpected + increment);
       }
 
       public TestChecker setActualNumberOfValues(Callable<Integer> actual) {
-      setActualInteger(NUMBER_OF_VALUES, actual);
-      intValues.get(NUMBER_OF_VALUES).expected = 0;
-      return this;
+        setActual(NUMBER_OF_VALUES, actual);
+        values.get(NUMBER_OF_VALUES).expected = 0;
+        return parent;
+      }
+    }
+    private final static String NUMBER_OF_VALUES = "numberOfValues";
+    public final ValuesToCompare<String> stringValues = new ValuesToCompare<>(this);
+    public final ValuesToCompare<Tuple> tupleValues = new ValuesToCompare<>(this);
+    public final IntegerValuesToCompare intValues = new IntegerValuesToCompare(this);
+    private boolean needManualWait = true;
+    private boolean useManualWait = true;
+
+    public TestChecker incNumberOfValues() {
+      return intValues.incNumberOfValues();
+    }
+
+    public TestChecker addToNumberOfValues(int increment) {
+      return intValues.addToNumberOfValues(increment);
+    }
+
+    public TestChecker setActualNumberOfValues(Callable<Integer> actual) {
+      return intValues.setActualNumberOfValues(actual);
     }
 
     public TestChecker setActualString(String name, Callable<String> actual) {
-      stringValues.computeIfAbsent(name, ActualAndExpected::new).actual = actual;
-      return this;
+      return stringValues.setActual(name, actual);
     }
 
     public TestChecker setCheckForString(String name, BiConsumer<String, String> check) {
-      stringValues.computeIfAbsent(name, ActualAndExpected::new).customCheck = check;
-      return this;
+      return stringValues.setCheck(name, check);
     }
 
     public TestChecker put(String name, String expected) {
-      stringValues.computeIfAbsent(name, ActualAndExpected::new).expected = expected;
-      return this;
+      return stringValues.put(name, expected);
     }
 
     public TestChecker setActualTuple(String name, Callable<Tuple> actual) {
-      tupleValues.computeIfAbsent(name, ActualAndExpected::new).actual = actual;
-      return this;
+      return tupleValues.setActual(name, actual);
     }
 
     public TestChecker setCheckForTuple(String name, BiConsumer<String, Tuple> check) {
-      tupleValues.computeIfAbsent(name, ActualAndExpected::new).customCheck = check;
-      return this;
+      return tupleValues.setCheck(name, check);
     }
 
     public TestChecker put(String name, Tuple expected) {
-      tupleValues.computeIfAbsent(name, ActualAndExpected::new).expected = expected;
-      return this;
+      return tupleValues.put(name, expected);
     }
 
     public TestChecker setActualInteger(String name, Callable<Integer> actual) {
-      intValues.computeIfAbsent(name, ActualAndExpected::new).actual = actual;
-      return this;
+      return intValues.setActual(name, actual);
     }
 
     public TestChecker setCheckForInteger(String name, BiConsumer<String, Integer> check) {
-      intValues.computeIfAbsent(name, ActualAndExpected::new).customCheck = check;
-      return this;
+      return intValues.setCheck(name, check);
     }
 
     public TestChecker put(String name, Integer expected) {
-      intValues.computeIfAbsent(name, ActualAndExpected::new).expected = expected;
-      return this;
+      return intValues.put(name, expected);
     }
 
     public TestChecker disableManualWait() {