Merge branch '36-feature-send-enpoint-for-non-terminal-using-implicit-nta' into 'dev'

Resolve "Feature: Send enpoint for non-terminal using implicit NTA"

Closes #36

See merge request !24

pages/docs/dsl.md

@@ -38,9 +38,6 @@ A breakdown of the parts of that syntax:
 
 ### Context-Free Endpoints
 
-!!! attention
-    Context-Free endpoints are currently only supported for receiving endpoints.
-
 An endpoint with only a non-terminal and without a target is called context-free endpoint.
 Specifying such an endpoint has several consequences:
 

pages/docs/inner-workings.md

@@ -19,3 +19,17 @@ One of the main aspects is `Intermediate` containing all attributes consumed by
 The other main aspect (which is currently not really used) is `IntermediateToYAML` containing the transformation from a `RagConnect` subtree to a `Document` subtree defined by `Mustache.relast` (located in `relast-preprocessor` submodule).
 This is used to generate a YAML file containing the data used by mustache.
 It can be used by the default mustache implementation together with the templates.
+
+# Implementation details
+
+In the following, details for special implementation topics are discussed.
+
+## forwarding
+
+When a nonterminal is used in a send endpoints, it needs an implicit forwarding attribute to work, because only _computed elements_ can be sent.
+Since the nonterminal itself should be sent, the generated attribute simply returns this nonterminal.
+
+However, changing any token within the whole subtree or changing the structure of the subtree must trigger a new message, upon computation of the forwarding attribute, all tokens are "touched" (their getter is called).
+This way, the dependency tracking registers a dependency between structure and tokens to the attribute.
+
+The attribute (as well as any other generated element) is prefixed with `_ragconnect_` to avoid potential name conflicts with user-specified elements.

pages/docs/using.md

@@ -4,14 +4,14 @@ The full example is available at <https://git-st.inf.tu-dresden.de/jastadd/ragco
 
 ## Preparation and Specification
 
-The following examples are inspired by the real test case [read1write2](https://git-st.inf.tu-dresden.de/jastadd/ragconnect-tests/-/tree/master/ragconnect.tests/src/test/01-input/read1write2)
+The following examples are inspired by real [test cases](https://git-st.inf.tu-dresden.de/jastadd/ragconnect-tests/-/tree/master/ragconnect.tests/src/test/01-input) read1write2 and tokenValueSend.
 The idea is to have two non-terminals, where input information is received on one of them, and - after transformation - is sent out by both.
 
 Let's use the following grammar:
 
 ```
 A ::= <Input:String> /<OutputOnA:String>/ B* ;
-B ::= /<OutputOnB:String>/ ;
+B ::= <OutputOnB:String> ;
 ```
 
 To declare receiving and sending tokens, a dedicated DSL is used:
@@ -35,28 +35,15 @@ Such mapping definitions can be defined for receiving tokens as well.
 In this case, they are applied before the value is set.
 If no mapping definition is given, or if the required type (depending on the communication protocol, see later) does not match, a "default mapping definition" is used to avoid boilerplate code converting from or to primitive types.
 
-Furthermore, let the following attribute definitions be given:
+Furthermore, let the following attribute definition be given:
 
 ```java
 syn String A.getOutputOnA() = "a" + getInput();
-
-syn String B.getOutputOnB() = "b" + input();
-inh String B.input();
-eq A.getB().input() = getInput();
 ```
 
-In other words, `OutputOnA` depends on `Input` of the same node, and `OutputOnB` depends on `Input` of its parent node.
-Currently, those dependencies can be explicitly written down, or incremental evaluation can be used.
-
-### Dependency tracking: Manually specified
-
-This specification happens also in the DSL (dependencies have to be named to uniquely identify them):
-
-```java
-// dependency definitions
-A.OutputOnA canDependOn A.Input as dependencyA ;
-B.OutputOnB canDependOn A.Input as dependencyB ;
-```
+In other words, `OutputOnA` depends on `Input` of the same node.
+This dependency is automatically inferred, if incremental evaluation is used.
+Otherwise, the deprecated manual dependencies must be used.
 
 ### Dependency tracking: Automatically derived
 
@@ -68,6 +55,15 @@ The value for `trace` can include other values besides `flush`.
 An experimental, optimized version can be selected using `--experimental-jastadd-329` reducing the risk of conflicts between concurrent attribute evaluations.
 However, this requires a version of JastAdd that resolved the [issue 329](https://bitbucket.org/jastadd/jastadd2/issues/329/add-event-for-completion-of-flush).
 
+### Deprecated Manual Dependency Specification
+
+Specification happens also in the DSL (dependencies have to be named to uniquely identify them):
+
+```java
+// dependency definition
+A.OutputOnA canDependOn A.Input as dependencyA ;
+```
+
 ## Using generated code
 
 After specifying everything, code will be generated if [setup properly](/adding).
@@ -83,15 +79,11 @@ a.addB(b1);
 a.addB(b2);
 ```
 
-If necessary, we have to set the dependencies as [described earlier](#dependency-tracking-manually-specified).
+If necessary, we have to set the dependencies as [described earlier](#deprecated-manual-dependency-specification).
 
 ```java
 // a.OutputOnA -> a.Input
 a.addDependencyA(a);
-// b1.OutputOnB -> a.Input
-b1.addDependencyB(a);
-// b2.OutputOnB -> a.Input
-b2.addDependencyB(a);
 ```
 
 Finally, we can actually _connect_ the tokens.
@@ -144,24 +136,24 @@ Non-terminal children can also be selected as endpoints (not only tokens).
 Receiving normal non-terminal children and optionals means to replace them with a new node deserialized from the received message.
 Sending them involves serializing a node, and sending this representation in a message.
 
-Suppose, the following (shortened) grammar is used (inspired from the testcase [tree](https://git-st.inf.tu-dresden.de/jastadd/ragconnect-tests/-/tree/master/ragconnect.tests/src/test/01-input/tree))
+Suppose, the following (shortened) grammar is used (inspired from the testcase tree and forwarding)
 
 ```
 Root ::= SenderRoot ReceiverRoot ;
-SenderRoot ::= <Input:int> /Alfa/ ;
-ReceiverRoot ::= Alfa ;
-Alfa ::= // some content ...
+SenderRoot ::= <Input:int> /A/ B ;
+ReceiverRoot ::= A ;
+A ::= // some content ...
+B ::= <Value> ;
 ```
 
-Now, the complete node of type `Alfa` can be sent, and received again using the following connect specification:
+Now, the complete node of types `A` and `B` can be sent, and received again using the following connect specification:
 
 ```
-send tree SenderRoot.Alfa ;
-receive tree ReceiverRoot.Alfa ;
+send SenderRoot.A ;
+send SenderRoot.B ;
+receive ReceiverRoot.A ;
 ```
 
-Currently, receiving and sending trees requires the explicit demarcation from tokens using the keyword `tree`.
-
 To process non-terminals, default mappings are provided for every non-terminal type of the used grammar.
 They use the JSON serialization offered by the RelAST compiler, i.e., interpret the message as a `String`, deserialize the content reading the message as JSON, or vice versa.
 Additional dependencies are required to use this feature, as detailed in [the compiler section](/compiler#treelist-endpoints).
@@ -173,82 +165,82 @@ When receiving list children, there are a few more options to match the connecti
 Suppose we use a similar grammar as above, i.e.:
 
 ```
-SenderRoot ::= /AlfaList:Alfa*/ /SingleAlfa:Alfa/;
-ReceiverRoot ::= Alfa* ;
+SenderRoot ::= /AList:A*/ /SingleA:A/;
+ReceiverRoot ::= A* ;
 ```
 
-Several options are possible:
+Several options are possible (please also refer to the specification of the [connect DSL](/dsl):
 
-### list
+### (empty)
 
-A message for a list endpoint can be interpreted as a complete list (a sequence of nodes of type `Alfa`) by using the `list` keyword instead of `tree`:
+A message for a list endpoint can be interpreted as a complete list (a sequence of nodes of type `A`) by not specifying any special keyword:
 
 ```
-receive list ReceiverRoot.Alfa ;
+receive ReceiverRoot.A ;
 ```
 
-### list + with add
+### with add
 
 Upon receiving the message, the deserialized list can also be appended to the existing list instead of replace the latter.
-This can be achieved using the keyword `with add` in addition to the keyword `list`:
+This can be achieved using the keyword `with add` :
 
 ```
-receive list with add ReceiverRoot.Alfa ;
+receive with add ReceiverRoot.Alfa ;
 ```
 
-### tree (indexed)
+### indexed
 
 A message for a list endpoint can also be interpreted as an element of this list.
 
 ```
-receive tree ReceiverRoot.Alfa ;
+receive tree ReceiverRoot.A ;
 ```
 
 Upon connection, the index of the deserialized element to set, has to be passed (`1` in the example below).
 The list must have enough elements once a message is received.
 
 ```java
-receiverRoot.connectAlfa("<some-url>", 1);
+receiverRoot.connectA("<some-url>", 1);
 ```
 
-### tree (wildcard)
+### indexed (wildcard)
 
-Similar to the `tree (indexed)` case above, messages are interpreted as an element of the list, but the connection can also be made using a "wildcard topic" and without an index.
+Similar to the `indexed` case above, messages are interpreted as an element of the list, but the connection can also be made using a "wildcard topic" and without an index.
 Then, once a message is received from a new concrete topic, the deserialized element will be appended to the list and this topic is associated with the index of the newly added element.
 Any further message from that topic will replace the element at the associated index.
 In the short example below, MQTT is used to with a wildcard topic, as `#` matches every sub-topic.
 
 ```java
-receiverRoot.connectAlfa("mqtt://<broker>/some/topic/#");
+receiverRoot.connectA("mqtt://<broker>/some/topic/#");
 
 // list is initially empty
-assertEquals(receiverRoot.getAlfaList(), list());
+assertEquals(receiverRoot.getAList(), list());
 // after receiving "1" on new topic "some/topic/one" (index 0)
-assertEquals(receiverRoot.getAlfaList(), list("1"));
+assertEquals(receiverRoot.getAList(), list("1"));
 // after receiving "other" on new topic "some/topic/two" (index 1)
-assertEquals(receiverRoot.getAlfaList(), list("1", "other"));
+assertEquals(receiverRoot.getAList(), list("1", "other"));
 // after receiving "new" on existing topic "some/topic/one" (index 0)
-assertEquals(receiverRoot.getAlfaList(), list("new", "other"));
+assertEquals(receiverRoot.getAList(), list("new", "other"));
 ```
 
-### tree (indexed/wildcard) + with add
+### indexed + with add
 
-Combining `tree` and `with add` results in a connection, where messages are interpreted as elements of the list, and new elements are appended to the existing list.
+Combining `indexed` and `with add` results in a connection, where messages are interpreted as elements of the list, and new elements are appended to the existing list.
 In that case, wildcard and non-wildcard connections behave in the same way, as no index has to be passed, and the element is always append at the end.
 Reusing the example from above, the following observations can be made.
 
 ```java
-receiverRoot.connectAlfa("mqtt://<broker>/some/topic/#");
+receiverRoot.connectA("mqtt://<broker>/some/topic/#");
 // or
-receiverRoot.connectAlfa("mqtt://<broker>/some/topic/one");
-receiverRoot.connectAlfa("mqtt://<broker>/some/topic/two");
+receiverRoot.connectA("mqtt://<broker>/some/topic/one");
+receiverRoot.connectA("mqtt://<broker>/some/topic/two");
 
 // list is initially empty
-assertEquals(receiverRoot.getAlfaList(), list());
+assertEquals(receiverRoot.getAList(), list());
 // after receiving "1" on topic "some/topic/one"
-assertEquals(receiverRoot.getAlfaList(), list("1"));
+assertEquals(receiverRoot.getAList(), list("1"));
 // after receiving "other" on topic "some/topic/two"
-assertEquals(receiverRoot.getAlfaList(), list("1", "other"));
+assertEquals(receiverRoot.getAList(), list("1", "other"));
 // after receiving "new" on topic "some/topic/one"
-assertEquals(receiverRoot.getAlfaList(), list("1", "other", "new"));
+assertEquals(receiverRoot.getAList(), list("1", "other", "new"));
 ```

ragconnect.base/build.gradle

@@ -22,7 +22,10 @@ mainClassName = 'org.jastadd.ragconnect.compiler.Compiler'
 
 repositories {
     mavenCentral()
-    jcenter()
+    maven {
+        name "gitlab-maven"
+        url "https://git-st.inf.tu-dresden.de/api/v4/groups/jastadd/-/packages/maven"
+    }
 }
 tasks.compileJava {
     options.release.set(11)
@@ -31,8 +34,8 @@ tasks.compileJava {
 dependencies {
     implementation project(':relast-preprocessor')
     implementation group: 'com.github.spullara.mustache.java', name: 'compiler', version: "${mustache_java_version}"
-//    runtimeOnly group: 'org.jastadd', name: 'jastadd', version: '2.3.5'
-    runtimeOnly fileTree(include: ['jastadd2.jar'], dir: '../libs')
+    runtimeOnly group: 'org.jastadd', name: 'jastadd2', version: '2.3.5-dresden'
+//    runtimeOnly fileTree(include: ['jastadd2.jar'], dir: '../libs')
     api group: 'net.sf.beaver', name: 'beaver-rt', version: '0.9.11'
 }
 

ragconnect.base/src/main/jastadd/Analysis.jrag

@@ -8,7 +8,7 @@ aspect Analysis {
         .count() > 1;
   }
   eq TypeEndpointTarget.isAlreadyDefined() {
-    return lookupTypeEndpointDefinitions(getType()).stream()
+    return lookupGivenTypeEndpointDefinitions(getType()).stream()
         .filter(containingEndpointDefinition()::matchesType)
         .count() > 1;
   }

ragconnect.base/src/main/jastadd/Errors.jrag

@@ -11,6 +11,10 @@ aspect Errors {
     when !getSend() && getEndpointTarget().isTokenEndpointTarget() && token().getNTA()
     to RagConnect.errors();
 
+  EndpointDefinition contributes error("Indexed based list access may only be used for type endpoint targets!")
+    when getIndexBasedListAccess() && !getEndpointTarget().isTypeEndpointTarget()
+    to RagConnect.errors();
+
   // if first mapping is null, then suitableDefaultMapping() == null
   EndpointDefinition contributes error("No suitable default mapping found for type " +
       ((getMappingList().isEmpty())
@@ -27,6 +31,9 @@ aspect Errors {
           token().effectiveJavaTypeUse())
     to RagConnect.errors();
 
+  UntypedEndpointTarget contributes error("Could not resolve endpoint target " + getTypeName() + "." + getChildName())
+    to RagConnect.errors();
+
   ContextFreeTypeEndpointTarget contributes error("Context-Free endpoint not allowed for root node " +
       getTypeDecl().getName() + "!")
     when getTypeDecl().occurencesInProductionRules().isEmpty()
@@ -34,7 +41,7 @@ aspect Errors {
 
   EndpointDefinition contributes error("Clash with implied, indexed endpoint definition of context-free endpoint in line " +
           clashingContextFreeEndpointDefinition().getStartLine() + "!")
-    when !getSend() && clashingContextFreeEndpointDefinition() != null
+    when clashingContextFreeEndpointDefinition() != null && clashingContextFreeEndpointDefinition().matchesType(this)
     to RagConnect.errors();
 
   DependencyDefinition contributes error("Dependency definition already defined for " + getSource().containingTypeDecl().getName() + " with name " + getID())

ragconnect.base/src/main/jastadd/Intermediate.jadd

@@ -7,16 +7,38 @@ aspect SharedMustache {
   // === RagConnect ===
   syn boolean RagConnect.configIncrementalOptionActive() = getConfiguration().getIncrementalOptionActive();
 
+  syn String RagConnect.configJastAddList() = getConfiguration().getJastAddList();
+
+  syn String RagConnect.configJastAddOpt() = getConfiguration().getJastAddOpt();
+
   syn boolean RagConnect.configLoggingEnabledForIncremental() = getConfiguration().getLoggingEnabledForIncremental();
 
   syn boolean RagConnect.configExperimentalJastAdd329() = getConfiguration().getExperimentalJastAdd329();
 
   syn String RagConnect.internalRagConnectPrefix() = "_ragconnect_";
 
+  syn String RagConnect.observerInstanceSingletonMethodName() = internalRagConnectPrefix() + "Observer";
+  syn String RagConnect.observerInstanceResetMethodName() = internalRagConnectPrefix() + "resetObserver";
+
   syn String RagConnect.rootNodeName() = getConfiguration().getRootNode().getName();
 
   // === EndpointDefinition ===
   syn String EndpointDefinition.lastResult() = lastDefinition().outputVarName();
+
+  syn boolean EndpointDefinition.typeIsList() = getEndpointTarget().typeIsList();
+
+  syn boolean EndpointDefinition.typeIsOpt() = getEndpointTarget().typeIsOpt();
+
+  // === attributes needed for computing above ones ===
+  syn boolean EndpointTarget.typeIsList() = false;
+  eq TypeEndpointTarget.typeIsList() {
+    return getType().isListComponent();
+  }
+
+  syn boolean EndpointTarget.typeIsOpt() = false;
+  eq TypeEndpointTarget.typeIsOpt() {
+    return getType().isOptComponent();
+  }
 }
 
 aspect MustacheDependencyDefinition {
@@ -60,8 +82,6 @@ aspect MustacheHandleUri { /* empty */ }
 
 aspect MustacheListAspect {
   // === RagConnect ===
-  syn String RagConnect.configJastAddList() = getConfiguration().getJastAddList();
-
   syn boolean RagConnect.hasTreeListEndpoints() {
     for (EndpointDefinition endpointDef : allEndpointDefinitionList()) {
       if (endpointDef.typeIsList()) {
@@ -96,14 +116,19 @@ aspect MustacheMappingApplicationAndDefinition {
       // only check if received list is not null
       return lastResult() + " == null";
     }
+    String toPrepend = "";
+    if (!getSend() && getIndexBasedListAccess() && typeIsList() && !getWithAdd()) {
+      // for wildcard (list) receive connect
+      toPrepend = "index >= 0 && ";
+    }
     if (getEndpointTarget().isTypeEndpointTarget() && type().isOptComponent()) {
       // use "hasX()" instead of "getX() != null" for optionals
-      return "has" + typeName() + "()" + " && " + preemptiveExpectedValue() + ".equals(" + lastResult() + ")";
+      return toPrepend + "has" + typeName() + "()" + " && " + preemptiveExpectedValue() + ".equals(" + lastResult() + ")";
     }
     if (lastDefinition().getMappingDefinition().getToType().isPrimitiveType() || lastDefinition().getMappingDefinition().isDefaultMappingDefinition()) {
-      return preemptiveExpectedValue() + " != null && " + preemptiveExpectedValue() + ".equals(" + lastResult() + ")";
+      return toPrepend + preemptiveExpectedValue() + " != null && " + preemptiveExpectedValue() + ".equals(" + lastResult() + ")";
     }
-    return preemptiveExpectedValue() + " != null ? " + preemptiveExpectedValue() + ".equals(" + lastResult() + ") : " + lastResult() + " == null";
+    return toPrepend + preemptiveExpectedValue() + " != null ? " + preemptiveExpectedValue() + ".equals(" + lastResult() + ") : " + lastResult() + " == null";
   }
 
   syn JastAddList<MInnerMappingDefinition> EndpointDefinition.innerMappingDefinitions() = toMustache().getInnerMappingDefinitionList();
@@ -144,19 +169,19 @@ aspect MustacheMappingApplicationAndDefinition {
   syn String MEndpointDefinition.firstInputVarName();
 
   eq MTokenReceiveDefinition.firstInputVarName() = "message";
-  eq MTokenReceiveDefinition.preemptiveExpectedValue() = getterMethodName() + "()";
+  eq MTokenReceiveDefinition.preemptiveExpectedValue() = getterMethodCall();
   eq MTokenReceiveDefinition.preemptiveReturn() = "return;";
 
-  eq MTokenSendDefinition.firstInputVarName() = getterMethodName() + "()";
-  eq MTokenSendDefinition.preemptiveExpectedValue() = lastValue();
+  eq MTokenSendDefinition.firstInputVarName() = getterMethodCall();
+  eq MTokenSendDefinition.preemptiveExpectedValue() = lastValueGetterCall();
   eq MTokenSendDefinition.preemptiveReturn() = "return false;";
 
   eq MTypeReceiveDefinition.firstInputVarName() = "message";
-  eq MTypeReceiveDefinition.preemptiveExpectedValue() = getterMethodName() + "()";
+  eq MTypeReceiveDefinition.preemptiveExpectedValue() = getterMethodCall();
   eq MTypeReceiveDefinition.preemptiveReturn() = "return;";
 
-  eq MTypeSendDefinition.firstInputVarName() = getterMethodName() + "()";
-  eq MTypeSendDefinition.preemptiveExpectedValue() = lastValue();
+  eq MTypeSendDefinition.firstInputVarName() = getterMethodCall();
+  eq MTypeSendDefinition.preemptiveExpectedValue() = lastValueGetterCall();
   eq MTypeSendDefinition.preemptiveReturn() = "return false;";
 
   eq MContextFreeTypeReceiveDefinition.firstInputVarName() = "message";
@@ -169,7 +194,7 @@ aspect MustacheMappingApplicationAndDefinition {
 
   syn String MEndpointDefinition.parentTypeName() = getEndpointDefinition().parentTypeName();
 
-  syn String MEndpointDefinition.getterMethodName() = getEndpointDefinition().getterMethodName();
+  syn String MEndpointDefinition.getterMethodCall() = getEndpointDefinition().getterMethodCall();
 }
 
 aspect MustacheRagConnect {
@@ -198,8 +223,26 @@ aspect MustacheRagConnect {
     return result;
   }
 
+  syn List<TypeDecl> RagConnect.allTypeDecls() {
+    return getProgram().typeDecls().stream()
+            .sorted(java.util.Comparator.comparing(TypeDecl::getName))
+            .collect(java.util.stream.Collectors.toList());
+  }
+
   // > allMappingDefinitions in Mappings.jrag
 
+  syn String RagConnect.observerInstanceFieldName() = internalRagConnectPrefix() + "ObserverInstance";
+
+  syn java.util.List<Component> TypeDecl.tokenComponents() {
+    return filteredComponents(Component::isTokenComponent);
+  }
+  syn java.util.List<Component> TypeDecl.normalComponents() {
+    return filteredComponents(comp -> comp.isTypeComponent() && !comp.isListComponent());
+  }
+  syn java.util.List<Component> TypeDecl.listComponents() {
+    return filteredComponents(Component::isListComponent);
+  }
+
   syn List<TokenComponent> RagConnect.tokenComponentsThatNeedProxy() {
     List<TokenComponent> result = new ArrayList<>();
     for (TokenComponent token : getProgram().allTokenComponents()) {
@@ -210,6 +253,8 @@ aspect MustacheRagConnect {
     return result;
   }
 
+  syn String RagConnect.touchedTerminalsMethodName() = internalRagConnectPrefix() + "touchedTerminals";
+
   syn List<TypeDecl> RagConnect.typeDeclsOfContextFreeEndpointTargets() {
     List<TypeDecl> result = new ArrayList<>();
     for (EndpointTarget target : givenEndpointTargetList()) {
@@ -233,11 +278,18 @@ aspect MustacheRagConnect {
   }
 
   syn nta JastAddList<EndpointDefinition> EndpointTarget.impliedEndpointDefinitions() = new JastAddList<>();
+//  eq TypeEndpointTarget.impliedEndpointDefinitions() {
+//    JastAddList<EndpointDefinition> result = super.impliedEndpointDefinitions();
+//    if (!getSend() || !typeIsList() || !getIndexBasedListAccess()) {
+//      return result;
+//    }
+//    // create a new endpoint
+//  }
   eq ContextFreeTypeEndpointTarget.impliedEndpointDefinitions() {
     JastAddList<EndpointDefinition> result = super.impliedEndpointDefinitions();
     EndpointDefinition containingDef = containingEndpointDefinition();
     for (TypeComponent typeComponent : getTypeDecl().occurencesInProductionRules()) {
-      List<EndpointDefinition> defsForTypeComponent = lookupTypeEndpointDefinitions(typeComponent);
+      List<EndpointDefinition> defsForTypeComponent = lookupGivenTypeEndpointDefinitions(typeComponent);
       if (!defsForTypeComponent.stream().anyMatch(containingDef::matchesType)) {
         // there is no user-defined endpoint definition for this typeComponent yet
         // -> create a new endpoint definition with the same options and mappings as the context-free def
@@ -258,6 +310,15 @@ aspect MustacheRagConnect {
     return result;
   }
 
+  private java.util.List<Component> TypeDecl.filteredComponents(java.util.function.Predicate<Component> filter) {
+    java.util.List<Component> result = new java.util.ArrayList<>();
+    for (Component comp : getComponentList()) {
+      if (filter.test(comp)) {
+        result.add(comp);
+      }
+    }
+    return result;
+  }
 }
 
 aspect MustacheReceiveAndSendAndHandleUri {
@@ -272,7 +333,18 @@ aspect MustacheReceiveAndSendAndHandleUri {
     if (getEndpointTarget().isTokenEndpointTarget()) {
       extra = lookupTokenEndpointDefinitions(token()).size() > 1 ? uniqueSuffix() : "";
     } else if (getEndpointTarget().isTypeEndpointTarget()) {
-      extra = lookupTypeEndpointDefinitions(type()).size() > 1 ? uniqueSuffix() : "";
+      // here it has to be checked if there are ANY typeEndpointDefinitions within all endpoints (including implied)
+      extra = lookupAllTypeEndpointDefinitions(type()).size() > 1 ? uniqueSuffix() : "";
+    } else if (getEndpointTarget().isContextFreeTypeEndpointTarget()) {
+      // here it has to be checked if there are ANY typeEndpointDefinitions within all endpoints (including implied)
+      boolean needExtra = false;
+      for (TypeComponent typeComponent : getEndpointTarget().asContextFreeTypeEndpointTarget().getTypeDecl().occurencesInProductionRules()) {
+        if (lookupAllTypeEndpointDefinitions(typeComponent).size() > 1) {
+          needExtra = true;
+          break;
+        }
+      }
+      extra = needExtra ? uniqueSuffix() : "";
     } else {
       extra = "";
     }
@@ -281,12 +353,21 @@ aspect MustacheReceiveAndSendAndHandleUri {
 
   syn String EndpointDefinition.entityName() = getEndpointTarget().entityName();
 
+  syn boolean EndpointDefinition.indexedSend() = getSend() && indexedList();
+  syn boolean EndpointDefinition.indexedList() = typeIsList() && getIndexBasedListAccess();
+
   syn String EndpointDefinition.getterMethodName() = getEndpointTarget().getterMethodName();
+  syn String EndpointDefinition.getterMethodCall() = getEndpointTarget().getterMethodCall();
+  syn String EndpointDefinition.realGetterMethodName() = getEndpointTarget().realGetterMethodName();
+  syn String EndpointDefinition.realGetterMethodCall() = getEndpointTarget().realGetterMethodCall();
 
   syn String EndpointDefinition.parentTypeName() = getEndpointTarget().parentTypeName();
 
   // === attributes needed for computing above ones ===
   syn String EndpointTarget.getterMethodName();
+  syn String EndpointTarget.getterMethodCall() = getterMethodName() + "()";
+  syn String EndpointTarget.realGetterMethodName() = getterMethodName() + "NoTransform";
+  syn String EndpointTarget.realGetterMethodCall() = realGetterMethodName() + "()";
   syn String EndpointTarget.parentTypeName();
   syn String EndpointTarget.entityName();
 
@@ -294,14 +375,23 @@ aspect MustacheReceiveAndSendAndHandleUri {
   eq TokenEndpointTarget.parentTypeName() = getToken().containingTypeDecl().getName();
   eq TokenEndpointTarget.entityName() = getToken().getName();
 
-  eq TypeEndpointTarget.getterMethodName() = "get" + getType().getName() + (typeIsList() ? "List" : "");
+  eq TypeEndpointTarget.getterMethodName() = getterMethodeNameHelper(true);
+  eq TypeEndpointTarget.getterMethodCall() = getterMethodName() + (containingEndpointDefinition().indexedList() && !containingEndpointDefinition().getWithAdd() ? "(index)" : "()") + (typeIsOpt() ? ".getChild(0)" : "");
+  eq TypeEndpointTarget.realGetterMethodName() = getterMethodeNameHelper(false);
+  eq TypeEndpointTarget.realGetterMethodCall() = realGetterMethodName() + (containingEndpointDefinition().indexedSend() ? "(index)" : "()");
   eq TypeEndpointTarget.parentTypeName() = getType().containingTypeDecl().getName();
   eq TypeEndpointTarget.entityName() = getType().getName() + (typeIsList() && !containingEndpointDefinition().getIndexBasedListAccess() ? "List" : "");
 
+  private String TypeEndpointTarget.getterMethodeNameHelper(boolean useForwarding) {
+    return (useForwarding && needForwardingNTA() ? forwardingNTA_Name() :
+            "get" + getType().getName() + (typeIsList() && (!containingEndpointDefinition().getIndexBasedListAccess() ||
+                    containingEndpointDefinition().getWithAdd()) ? "List" : "") + (typeIsOpt() ? "Opt" : "")
+                    + (needForwardingNTA() ? "NoTransform" : ""));
+  }
+
   eq ContextFreeTypeEndpointTarget.getterMethodName() = null;
   eq ContextFreeTypeEndpointTarget.parentTypeName() = getTypeDecl().getName();
   eq ContextFreeTypeEndpointTarget.entityName() = "";
-
 }
 
 aspect MustacheReceiveDefinition {
@@ -319,14 +409,7 @@ aspect MustacheReceiveDefinition {
 
   syn String EndpointDefinition.resolveInListMethodName() = ragconnect().internalRagConnectPrefix() + "_resolve" + entityName() + "InList";
 
-  syn boolean EndpointDefinition.typeIsList() = getEndpointTarget().typeIsList();
-
   // === attributes needed for computing above ones ===
-  syn boolean EndpointTarget.typeIsList() = false;
-  eq TypeEndpointTarget.typeIsList() {
-    return getType().isListComponent();
-  }
-
   syn String EndpointDefinition.uniqueSuffix() = getSend() ? "Send" : "Receive";
 }
 
@@ -335,7 +418,14 @@ aspect MustacheSendDefinition {
   syn boolean RagConnect.configLoggingEnabledForWrites() = getConfiguration().getLoggingEnabledForWrites();
 
   // === EndpointDefinition ===
-  syn String EndpointDefinition.lastValue() = senderName() + ".lastValue";
+  syn String EndpointDefinition.lastValueGetterCall() = senderName() + ".getLastValue(" +
+    (getIndexBasedListAccess() ? "index" : "") + ")";
+
+  syn String EndpointDefinition.lastValueSetter() = senderName() + ".setLastValue";
+
+  syn boolean EndpointDefinition.needForwardingNTA() = getEndpointTarget().needForwardingNTA();
+  syn String EndpointDefinition.forwardingNTA_Name() = getEndpointTarget().forwardingNTA_Name();
+  syn String EndpointDefinition.forwardingNTA_Type() = getEndpointTarget().forwardingNTA_Type();
 
   syn String EndpointDefinition.senderName() = getEndpointTarget().senderName();
 
@@ -348,6 +438,24 @@ aspect MustacheSendDefinition {
   syn String EndpointDefinition.writeMethodName() = toMustache().writeMethodName();
 
   // === attributes needed for computing above ones ===
+  syn boolean EndpointTarget.needForwardingNTA() = false;
+  eq TypeEndpointTarget.needForwardingNTA() = containingEndpointDefinition().getSend() && !getType().getNTA();
+
+  syn String EndpointTarget.forwardingNTA_Name() = null;
+  eq TypeEndpointTarget.forwardingNTA_Name() = ragconnect().internalRagConnectPrefix() + getType().getName();
+
+  syn String EndpointTarget.forwardingNTA_Type() = null;
+  eq TypeEndpointTarget.forwardingNTA_Type() = getType().forwardingNTA_Type(
+          containingEndpointDefinition().getIndexBasedListAccess());
+
+  syn String TypeComponent.forwardingNTA_Type(boolean indexBasedListAccess);
+  eq NormalComponent.forwardingNTA_Type(boolean indexBasedListAccess) = getTypeDecl().getName();
+  eq OptComponent.forwardingNTA_Type(boolean indexBasedListAccess) =
+          ragconnect().configJastAddOpt() + "<" + getTypeDecl().getName() + ">";
+  eq ListComponent.forwardingNTA_Type(boolean indexBasedListAccess) = indexBasedListAccess ?
+          getTypeDecl().getName() :
+          ragconnect().configJastAddList() + "<" + getTypeDecl().getName() + ">";
+
   syn String EndpointTarget.senderName();
   eq TokenEndpointTarget.senderName() = ragconnect().internalRagConnectPrefix() + "_sender_" + getToken().getName();
   eq TypeEndpointTarget.senderName() = ragconnect().internalRagConnectPrefix() + "_sender_" + getType().getName();
@@ -426,6 +534,14 @@ aspect MustacheTokenComponent {
 aspect MustacheTypeDecl {
   // === TypeComponent ===
   syn String TypeComponent.parentTypeName() = containingTypeDecl().getName();
+  syn String TypeComponent.disconnectMethodName() {
+    List<TypeEndpointTarget> typeEndpointTargets = getTypeEndpointTargets();
+    if (typeEndpointTargets.isEmpty()) {
+      return "MISSING_ENDPOINT";
+    } else {
+      return typeEndpointTargets.get(0).containingEndpointDefinition().disconnectMethodName();
+    }
+  }
 
   // === TypeDecl ===
   syn List<TypeComponent> TypeDecl.occurencesInProductionRules() {
@@ -443,7 +559,8 @@ aspect MustacheTypeDecl {
 }
 
 aspect AttributesForMustache {
-  syn String MEndpointDefinition.lastValue() = getEndpointDefinition().lastValue();
+  syn String MEndpointDefinition.lastValueGetterCall() = getEndpointDefinition().lastValueGetterCall();
+  syn String MEndpointDefinition.lastValueSetter() = getEndpointDefinition().lastValueSetter();
 
   // token and type are potentially dangerous because asXEndpointTarget can return null
   syn TokenComponent EndpointDefinition.token() = getEndpointTarget().asTokenEndpointTarget().getToken();
@@ -473,8 +590,8 @@ aspect AttributesForMustache {
     return isSend ? new MContextFreeTypeSendDefinition() : new MContextFreeTypeReceiveDefinition();
   }
   MEndpointDefinition UntypedEndpointTarget.createMEndpointDefinition(boolean isSend) {
-    throw new RuntimeException("Untyped endpoint target type, typeName= " +
-        getTypeName() + ", childName=" + getChildName());
+    throw new RuntimeException("Could not resolve endpoint target '" +
+        getTypeName() + "." + getChildName() + "'");
   }
 }
 

ragconnect.base/src/main/jastadd/IntermediateToYAML.jrag

@@ -100,7 +100,8 @@ aspect IntermediateToYAML {
     result.put("parentTypeName" , parentTypeName());
 
     if (getSend()) {
-      result.put("lastValue" , lastValue());
+      result.put("lastValueGetterCall" , lastValueGetterCall());
+      result.put("lastValueSetter" , lastValueSetter());
       result.put("senderName" , senderName());
       result.put("shouldSendValue" , shouldSendValue());
       result.put("tokenResetMethodName" , tokenResetMethodName());

ragconnect.base/src/main/jastadd/Mappings.jrag

@@ -236,12 +236,8 @@ aspect Mappings {
 
   // --- suitableSendDefaultMapping ---
   syn DefaultMappingDefinition EndpointDefinition.suitableSendDefaultMapping() {
-    if (getEndpointTarget().isTypeEndpointTarget()) {
-      try {
-        TypeDecl typeDecl = program().resolveTypeDecl(targetTypeName());
-        return typeIsList() && !getIndexBasedListAccess() ? ragconnect().defaultListTreeToBytesMapping() : ragconnect().defaultTreeToBytesMapping(typeDecl.getName());
-      } catch (Exception ignore) {
-      }
+    if (getEndpointTarget().isTypeEndpointTarget() && typeIsList() && !getIndexBasedListAccess()) {
+      return ragconnect().defaultListTreeToBytesMapping();
     }
     switch (targetTypeName()) {
       case "boolean":
@@ -270,9 +266,9 @@ aspect Mappings {
       default:
         try {
           TypeDecl typeDecl = program().resolveTypeDecl(targetTypeName());
-          // TODO: also support list-types, if list is last type
           return ragconnect().defaultTreeToBytesMapping(typeDecl.getName());
         } catch (Exception ignore) {
+          // exception should be logged to debug/fine
         }
         System.err.println("Could not find suitable default mapping for " + targetTypeName() + " on " + this);
         return null;

ragconnect.base/src/main/jastadd/NameResolution.jrag

@@ -13,15 +13,22 @@ aspect RagConnectNameResolution {
     return result;
   }
 
-  // --- lookupTypeEndpointDefinition ---
-  inh java.util.List<EndpointDefinition> EndpointDefinition.lookupTypeEndpointDefinitions(TypeComponent type);
-  inh java.util.List<EndpointDefinition> EndpointTarget.lookupTypeEndpointDefinitions(TypeComponent type);
-  eq RagConnect.getConnectSpecificationFile().lookupTypeEndpointDefinitions(TypeComponent type) = lookupTypeEndpointDefinitions(type);
-  syn java.util.List<EndpointDefinition> RagConnect.lookupTypeEndpointDefinitions(TypeComponent type) {
+  // --- lookupGivenTypeEndpointDefinition ---
+  inh java.util.List<EndpointDefinition> EndpointDefinition.lookupGivenTypeEndpointDefinitions(TypeComponent type);
+  inh java.util.List<EndpointDefinition> EndpointTarget.lookupGivenTypeEndpointDefinitions(TypeComponent type);
+  eq RagConnect.getConnectSpecificationFile().lookupGivenTypeEndpointDefinitions(TypeComponent type) = lookupTypeEndpointDefinitions(type, true);
+
+  // --- lookupAllTypeEndpointDefinition ---
+  inh java.util.List<EndpointDefinition> EndpointDefinition.lookupAllTypeEndpointDefinitions(TypeComponent type);
+  inh java.util.List<EndpointDefinition> EndpointTarget.lookupAllTypeEndpointDefinitions(TypeComponent type);
+  eq RagConnect.getConnectSpecificationFile().lookupAllTypeEndpointDefinitions(TypeComponent type) = lookupTypeEndpointDefinitions(type, false);
+
+  syn java.util.List<EndpointDefinition> RagConnect.lookupTypeEndpointDefinitions(TypeComponent type, boolean onlyGiven) {
     java.util.List<EndpointDefinition> result = new java.util.ArrayList<>();
-    for (EndpointTarget target : givenEndpointTargetList()) {
+    for (EndpointDefinition def : onlyGiven ? givenEndpointDefinitionList() : allEndpointDefinitionList()) {
+      EndpointTarget target = def.getEndpointTarget();
       if (target.isTypeEndpointTarget() && target.asTypeEndpointTarget().getType().equals(type)) {
-        result.add(target.containingEndpointDefinition());
+        result.add(def);
       }
     }
     return result;

ragconnect.base/src/main/jastadd/Printing.jrag

 aspect Printing {
   syn String MappingDefinitionType.prettyPrint();
-  eq JavaMappingDefinitionType.prettyPrint() = getType().getName();
-  eq JavaArrayMappingDefinitionType.prettyPrint() = getType().getName() + "[]";
+  eq JavaMappingDefinitionType.prettyPrint() = getType().prettyPrint();
+  eq JavaArrayMappingDefinitionType.prettyPrint() = getType().prettyPrint() + "[]";
 
   syn String JavaTypeUse.prettyPrint() {
     StringBuilder sb = new StringBuilder();

ragconnect.base/src/main/jastadd/RagConnect.relast

@@ -12,7 +12,7 @@ rel TokenEndpointTarget.Token <-> TokenComponent.TokenEndpointTarget*;
 TypeEndpointTarget : EndpointTarget;
 rel TypeEndpointTarget.Type <-> TypeComponent.TypeEndpointTarget*;
 ContextFreeTypeEndpointTarget : EndpointTarget;
-rel ContextFreeTypeEndpointTarget.TypeDecl <-> TypeDecl.ContextFreeTypeEndpointTarget?;
+rel ContextFreeTypeEndpointTarget.TypeDecl <-> TypeDecl.ContextFreeTypeEndpointTarget*;
 UntypedEndpointTarget : EndpointTarget ::= <TypeName> <ChildName>;  // only used by parser
 // to be integrated:
 //AttributeEndpointTarget : EndpointTarget ::= <Name> ;
@@ -36,6 +36,7 @@ Configuration ::=
 <LoggingEnabledForWrites:boolean>
 <LoggingEnabledForIncremental:boolean>
 <JastAddList:String>
+<JastAddOpt:String>
 <IncrementalOptionActive:boolean>
 <ExperimentalJastAdd329:boolean>;
 rel Configuration.RootNode -> TypeDecl ;

ragconnect.base/src/main/jastadd/parser/ParserRewrites.jrag

@@ -39,4 +39,5 @@ aspect ParserRewrites {
   eq UntypedEndpointTarget.isAlreadyDefined() = false;
   eq UntypedEndpointTarget.entityIsNormalAttribute() = false;
   eq UntypedEndpointTarget.targetTypeName() = "<untyped.targetTypeName>";
+  eq UntypedEndpointTarget.isTypeEndpointTarget() = false;
 }

ragconnect.base/src/main/jastadd/parser/RagConnect.parser

@@ -54,6 +54,7 @@ EndpointDefinition endpoint_definition
 
 EndpointDefinition endpoint_definition_type
   = SEND endpoint_target.t                        {: return createEndpointDefinition(t, true,  false, false); :}
+  | SEND INDEXED endpoint_target.t                {: return createEndpointDefinition(t, true,  true,  false); :}
   | RECEIVE endpoint_target.t                     {: return createEndpointDefinition(t, false, false, false); :}
   | RECEIVE INDEXED endpoint_target.t             {: return createEndpointDefinition(t, false, true,  false); :}
   | RECEIVE WITH ADD endpoint_target.t            {: return createEndpointDefinition(t, false, false, true ); :}

ragconnect.base/src/main/java/org/jastadd/ragconnect/compiler/Compiler.java

@@ -41,13 +41,22 @@ public class Compiler extends AbstractCompiler {
 
   @Override
   protected int compile() throws CompilerException {
+    compile0();
+    return 0;
+  }
+
+  /**
+   * Compiles with given options. Either successful, or throws exception upon failure.
+   * @throws CompilerException if something went wrong
+   */
+  private void compile0() throws CompilerException {
     if (getConfiguration().shouldPrintVersion()) {
       System.out.println(readVersion());
-      return 0;
+      return;
     }
     if (getConfiguration().shouldPrintHelp()) {
       getConfiguration().printHelp(System.out);
-      return 0;
+      return;
     }
     if (optionVerbose.value()) {
       LOGGER.setLevel(Level.FINE);
@@ -64,11 +73,15 @@ public class Compiler extends AbstractCompiler {
     }
 
     if (!optionRootNode.isMatched()) {
-      return error("Root node not specified");
+      throw new CompilerException("Root node not specified");
     }
 
     RagConnect ragConnect = parseProgram(getConfiguration().getFiles());
+    try {
       setConfiguration(ragConnect);
+    } catch (RuntimeException re) {
+      throw new CompilerException("Failed to parse all files", re);
+    }
 
     if (!ragConnect.errors().isEmpty()) {
       StringBuilder sb = new StringBuilder("Errors:\n");
@@ -83,7 +96,7 @@ public class Compiler extends AbstractCompiler {
       String yamlContent = ragConnect.toYAML().prettyPrint();
       LOGGER.fine(yamlContent);
       writeToFile(getConfiguration().outputDir().toPath().resolve("RagConnect.yml"), yamlContent);
-      return 0;
+      return;
     }
 
     LOGGER.fine("Writing output files");
@@ -109,9 +122,13 @@ public class Compiler extends AbstractCompiler {
       Path outputFile = getConfiguration().outputDir().toPath().resolve(grammarFile.getFileName());
       writeToFile(outputFile, grammarFile.generateAbstractGrammar());
     }
-    writeToFile(getConfiguration().outputDir().toPath().resolve("RagConnect.jadd"),
-        generateAspect(ragConnect));
-    return 0;
+    String aspectCode;
+    try {
+      aspectCode = generateAspect(ragConnect);
+    } catch (RuntimeException re) {
+      throw new CompilerException("Could not generate RagConnect aspect", re);
+    }
+    writeToFile(getConfiguration().outputDir().toPath().resolve("RagConnect.jadd"), aspectCode);
   }
 
   public static void main(String[] args) {
@@ -272,6 +289,10 @@ public class Compiler extends AbstractCompiler {
     return new File(filename).getName();
   }
 
+  /**
+   * Set all configuration values.
+   * @param ragConnect the RagConnect instance to set configuration values
+   */
   private void setConfiguration(RagConnect ragConnect) {
     ragConnect.setConfiguration(new Configuration());
     ragConnect.getConfiguration().setLoggingEnabledForReads(optionLogReads.value());
@@ -284,10 +305,18 @@ public class Compiler extends AbstractCompiler {
     ragConnect.getConfiguration().setIncrementalOptionActive(incrementalOptionActive);
     LOGGER.fine(() -> "ragConnect.getConfiguration().IncrementalOptionActive = " + incrementalOptionActive);
 
-    // reuse "--List" option of JastAdd
+    // reuse "--List" and "--Opt" options of JastAdd
     ragConnect.getConfiguration().setJastAddList(this.getConfiguration().listType());
+    ragConnect.getConfiguration().setJastAddOpt(this.getConfiguration().optType());
 
-    ragConnect.getConfiguration().setRootNode(ragConnect.getProgram().resolveTypeDecl(optionRootNode.value()));
+    final TypeDecl rootNode;
+    try {
+      rootNode = ragConnect.getProgram().resolveTypeDecl(optionRootNode.value());
+    } catch (RuntimeException re) {
+      // root node was not found
+      throw new RuntimeException("Could not resolve root node '" + optionRootNode.value() + "'!", re);
+    }
+    ragConnect.getConfiguration().setRootNode(rootNode);
 
     // Handler ::= <ClassName> <UniqueName> <InUse:boolean>;
     ragConnect.addHandler(new Handler("MqttHandler.jadd", "MqttServerHandler", "mqtt", optionProtocols.hasValue(OPTION_PROTOCOL_MQTT)));
@@ -317,4 +346,11 @@ public class Compiler extends AbstractCompiler {
     m.execute(new java.io.PrintWriter(new org.jastadd.ragconnect.compiler.AppendableWriter(sb)), ragConnect);
     return sb.toString();
   }
+
+  @Override
+  protected int error(String message) {
+    LOGGER.log(Level.SEVERE, message);
+    return 1;
+  }
+
 }

ragconnect.base/src/main/resources/handler.mustache

@@ -13,6 +13,10 @@ aspect RagConnectHandler {
     {{#Handlers}}
     {{#InUse}}{{fieldName}}.close();{{/InUse}}
     {{/Handlers}}
+    {{#configIncrementalOptionActive}}
+    trace().setReceiver({{observerInstanceSingletonMethodName}}().oldReceiver);
+    {{observerInstanceResetMethodName}}();
+    {{/configIncrementalOptionActive}}
   }
 
 {{#mqttHandler}}
@@ -62,7 +66,7 @@ aspect RagConnectHandler {
   class RagConnectPublisher {
     java.util.List<Runnable> senders = new java.util.ArrayList<>();
     java.util.Map<RagConnectToken, Runnable> tokenToSender;
-    byte[] lastValue;
+    private byte[] lastValue;
 
     void add(Runnable sender, RagConnectToken connectToken) {
       if (tokenToSender == null) {
@@ -73,24 +77,89 @@ aspect RagConnectHandler {
     }
 
     boolean remove(RagConnectToken token) {
-      if (tokenToSender == null) {
-        System.err.println("Removing sender before first addition for " + token.entityName + " at " + token.uri);
+      String errorMessage = internal_remove(token);
+      if (errorMessage == null) {
+        return true;
+      } else {
+        System.err.println(errorMessage);
         return false;
       }
+    }
+
+    /**
+    * (internal) Removes the token, returning an error message if there is one.
+    * @param token the token to be removed
+    * @return an error message (upon error), or null (upon success)
+    */
+    String internal_remove(RagConnectToken token) {
+      if (tokenToSender == null) {
+        return "Removing sender before first addition for " + token.entityName + " at " + token.uri;
+      }
       Runnable sender = tokenToSender.remove(token);
       if (sender == null) {
-        System.err.println("Could not find connected sender for " + token.entityName + " at " + token.uri);
-        return false;
+        return "Could not find connected sender for " + token.entityName + " at " + token.uri;
       }
       boolean success = senders.remove(sender);
       if (senders.isEmpty()) {
         lastValue = null;
       }
-      return success;
+      return success ? null : "Could not remove sender for " + token.entityName + " at " + token.uri;
     }
 
     void run() {
       senders.forEach(Runnable::run);
     }
+
+    byte[] getLastValue() {
+      return lastValue;
+    }
+
+    void setLastValue(byte[] value) {
+      this.lastValue = value;
+    }
+  }
+
+  class RagConnectMappingPublisher {
+    java.util.Map<Integer, RagConnectPublisher> publishers = new java.util.HashMap<>();
+
+    void add(Runnable sender, int index, RagConnectToken connectToken) {
+      publishers.computeIfAbsent(index, ignoredIndex -> new RagConnectPublisher()).add(sender, connectToken);
+    }
+
+    boolean remove(RagConnectToken token) {
+      // publishers.forEach((index, publisher) -> publisher.remove(token));
+      // remove token from each publisher, at least one has to successfully remove the token to make this call a success
+      boolean result = false;
+      java.util.List<String> errorMessages = new java.util.ArrayList<>();
+      for (RagConnectPublisher publisher : publishers.values()) {
+        String errorMessage = publisher.internal_remove(token);
+        if (errorMessage == null) {
+          result = true;
+        } else {
+          errorMessages.add(errorMessage);
+        }
+      }
+      if (!result) {
+        // only print error message, if all publishers failed to remove the token
+        errorMessages.stream().forEachOrdered(System.err::println);
+      }
+      return result;
+    }
+
+    void run(int index) {
+      java.util.Optional.ofNullable(publishers.get(index)).ifPresent(RagConnectPublisher::run);
+    }
+
+    byte[] getLastValue(int index) {
+      RagConnectPublisher publisher = publishers.get(index);
+      if (publisher == null) {
+        return null;
+      }
+      return publisher.getLastValue();
+    }
+
+    void setLastValue(int index, final byte[] value) {
+      java.util.Optional.ofNullable(publishers.get(index)).ifPresent(publisher -> publisher.setLastValue(value));
+    }
   }
 }

ragconnect.base/src/main/resources/ragconnect.mustache

@@ -32,6 +32,39 @@ aspect RagConnect {
   {{> typeDecl}}
   {{/typeDeclsOfContextFreeEndpointTargets}}
 
+  {{! --- touchedTerminals ---}}
+  {{#allTypeDecls}}
+    {{Name}} {{Name}}.{{touchedTerminalsMethodName}}() {
+    {{#tokenComponents}}
+      get{{Name}}();
+    {{/tokenComponents}}
+    {{#normalComponents}}
+      get{{Name}}().{{touchedTerminalsMethodName}}();
+    {{/normalComponents}}
+    {{#listComponents}}
+      for ({{#TypeDecl}}{{Name}}{{/TypeDecl}} element : get{{Name}}List()) {
+        element.{{touchedTerminalsMethodName}}();
+      }
+    {{/listComponents}}
+    return this;
+  }
+  {{/allTypeDecls}}
+  ASTNode ASTNode.{{touchedTerminalsMethodName}}() {
+    return this;
+  }
+  {{configJastAddList}}<T> {{configJastAddList}}.{{touchedTerminalsMethodName}}() {
+    for (T child : this) {
+      child.{{touchedTerminalsMethodName}}();
+    }
+    return this;
+  }
+  {{configJastAddOpt}}<T> {{configJastAddOpt}}.{{touchedTerminalsMethodName}}() {
+    if (getChild(0) != null) {
+      getChild(0).{{touchedTerminalsMethodName}}();
+    }
+    return this;
+  }
+
   {{> ListAspect}}
 
   public void {{rootNodeName}}.ragconnectCheckIncremental() {
@@ -57,12 +90,17 @@ aspect RagConnectObserver {
       final RagConnectToken connectToken;
       final ASTNode node;
       final String attributeString;
+      final boolean compareParams;
+      final Object params;
       final Runnable attributeCall;
 
-      RagConnectObserverEntry(RagConnectToken connectToken, ASTNode node, String attributeString, Runnable attributeCall) {
+      RagConnectObserverEntry(RagConnectToken connectToken, ASTNode node, String attributeString,
+                              boolean compareParams, Object params, Runnable attributeCall) {
         this.connectToken = connectToken;
         this.node = node;
         this.attributeString = attributeString;
+        this.compareParams = compareParams;
+        this.params = params;
         this.attributeCall = attributeCall;
       }
     }
@@ -96,11 +134,21 @@ aspect RagConnectObserver {
     }
 
     void add(RagConnectToken connectToken, ASTNode node, String attributeString, Runnable attributeCall) {
+      internal_add(connectToken, node, attributeString, false, null, attributeCall);
+    }
+    void add(RagConnectToken connectToken, ASTNode node, String attributeString, Object params, Runnable attributeCall) {
+      internal_add(connectToken, node, attributeString, true, params, attributeCall);
+    }
+
+    private void internal_add(RagConnectToken connectToken, ASTNode node, String attributeString,
+        boolean compareParams, Object params, Runnable attributeCall) {
       {{#configLoggingEnabledForIncremental}}
-      System.out.println("** observer add: " + node + " on " + attributeString);
+      System.out.println("** observer add: " + node + " on " + attributeString + (compareParams ? " (parameterized)" : ""));
       {{/configLoggingEnabledForIncremental}}
-      observedNodes.add(new RagConnectObserverEntry(connectToken, node, attributeString, attributeCall));
+      observedNodes.add(new RagConnectObserverEntry(connectToken, node, attributeString,
+                                                    compareParams, params, attributeCall));
     }
+
     void remove(RagConnectToken connectToken) {
       observedNodes.removeIf(entry -> entry.connectToken.equals(connectToken));
     }
@@ -127,7 +175,7 @@ aspect RagConnectObserver {
         entryQueue.clear();
         startEntry = null;
         {{#configLoggingEnabledForIncremental}}
-        System.out.println("** observer process (" + entriesToProcess.length + "): " + node + " on " + attribute);
+        System.out.println("** observer process (entries: " + entriesToProcess.length + "): " + node + " on " + attribute);
         {{/configLoggingEnabledForIncremental}}
         for (RagConnectObserverEntry entry : entriesToProcess) {
           entry.attributeCall.run();
@@ -146,7 +194,7 @@ aspect RagConnectObserver {
       {{/configLoggingEnabledForIncremental}}
       // iterate through list, if matching pair. could maybe be more efficient.
       for (RagConnectObserverEntry entry : observedNodes) {
-        if (entry.node.equals(node) && entry.attributeString.equals(attribute)) {
+        if (entry.node.equals(node) && entry.attributeString.equals(attribute) && (!entry.compareParams || java.util.Objects.equals(entry.params, params))) {
           // hit. call the attribute/nta-token
           {{#configLoggingEnabledForIncremental}}
           System.out.println("** observer hit: " + entry.node + " on " + entry.attributeString);
@@ -162,13 +210,16 @@ aspect RagConnectObserver {
     }
   }
 
-  private static RagConnectObserver ASTNode.{{internalRagConnectPrefix}}ObserverInstance;
-  RagConnectObserver ASTNode.{{internalRagConnectPrefix}}Observer() {
-    if ({{internalRagConnectPrefix}}ObserverInstance == null) {
+  private static RagConnectObserver ASTNode.{{observerInstanceFieldName}};
+  RagConnectObserver ASTNode.{{observerInstanceSingletonMethodName}}() {
+    if ({{observerInstanceFieldName}} == null) {
       // does not matter, which node is used to create the observer as ASTState/tracing is also static
-    {{internalRagConnectPrefix}}ObserverInstance = new RagConnectObserver(this);
+      {{observerInstanceFieldName}} = new RagConnectObserver(this);
+    }
+    return {{observerInstanceFieldName}};
   }
-    return {{internalRagConnectPrefix}}ObserverInstance;
+  void ASTNode.{{observerInstanceResetMethodName}}() {
+    {{observerInstanceFieldName}} = null;
   }
 }
 {{/configIncrementalOptionActive}}

ragconnect.base/src/main/resources/receiveDefinition.mustache

@@ -34,7 +34,7 @@ public boolean {{parentTypeName}}.{{connectMethodName}}(String {{connectParamete
  {{#typeIsList}}
   {{^IndexBasedListAccess}}
    {{#WithAdd}}
-    {{getterMethodName}}().addAll({{lastResult}});
+    {{getterMethodCall}}.addAll({{lastResult}});
    {{/WithAdd}}
    {{^WithAdd}}
     set{{entityName}}({{lastResult}});
@@ -43,7 +43,7 @@ public boolean {{parentTypeName}}.{{connectMethodName}}(String {{connectParamete
   {{#IndexBasedListAccess}}
     {{lastResult}}.set{{idTokenName}}(topic);
    {{#WithAdd}}
-    {{getterMethodName}}().add({{lastResult}});
+    {{getterMethodCall}}.add({{lastResult}});
    {{/WithAdd}}
    {{^WithAdd}}
     set{{entityName}}({{lastResult}}, index);
@@ -70,16 +70,16 @@ public boolean {{parentTypeName}}.{{connectMethodName}}(String {{connectParamete
 */
 public boolean {{parentTypeName}}.{{connectMethodName}}(String {{connectParameterName}}) throws java.io.IOException {
   java.util.function.BiConsumer<String, byte[]> consumer = (topic, message) -> {
+    int index = {{resolveInListMethodName}}(topic);
     {{> mappingApplication}}
 {{#configLoggingEnabledForReads}}
     System.out.println("[Receive] " + {{connectParameterName}} + " (" + topic + ") -> {{entityName}} = " + {{lastResult}});
 {{/configLoggingEnabledForReads}}
     {{lastResult}}.set{{idTokenName}}(topic);
-    int resolvedIndex = {{resolveInListMethodName}}(topic);
-    if (resolvedIndex == -1) {
+    if (index == -1) {
       add{{entityName}}({{lastResult}});
     } else {
-      set{{entityName}}({{lastResult}}, resolvedIndex);
+      set{{entityName}}({{lastResult}}, index);
     }
   };
   return {{internalConnectMethodName}}({{connectParameterName}}, consumer);

ragconnect.base/src/main/resources/sendDefinition.mustache

-private RagConnectPublisher {{parentTypeName}}.{{senderName}} = new RagConnectPublisher();
+private RagConnect{{#IndexBasedListAccess}}Mapping{{/IndexBasedListAccess}}Publisher {{parentTypeName}}.{{senderName}} = new RagConnect{{#IndexBasedListAccess}}Mapping{{/IndexBasedListAccess}}Publisher();
 
-public boolean {{parentTypeName}}.{{connectMethodName}}(String {{connectParameterName}}, boolean writeCurrentValue) throws java.io.IOException {
+public boolean {{parentTypeName}}.{{connectMethodName}}(String {{connectParameterName}}{{#IndexBasedListAccess}}, int index{{/IndexBasedListAccess}}, boolean writeCurrentValue) throws java.io.IOException {
   {{>handleUri}}
   RagConnectToken connectToken = new RagConnectToken(uri, "{{entityName}}");
   boolean success;
@@ -12,13 +12,13 @@ public boolean {{parentTypeName}}.{{connectMethodName}}(String {{connectParamete
       final String topic = {{attributeName}}().extractTopic(uri);
       {{senderName}}.add(() -> {
         {{#configLoggingEnabledForWrites}}
-        System.out.println("[Send] {{entityName}} = " + {{getterMethodName}}() + " -> " + {{connectParameterName}});
+        System.out.println("[Send] {{entityName}} = " + {{getterMethodCall}} + " -> " + {{connectParameterName}});
         {{/configLoggingEnabledForWrites}}
-        handler.publish(topic, {{lastValue}});
-        }, connectToken);
-      {{updateMethodName}}();
+        handler.publish(topic, {{lastValueGetterCall}});
+        }{{#IndexBasedListAccess}}, index{{/IndexBasedListAccess}}, connectToken);
+      {{updateMethodName}}({{#IndexBasedListAccess}}index{{/IndexBasedListAccess}});
       if (writeCurrentValue) {
-        {{writeMethodName}}();
+        {{writeMethodName}}({{#IndexBasedListAccess}}index{{/IndexBasedListAccess}});
       }
       success = true;
       break;
@@ -28,8 +28,8 @@ public boolean {{parentTypeName}}.{{connectMethodName}}(String {{connectParamete
   {{#InUse}}
     case "rest":
       success = {{attributeName}}().newGETConnection(connectToken, () -> {
-        {{updateMethodName}}();
-        return new String({{lastValue}});
+        {{updateMethodName}}({{#IndexBasedListAccess}}index{{/IndexBasedListAccess}});
+        return new String({{lastValueGetterCall}});
       });
       break;
   {{/InUse}}
@@ -41,16 +41,24 @@ public boolean {{parentTypeName}}.{{connectMethodName}}(String {{connectParamete
   if (success) {
     connectTokenMap.add(this, false, connectToken);
     {{#configIncrementalOptionActive}}
-    {{internalRagConnectPrefix}}Observer().add(connectToken, this, "{{getterMethodName}}", () -> {
-      if (this.{{updateMethodName}}()) {
-        this.{{writeMethodName}}();
+        {{!todo maybe getterMethodName needs to be change for indexed send}}
+      {{observerInstanceSingletonMethodName}}().add(
+      connectToken,
+      this,
+      "{{getterMethodName}}{{#IndexBasedListAccess}}_int{{/IndexBasedListAccess}}",
+      {{#IndexBasedListAccess}}index,{{/IndexBasedListAccess}}
+      () -> {
+        if (this.{{updateMethodName}}({{#IndexBasedListAccess}}index{{/IndexBasedListAccess}})) {
+          this.{{writeMethodName}}({{#IndexBasedListAccess}}index{{/IndexBasedListAccess}});
         }
-    });
+      }
+    );
     {{/configIncrementalOptionActive}}
   }
   return success;
 }
 
+{{!todo check if index parameter is needed here for typeIsList and indexBasedListAccess}}
 public boolean {{parentTypeName}}.{{disconnectMethodName}}(String {{connectParameterName}}) throws java.io.IOException {
   {{>handleUri}}
   java.util.List<RagConnectToken> connectTokens = connectTokenMap.removeAll(this, false, uri, "{{entityName}}");
@@ -59,7 +67,7 @@ public boolean {{parentTypeName}}.{{disconnectMethodName}}(String {{connectParam
     return false;
   }
   {{#configIncrementalOptionActive}}
-  connectTokens.forEach(token -> {{internalRagConnectPrefix}}Observer().remove(token));
+  connectTokens.forEach(token -> {{observerInstanceSingletonMethodName}}().remove(token));
   {{/configIncrementalOptionActive}}
   RagConnectDisconnectHandlerMethod disconnectingMethod;
   switch (scheme) {
@@ -88,16 +96,20 @@ public boolean {{parentTypeName}}.{{disconnectMethodName}}(String {{connectParam
   return success;
 }
 
-protected boolean {{parentTypeName}}.{{updateMethodName}}() {
+protected boolean {{parentTypeName}}.{{updateMethodName}}({{#IndexBasedListAccess}}int index{{/IndexBasedListAccess}}) {
   {{^shouldSendValue}}
   {{tokenResetMethodName}}();
   {{/shouldSendValue}}
   {{> mappingApplication}}
-  {{lastValue}} = {{lastResult}};
+  {{lastValueSetter}}({{#IndexBasedListAccess}}index, {{/IndexBasedListAccess}}{{lastResult}});
   // normally we would return true here. unless no connect method was called so far to initialize {{senderName}} yet
   return {{senderName}} != null;
 }
 
-protected void {{parentTypeName}}.{{writeMethodName}}() {
-  {{senderName}}.run();
+protected void {{parentTypeName}}.{{writeMethodName}}({{#IndexBasedListAccess}}int index{{/IndexBasedListAccess}}) {
+  {{senderName}}.run({{#IndexBasedListAccess}}index{{/IndexBasedListAccess}});
 }
+
+{{#needForwardingNTA}}
+syn {{{forwardingNTA_Type}}} {{parentTypeName}}.{{forwardingNTA_Name}}({{#IndexBasedListAccess}}int index{{/IndexBasedListAccess}}) = {{realGetterMethodCall}}.{{touchedTerminalsMethodName}}();
+{{/needForwardingNTA}}