<p>To transform the grammar extension we provide a preprocessor for JastAdd This preprocessor including its source code is provided in the <code>preprocessor</code> subdirectory.</p>
<p>Its usage is: - Running the preprocessor <code>java -jar relast-compiler.jar</code> Run preprocessor on train benchmark (output written to standard output):</p>
<h3id="structure-of-the-train-benchmark">Structure of the Train Benchmark</h3>
<h3id="structure-of-the-train-benchmark">Structure of the Train Benchmark</h3>
<p>The benchmark is able to measure different scenarios specified by configurations with several kinds of parameters:</p>
<p>The benchmark is able to measure different scenarios specified by configurations with several kinds of parameters:</p>
...
@@ -97,4 +109,60 @@
...
@@ -97,4 +109,60 @@
<p><strong>Please Note: Reproducing the graphs as presented in the paper and supplied here takes a very long time depending on the utilized hardware. It is strongly suggested to run the benchmark with a smaller maximum problem size, less repetitions, and a shorter timeout.</strong> Most results of the benchmark are observable with more restricted setup as well. In the following, we will provide a suggested way to run the benchmark in different sizes.</p>
<p><strong>Please Note: Reproducing the graphs as presented in the paper and supplied here takes a very long time depending on the utilized hardware. It is strongly suggested to run the benchmark with a smaller maximum problem size, less repetitions, and a shorter timeout.</strong> Most results of the benchmark are observable with more restricted setup as well. In the following, we will provide a suggested way to run the benchmark in different sizes.</p>
<p>To reproduce the measurements, there are several options. We provide a prepared Docker image that can be run directly. Alternatively, it is, on course, also possible to simply run the provided gradle build scripts. However, since there are some software requirements imposed by the benchmark, particularly for creating the diagrams using R. We stronly suggest running the Docker variant.</p>
<p>To reproduce the measurements, there are several options. We provide a prepared Docker image that can be run directly. Alternatively, it is, on course, also possible to simply run the provided gradle build scripts. However, since there are some software requirements imposed by the benchmark, particularly for creating the diagrams using R. We stronly suggest running the Docker variant.</p>
<h4id="running-the-benchmark-with-docker">Running the Benchmark with Docker</h4>
<h4id="running-the-benchmark-with-docker">Running the Benchmark with Docker</h4>
<h5id="loading-the-docker-image">Loading the Docker Image</h5>
<ul>
<li>Variant 1 (recommended): Load the provided docker image
<ul>
<li>Prerequisites: An installation of Docker in the <code>PATH</code></li>
<li>Steps:
<ul>
<li>Unpack the provided archive and open a terminal in the extracted directory</li>
<li><code>docker run -it trainbenchmark</code></li>
</ul></li>
</ul></li>
</ul>
<h5id="running-the-docker-image">Running the Docker Image</h5>
<ul>
<li><code>docker run -it trainbenchmark</code> To make the results available outside the container, run <code>docker run -it -v "$PWD"/docker-results:/trainbenchmark/results:Z -v "$PWD"/docker-diagrams:/trainbenchmark/diagrams:Z trainbenchmark</code></li>
<li>A command prompt is opened and some information is displayed</li>
<li>Follow the instructions below</li>
</ul>
<h4id="running-the-benchmark-directly">Running the Benchmark directly</h4>
<h4id="running-the-benchmark-directly">Running the Benchmark directly</h4>
<ul>
<li>For running a standard run,
<ul>
<li><code>./run_small</code></li>
<li><code>./run_medium</code></li>
<li><code>./run_full</code></li>
</ul></li>
<li>For running a custom run,
<ul>
<li>run <code>./gradlew preprocess</code> to generate the</li>
<li>configure the scripts either by running <code>./scripts/configure.sh 1 <MAXSIZE><TIMEOUT in s><REPETITIONS></code> Where MAXSIZE is one of 2,4,8,16,32,64,128,256,512,1024. The larger sizes use <strong>a lot of</strong> disk space!</li>
<li><em>Alternatively</em>, run ``</li>
<li>run <code>./gradlew initScripts</code></li>
<li>run <code>./gradlew generate</code></li>
<li>run the benchmark
<ul>
<li>run <code>./gradlew individualInjectBenchmark</code> for the <em>inject</em> scenarios</li>
<li>run <code>./gradlew individualRepairBenchmark</code> for the <em>repair</em> scenarios</li>
</ul></li>
<li>Plot the diagrams for the current run: <code>./gradlew plotIndividual</code></li>
</ul></li>
<li>The resulting data are in the <code>results</code> and the <code>diagrams</code> folder
<ul>
<li>When running with docker, the data are also in <code>docker-results</code> and <code>docker-diagrams</code> on the host machine.</li>