diff --git a/bibliography.bib b/bibliography.bib index 7d2190dba17be94afad29b6d3e8f450660b6f3f0..f7dd294535f377937ab515beb258aad36f878f26 100644 --- a/bibliography.bib +++ b/bibliography.bib @@ -105,4 +105,13 @@ isbn = {978-81-203-5068-7} pages={166--200}, year={2009}, organization={Springer} -} \ No newline at end of file +} + +@inproceedings{gotz2018jastadd, + title={A JastAdd-and ILP-based Solution to the Software-Selection and Hardware-Mapping-Problem at the TTC 2018.}, + author={G{\"o}tz, Sebastian and Mey, Johannes and Sch{\"o}ne, Ren{\'e} and A{\ss}mann, Uwe}, + booktitle={TTC@ STAF}, + pages={31--36}, + year={2018} +} + diff --git a/sections/benefits.tex b/sections/benefits.tex index ea5b207c698edf8c6e1174ccffebae16feaa3c00..0218de0352f72f89d1435d6c6ac30c55102055c5 100644 --- a/sections/benefits.tex +++ b/sections/benefits.tex @@ -1,5 +1,42 @@ \section{Benefits} \label{sec:benefits} +\begin{description} +\item{\textbf{RQ3 : Which advantages can RAG provide with its features at expressing testing approaches?}} +\end{description} + + + + +% Helpful attributes (Collection, inherited, easier return with =) +During the implementation phase of parameter inference, RAG was beneficial at writing codes. At writing codes for parser and back-transformation, attributes could not be applied, because the functional parts change the structure of AST, which conflicts to the definition of attribute and also in JastAdd. So, it could not be realized that attributes can be practical. But, following attributes and features of JastAdd were helpful to construct the generators and implement them: + +\begin{itemize} +\item As long as a returning value can be called or storable by an AST node, the syntax in RAG is reduced with an equals sign and simpler than in common programming language. Therefore, the source codes are compact. To compare how RAG could be implemented effectively, see the implementation of an attribute in this work in Listing~\ref{lst:attribute-infParam} and how this attribute is defined in common Java description in Listing~\ref{lst:java-infParam}. +\item To iterate all paths available in OpenAPI, an inherited attribute facilitated the writing effort. Paths Objects are child nodes of OpenAPI Object, so, all that should have been done was only two lines of code instead of writing an extra for loop. See Listing~\ref{lst:inherited}. +\item Nodes described as References could refer objects with a collection attribute. Listing~\ref{lst:rago-ast} shows how attributes for such purpose look like. +\end{itemize} + +\begin{lstlisting}[language=JRAG,label={lst:attribute-infParam},caption={Attribute InferredParameter.value()}] +syn String InferredParameter.value() = getParameter().substring( getParameter().indexOf("?") + 1 ); +\end{lstlisting} + +\begin{lstlisting}[language=JRAG,label={lst:java-infParam},caption={InferredParameter.value() in Java}] +Class InferredParameter { +... +constructor; +... + public String value() { + return getParameter().substring( getParameter().indexOf("?") + 1 ); + } +} +\end{lstlisting} +\begin{lstlisting}[language=JRAG,label={lst:inherited},caption={Inherited Attribute in Paths Object}] +inh boolean PathsObject.inferUrl(); +eq OpenAPIObject.getPathsObject(int i).inferUrl(){ + ... +} +\end{lstlisting} + \begin{comment} > To evaulate how well-suited RAGs are for API testing, we revisit the research questions... diff --git a/sections/feasibility.tex b/sections/feasibility.tex index d19c9a17fbc6e348744cc4964648e7e6ea5a8b4c..188adc166e7ff469c8111a04b31f11517f2a01a0 100644 --- a/sections/feasibility.tex +++ b/sections/feasibility.tex @@ -5,64 +5,16 @@ \end{description} The results of current existing approaches are shown in Table 3.1. %\cite{tab:appr-sum} -As we can see, most of current approaches to test OpenAPI are in black-box and use Fuzzing method to generate test cases. There are differences between approaches at test generation phase. Several of them inference parameters or operations or use test model generation. They also have differences at bug types. They all consider 500 status codes as bugs, but, three of them consider 200 status codes in test cases expected errors and 4xx status codes in test cases expected successful responses as bugs. Exact bug reports of approaches which experimented with industrial APIs excluded Property-based Approach formulate that they can be interesting research targets combined with RAG. +As we can see, most of current approaches to test OpenAPI are in black-box and use Fuzzing method to generate test cases. There are differences between approaches at test generation phase. + +% 2 +Several of them inference parameters or operations or use test model generation. They also have differences at bug types. They all consider 500 status codes as bugs, but, three of them consider 200 status codes in test cases expected errors and 4xx status codes in test cases expected successful responses as bugs. Exact bug reports of approaches which experimented with industrial APIs excluded Property-based Approach formulate that they can be interesting research targets combined with RAG. \begin{description} \item{\textbf{RQ2 : Are suggested testing approaches from the literatures also available in RAG?}} \end{description} -To determine an answer of this question, we have constructed a data structure for OpenAPI specification in \Cref{ch:rago} to parse OpenAPI documents and validated the parser with 98 commercial APIs, i.e. the parser taken an OpenAPI document returns the same elements except elements with empty values and sibling elements of references in an OpenAPI document. The input and output documents still semantically are same. In \Cref{ch:stm}, we have also implemented two Fuzzing approaches (Random Testing, Parameter Inference). Parameter Inference is motivated by Specification-based Approach\cite{ed2018automatic} and RESTTESTGEN\cite{viglianisi2020resttestgen}, where seperate properties of responses are collected and used as inputs in a parameter with the same schema of a response. \\ - - -\begin{comment} -> To evaulate how well-suited RAGs are for API testing, we revisit the research questions... - -### 6.1 Feasibility - -RQ1 RQ2 - -- current testing: mostly fuzzing -- different additions: - - inference - - test model -- this can be solved with the RAG tool presented in \cref{chap:rago} -- in \cref{sec:inf} we show how inference can be done -- \cite{ttc-rag-von-ceur} for test model/model connections - - http://ceur-ws.org/Vol-2310/ A JastAdd- and ILP-based Solution to the Software-Selection and Hardware-Mapping-Problem at the TTC 2018 - -The results of Nominal Tester module on the set of case -studies are shown in Table I. As we can see, all the 87 case -studies (first line) have been subject to automated test case -generation and for 62 of them (second line) at least one test -for a nominal execution scenario (status code 2xx) could be -automatically generated. For 20 case studies, the test case -automatically generated by RESTTESTGEN exposed errors -that were not properly handled by the REST API (status -code 5xx). On 66 case studies, invalid response messages -were observed, they are responses inconsistent with their -schema defined in the Swagger. -Table II shows a more detailed perspective, focusing on -the case study operations. Among the total 2,612 operations, for 2,560 of them test cases could be generated by -RESTTESTGEN. The untested operations are due to some -failure of the tool, such as unsupported input parameter -generation (e.g., files to be uploaded). -In particular, automatically generated test cases found a -nominal execution for 625 of them (status code 2xx) and an -ungraceful error for 151 (status code 5xx). -For 435 test cases the status code was 4xx, but they are not -shown in the table, because they were hard to classify with -a black-box access. In fact, they might be graceful errors -due to programming defects, or just rejected requests due to -failures by RESTTESTGEN in generating appropriate inputs. -Test cases with validation errors are still a majority: in -1,733 tests the response did not match the declared schema. -Considering these results, we can formulate the following -answer to RQN : -The Nominal Tester module of RESTTESTGEN is effective -in automatically generating test cases with black box access, because it was able to test 2,560 operations -out of 2,612 operations on real world REST APIs. These -tests exposed 151 faults in the form of not correctly -handled internal errors and 1,733 inconsistent response -messages. +To determine an answer of this question, we have constructed a data structure for OpenAPI specification in \Cref{ch:rago} to parse OpenAPI documents and validated the parser with 98 commercial APIs, i.e. the parser taken an OpenAPI document returns the same elements except elements with empty values and sibling elements of references in an OpenAPI document. The input and output documents still semantically are same. In \Cref{ch:stm}, we have also implemented two Fuzzing approaches (Random Testing, Parameter Inference). Parameter Inference is motivated by Specification-based Approach\cite{ed2018automatic} and RESTTESTGEN\cite{viglianisi2020resttestgen}, where seperate properties of responses are collected and used as inputs in a parameter with the same schema of a response. -\end{comment} \ No newline at end of file +% 3 +Several approaches are based on their own metamodels. It is not clear yet, whether such model-based approaches could be developed in RAGs. While implementing metamodel generations of approaches in RAGs, a JastAdd- and ILP-based framework \cite{gotz2018jastadd} could be helpful to analyze and solve model problems, because it gives an optimized model solution for a problem described in AST model. Particularly, the OpenAPI structure implemented in this work is much smaller than models denied in \cite{gotz2018jastadd}. So, it could produce an interesting result for model problems in OpenAPI testing. \ No newline at end of file diff --git a/sections/ragoGram.tex b/sections/ragoGram.tex index 0f60f7807e8cbece899d75e2a7985a7102c1eca3..691d2e0cb8cf805be3215feb2e93859063416da6 100644 --- a/sections/ragoGram.tex +++ b/sections/ragoGram.tex @@ -3,17 +3,34 @@ As mentioned in \Cref{sec:openapi}, OpenAPI specifications are written in structured JSON or YAML and do not hold on strict definitions (i.e. programming language-agnostic). It means that specifications can be described and implemented in any programming language or grammar, which also applies to RAG. % AST -To use OpenAPI in RAG, it is firstly necessary to rewrite the OpenAPI structure in an AST. We have constructed this AST in 95 AST-nodes to define 30 objects. The version of OpenAPI considered in this framework is 3.0.0. To have a better overview, the definition of Parameter Object in our AST is shown in Listing~\ref{lst:rago-ast} and can be compared with the definition in the OpenAPI official GitHub\footnote{\url{https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.0.md}}. +To use OpenAPI in RAG, it is firstly necessary to rewrite the OpenAPI structure in an AST. We have constructed this AST in 95 AST-nodes to define 30 objects. The version of OpenAPI considered in this framework is 3.0.0. To have a better overview, the definition of Parameter Object in our AST is shown in Listing~\ref{lst:rago-ast} and can be compared with the definition in the OpenAPI official GitHub\footnote{\url{https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.0.md}}. Every Reference Object has a String token named Ref and refers an object in the OpenAPI document with attributes in Listing~\ref{lst:rago-reference}. \begin{lstlisting}[language=AST,label={lst:rago-ast},caption={Parameter Object}] abstract ParameterOb; ParameterReference : ParameterOb ::= <Ref>; ParameterObject : ParameterOb ::= <Name> <In> <Description> <Required:Boolean> <DeprecatedBoolean:Boolean> <AllowEmptyValue:Boolean> <Style> <Explode:Boolean> <AllowReserved:Boolean> [SchemaOb] <Example:Object> ExampleTuple* ContentTuple* Extension*; -\end{lstlisting} \label{lst:rago-one} +\end{lstlisting} -During transferring the structure from OpenAPI to RAG, several properties of JastAdd could be clearly noticed: +\newpage + +\begin{lstlisting}[language=JRAG,label={lst:rago-reference},caption={Attributes for Reference}] +coll List<ParameterTuple> OpenAPIObject.parameterTuples() [new ArrayList<>()] root OpenAPIObject; + ParameterTuple contributes this + to OpenAPIObject.parameterTuples(); + +syn ParameterObject ParameterOb.parameterObject(); +eq ParameterObject.parameterObject() = this; +eq ParameterReference.parameterObject() { + for( ParameterTuple t : root().parameterTuples() ){ + if( t.getKey().equals(getRef().substring(getRef().lastIndexOf("/")+1, getRef().length())) ) + return t.getParameterOb().parameterObject(); + } + return new ParameterObject(); +} +\end{lstlisting} + +During transferring the structure from OpenAPI to RAG, several properties of JastAdd were detectable: \begin{description} \item JastAdd does not support any map structure. So, nodes derived from maps in OpenAPI must be configured in tuples (List of a named tuple containing a key and a value). \item Extensions which are properties of an object class in OpenAPI are also defined in nodes. -\item If a separate object is a reference object, it saves only a String token and calls the corresponding object, if needed. (Listing~\ref{lst:rago-ast}). \end{description} \ No newline at end of file diff --git a/sections/ragoProcess.tex b/sections/ragoProcess.tex index 859f7e2af2ec0ebcc28b3ae6ca1fb92743085a32..d34ca920c883091262340f38183e2e74099b679d 100644 --- a/sections/ragoProcess.tex +++ b/sections/ragoProcess.tex @@ -13,6 +13,8 @@ Before working with attribute definitions, an input specification in JSON or YAM %Parse Java -> RAG The next step of processing this framework is transferring parsed objects in java into AST nodes. This step is done with attribute definitions in jrag file. A small definition example of Parameter Object is in Listing~\ref{lst:parser}. + +\newpage \begin{lstlisting}[language=JRAG,label={lst:parser},caption={Parser for Parameter Object}] { ... @@ -32,7 +34,7 @@ The next step of processing this framework is transferring parsed objects in jav First, "parameter" is an object parsed with openapi4j and "parameterObject" is a node which should describe the structure of Parameter Object in RAG. It checks whether values in "parameter" exist and are set into the matching node in the AST (in this case "parameterObject"). At the end of the attribute, it returns the node. %Transfer back RAG -> Java for validation -To see if the parser implemented with JastAdd works correctly, parsed nodes in an AST that describes OpenAPI structure must be transferred back into the structure of openapi4j and validated. The validation of objects after processing is explained in \Cref{sec:ragotest}. The way of definitions to transfer RAG into openapi4j is similar to the parser, it only provides the opposite direction of implementation. Listing~\ref{lst:backtransf} allows to compare itself with Listing 4.2. +To see if the parser implemented with JastAdd works correctly, parsed nodes in an AST that describes OpenAPI structure must be transferred back into the structure of openapi4j and validated. The validation of objects after processing is explained in \Cref{sec:ragotest}. The way of definitions to transfer RAG into openapi4j is similar to the parser, it only provides the opposite direction of implementation. Listing~\ref{lst:backtransf} allows to compare itself with Listing~\ref{lst:parser}. \begin{lstlisting}[language=JRAG,label={lst:backtransf},caption={Back-Transformation for Parameter Object}] { ... diff --git a/sections/ragoTest.tex b/sections/ragoTest.tex index d0248f60dcb998f4f8c2e99a42af47d70d848c73..d3c409172df116791d5d8db1baec6a6bd3a81d46 100644 --- a/sections/ragoTest.tex +++ b/sections/ragoTest.tex @@ -1,10 +1,10 @@ \section{Validation} \label{sec:ragotest} % General -For correct (re-)constructions, objects after the processing phase must be the same before. In that sense, a generated JSON or YAML must be equivalent to the given OpenAPI document. Thankfully, there are several practical libraries to compare two JSONs (JsonNode, JsonDiff, JsonPath) and conditions for the assertion were uncomplicated. As a result, we have validated the functionality of the structure transfer, while 100 APIs from api.guru$^{3}$ are constructed in RAG and reconstructed in openapi4j. +For correct (re-)constructions, objects after the processing phase must be the same before. In that sense, a generated JSON or YAML must be equivalent to the given OpenAPI document. Thankfully, there are several practical libraries to compare two JSONs (JsonNode, JsonDiff, JsonPath) and conditions for the assertion were uncomplicated. As a result, we have validated the functionality of the structure transfer, while 100 APIs from apis.guru\footnote{\url{https://apis.guru/}} are constructed in RAG and reconstructed in openapi4j. % Differences -In the validation, differences with empty values and differences in Reference Objects are excluded. After the observation of OpenAPI documents, we have noticed that every author has an individual implementing behavior (i.e. description parts are always initialized in some APIs or nodes with empty values do not exist in others). Sibling elements of references do not provide semantical differences. So, it was not sensible to generate strictly equivalent values.$^{4}$ Its implementation is shown in Listing 4.4. +In the validation, differences with empty values and differences in Reference Objects are excluded. After the observation of OpenAPI documents, we have noticed that every author has an individual implementing behavior (i.e. description parts are always initialized in some APIs or nodes with empty values do not exist in others). Sibling elements of references do not provide semantical differences. So, it was not sensible to generate strictly equivalent values\footnote{\url{https://swagger.io/docs/specification/using-ref/}}. Its implementation is shown in Listing~\ref{lst:validation}. \begin{lstlisting}[language=JRAG,label={lst:validation},caption={Validation Method}] JsonNode diff = JsonDiff.asJson(expectedNode, actualNode); diff --git a/sections/stm2.tex b/sections/stm2.tex index 58127bdd6bbb68211b64043b36ac9d81a0cb7fca..fbf428789b81682f3e389f449897df2b2840739e 100644 --- a/sections/stm2.tex +++ b/sections/stm2.tex @@ -2,8 +2,7 @@ Random testing is a one of easiest way to test API and can be useful in some situations. However, it is not effective in REST API testing, because the coverage of the tested API would be particularly low and random values are unusually valid. During the observation in \Cref{sec:stm1}, it was clear to see that random testing mostly produces only requests that receive only 4xx HTTP stauts codes from commercial APIs. % General -To solve this problem, most of REST API testing approaches use a stateful process, because it enables to analyze properties of APIs and infer inputs which are more appropriate than random inputs. There are several suggestions in \Cref{ch:curr-appr}, this framework investigates a inference of parameters with operation dependency motivated by Specification-based Approach \cite{ed2018automatic} and RESTTESTGEN \cite{viglianisi2020resttestgen}. Generally, it collects all responses and inferences parameters contributing the same schema of a succesful response. \\ -If there is a schema set in a request and a response, parameters of them are inferred by three strategies: +To solve this problem, most of REST API testing approaches use a stateful process, because it enables to analyze properties of APIs and infer inputs which are more appropriate than random inputs. There are several suggestions in \Cref{ch:curr-appr}, this framework investigates a inference of parameters with operation dependency motivated by Specification-based Approach \cite{ed2018automatic} and RESTTESTGEN \cite{viglianisi2020resttestgen}. Generally, it collects all responses and inferences parameters contributing the same schema of a succesful response. If there is a schema set in a request and a response, parameters of them are inferred by three strategies: \begin{itemize} \item Case insensitive @@ -12,8 +11,7 @@ If there is a schema set in a request and a response, parameters of them are inf \end{itemize} % Process -In the implementation of this work, case insensitive comparison and id completion are utilized to create the basic functionality. Stemming algorithm can be also extended in the future. \\ -The follwing code in Listng~\ref{lst:parameter-inference} shows how the parameter inference is compiled with predefined attributes: +In the implementation of this work, case insensitive comparison and id completion are utilized to create the basic functionality. Stemming algorithm can be also extended in the future. The follwing code in Listng~\ref{lst:parameter-inference} and List~\ref{lst:parameter-dict} shows how the parameter inference is compiled with predefined attributes: \begin{lstlisting}[language=JRAG,label={lst:parameter-inference},caption={Parameter Inference}] generateRequests(); @@ -32,61 +30,43 @@ List<String> paths = new ArrayList<>(); for (ParameterOb o : operationObject.getParameterObs()) { ParameterObject p = o.parameterObject(); if (p.getIn().equals("path")) - paths = p.addinfPathParameters(pathRef, paths); + paths = p.addInfPathParameters(pathRef, paths); else if (p.getIn().equals("query")) - paths = p.addinfQueryParameters(pathRef, paths); + paths = p.addInfQueryParameters(pathRef, paths); } for (String path : paths) connect(path); \end{lstlisting} -Before it starts with the parameter inference, the random testing generator of \Cref{sec:stm1} is executed first (Line 1). During this execution, the status code of a response is checked if it is a successful response with 200 status code (Line 4). Afterwards, the response schema of returned values is also checked. If it is in type array, the function "writeDictionary" is iterated, otherwise it only executed once (Line 6-9). "writeDictionary" is not attributed, because it would be critical for the rule of RAG that an AST structure must not be edited, while attributes are computed. However, the function "writeDictionary" saves the returend values of a successful response in seperate properties and write them in a dictionary (e.g. properties "id" and "name" are seperately stored with their value in the dictionary). If the schema of a response provides a reference of a schema object, the field name gets a name of a reference as prefix. Subsequently, this implementation does the similar way of execution in random testing at the generation phase. Firstly, it iterates all parameter objects (Line 14) and examines whether the parameter type is Path or Query (Line 15-18). Both attributes return URIs with parameter values inferred by the dictionary and case insensitive comparison. Generated URIs are put in a list. Lastly, the generator attribute sends requests with the URIs and starts with observation (Line 20-21). +\begin{lstlisting}[language=JRAG,label={lst:parameter-dict},caption={Attribute writeDictionary}] -% Result -As results, the test case generator with parameter inference implemented in this framework could create maximum over 300 acceptable URIs for the parameter petId in the selected API, pet store, at the operation getPetById. It generated also numerous requests denied by the server. After the observation of several execution iterations, it can be assumed that the API with this operation sends 200 or 404 status codes randomly or according to some rules, because a same URI provided status codes in that way. - -% Helpful attributes (Collection, inherited, easier return with =) -During the implementation phase of parameter inference, RAG was beneficial at writing codes. At writing codes for parser and back-transformation, attributes could not be applied, because the functional parts change the structure of AST, which conflicts to the definition of attribute and also in JastAdd. So, it could not be realized that attributes can be practical. But, following attributes and features of JastAdd were helpful to construct the generators and implement them: - -\begin{itemize} -\item As long as a returning value can be called or storable by an AST node, the syntax in RAG is reduced with an equals sign and simpler than in common programming language. Therefore, the source codes are compact. To compare how RAG could be implemented effectively, see the implementation of an attribute in this work in Listing~\ref{lst:attribute-infParam} and how this attribute is defined in common Java description in Listing~\ref{lst:java-infParam}. -\item To iterate all paths available in OpenAPI, an inherited attribute facilitated the writing effort. Paths Objects are child nodes of OpenAPI Object, so, all that should have been done was only two lines of code instead of writing an extra for loop. See Listing~\ref{lst:inherited}. -\item As its name, Reference Attribute Grammar, every AST node can be referenced by an attribute. For instance, with a collection attribute. A reference defined in Listing~\ref{lst:rago-ast} contains only a String variable named Ref, it still could provide its Object like in Listing~\ref{lst:coll-param}. -\end{itemize} - -\begin{lstlisting}[language=JRAG,label={lst:attribute-infParam},caption={Attribute InferredParameter.value()}] -syn String InferredParameter.value() = getParameter().substring( getParameter().indexOf("?") + 1 ); -\end{lstlisting} - -\begin{lstlisting}[language=JRAG,label={lst:java-infParam},caption={InferredParameter.value() in Java}] -Class InferredParameter { -... -constructor; -... - public String value() { - return getParameter().substring( getParameter().indexOf("?") + 1 ); - } +syn String OperationObject.writeDictionary(SchemaOb schema, String resp) { + ... + return inferredParameter; } -\end{lstlisting} -\begin{lstlisting}[language=JRAG,label={lst:inherited},caption={Inherited Attribute in Paths Object}] -inh boolean PathsObject.inferUrl(); -eq OpenAPIObject.getPathsObject(int i).inferUrl(){ + +syn List<String> OperationObject.writeDictionaryWithArray(SchemaOb schema, String resp) { + List<String> list = new ArrayList<>(); + Iterator<JsonNode> props = parseArrayNode(resp).elements(); ... + while(props.hasNext()) + list.add(writeDictionary(schema.itemsSchema(), props.next().toString())); + return list; } -\end{lstlisting} +\end{lstlisting} -\begin{lstlisting}[language=JRAG,label={lst:coll-param},caption={Collection Attribute in Parameter Object}] -coll List<ParameterTuple> OpenAPIObject.parameterTuples() [new ArrayList<>()] root OpenAPIObject; - ParameterTuple contributes this - to OpenAPIObject.parameterTuples(); -... -syn ParameterObject ParameterOb.parameterObject(); -eq ParameterObject.parameterObject() = this; -eq ParameterReference.parameterObject() { - for( ParameterTuple t : root().parameterTuples() ){ - if( t.getKey().equals(getRef().substring(getRef().lastIndexOf("/")+1, getRef().length())) ) - return t.getParameterOb().parameterObject(); +\begin{lstlisting}[language=JRAG,label={lst:parameter-case},caption={Case insensitive comparison}] +syn List<String> ParameterObject.addinfPathParameters(String pathRef,List<String> paths){ + for(InferredParameter i:root().collectInferredParameters()){ + String pathPart=pathRef.substring(pathRef.indexOf("{"),pathRef.indexOf("}")+1); + if(getName().equalsIgnoreCase(i.name())) + paths.add(pathRef.replace(pathPart,i.value())); } - return new ParameterObject(); + return paths; } \end{lstlisting} + +Before it starts with the parameter inference, the random testing generator of \Cref{sec:stm1} is executed first (Line 1, Listing~\ref{lst:parameter-inference}). During this execution, the status code of a response is checked if it is a successful response with 200 status code (Line 4, Listing~\ref{lst:parameter-inference}). Afterwards, the response schema of returned values is also checked. If it is in type array, the function "writeDictionary" is iterated, otherwise it only executed once (Listing~\ref{lst:parameter-dict}). The attribute "writeDictionary" saves the returend values of a successful response in seperate properties and write them in a dictionary (e.g. properties "id" and "name" are seperately stored with their value in the dictionary). If the schema of a response provides a reference of a schema object, the field name gets a name of a reference as prefix. Subsequently, this implementation does the similar way of execution in random testing at the generation phase. Firstly, it iterates all parameter objects (Line 14, Listing~\ref{lst:parameter-inference}) and examines whether the parameter type is Path or Query (Line 15-18, Listing~\ref{lst:parameter-inference}). Both attributes return URIs with parameter values inferred by the dictionary and case insensitive comparison (Listing~\ref{lst:parameter-case}). Generated URIs are put in a list. Lastly, the generator attribute sends requests with the URIs and starts with observation (Line 20-21, Listing~\ref{lst:parameter-inference}). + +% Result +As results, the test case generator with parameter inference implemented in this framework could create maximum over 300 acceptable URIs for the parameter petId in the selected API, pet store, at the operation getPetById. It generated also numerous requests denied by the server. After the observation of several execution iterations, it can be assumed that the API with this operation sends 200 or 404 status codes randomly or according to some rules, because a same URI provided status codes in that way. \ No newline at end of file