In Part 2 of this series we checked the requirements using NLP techniques in order to validate the use of domain specific concepts in their free text parts. In this part we will annotate the domain specific concepts and provide the user quick fixes to synchronize them with the glossary.
To provide usefull quick fixes, we have to take the current state of the glossary into account. For example, if a verb is found which exists in the glossary as name of a function, we will provide a quick fix which encloses the verb with quotation marks. Respectively the grammar defined in part 1 of our series this leads to a conversion of the verb from the typeWORD to the type STRING. In our model this means the verb becomes a reference to the function defined in the glossary and an association between the requirement containing the verb and the function is established.
In order to keep the glossary consistent during such quick fix operations, we will extend the use of NLP techniques. To avoid the creation of multiple glossary concepts for the same word in different inflections we will use stemming. Stemming means finding the root (stem) of a word for inflected forms, for example the singular for a plural word. On base of the comprehension between the found concept and his stem it is possible to decide if the found concept is added as a new concept or as a synonym for an existing concept.
To take all of such conditions into account and decide which quick fixes are provided to the user, we will define a decision table. The realization of the quick fix will be shown exemplary for Functions and one of the rules described in the decision table. This quick fix will require the selection of an existing functions by the user. Therefore, we will integrate a choose dialog into it.
The following decision table contains six rules. Each rule describes one constellation of conditions which leads to a specific quick fix. Conditions which have to apply are marked with Y (Yes). Conditions which have not to apply withN (No), ignored conditions with -. The quick fix that will be shown, if the constellation of conditions for the rule is fulfilled, is marked with X.
| Available quick fixes if a concept was found in freetext | R1 | R2 | R3 | R4 | R5 | R6 |
|---|---|---|---|---|---|---|
| Conditions | ||||||
| Name of concept exists in glossary as synonym | Y | N | N | N | N | N |
| Steam of concept exists in glossary as concept | - | Y | Y | N | N | N |
| Name equals stem | - | Y | N | Y | N | - |
| User choses "add to glossary as concept" | - | - | - | Y | Y | N |
| User choses "add to existing concept as synonym" | - | - | - | N | N | Y |
| Quick fixes | ||||||
| Create concept and add it to the glossary | X | |||||
| Create synonym and add it to the concept whose Name equals the steam | X | |||||
| Reference existing concept | X | |||||
| Reference existing synonym | X | |||||
| Create concept with the steam as name and add the concept as synonym | X | |||||
| User choses existing concept. The concept will be added as synonym to this concept | X |
For example the rule 6 (R6) leads to a quick fix where the user has to choose an existing concept. The found concept will be added as synonym to the existing concept. This quick fix is only shown if the concept found is not present in the glossary (not as synonym and not as concept) and the user choose to add this concept to an existing concept as synonym. It does not matter in this case if the concept is the stem or an inflected form of the word. On the contrary, for the rules 4 and 5 this does matter. If the user chooses "add to glossary as concept" and the name of the concept is the stem of the word, we provide a quick fix which creates a new concept with the name of the found concept as name (R4). Otherwise, if the name of the new concept is not the stem and the user chooses the same, we provide a quickfix which creates a new concept with the steam as name and add the found concept as synonym to the newly created (R5). The final step is always to encapsulate the found concept in quotation marks and therefore reference a glossary concept or one of its synonyms.
The behavior described in the previous section will be realized by a validator and a quick fix provider. The validator is responsible for the evaluation of the conditions and the annotation of the found concepts with a warning that can be fixed by one or more quick fixes.
The following check finds verbs in the RequirementEnd similar to that one described in part two of this series. It collects all information needed for the evaluation of the conditions described in the decision table.
@Check(CheckType.NORMAL)
def extractFunctionFromObjectWithDetails(RequirementEnd end) {
val text = end.objectWithDetails
val string = converter.textToString(text)
val pattern = '(?$verb[pos:VB|pos:VBD|pos:VBG|pos:VBN|pos:VBP|pos:VBZ])'
val result = tokenRegex.match(string, pattern)
val tokensByGroup = result.tokensByGroup
val verbGroupsFound = tokensByGroup.get("verb")
for (verbs : verbGroupsFound) {
var String verb = verbs.head.word
var String lemma = verbs.head.lemma
var verbPosition = verbs.head.begin
if (!isReference(verb, text)) {
val existingFunctions = getParentForEObject(end, Root).glossary.concepts.filter(typeof(Function)).toList
showFunctionQuickFixes(existingFunctions, text, verb, verbPosition, lemma)
}
}
}
For each verb found the check retrieves the verb and the lemma from the NLP result and stores the position of the verb in the list of found verbs. This position will be used later to determine the position of the verb in the free text that will be annotated. If the found verb is not a reference to a concept or synonym, the check retrieves all existing functions from the glossary. With this information the method showFunctionQuickFixes is called.
showFunctionQuickFixes determines the quick fix to show by evaluating the first three conditions of the decision table. The conditions of rule 6 are the same than the conditions for the rules 5 and 6 except the ones related to user interaction. Therefore, we provide additionally the quick fix resulting from rule 6 if the rules 4 and 5 apply and let the user choose one of them.
def showFunctionQuickFixes(List<Function> functions, TextWithConceptsOrSynonyms text, String verb, int verbPos, String lemma) {
val synonyms = functions.map[f|f.synonyms].flatten.toList
val synonymNames = synonyms.map[name].toList
val functionNames = functions.map[name].toList
val offset = calculateOffset(text, verb, verbPos);
val length = verb.length
val message = "Function " + verb + " found"
if (synonymNames.contains(verb)) {
// rule 1
acceptWarning(message, text, offset, length, REFERENCE_CONCEPT_OR_SYNONYM, verb)
} else if (functionNames.contains(lemma) && verb.equals(lemma)) {
// rule 2
acceptWarning(message, text, offset, length, REFERENCE_CONCEPT_OR_SYNONYM, verb)
} else if (functionNames.contains(lemma) && !verb.equals(lemma)) {
// rule 3
acceptWarning(message, text, offset, length, ADD_AS_SYNONYM_FOR_EXISTING_FUNCTION, verb, lemma)
} else if (!functionNames.contains(lemma) && verb.equals(lemma)) {
// rule 4 & 6
acceptWarning(message, text, offset, length, ADD_AS_NEW_FUNCTION, verb, lemma)
acceptWarning(message, text, offset, length, CHOOSE_FUNCTION_AND_ADD_AS_SYNONYM, verb)
} else if (!functionNames.contains(lemma) && !verb.equals(lemma)) {
// rule 5 & 6
acceptWarning(message, text, offset, length, CREATE_NEW_FUNCTION_AND_ADD_AS_SYNONYM, verb, lemma)
acceptWarning(message, text, offset, length, CHOOSE_FUNCTION_AND_ADD_AS_SYNONYM, verb)
}
}
If a rule applies, we annotate a range of the resource with a warning using the method acceptWarning. We provide this method with a warning message, the object to annotate (text) as well as the offset and length of the annotation range which means the position and the length of the found verb in the textual representation of theRequirementEnd. Furthermore, we have to provide an issue code which is used to identify the corresponding quick fix (e.g. REFERENCE_CONCEPT_OR_SYNONYM) and additional issue data (verb, lemma) that is needed by the quick fix to solve the issue. The issue codes are stored in public static variables.
public static val ADD_AS_NEW_FUNCTION = "INTRODUCE_FUNCTION" public static val CREATE_NEW_FUNCTION_AND_ADD_AS_SYNONYM = "INTRODUCE_FUNCTION_AND_SYNONYM" public static val CHOOSE_FUNCTION_AND_ADD_AS_SYNONYM = "INTRODUCE_FUNCTION_SYNONYM" public static val ADD_AS_SYNONYM_FOR_EXISTING_FUNCTION = "INTRODUCE_SYNONYM_FOR_EXISTING_FUNCTION" public static val REFERENCE_CONCEPT_OR_SYNONYM = "REFERENCE_CONCEPT_OR_SYNONYM"
We only have to define five issue codes, because the quick fix for the referencing of existing concepts and synonyms is the same.
The quick fixes are implemented in the generated class <YourLanguageName>QuickfixProvider located in the .uiproject. It is possible to provide multiple fixes for one issue code. A fix is annotated with the annotation @Fix and a reference to the issue code defined in the validator.
The following example shows the fix for the rule 6. The first parameter of the accept method provided by theIssueResolutionAcceptor is the issue which contains data related to the issue. For example the line number, the offset, the severity and the additional issue data. The three following parameters are the label, the description and the icon of the quick fix shown in the UI. To implement the fix we have to modify the document content. The modification is implemented in the IModification which provides access to the document by a IModificationContext(context).
@Fix(MyNaturalLanguageValidator::CHOOSE_FUNCTION_AND_ADD_AS_SYNONYM)
def chooseFunctionAndAddAsSynonym(Issue issue, IssueResolutionAcceptor acceptor) {
acceptor.accept(issue, 'Choose Function and add as synonym', 'Choose Function and add as synonym', 'choose.png') [ context |
val verb = issue.data.head
val modified = context.xtextDocument.modify(
[ resource |
val model = resource.contents.filter(typeof(Root)).head
val functions = model.glossary.concepts.filter(typeof(Function)).toList
val functionNames = functions.map[name].toList
val dialog = showSelectionDialog(functionNames, DIALOG_TITLE.replace(":CONCEPT", "Functions"))
if (dialog.result == null) {
return false
}
val chosenFunctionName = dialog.result.head as String
val chosenFunction = functions.filter(a|a.name.equals(chosenFunctionName)).head
val synonym = <YourLanguageName>Factory.eINSTANCE.createFunctionSynonym
synonym.name = verb
chosenFunction.synonyms.add(synonym)
])
if (modified){
val reference = "\"" + verb + "\""
val text = context.xtextDocument.get(issue.offset, issue.length)
val positionInText = text.indexOf(verb)
context.xtextDocument.replace(issue.offset + positionInText, verb.length, reference)
}
]
}
For this fix, we are using Xtext’s IDocument API. After retrieving the verb from the issue data, we call the modifymethod on the document providing an implementation of the IUnitOfWork interface which overrides the methodboolean exec(XtextResource resource). A unit of work acts as a transaction block, which can be used to get read and write access to the parsed model of the document. Inside the exec- method we collect all existing function names and show a selection dialog. The dialog allows the user to choose the name of the function for that we will create the synonym.
def showSelectionDialog(List<String> elements, String title) {
val shell = PlatformUI.getWorkbench().getActiveWorkbenchWindow().getShell();
val dialog = new ConceptSelectionDialog(shell, elements);
dialog.setInitialPattern("?*")
dialog.setTitle(title)
dialog.open
return dialog
}
To initialize the ConceptSelectionDialog we retrieve the workbench window's shell and provide the function names. After the customization of the title and the search pattern we open the dialog. The open method is blocking. This means it waits until the window is closed by the end user. The ConceptSelectionDialog allows the user to select and search for the function by its name. It is an implementation of the abstract class FilteredItemsSelectionDialog which is explained in the Eclipse Platform Plug-in Developer Guide.
After the user closed the dialog, we return the dialog and make sure the user selected a function, otherwise we will stop the modification by returning false. For the chosen function we create the new synonym with the verb as name. For the creation of the synonym we use the singleton model factory instance of our language.
If the modification was successful, we encapsulate the verb in quotations marks to convert it into a reference. In the next step, we get the position of the verb inside the annotated text and finally we replace the verb by thereference to link it with the newly created synonym.
In this part we saw how to support the user keeping the glossary and the domain specific concepts mentioned in the requirements in sync. We defined a set of rules which define under what conditions which quick fix is provided to the user. To evaluate the conditions we implemented a validator. Furthermore, this validator annotates the findings and provides the correct quick fix. The quick fix itself showed how to change the current document and its model in a transactional safe way, how to create new model elements and how to integrate dialogs into quick fixes.
This series showed how to create a controlled natural language for software requirements using Xtext. The language controls the use of natural language by a fixed syntax for requirements which contains free text parts in order to allow the user to express the requirements in natural language. These free text parts are validated using natural language processing techniques. Furthermore, the language supports the user through providing semi-automatic quick fixes in order to keep the glossary in sync with the requirements and therefore to establish a common language for the domain.