Xbase Customization: Redefining operator keywords

If you use Xbase in your Xtext-based DSL, you are usually satisfied with the set of operators the expression language defines. They are closely related to what you are used to in Java or similar languages. But what if that's not enough and you need to customize keywords for operators? Here is how it's done.

In a customer’s recent workshop the customer wished to have custom keywords for some operators. For example, the operator && should be alternatively presented as AND, and the || operator as OR.

To demonstrate this customization we’ll start with Xtext’s famous Domainmodel Example. The Domainmodel.xtext grammar is derived from org.eclipse.xtext.xbase.Xbase and the Operation rule uses an XBlockExpression for the Operation’s body:

grammar org.eclipse.xtext.example.domainmodel.Domainmodel with org.eclipse.xtext.xbase.Xbase
 
...
 
Operation:
    'op' name=ValidID '(' (params+=FullJvmFormalParameter (',' params+=FullJvmFormalParameter)*)? ')' (':' type=JvmTypeReference)?
        body=XBlockExpression;

Unit Test

We’ll extend the language test-driven, thus we first create a unit test that uses the new feature, but will fail first until we have successfully implemented it. Fortunately there is already a suitable test class in project org.eclipse.xtext.example.domainmodel.tests.

We extend the class ParserTest.xtend:

Note the Operation body, the expression uses both presentations of the And-operator.

return s!= null && s.length > 0 AND s.startsWith("bar")

When the ParserTest is executed it will now fail, of course, but only for the AND keyword:

java.lang.AssertionError: Expected no errors, but got :
ERROR (org.eclipse.xtext.diagnostics.Diagnostic.Linking) 
'The method or field AND is undefined' 
on XFeatureCall, offset 119, length 3
ERROR (org.eclipse.xtext.xbase.validation.IssueCodes.unreachable_code) 'Unreachable expression.' on XFeatureCall, offset 119, length 3

	at org.junit.Assert.fail(Assert.java:88)
	at org.eclipse.xtext.junit4.validation.ValidationTestHelper.assertNoErrors(ValidationTestHelper.java:187)
	at org.eclipse.xtext.example.domainmodel.tests.ParserTest.testOverriddenKeyword(ParserTest.java:268)
...

Overloading the operator rules

Looking at Xbase.xtext shows that Xbase defines separate data type rules for operators:

OpOr:
    '||';
OpAnd:
    '&&';

Xtext’s Grammar Mixin feature allows a redefinition of those rules. The obvious customization is in Domainmodel.xtext: 

OpOr:
    '||' | 'OR';
OpAnd:
    '&&' | 'AND';

Regenerating the language’s Xtext implementation makes those keywords available to the syntax. However, the unit test still fails, but now with a different error:

java.lang.AssertionError: Expected no errors, but got :
ERROR (org.eclipse.xtext.diagnostics.Diagnostic.Linking) 
'AND cannot be resolved.' on XBinaryOperation, offset 119, length 3

	at org.junit.Assert.fail(Assert.java:88)
	at org.eclipse.xtext.junit4.validation.ValidationTestHelper.assertNoErrors(ValidationTestHelper.java:187)
	at org.eclipse.xtext.example.domainmodel.tests.ParserTest.testOverriddenKeyword(ParserTest.java:268)

OperatorMapping

The missing piece is a customization of class OperatorMapping. Thus we create a subclass OperatorMappingCustom with constant QualifiedNames for the additional operator keywords and bind it in DomainmodelRuntimeModule:

A naive approach is here to overload the initializeMapping(), as the Javadoc suggests (“Clients may want to override #initializeMapping() to add other operators.“).

But this fails again:

com.google.inject.CreationException: Guice creation errors:

1) Error injecting constructor, java.lang.IllegalArgumentException: value already present: operator_and
  at org.eclipse.xtext.example.domainmodel.OperatorMappingCustom.(Unknown Source)
  at org.eclipse.xtext.example.domainmodel.OperatorMappingCustom.class(Unknown Source)
  while locating org.eclipse.xtext.example.domainmodel.OperatorMappingCustom
  while locating org.eclipse.xtext.xbase.scoping.featurecalls.OperatorMapping
    for field at org.eclipse.xtext.xbase.util.XExpressionHelper.operatorMapping(Unknown Source)
  while locating org.eclipse.xtext.xbase.util.XExpressionHelper
    for field at org.eclipse.xtext.xbase.validation.XbaseValidator.expressionHelper(Unknown Source)
  at org.eclipse.xtext.service.MethodBasedModule.configure(MethodBasedModule.java:57)
  while locating org.eclipse.xtext.example.domainmodel.validation.DomainmodelJavaValidator
Caused by: java.lang.IllegalArgumentException: value already present: operator_and
	at com.google.common.collect.HashBiMap.put(HashBiMap.java:237)
	at com.google.common.collect.HashBiMap.put(HashBiMap.java:214)
	at org.eclipse.xtext.example.domainmodel.OperatorMappingCustom.initializeMapping(OperatorMappingCustom.java:25)
	at org.eclipse.xtext.xbase.scoping.featurecalls.OperatorMapping.(OperatorMapping.java:121)
	at org.eclipse.xtext.example.domainmodel.OperatorMappingCustom.(OperatorMappingCustom.java:18)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	...

Instead, the method getMethodName() must be overridden and delegate the invocations of the new operators to the existing ones. The resulting OperatorMappingCustom is then:

Finally, the unit test will execute successful.

Summary (tl;dr)

Xbase allows customization of operators by overriding the operator’s data type rule from Xbase.xtext. This adds the operator keywords to the language, but fails at runtime. Additionaly the class OperatorMapping must be customized and method getMethodName() overloaded.