How to fix Artop memory problems

In the automotive domain, itemis and others make intensive use of the community project Artop (Autosar Tool Platform) – an Eclipse (EMF) based implementation of the AUTOSAR meta-model. Artop supports the conversion of models between the many revisions of the AUTOSAR standard. 

We have encountered situations, where an Artop based application needs huge amounts of memory – here is an analysis of the technical reasons.

AUTOSAR revisions and Artop revision conversion

Any given product built on top of Artop will make use of a specific implementation of the AUTOSAR meta-model (e.g. 4.3.1). Artop supports the conversion between these revisions upon loading and saving (i.e. from or to 4.3.0). In our case, the application is based on a custom built Artop for some internally used pre-released AUTOSAR versions. We work on a revision called 4.3.1, but we also have to save 4.3.0.

Upon saving Artop will do a conversion in the following steps:

1. The standard Eclipse/EMF implementation will create a memory-optimized XML-representation of the AUTOSAR model (a XMLStringImpl instance). This is used to avoid duplication of string values.
2. Artop will then take this as an input and create a JDOM representation of the XML internally.
3. Artop will invoke the revision converter for all elements of the JDOM tree, to allow for conversion (this is specific for each combination of from/to revisions).
4. Artop will finally serialize the resulting JDOM to disk as XML.

With production data, we have seen Artop consume huge amounts of memories. One of the reasons is the JDOM representation as created in Step 2.

Quickfix

Obviously the quickest fix is to increase the memory available to the JVM with the -Xmx option. However, we still had memory problems and have started to create patches for Artop.

Minimal invasive fix

We have several approaches, with different impacts on Artop architecture. Our first fix concentrates on minimizing the memory required by the JDOM model. It seems that the JDOM creation is not memory optimized and each JDOM element created uses its own string instances. JDOM2 provides a JDOM builder with string caching, which is not available to us. So we created our own implementation making use of Java 7 / Java 8's improved String interning.

While we were at it, we also simplified Artop a bit: The original code uses an additional thread (for technical reasons) for creating the JDOM – we were able to remove that by implementing a simple Reader class that takes data directly from the XMLStringImpl.

Possible Redesign

Since EMF does not only support using the XMLStringImpl for serialization, but also using the DOM classes from the w3c packages, it could be an optimization to use those directly. But since the conversion APIs are based on JDOM, this will need API changes (converting w3c DOM to JDOM does not seem to be an option, since we would be duplicating DOM trees, again).

Result

In a combination of increasing available memory and reducing the memory footprint with our patches, we resolved the out-of-memory issues. In situations like these, we not only built products and toolchains on top of Artop, but also work actively to improve the overall code base for the community.

You want to learn more about our projects in the automotive domain?