On November 18th the Eclipse Xtext project added a new source repository eclipse/xtext-umbrella. This article describes the reasons for this additional repository and what is developed in there.
In the past Eclipse Xtext was developed in a single repository, first at eclipse.org and since 2015 at GitHub in the repository eclipse/xtext. After Eclipse Neon and Xtext 2.10 the code was splitted into several separate repositories:
While these repositories contain the majority of the code there is still the need for some code that is related to all sub projects and has to be placed in a central “umbrella” project: release engineering, development tools, documentation, website, etc. This code remained in the eclipse/xtext repository.
When developing on Xtext, committers and contributors should have a similar development environment and all relevant Xtext sources. The Xtext project therefore provides an Oomph setup which is automatically available in Oomph’s Eclipse product catalog. All sources also include the mentioned cross-cutting-concern projects for build, documentation, etc. Checking out the eclipse/xtext repository is easy with Oomph. However, the cloned and checked out repository takes 550MB and also cloning takes a bit. The pain of this large download could be lowered if the repository would be cloned shallow. Unfortunately Oomph is not able to configure the Git Clone Task for that and JGit – which is used in the background – isn’t either (see Bug#475615, Bug#507324).
This would be not a problem if developers could choose this repository optionally like any other sub project in Xtext’s setup:
But there is an important requirement for a permanent availability of the umbrella repository.
Xtext’s project structure is now due to the repository split a multi-repository multi-project structure. During the development in the IDE it is important that sources from an upstream repository could be changed and seamlessly used by a project in a downstream repository. For example, if a developer debugs code in Xtend (repository eclipse/xtext-xtend) and changes a class in Xbase (repository eclipse/xtext-extras), the change has to be used immediately. This sounds natural, but with Gradle inter-project dependencies can only be resolved within one composite build and the composite is only defined per repository. In order to resolve now cross-repository dependencies, there must be a composite that aggregates the composites of each repository.
The Gradle composite has to be placed in an umbrella repository (xtext-umbrella/releng/gradle-composite). Placing that in the eclipse/xtext repository would require to clone this large repository always although just a very small bit of code is actually needed.
Xtext’s Oomph setup has now been updated. Users that set up a development environment with the Eclipse Installer like described in the Contribution Guide will automatically benefit from the new setup. The setup will now additionally clone the eclipse/xtext-umbrella repository and import the projects from there for any combination of sub projects selected in the Oomph wizard.
During tests of the new setup it becomes apparent that users should pay special attention to the Variables page and therefore the Git clone location rule. The value should be
${workspace.location/.git/}${@id.remoteURI|gitRepository}
if the repositories should be checked out within the workspace. On another machine I recognized that the Git branch was another directory layer, but this would break the composite build right now. It is possible that we will make this even more flexible later.
The Eclipse Xtext project has now a new repository eclipse/xtext-umbrella, which contains sub project independent sources. A new repository became necessary since a new Gradle composite build was required and must be always checked out. Cloning the old main repository takes too much space and time to be reasonable for these sources.
Contributors can now use the new project structure by setting up their environment with the updated Oomph setup.
The new Gradle composite build became mandatory to resolve cross-repository dependencies in the IDE. In a future blog post I will go into some details on this requirement and how and why we created the composite build.
More details can be found in the related discussions in the following issues and PRs at GitHub:
You want to learn more about the project? Check these blogs: