Wednesday, September 29, 2010

Xtext in Indigo

We've started development on the next release of Xtext just four weeks ago. Prior to that we did the service release and some internal prototyping and evaluation for upcoming features. We also already have had our yearly kick-off meeting in order to define the main goals for the next release.

With this blog post I'ld like to share what we are up to. Feedback is welcome.

Xtext Version 2.0

Yes, that's right. We are aiming at a new major version. This means lots of
great new features and improvements as well as incompatible changes. While our API lifecycle contract is, that nothing remains compatible between two major versions, we of course are aware of what API most users use. So we reconsider any changes and don't do them easily.
The differences might turn out to be comparable to those we had between the last two releases, which were considered easy to migrate to.

What we did so far

During M2 we did a lot of clean up. We removed any deprecated code and incremented the bundle versions for those with incompatible changes to 2.0.0.
Also the whole outline view infrastructure as well as the auto editing has been rewritten. The new outline view is cleaner, faster and simpler. In addition Xtext now leverages JFace partitions. That is one can register different Jface service implementations, such as content assist, per partition. By default there are distinct partitions for comments and string literals as well as the default partition type for everything else. See the full list of solved bugzillas here.

Milestone builds

Yesterday we had our M2 of the current indigo stream. We don't have a promoted milestone build, as we had some issues with the build servers. Please use the nightly builds in case you want to give the latest changes a spin.

Expect the first promoted milestone build (M3) of the current development stream of Xtext in six weeks (November 9th).

What's next? (Plan for Indigo)

The development plan for Indigo is laid out around one big topic, which is called Xbase. Xbase is an expression language bound to the JVM, which will be shipped with Xtext and can be embedded and extended in other Xtext-based languages. It is basically a simplified version of Java's expressions but also adds some new concepts. I will post a separate blog entry on Xbase tomorrow. Note, that Xbase is a research project we are doing together with Christian-Albrecht University of Kiel which is sponsored by the Federal Ministry of Education and Research.

Refactorings & Clean ups

As you might know, we usually implement most functionality twice (or more often), because we are simply too stupid to get it right in the first place :-).
Even with Xtext 1.0.0 this is not different, so we have identified the following parts of the framework which we want to clean up or even re-implment:

  • Outline view (done)

  • Auto Editing (done)

  • Bracket Matching (done)

  • Serializer

  • Formatter

  • some smaller things

New Features

But hey, we know most people are interested in new features. So this is what we want to work on in addition to Xbase:

  • Generic Rename Refactoring

  • Additional Import Namespace Functionality (organize imports, add import, warnings and errors)

  • Default Implementation and Convenience API for Text Hovers

  • Lexer and Parser Fragments

  • View Framework (Convenient way to define tree views, such as call or type hierarchies)

More Languages

We of course eat our own dog food and therefore will implement a couple of new languages based on Xbase.

One is a textual notation to define Ecore elements similar to EMFatic, but where one can specify the behavior in derived EStructuralFeatures and EOperations in the language.

Another is a successor to Xpand and Xtend, called Xtend 2. That one comes with a couple of very interesting features I'll talk about in subsequent blog posts.

We are also thinking about having DSLs for several different view points in Xtext itself, such as scoping. Don't miss tomorrow's blog post on Xbase. :-)

Disclaimer: As usual this is what we would like to do and defines the general direction. But as we don't have any fortuneteller in our team, we might (will) adjust the plan slightly as we go.


  1. "We are also thinking about having DSLs for several different view points in Xtext itself, such as scoping."

    Ah, nice to see that one in there ;)

  2. "While our API lifecycle contract is, that nothing remains compatible between two major versions..."

    If nothing remains compatible, you should be aware that eventually nobody wants to use Xtext anymore. It is not our business to permanently rework our solution to make it running on a new Xtext version again. We have more important things to do.

  3. @Peter: In practice we don't change everything. I just want to be explicit about the contract. I do so just to make sure people who can't invest in one or two days of migration effort know so in advance.

    We don't change things lightly but keeping everything binary compatible forever won't be something we are able to do. The more mature the framework is the less drastically the changes will be, but it's still too early to have *everything* carved in stone.

    Also we always have provided guidance for migration and help everybody to update to the latest version. I'm pretty sure that most of our users take the minor migration effort with pleasure given the vast improvements we have delivered with each version (and we couldn't if everything had to be compatible).

    And also note, that we maintain previous releases. So in case you're facing a serious bug in Xtext 1.0.0 chances are very good that it has already been fixed with the binary compatible service release (1.0.1). And if not and it really is a show stopper, we will be glad to have another service release for that stream.

  4. is there any beta or snapshot for trial?
    you still use antlr as parser, or?

  5. The previsouly announced Xtend Beta also contains Xtext 2.0 (
    Yes, Xtext uses Antlr.