Friday, December 03, 2010

Xtend 2 - the Successor to Xpand

M2T Xpand is one of the widest used template languages for code generation. It is statically typed, supports polymorphic dispatching of template definitions and provides a very convenient way to navigate through models. The navigation convenience mostly comes from the built-in higher order functions for collection types, the possibility to have extension functions using Xtend and some syntactic sugar for common expressions.
Also Xpand is known to provide a very open architecture. There are several hooks in the interpreter to integrate profilers, pretty printers or whatever comes to mind. That's all very nice and has led to quite some success.

But there are also some issues with Xpand and Xtend (Just in case you don't know I am not blaming other people's work, because I'm the author of Xpand, Xtend and Check)

  • Performance - Xpand and its siblings are interpreted and the interpreter was written without much performance considerations, i.e. is slow.
  • Tooling - The tooling is hand-made with JFace and Co, the code is old, not well tested and unfortunately a bit buggy. 
  • Ugly conceptual short cuts - built-in higher order functions instead of closures, built-in collection types instead of generics, and a couple of other unsound semantics which pop up occasionally and bite the user frequently.
Xtend 2
The reason I started work on Xtext in the first place was, because it had been so painful to write the IDE support for Xpand, Xtend and Check. I thought that there must be some higher abstraction to derive functionality like syntax coloring, content assist, outline views, etc. from a language description. So I wrote the first prototype of Xtext in 2006 and luckily found some friends and a sponsor to create what was chosen to be the most innovative project at Eclipse this year.

Today Xtext is ready to be the basis for a rewritten version of Xpand. We call it Xtend 2 as it is more like a successor to Xtend which allows to have Xpand's template syntax as an expression. After having designed Xpand and Xtend and having seen so many different applications of it with different requirements and problems to solve, we know what features are helpful and which are problematic. Also during the last years with Xtext I have learned so much about language and IDE design and so did my team. All the languages we built for customers and all the languages we saw people developing with Xtext have helped to get a good feeling of what is good language design and what is not. We are now in a promising position to design a very, very nice language.

And indeed we are all thrilled about the concepts we put together so far. I'm writing up the language specification these days and will post it as soon as it is more consumable state. Today I wanted to share my excitement and give you an idea of what it is going to look like.

What are the main differences?

Based on Xbase

Xtend 2 is of course using the expressions defined in Xbase. That basically means it is tightly integrated with Java, has closures, full generics and a couple of very neat features like operator overloading or the super cool switch statement.

Compiles to Java
Xtend2 files are directly compiled to Java classes. In fact they are compiled to Java source code, which means that you can read (and debug) what the compiler makes out of your Xtend 2 coding. We will put much effort in making the generated code very readable and performant at the same time.

Contains successful features from Xpand and Xtend...
The cool stuff from Xpand and Xtend will be there such as polymorphic dispatch, extension methods or the template syntax.

... but they are even better now
For instance extension methods can be any static Java function and more important any methods from injected objects. This basically allows you to exchange the implementation of any extension methods only by changing the DI configuration. This in turn replaces the whole AOP story we have in Xpand and Xtend.

"Simplicity is the Ultimate Sophistication"
You'll see a lot of other small improvements, but the coolness of the language doesn't come from the amount of features but from the short and carefully chosen list of concepts working consistenly together. It is the overall composition of concepts and the holistic experience (tooling, etc.) which gives a language its "feel". And although I don't have the language in my hands right now, it already feels fantastic. :-)

We want to show this at EclipseCon

There is a submission for a tutorial on Xtend 2 we submitted for EclipseCon 2011. My favorite conference at my favorite location. It is extremely important to us to be able to share with you guys what we have been working on for the last months (actually years if you count the development of Xtext in).
Chris asked people to use the comment system to provide feedback, in order to help the committee composing a great program. So please, if you are interested in attending a hands on tutorial for Xtend 2 at EclipseCon 2011, tell them. :-)