RE: The Hidden Costs of Domain-Specific Languages

Today, Johan Den Haan pointed me to a post about "Hidden Costs" of DSLs via twitter.
The post starts with the following statement:

While I am a big fan of Domain-Specific Modeling, I am openly not in favor of Domain-Specific Languages.

... which leaves me puzzled. I'm pretty sure that the DSM guys call their abstractions DSLs as well.
Anyway, the post itself is not particularly interesting, but at the end of it there are a couple of questions about DSLs. This sort of questions which I've heard often and which contain a lot of miss- and preconceptions. Thought it was a good opportunity to answer them.

So here are the questions and my answers:

If we take so much time to master a general purpose language, should we invest a comparable amount of time in limited-use languages (i.e. DSL)?

No, you shouldn't and you won't. DSLs are much easier to learn, since they are limited and focused. The important thing is that you have to understand the domain. But that is necessary in any case, no matter if you translate everything to a general purpose language or write it down like it is meant to be (i.e. using a DSL). If you know the domain very well, but are not able to grasp the DSL, than the DSL might not be that good.

How can we get support for a DSL, apart from its own creators?

How can you get support for an API, apart from its creator? Well, by looking at the code or any existing documentation, or by asking any other users. In many aspects DSLs are comparable to libraries or frameworks.

Where’s community support?

It depends on whether it is a widely used DSL (i.e. there is a community) or one you use and have defined in your one-man project. For the DSLs used in Ruby on Rails there's much community support, for instance. Same for the DSLs we have in Xtext.
Where's the community support for your homegrown libraries?

What happens after the departure of the language’s creator?

It is like with any other artifact in your software system: A DSL definition needs to be clean, understandable and maintainable. If you know that an important team member is leaving the project, make sure she's not taking any exclusive knowledge with her. That is she should coach other people before she lefts.

What’s BNF? Do I need it?

see http://en.wikipedia.org/wiki/Backus%E2%80%93Naur_Form :-)

DSL critics say really useful DSLs are hard and expensive to create. DSL supporters answer that DSLs are not designed, they evolve. Well, won’t any of those “evolutionary steps” risk breaking the entire development based on that DSL, much like a broken app build?

Useful DSLs can be very simple and at the same time a very complex DSL can be absolutely useless. It's not the complexity of a DSL, which makes it useful, but whether how good it is to solve the problem at hand. KISS and YAGNI are important principles here as well.
When defining concepts and abstractions it is key to find a good compromise between simplicity (abstraction) and re-usability. The simpler a DSL and the higher its potential for reuse the better. You're hitting a sweet spot, then. You start leaving that sweet spot as soon as you add concepts to the DSL which are only used by a small subset of the users. Often it is better to have two simple DSLs in two projects, than putting everything into one big DSL.

Back to the question: I don't think that evolution and design is mutual exclusive. Don't everybody design software systems incrementally these days.
Changing a textual DSL is not a big problem in practice. At least not as long as you have a decent test suite (which you should have no matter if you use DSLs or not). However, managing compatibility and versioning can become quite complicated, but again it's the same as with APIs.

Will the evolution in the language be severe enough to trash what has been done so far?

You can shoot your foot, if you want. But you don't need a DSL to do so.

Can you imagine yourself developing a complex C++ software system while C++ itself was still being designed and developed?

Can you imagine writing a complex C++ software system while the standard libraries are still being designed and developed?
I can imagine, but I am not keen on doing that. That is why we have things like versioning and that is also why people think about compatibility when enhancing public programming interfaces. Such interfaces can be a method signature, but it can also be a syntax of a language. It doesn't matter. If it breaks it breaks. And the designer has to be explicit about the contract of that interface as well as its life cycle.

Xtext - Using existing Ecore models

Maybe it's because it was not possible in the old oAW Xtext or because the wizard suggests a grammar which makes use of the "generated Ecore model"-feature. But somehow some people tend to think that you cannot use existing Ecore models with Xtext. YOU CAN!

The feature is described in the documentation, but maybe it got too few words (Which is not because we don't use it and don't want you to use it, but because it's so simple).

In the following I want to show you how to do this. You'll see that it is relatively straight forward.
Let's assume you have an existing Ecore model, describing a library of books (sounds familiar?;-)):

In order to create a textual syntax with Xtext you only need to create an Xtext project and change the proposed grammar to something like this :

So if you want to reuse an existing ecore model, it is just a matter of switching from a "generated Ecore"-statement to an "import"-statement. Note that as soon as you do this, the grammar is statically analyzed and you'll get all kinds of compiler errors and warnings for missing attributes and classes in the imported Ecore models (you can import multiple ecore models, of course).

Next, you run Xtext's generator and you'll get an EMF Resource writing and reading instances of your Ecore model to the textual syntax defined in aboves Xtext grammar.
In addition you'll get an editor for your new language :


You can, of course, still use any EMF-based editor, since it's just an EMF model:


Note, that you cannot only import multiple Ecore models but can even mix generated and imported Ecore models in the same grammar.

To sum it up: The feature we have in Xtext to derive an Ecore model from the grammar description, is a very convenient option to get you started. If you have one or more existing Ecore models, you won't want and need to use it. But if you start developing a new language this feature speeds up your turn-arounds dramatically.

Xtext - Tour Dates

I just realized that there's an invasion of Xtext presentations and demos about to overrun the world during the next couple of weeks. And all of them are free of charge!

Date	Country	City	Title	Presenter
2009/11/16	USA	New York	Modeling Day: Building DSLs with Xtext	Heiko Behrens
2009/11/16	USA	New York	Modeling Day: Converging Textual and Graphical Editors	Moritz Eysholdt
2009/11/17	France	Paris	Demo Camp: Building DSLs with Xtext	Sebastian Zarnekow
2009/11/17	Germany	Leipzig	Demo Camp: Und heute generier' ich	Alexander Nittka
2009/11/18	Germany	Hamburg	Java User Group: Building DSLs with Xtext	Sven Efftinge, Sebastian Zarnekow
2009/11/18	Canada	Toronto	Modeling Day: Building DSLs with Xtext	Heiko Behrens
2009/11/18	Canada	Toronto	Modeling Day: Converging Textual and Graphical Editors	Moritz Eysholdt
2009/11/23	Germany	Berlin	Demo Camp: Building DSLs with Xtext	Peter Friese
2009/11/26	Germany	Frankfurt	Demo Camp: Using Xtext in a large scale modeling and implementation project	Nicolai Busse
2009/11/26	Germany	Stuttgart	Demo Camp: Textual DSLs with Eclipse Xtext (and a little bit code code generation with Xpand)	Markus Völter
2009/11/30	Austria	Vienna	Demo Camp: Xtext	Michael Clay
2009/12/04	Germany	Hamburg	Demo Camp: Combining graphics and text in model editors	Jan Köhnlein
2009/12/10	Italy	Florence	Demo Camp: Xtext, a Textual Modeling Framework	Francesco Guidieri

Xtext in the automotive industry

At this year's Eclipse Summit Europe there was a very nice Xtext-related talk given by Sebastian Benz who is working at BMW Car IT.
The talk was about how they used Xtext in order to develop a textual syntax and corresponding IDE for the AUTOSAR standard. The name of the project is ARText and it is freely available to all AUTOSAR members as part of the Artop project.

One of the biggest challenges when implementing the AUTOSAR standard is to create scalable solutions. AUTOSAR projects might become very, very big and in order to work with such huge projects the tools need to perform very well. In the modeling world people usually strife for repository-based (i.e. database-based) solutions as soon as projects get really big. However, the folks at BMW Car IT wanted to develop AUTOSAR projects in a traditional text-based manner and given all the good experiences with tools like JDT or IntelliJ, it's clear that text-based IDEs can scale very well.

In order to implement the language they first had tried the old Xtext version from oAW, which was way too slow. Then, later, when the new TMF Xtext came around they gave it another try and saw that the performance had been improved significantly. Sebastian showed the following slide in his talk:
After that he explained how they have solved a couple of other problems, such as supporting different releases of the standard or making the language extendable. All the solutions looked very nice, they must have some very skilled people at BMW Car IT.

He compared working with the state-of-the-art commercial graphical modeling tool (What's the name of it?) and the ARText IDE and found that the use of ARText reduces development time by about 40%.

In the end there was this nice summary slide I don't want to withhold:

It was a very nice talk and of course a pleasure to see that Xtext is used by such smart people in such an interesting environment. And even better it seems that they've had as much fun using Xtext as we had and still have when developing (and using) it. :-)

Btw.: If you have other interesting applications of Xtext or Eclipse Modeling in general, please contact me. (Even if you don't want me to blog about it ;-))

Edited: Sebastian has recently posted a screencast showing ARText in action:

ARText in Action! from Sebastian Benz on Vimeo.