Thursday, October 09, 2014

Eclipse Xtext goes IntelliJ IDEA

The Xtext framework has been in development for six years now. With a team size between five and nine people (we are now at nine) plus many contributors working on it, it has become a successful framework for development of domain specific languages and programming languages in general. From parsers, linkers and compilers to interpreters, type-checkers and full IDE-support, the framework covers everything a grown up language infrastructure needs. So far however, the IDE part has been Eclipse-only. This will change now.

The Eclipse based IDE support of Xtext is very mature. We will further improve robustness and performance and might add one or the other feature in the future, but the Eclipse-support really has come to a state where not much is missing. Personally I like Eclipse as a code base as well as a tool, but the reduced activity in the platform itself has resulted in a noticeable decline of excitement and we see more and more devs preferring IntelliJ IDEA for their Java development. Also there are many shops where the IDE is not set, so developers can choose. Those teams with mixed IDEs suffer from lacking DSL tool support for their non-Eclipse IDEs, of course.

Long story short, this summer we started working on support for IntelliJ IDEA. It shall work without requiring duplicate implementation for the main aspects. So as an Xtext user you can still work against the core Xtext APIs and any existing Xtext language is going to work in IDEA without further ado. A thin layer translates the information to IntelliJ’s concepts (PSI, etc.). Of course everything is composed with dependency injection, such that IntelliJ-specific optimisations and customisations are still possible. The work is coming along nicely and we hope that a first public beta version can be released early next year. Of course this includes IntelliJ IDEA support for Xtend as well!

Here’s a short screencast showing a bit what’s already working.

Xtext on IntelliJ IDEA from Xtext Team on Vimeo.

More platforms to come

Although the current focus is IntelliJ (and Eclipse of course) we think in the long term it is important for the Xtext framework to support other platforms as well. Next up will be serious support for web browsers, which is especially interesting for the many DSLs targeting non-developers (i.e. “business DSLs”).

Tuesday, April 15, 2014

XtextCON Update

XtextCON 2014 is still 40 days away, but I have to announce that ...

We are sold out

Initially I planned for 80 attendees. It turned out that was much too small, so we have added 30 more tickets as the event and venue can handle 110 people without problems. Today we have 112 registrations and since I want to make sure that everyone has an excellent time at this first XtextCON we closed registration today. I'm really sorry if you haven't booked yet but wanted to do so. We'll likely do an XtextCON next year again. 

New Sessions

We have added some really cool new sessions. All in all XtextCON will feature 18 Speakers and 28 sessions in two tracks. The added sessions are:
 - Xtext + Sirius : <3 (Cedric Brun)
 - Xtext + CDO - Does it blend? (Stefan Winkler)
 - Oomph - Automatically Provision a Project-Specifc IDE (Eike Stepper, Ed Merks)
 - 3D Modeling with Xtext (Martin Nilsson, Esa Ryh√§nen)
 - Handle Based Models for Xtext with Handly (Vladimir Piskarev)

Checkout the updated program.

Thursday, February 13, 2014

Announcing XtextCON 2014

I'm super stoked to announce the first edition of XtextCON, an annual conference on Eclipse Xtext.


With close to 6000 newsgroup items per year the Xtext community is one of the largest and most active at Eclipse.org. Although conferences such as EclipseCON feature many talks on Xtext, they usually cannot go into much detail as those conferences are targeting many different technologies. XtextCON should be an event where we can dive deeper. It is a community event and is meant to help users solving technical problems, get feedback on design ideas and learn about advanced topics.
XextCON is targeting beginners as well as advanced users. Beginners get the opportunity to attend a one-day workshop before the conference. So they get up to speed for the deeper talks at the main conference while those who are experienced with Xtext but feel like there knowledge is a bit outdated will get a quicker heads-up in the morning of day one.
The program covers many different aspects such as scoping, linking, how the index works, how to build type system, and many more. Also we have talks on graphical editing or generally how to improve the performance of your languages, code generators and IDEs. Btw: Thanks to the many attendees of the survey, it helped a lot to put the program together.

Xtext Clinic

In addition to the two main tracks, we will run a special Xtext-clinic on both days, where attendees get the opportunity to discuss their individual Xtext-related issues with committers and experts.

The Location

The event takes place in Kiel at the ATLANTIC hotel, which sits directly at the beautiful harbour of Kiel face to face with our office. Kiel is a small city in northern Germany at the baltic sea and May is the best time to visit it. The area around it is one of the most beautiful places at this time.
It could be a good idea to bring your family and spend the weekend here as well.
So, check out more details on the website and make sure to make a reservation early. Tickets are limited! Looking forward to see you there!








Monday, January 27, 2014

Quick Survey about XtextCON

Dear Xtext-Community!

we think it's time for an annual meet-up / conference ... what ever, solely on Xtext. The first one is planned to take place on May 26 and 27 and I'm in the process of building a website including the program. For that I need your help :-)

I have compiled a list of Xtext-related topics, and would like to ask you to give feedback about which you find interesting, and also which are missing. So, in case consider attending XtextCon, please take the survey below (it's just one question). Also please forward to you colleagues. the more feedback we have the better the program can be.


Also: If you are interested in presenting at XtextCON, please drop me a mail (xtextcon at itemis dot com). Thanks! :-)

Wednesday, October 16, 2013

Xtend & Android - Sneak Preview

In preparation of upcoming presentations for EclipseCon and W-JAX on that matter, I dived deeper into Android development and worked improving common programming idioms in that area. I'm contributing to Toby Kurien's Xtendroid and in this post I want to give you an idea of how Xtend can help developing Android applications.

Binding Android Layouts to Code

In Android user interfaces are mostly developed using Android's XML format for layouts. You can program them using Java but that's not what people do most of the time for various reasons. One of course being the verbosity of Java, but also the tools around the layout XML are quite helpful, especially when you have to deal with internationalization and need to support a wide range of devices.

Of course XML alone doesn't cut it which is why you have to bind the stuff declared in XML to Java code somehow in order to add listeners and other dynamic functionality. By default this is quite cumbersome as you have to deal with untyped code (cast) and a lot of integer constants. Some thirdparty frameworks try to reduce this problem by "injecting" the needed views reflectively, but that doesn't really help much. It's still not type safe and therefore error prone and you'll have to redeclare (i.e. duplicate) what you have declared in XML already.

With Xtend you can use an active annotation, which just points to the xml layout file. An active annoation is a macro annotation, which allows for participating in the compilation. This one will look into the referenced layout XML and declare local fields for any elements attributed with an '@id'. It also will derive an interface with all onclick methods declared in XML and make the current class implement it. As a result there is zero duplication and 100% type safety. The best is that the IDE knows about the changes and provides you with respective compilation problems, quick fixes, content assist and also the outline shows what has been derived from the XML layout file.

This is best shown in a small demo:

Xtend and Android - Binding XML Layout and Code from Xtext Team on Vimeo.

Wednesday, September 04, 2013

Better I18n in Java

Internationalization is not the most loved topic for Java developers. This is merely because it makes code less readable and introduces the need to work with property files and constants. As a result there is no static typing in that area and we are in need to get supported by tools, such as IDEs. In this blog post I want to show how this can be elegantly solved with Xtend.

I18n - The JDK Way

The JDK offers well working facilities for externalizing messages and formatting values such as dates and currencies in a proper localized way. For externalizing messages the class ResourceBundle is your friend, as it not only reads and parses property files but also supports flexible way of how property values are looked up based on Locales. The other interesting bit is the MessageFormat hierarchy which contains strategies to turn data types into localized strings.

The provided functionality is very powerful but enforces a cumbersome programming model. The standard way is to have a bunch of property files and whenever you need to get the contained values you use a ResourceBundle which has to be created like this:

ResourceBundle messages = ResourceBundle.getBundle("MyMessages", localeToUse);

Now you obtain the messages using the property keys, which are just strings. To minimize very likely spelling problems and it is good practice to have a declaring all the keys as constants. Then you only have to declare them once in the code and once for each property file.

public class MyMessagesKeys {
  public final static GREETING = "greeting";
  ...
}

Clients can access the values using the constants :

messages.getString(MyMessagesKeys.MY_KEY);

A common requirement and feature is to have variables in the messages which should provided by the program. These arguments can even be of different types, where the MessageFormat types come in. But again the Java compiler doesn't know if there are any arguments, nor how many and of which types they are. So you are back in untyped land and will learn about your errors only at runtime.

Let's see how we can improve the API for the JDK classes, such that we get rid of all the duplication and weakly typed access.

DRY With Xtend

Xtend is not a language which is meant to be used instead of Java, but is an extension to Java. You should use it in places where Java just doesn't cut it, like in this example.

In the following we're going to built a statically typed API to be used from Java by means of an active annotation. The basic idea is, that we make the compiler turn a constant containing a message into a Java facade and a corresponding properties file. Consider the following declaration of messages:

@Externalized class MyMessages {
  val GREETING = "Hello {0}!"
  val DATE_AND_LOCATION = "Today is {0,date} and you are in {1}"
}

This is all the developer should need to declare. Now we implement an active annotation, that turns the field declarations into static methods. It even adds typed arguments for the placeholders in the messages. That is the generated Java provides the following sigatures:

// Generated Java signatures

/**
 * Hello {0}!
 */
public static String GREETING(String arg0) { ... }

/**
 * Today is {0,date} and you are in {1}
 */
public static String DATE_AND_LOCATION(Date arg0, String arg1) { ... }

The actual values are written to a *.properties file and the implementation of the static methods use a ResourceBundler to look them up. So in case you need a locale-specific translation, you only need to add an additional *.properties file. Even that could be easily validated for miss-spelled or non declared keys. But I'll leave this feature as an exercise for other people :-).

Java clients can now access the messages in a statically typed way, and if you are working within Eclipse you even get the default message displayed when hovering over a property.

Building The @Externalized Annotation

Developing the active annotation is relatively easy. First you need to create a separate Java project. Although an active annotation is just library, it has to sit in an upstream project or jar, i.e. cannot be used within the same project. We would run into chicken 'n egg problems during compilation if that was allowed.

In that project create a new Xtend file and name it Externalized.xtend. We first declare the annotation and annotate it with @Active so the compiler is aware of it being an active annotation. We also have to provide a processor class which is executed by the compiler:

@Active(ExternalizedProcessor) annotation Externalized {}

class ExternalizedProcessor extends AbstractClassProcessor {

  override doTransform(MutableClassDeclaration clazz, extension TransformationContext ctx) {
    // To be implemented ...
  }

  override doGenerateCode(ClassDeclaration clazz, extension CodeGenerationContext ctx) {
    // To be implemented ...
  }

}

As you can see two callback methods have been overridden, which correspond to certain phases in the compiler. The first phase we participate in is doTransform in which the annotated classes can be mutated. The second phase doGenerateCode allows us to write to the file system. We will later write the properties files during this phase. The doGenerateCode phase has been introduced in version 2.4.3, so make sure you have updated accordingly if you want to use this.

Step 1: Transforming The Fields Into Methods

We want to turn all field declarations into static methods, so we first iterate over the declared fields of the given class.

override doTransform(MutableClassDeclaration clazz, extension TransformationContext ctx) {

  for (field : clazz.declaredFields) {

    // get the actual value of the field
    val initializer = field.initializerAsString

    // create a message format object for that value
    val msgFormat = try {
      new MessageFormat(initializer)
    } catch(IllegalArgumentException e) {
      field.initializer.addError("invalid format : " + e.message)
      new MessageFormat("")
    }

    // check the syntax and report back in case of problems
    val formats = msgFormat.formatsByArgumentIndex
    if(msgFormat.formats.length != formats.length) {
      field.initializer.addWarning('Unused placeholders. They should start at index 0.')
    }

    // add a method using the field's name
    clazz.addMethod(field.simpleName) [

      // return type is always string and the method is static
      returnType = string
      static = true

      // add parameters for the given arguments in the message format object
      formats.forEach [ format, idx |
        addParameter("arg" + idx,
          switch format {
            NumberFormat: primitiveInt
            DateFormat: Date.newTypeReference()
            default: string
          })
      ]
      
      // add the value as a comment, for documentation purpose
      docComment = initializer
      
      // add the actual body. It's generated Java code.
      val params = parameters
      body = [
        '''
          try {
            String msg = RESOURCE_BUNDLE.getString("«field.simpleName»");
            «IF formats.length > 0»
              msg = «toJavaCode(MessageFormat.newTypeReference)»
                    .format(msg,«params.map[simpleName].join(",")»);
            «ENDIF»
            return msg;
          } catch («toJavaCode(MissingResourceException.newTypeReference)» e) {
            // TODO error logging
            return "«initializer»";
          }
        ''']
    ]
  }

  // now that we have the methods we can just remove the fields, 
  // as they are no longer needed
  clazz.declaredFields.forEach[remove]

  // add a static ResourceBundle to be used from the methods we just created.
  clazz.addField("RESOURCE_BUNDLE") [
    static = true
    final = true
    type = ResourceBundle.newTypeReference
    initializer = ['''ResourceBundle.getBundle("«clazz.qualifiedName»")''']
  ]
}

Now we have successfully transformed an @Externalized class written in Xtend into a Java class containing corresponding static methods.

Step 2: Generating The Properties File

The last thing we need to do now is to write the values of the fields into the properties file. For this we participate in the code generation phase by overriding doGenerateCode. While during doTransform we got a mutable Java representation of the original sources, in this phase we get the unmodifiable original source passed in.

override doGenerateCode(ClassDeclaration clazz, extension CodeGenerationContext ctx) {
    
  // obtain the target folder for the given compilation unit
  val targetFolder = clazz.compilationUnit.filePath.targetFolder
    
  // compute the path for the properties file
  val file = targetFolder.append(clazz.qualifiedName.replace('.', '/') + ".properties")

  // write the contents to the file
  file.contents = '''
    «FOR field : clazz.declaredFields»
      «field.simpleName» = «field.initializerAsString»
    «ENDFOR»
  '''
}

Check Out The New Release

The described example is included in the "active annotations examples" coming with latest release.

Xtend and Xtext 2.4.3 Is Out!

We have released version 2.4.3 of Xtext and Xtend. As you can see by the minor increment it's a fully backwards compatible bugfix release. For this release more than 120 bugzillas have been solved, among them some very important enhancements:

Active Annotations : New Provisional File API

The still provisional active annotations API has gotten a new compiler phase and API which allows users to interact with the file system during compilation. This on the one hand is useful to generate signatures based on configuration files or generally project resources and on the other hand allows to update or fully generate such resources based on code. I have written a separate blog post describing an example.

Active Annotations : Polishing, Fixes and Constrain Enforcements

The API has been further refined and polished. Many contracts already described in the JavaDoc are now enforced such that you get early feedback when violating them.

Full Support For Unicode

Xtend now fully supports Unicode in identifiers. When translated to Java unicode characters get escaped only when the target encoding cannot display them. The same mechanism is used for string literals. We haven't pulled this up to Xbase for now, because it would have required clients to regenerate their language infrastructure. This is planned for the next release (2.5).

Content Assist

There were some important context where content assist didn't work. That has been fixed. Also some performance optimizations have been made. That said it's still not where we want it to be. So stay tuned.

Auto Edit

Severe performance problems with large documents have been identified and solved.

Rename Refactoring, Organize Imports, Formatting

Various enhancements have been made and bugs have been fixed in this area.

Xcore Support

There were some URI-related problems with using Xcore, which in some situations were a real showstopper. They are fixed now.

URI Handling

A problem that was introduced with 2.4.2, which made the build fail with URI comparison exception, has been fixed. As a result when generating an Xtext language the *.ecore and *.genmodel files are now generated to a different location.

... and many more

Check the full list.