Vert.x and Xtend

Vert.x is a framework for asynchonous, scalable, concurrent applications. It’s conceptually very similar to node.js but runs on the JVM leveraging its native support for multiple threads.

Its programming model is based on a tell don't ask paradigm, that is you never wait for responses (and thereby block the current thread), but tell what to do as soon as a response is available by passing a handler. It also isolates state (even static one) so no locking is required.

Vert.x comes with special APIs for Java, JavaScript, Ruby and Groovy. They also plan to support Scala and Python.

So what about Eclipse's Xtend?

Xtend is not just another JVM language but an alternative way to write Java applications. It translates to comprehensible Java source code and unlike other JVM languages Xtend is 100% interoperable with Java and is designed to work great with existing Java APIs. That’s why no special API for Xtend is required.

The language is statically typed and features very advanced IDE support which tightly integrates with Java projects. A release of Xtend with lots of cool new features (e.g. debugging support) is just a couple of days away (June 27).

Some Examples

I have converted the Java examples shipped with vert.x to Xtend. The code is on github and is a fully working Eclipse project including working launch configurations to run and debug all the examples. If you are used to working in Eclipse, this is an easy way to try Vert.x : Just clone & import (using the "Import existing Projects into workspace" wizard) and you are done (Thanks to Doug for explaining the setup).

In the following I want to compare a couple of Java code snippets with their equivalent Xtend code, to demonstrate the expressiveness.

Https Example

The https example consists of two classes one for the client and one for the server. The client simple executed a get request on port 4443 and the server 'localhost' and prints the received data to System.out. Here's the Java code:

// Java
@Override
public void start() {
  vertx.createHttpClient()
    .setSSL(true)
    .setTrustAll(true)
    .setPort(4443)
    .setHost("localhost")
    .getNow("/", new Handler<HttpClientResponse>() {
      public void handle(HttpClientResponse response) {
        response.dataHandler(new Handler<Buffer>() {
          public void handle(Buffer data) {
            System.out.println(data);
          }
        });
      }
    });
  }

The Java API allows to configure the HttpClient using chained calls to setter methods. The last call is the get-request (getNow) where the path as well as a response handler is provided. Instead of blocking the current thread and waiting for the response you just tell what to do when the response is available. The Java API expects an instance of Handler<HttpClientResponse> for that.

Xtend allows to use the very same API with the following code:

// Xtend
override start() {
  vertx.createHttpClient => [
    SSL = true
    trustAll = true
    port = 4443
    host = "localhost"
    getNow("/") [
      dataHandler [ data |
        println( data )
      ]
    ]
  ]
}

First an http client is created, instead of using the chained setters (we could), we use the with-operator (=>) which allows us to use and initialize the http client from the left hand side within a lambda expression (the block in squared brackets). Within the lambda the HttpClient is bound to the implicit variable 'it', which like the self reference 'this' can be omitted when used as a receiver.
Next we call the setters using assignments, that is the expression

    SSL = true

is translated and equivalent to

    it.setSSL(true)

The interesting part is how we pass the response handler in Xtend :

    getNow("/") [
      dataHandler [ data |
        println( data )
      ]
    ]

The call to getNow(String, Handler<HttpClientResponse>) is done by passing just the string in the parentheses and a lambda expression right after the call. You need to understand two things here:

1. If the last argument of a feature call is a lambda expression it can be passed after the method call.
2. A lambda expression automatically coerces to the expected target type if it's an interface with just one method (which is a common idiom not only in vert.x).

Within the response handling a data handler is registered by calling setDataHandler(Handler<Buffer>) on the response object. Note that since this method gets just one lambda expression passed you don't have to write the parenthesis (in Xtend empty parenthesis are optional). Also note, that this time we gave the parameter a name (data) since I found it sightly more readable than :

    getNow("/") [
      dataHandler [
        println( it )
      ]
    ]

The Java implementation of the server prints out the header keys to the console and returns a tiny document:

// JAVA
public void start() {
  vertx.createHttpServer()
    .setSSL(true)
    .setKeyStorePath("server-keystore.jks")
    .setKeyStorePassword("wibble")
    .requestHandler(new Handler<HttpServerRequest>() {
      public void handle(HttpServerRequest req) {
        System.out.println("Got request: " + req.uri);
        System.out.println("Headers are: ");
        for (String key : req.headers().keySet()) {
          System.out.println(key + ":" + req.headers().get(key));
        }
        req.response.headers().put("Content-Type", "text/html; charset=UTF-8");
        req.response.setChunked(true);
        req.response.write("<html><body><h1>Hello from vert.x!</h1></body></html>", "UTF-8").end();
      }
    }).listen(4443);
  }

Translated to Xtend and applying the same patterns we used for the client this looks like the following:

// Xtend
override start() {
  vertx.createHttpServer => [
    SSL = true
    keyStorePath = "server-keystore.jks"
    keyStorePassword = "wibble"
    requestHandler [
      println("Got request: " + uri)
      println("Headers are: ")
      for (it : headers.entrySet) {
        println(key + ":" + value)
      }
      response.headers.put("Content-Type", "text/html; charset=UTF-8")
      response.chunked = true
      response.write('''
        <html>
          <body>
            <h1>Hello from vert.x!</h1>
          </body>
        </html>
        ''', "UTF-8").end
      ]
    listen(4443) 
  ]
}

There are two things worth mentioning in addition to what we've discussed in the first example:

• You can use the variable name it every where. For instance in a for loop like in the code snippet above.
  
• Xtend supports multiline string literals. The common literals using single quote or double quote can be multiline as well, but by using triple single quotes you get smart whitespace handling. That is the indentation before the tag will be pruned for all lines, i.e. the result will be well formatted. Also the triple quotes allow for having interpolation expressions, which are not used in this example.

Can we do even more?

Although the standard Java API already works very well there's room for special Xtend API. Vert.x for instance communicates with JSON objects a lot, which you might want to declare easily. Xtend doesn't support native JSON syntax, but you can design powerful APIs using a combination of extension methods and operator overloading. I added just three methods to be able to construct instances of vert.x's JsonObjects like this:

class JsonExample extends Verticle {
  
  override start() {
    val eb = vertx.eventBus
    val pa = 'vertx.mongopersistor'
    val albums = _(
      _(
        'artist'- 'The Wurzels',
        'genre'- 'Scrumpy and Western',
        'title'- 'I Am A Cider Drinker',
        'price'- 0.99,
        'categories'- _('action', 'comedy')
      ),
      _(
        'artist'- 'Vanilla Ice',
        'genre'- 'Hip Hop',
        'title'- 'Ice Ice Baby',
        'price'- 0.01
      ),
      _(
        'artist'- 'Ena Baga',
        'genre'- 'Easy Listening',
        'title'- 'The Happy Hammond',
        'price'- 0.50
      ),
      _(
        'artist'- 'The Tweets',
        'genre'- 'Bird related songs',
        'title'- 'The Birdy Song',
        'price'- 1.20
      )
    )
    
    // First delete everything
    eb.send(pa, _('action'- 'delete', 'collection'- 'albums', 'matcher'- _()))
    eb.send(pa, _('action'- 'delete', 'collection'- 'users', 'matcher'- _()))
    
    // Insert albums - in real life price would probably be 
    // stored in a different collection, but, hey, this is a demo.
    
    for (album : albums) {
      eb.send(pa, _(
        'action'- 'save',
        'collection'- 'albums',
        'document'- album
      ))
    }
    
    // And a user
    eb.send(pa, _(
      'action'- 'save',
      'collection'- 'users',
      'document'- _(
        'firstname'- 'Tim',
        'lastname'- 'Fox',
        'email'- 'tim@localhost.com',
        'username'- 'tim',
        'password'- 'password'
      )
    ))
  }  
}

These are the three extension methods:

  def static <T> Pair<String,T> operator_minus(String key, T value) {
    new Pair(key, value)
  }
  
  def static JsonArray _(Object ... entries) {
    val result = new JsonArray
    for (e : entries) {
      switch e {
        String : result.addString(e)
        Number : result.addNumber(e)
        Boolean : result.addBoolean(e)
        JsonObject : result.addObject(e)
        JsonArray : result.addArray(e)
      }
    }
    return result
  }
  
  def static JsonObject _(Pair<String, ?> ... entries) {
    val result = new JsonObject
    for (e : entries) {
      switch value : e.value {
        String : result.putString(e.key, value)
        Number : result.putNumber(e.key, value)
        Boolean : result.putBoolean(e.key, value)
        JsonObject : result.putObject(e.key, value)
        JsonArray : result.putArray(e.key, value)
      }
    }
    return result
  }

Xtend - The Movies Example

The upcoming release of the new version of Xtend is just a couple of days away (June 27). Although the technical (OSGi) version is 2.3, for me it's really more like a 1.0 release. It now has everything you need to write beautiful Java programs, like the little movies example we have been using in some workshops we did recently.

The movies example is also included in the example project that will be shipped with the Eclipse plug-in and is about reading a file of movie data in and doing some queries on it.

The Data

The movie database is a plain text file (data.csv) with data sets describing movies. Here's an example data set:

Naked Lunch  1991  6.9  16578  Biography  Comedy  Drama  Fantasy

The values are separated by two spaces. The columns are :

• title
• year
• rating
• numberOfVotes
• categories (where any number of categories is allowed)

Let's start by declaring a data type Movie reflecting the data set:

@Data class Movie {
  String title
  int year
  double rating
  long numberOfVotes
  Set categories 
}

The @Data annotation will turn this class into a value object, that is the compiler will create

• a getter-method for each field,
• a hashCode()/equals() implementation,
• implementation of Object.toString() and
• a constructor accepting values for all fields in the declared order.

Parsing The Data

Let's now define another class which reads the text file into a list of movies so that we can do some analysis on the data. We will access the data from within a JUnit test, so simply initializing a field is appropriate:

import java.io.FileReader
import java.util.Set
import static extension com.google.common.io.CharStreams.*

class Movies {

  val movies = new FileReader('data.csv').readLines.map[ line |
    val segments = line.split('  ').iterator
    return new Movie(
      segments.next, 
      Integer::parseInt(segments.next), 
      Double::parseDouble(segments.next), 
      Long::parseLong(segments.next), 
      segments.toSet
    )
  ]
}

The field's type (List) is inferred from the expression on the right hand-side and we want the field to be final, so we declare it as a value using the keyword code val.

The initialization on the right hand side first creates a fresh instance of java.io.FileReader. Then the method readLines() is invoked on it. But if you have a look at FileReader you won't find such a method. It's in fact a static method coming from Google Guava's CharStream and is imported as an extension :

import static extension com.google.common.io.CharStreams.*

CharStream.readLines(Reader) returns a List on which we call another extension method called map. That one is defined in Xtend's runtime and is always imported and therefore automatically available on all lists. The map-method expects a function as the parameter. It invokes that function for each value in the list and returns a list containing the results of the function invocations.

Function objects are created using lambda expression (the code in squared brackets). Within the lambda we process a single line from the text file and turn it into a movie by splitting the string using the separator and calling iterator() on the result. As you might know java.lang.String.split(String) returns a string array (String[]). But as Xtend auto-converts arrays to lists when needed, we can call iterator() on it.

val segments = line.split('  ').iterator

Now we use the iterator to create an instance of Movie:

return new Movie (
  segments.next, 
  Integer::parseInt(segments.next), 
  Double::parseDouble(segments.next), 
  Long::parseLong(segments.next), 
  segments.toSet
)

Answering Some Questions

Now that we've the text file turned into a List, we are ready to do some queries on it. We use JUnit to make the individual expressions executable.

Question 1: How Many Action Movies Are Contained?:

@Test def void numberOfActionMovies() {
  assertEquals(828, 
    movies.filter[categories.contains('Action')].size)
}

It's using the extension method filter to filter the movies. The lambda expression checks whether the current movie's categories contains the entry 'Action'. Note that unlike the lambda we used to turn the lines in the file into movies, we haven't declared a parameter name this time. We could have given the parameter an explicit name 'movie' by writing the following:

assertEquals(828, movies.filter[movie | movie.categories.contains('Action')].size)

But if we leave out the name and the vertical bar the variable is automatically named 'it' which (like this) is an implicit variable. That's why we can either write

assertEquals(828, movies.filter[it.categories.contains('Action')].size)

or just

assertEquals(828, movies.filter[categories.contains('Action')].size)

Lastly we call size on the resulting iterable, which again is an extension method (java.lang.Iterable doesn't define such a method).

Question 2: What's The Year The Best Movie From The 80ies Was Released.

@Test def void yearOfBestMovieFrom80ies() {
  assertEquals(1989, 
    movies.filter[(1980..1989).contains(year)].sortBy[rating].last.year)
}

Here we filter out all movies where the year is not included in the range from 1980 to 1989 (the 80ies). The range-operator (..) again is an extension defined for two ints and returns an instance of org.eclipse.xtext.xbase.lib.IntegerRange.

The resulting iterable then is sorted by the rating of the movies. Since it's sorted in ascending order, we take the last movie from the list and return its year.

We could have sorted descending and take the head of the list as well (note the minus sign):

movies.filter[(1980..1989).contains(year)].sortBy[-rating].head.year

Btw. the calls to movie.year as well as movie.categories in the previous example of course access the corresponding getter methods, which were generated because of the @Data annotation.

Question 3: The Sum Of All Votes Of The Top Two Movies

@Test def void sumOfVotesOfTop2() {
  assertEquals(47_229, 
    movies.sortBy[-rating].take(2).map[numberOfVotes].reduce[a, b| a + b])
}

First the movies are sorted by rating, then we take the best two. Next the list of movies is turned into a list of their numberOfVotes using the map function. Now we have a List which can be reduced to a single Integer by adding the values.

Distance / Time = Speed

I blogged about introducing a Distance datatype using Xtend some time ago, which allows you to write the following:

@Test def distances() {

  assertEquals(15.km, 13.km + 2_000.m)

  assertEquals(30.km, (13.km + 2_000.m) * 2)

}

We've enhanced this example a bit and put it into the Xtend tutorial project, which you can materialize into your workspace using Eclipse's example wizard.

We added two more datatypes Time and Speed, so that you now can write things like this:

@Test def speed() {

  assertEquals(42.km/h, (40_000.m + 2.km) / 60.min)

}

The datatype Time is defined similarly to the Distance type. It defines operators and some static methods, which when imported as extension method allows you to write things like 60.min etc.
In order to be able to divide some Distance by some Time (i.e. create Speed), you need to define a static operator like so :

@Data class Speed {

  BigDecimal mmPerMsec

  def static operator_divide(Distance d, Time t) {

    new Speed(d.mm / t.msec)

  }

}

Just import this as an extension method and you are able to write :
  (40_000.m + 2.km) / 60.min

Finally we also want to write Speed literals like 42.km/h.

Can you guess how this is done?

(If not, it's in the example :-)
Eclipse updatesite: http://download.eclipse.org/modeling/tmf/xtext/updates/composite/latest/ )