Groovy, Scala, Java, Xtend - a stupid comparison

Disclaimer: This is probably the worst possible way to compare execution performance. I'm posting it nonetheless since the result was significant. Judge for yourself.

Today I took some time to check out the latest Eclipse plugins for both Groovy and Scala. I mainly wanted to know how nice the tooling is and played around with it a bit. I had been doing some Scala a couple of years ago, but since the Eclipse plugin not even existed those days it was a bit too painful for me to get used to it.

Groovy's tooling looks very good. There are just a few oddities but mainly I think it's well done... in contrast to the Scala plugin (I used 2.0.0 RC4). It holds a lot of surprises. For instance, with the Scala-IDE installed you'll get Scala-related template proposals in the Java and the Groovy editor. Still Scala the language would be my personal choice if I had to decide between the two. Anyway, this post is not about language features or Eclipse-plugin quality, but about what I experienced regarding runtime performance.

I knew Groovy's performance is bad. But I expected the performance problems to diminish, if I use a scenario using dynamic dispatch and mostly untyped objects. So I wrote the following brain dead code:

class Groovy {

  public static void main(String[] args) {
    def absoluteResult = []
    def before = System.currentTimeMillis()
    for (times in 1..10000) {
      def result = ['foo', 23, true]
      for (y in result) {
        absoluteResult += foo(y)
      }
    }
    println("Took : "+(System.currentTimeMillis() - before)
      +" ms, number of elements : "+absoluteResult.size);
  }

  static String foo(String s) {
    'String'
  }
  
  static String foo(Boolean s) {
    'Boolean'
  }
  
  static String foo(Integer s) {
    'Integer'
  }

}

It doesn't do anything interesting but leverages Groovy's built-in way of resolving overloaded methods using the runtime type. I thought it's fair to use this feature in a comparison, also because Xtend's dispatch methods have the same behavior. The result of the code above would be a list containing the string 'String', 'Boolean', and 'Integer' each 10.000 times.

When I run this code on my machine, it prints something like the following:

Took : 2025 ms, number of elements : 30000

A possible Xtend version

This is how I could achieve the same using Xtend:

class Xtend {
  def static void main(String[] args) {
    val absoluteResult = newArrayList()
    val before = System::currentTimeMillis()
    for (times : 1..10000) {
      val result = newArrayList('foo', 23, true)
      for (y : result) {
        absoluteResult += foo(y)
      }
    }
    println("Took : "+(System::currentTimeMillis() - before)
           +" ms, number of elements : "+absoluteResult.size)
  }

  def static dispatch foo(String s) {
    'String'
  }
  
  def static dispatch foo(Boolean s) {
    'Boolean'
  }

  def static dispatch foo(Integer s) {
    'Integer'
  }
}

On a first glance it looks very similar. Note, the dispatch keyword, which turns a set of overloaded methods into dynamically dispatched methods (i.e. the default Groovy behavior).
In contrast to the Groovy version this one is fully statically typed and as it turned out much faster. (Again as you see my measurement technique is everything but professional, but given such a big difference I dare to say it's faster).

It prints something like:

Took : 43 ms, number of elements : 30000

I also did a Scala version:

object Scala extends Application {
  val absoluteResult = MutableList[Any]()
  val before = System.currentTimeMillis()
  for (times <- 0 until 10000) {
    val result = List("foo", 23, true)
    for (y <- result) {
      absoluteResult += foo(y)
    }
  }
  println("Took : "+(System.currentTimeMillis() - before)
      +" ms, number of elements : "+absoluteResult.size)

  def foo(obj : Any) =
    obj match {
          case _:String => "String"
          case _:Boolean => "Boolean"
          case _:Integer => "Integer"
          case _ => throw new IllegalArgumentException()
    }
}

I got this kind of result:

Took : 130 ms, number of elements : 30000

And finally a Java version:

public class Java {
  public static void main(String[] args) {
    List<Object> absoluteResult = new ArrayList<Object>();
    long before = System.currentTimeMillis();
    for (int i=0; i < 10000; i++) {
      List<Object> result = new ArrayList<Object>();
      result.add("foo");
      result.add( 23);
      result.add(true);
      for (Object y : result) {
        absoluteResult.add(foo(y));
      }
    }
    System.out.println("Took : "+(System.currentTimeMillis() - before)
      +" ms, number of elements : "+absoluteResult.size());
  }

  static String foo(Object s) {
    if (s instanceof String) {
      return "String";
    } else if (s instanceof Boolean) {
      return "Boolean";
    } else if (s instanceof Integer) {
      return "Integer";
    } else {
      throw new IllegalArgumentException();
    }
  }
}

Which prints:

Took : 40 ms, number of elements : 30000