"We are almost at a stage where only double PhDs in math & comp.sc can qualify for Java compiler writers !"
(Srikanth S. Adayapalam (from Eclipse JDT) on the complexity of type inference in Java 8)
As Xtend implements Java's typesystem, we have to deal with exactly the same amount of complexity ... oh wait it's even better as Xtend has
- advanced type inference,
- sugared feature calls for properties,
- operator overloading,
- an implicit 'it' and
- extension methods (which are not only static as in C#)
- etc.
Write once, run ...
We are used to writing key parts of our software twice (at least!). The old type system did its job in the beginning but as we added more and more corner cases and complex type inference scenarios with lambdas and such it started to become too slow and too complex to be further enhanced. Good we have a solid test suite so we actually noticed the point at which even harmless looking changes broke existing tests... To make a long story short: Last year in April we decided to rewrite it from scratch.
The Lonesome Hero
Sebastian had some very promising ideas around a completely different architecture which would not only improve performance but also would allow everyone to better understand and debug the code again. We couldn't afford stopping all other developments and maintenance so one team member had to take the journey while the others were working on cool new features. Almost a year ago Sebastian started work on a complete rewrite. We discussed language semantics from time to time and such things but I rarely looked at the actual code. Most of the semantics were clear as it is based on the Java Language Specification and could be proved with the different Java compilers. So he was climbing this huge mountain all alone.
I know it was super risky and that it's not a good idea to rely such an important part of a project on just one person. If he had failed we had wasted a year. But I knew he would do it (And if he couldn't I don't know who else could).
It has been a long time and we even had to postpone the release which was initially planned for December. Finally by the end of January the new type system implementation was in a state that it could be activated and replace the old one. The switch was super smooth. It just worked.
Here is the corresponding bugzilla, which has been closed just yesterday. It blocked 95 downstream bugs! Most of them are now fixed, we just had to go through them and test, verify and close them. Not only that, the performance was already as good as the old one in the beginning without any further profiling and improvements. With just a little optimisation we were already able to improve the performance by 25%.
Also worth noting is, that the whole team was able to start working in the new code base after just two days of introduction. The code definitely is much easier to grasp and to debug yet covers much more of the essential complexity. Also we have an even more comprehensive test suite now. With over 20.000 unit tests you can confident that you don't accidentally break something. All tests run against the lower version bounds as well as the upper version bounds of our dependencies. There's no way software like this can be maintained and evolved without such a test suite.
The new type system is an incredible piece of engineering and a huge success. It's the basis for an awesome upcoming release of Xtend with many cool new features and enhancements.
Sebastian, I owe you one and so does the growing Xtend community!
