Friday, April 15, 2011

Dear Java killers,

I love to see all the languages for the JVM popping up on almost a daily basis. Language design is big fun and it's the ultimate thing for a developer to write programs in a language you've built your own. :-)

I especially like the languages which try out new ideas and don't take themselves too seriously, because thinking out of the box is the best you can do in order to push things forward.

However there are also a couple of languages who try hard to attract a certain audience: The Java community. The number of Java developers out there is huge (it had been about 9 million two years ago) so if you could make only a part of this crowd love and use your language it would be a huge  success.
Languages like Groovy, Scala, Clojure, Gosu, Fantom, or lately (but not yet existent) Ceylon and Kotlin more or less try to attract this group. Actually I shouldn't really count Clojure in, because it is so different (I like it though). But all the other languages seem to try to be a better Java for Java developers.

Here are my seven most important Dos and Don'ts you should consider when developing a language for the Java community:

1) Don't make unimportant changes

People are reluctant to change. You should leverage existing knowledge where possible. Changing keywords because you think they sound better might be good for your ego but not for the acceptance of your language. Most languages fall into this trap somehow. The best example is the recently announced Ceylon, which renames all kinds of Java keywords with little reason. For instance the keywords extends and implements are different. In an interview on InfoQ Gavin King explains why he made that decision:
"Java is irregular here. In a class definition, the list of one or more interfaces follows the keyword "implements". In an interface the list of one or more interfaces follows the keyword "extends", which by coincidence may also appear in a class definition where it means something completely different (it is followed by a single class). Ceylon is regular. Lists of types always follow "satisfies". The keyword "extends" is always followed by an expression that instantiates the superclass."
Given that over nine million people are used to it and absolutely understand the semantics, I'd consider this an unfortunate decision.

2) Static Typing


The Java folks are used to top-notch tooling based on static type information. I don't think that errors found by static typing are actually that important, but the tooling you can build on top of this information is. And again it is a matter of the audience you are trying to attract. I wouldn't try to come up with a new statically-typed Ruby, since that doesn't make much sense. I know Mirah and it's not bad. My point is that the Ruby community is not interested in static typing.
Static typing is an important part of the Java culture and the problems it causes can be reduced (see my other advices). Besides that Groovy generally looks a bit overgrown, I'd say the lack of static typing has been a problem for the success of Groovy. Obviously the Groovy folks have understood that and try to fix it with Groovy++.

3) Don't touch generics

Static type systems are complicated. I don't think that more than 1% of all Java developers have really understood generics. All this co- and contra variance, wildcards, raw types, upper and lower bounds and the involved conversion rules are super complicated.
However If asked most Java developers think they know generics and feel quite comfortable with it.
I'm glad that languages like Scala do important work in this area by trying different approaches and having more sophisticated type systems. That's important for the industry but it scares off and there are nine million people out there who feel comfortable with an existing, quite sound type system (aside from very few practically irrelevant problems).

After all generics in Java are rather sound, it's just that the problem they solve is essentially complex.

Also note that reified generics are a bad idea. Type arguments are static type information. They are lying at runtime and the situations where they are helpful are rare. This also breaks interoperability with other JVM languages btw.

4) Use Type Inference

The biggest problem with static typing and generics is that people think that this is the reason why Java code is so cluttered with redundant type signatures. That's not true! It is just a language design decision that one has to be very explicit (or ceremonial) about everything he/she writes down.

A modern statically-typed language should make heavy use of type inference. That is in most situations you give the developer the choice whether to put a type signature or not. Code needs to be concise and readable.

Ceremonial languages like Java don't esteem their users, they think they are stupid and therefore force them to do all kinds of repetitive things. I'm glad the way language or library users are treated has changed in recent years such that developers are given more responsibility for what they do. Only then you can expect quality. The ceremonial style is the biggest problem with Java which urgently needs to be fixed.

5) Care about Tool Support (IDE)

Your language won't be successful within the Java community if it lacks decent tool support. Scala ignored this for quite some time and are now trying to catch up a bit.
My advice is to start working on the IDE while designing the language, because the IDE is an important part of the overall user experience people have with your language. Many syntactical decisions can have a great influence on how the tooling works. A simple example:

In Java when declaring a local variable, Eclipse first proposes you with a type and then afterwards with a name proposal using the simple name of the field's type :

Name proposals in Eclipse
This is simply not possible if you switch the order of type and name like in Scala ( name : Type ), because you first have to type the name.

This is just a simple example, there are other problems hiding in the complexity of a language implementation. Make sure you have an experienced IDE developer in your team and care about the holistic user experience of your language. For Java folks the IDE is an important part of it.

6) Closures

There is no way your language can be successful without closures. Besides the ceremonial verbosity and inflexible syntax of Java the lack of closures is where Java suffers most.
Also for closures type inference and a concise syntax are essential.

7) Get rid of old unused concepts


Removing unnecessary concepts is also a good idea. Java was designed for the C/C++ crowd and was meant to be used in embedded devices and the web. It turned out that it is used quite differently today and most Java developers really don't care about bit shifting, break and continue statements and fall-through switches anymore. Remove them!

APPENDIX

A personal note to the Gavin Kings in this world:

Eclipse Xtext is a language development framework, which not only helps implementing parsers, linkers, compilers and interpreters but also a top-notch Eclipse IDE. You should use it for any kind of language development, because it boosts your productivity and is fun to use.

A general note on Xtext:

Of course there is no mass-market for programming languages. Although Xtext handles them quite well and is used for commercial programming language IDEs, the main focus is to allow people to easily define small, focussed domain-specific languages. The good news is that we have developed a concise, statically typed Java like expression language library supporting everything I mentioned before. And *you* can easily embed it into your DSL! See what you can do with it:


34 comments:

  1. I would add that you need some level of functional programming ability such as maps, folds, pattern matching, currying, and functions as first-class objects.

    ReplyDelete
  2. Thanks for this article. Quite amusing and straight to the point.

    ~Karsten

    ReplyDelete
  3. The example used for your fifth point is inconsistent with your fourth. If you're using type inference then the IDE won't know the type until your past the assignment operator i.e. val myVal = ...

    Other than that, I'd agree with all your points.

    ReplyDelete
  4. Basically a good post, but the other question is: Are people who are too dumb to understand even the Java type system really a target audience? I don't think so.

    ReplyDelete
  5. Java 7 does have some movement toward type inference in Project COIN.

    "There is no way your language can be successful without closures." Seems a bit extreme--after all, 9 million Java developers disagree :-)

    I would LOVE to have something Java DID NOT bring over from C -- unsigned bytes. Would make things like network programming so much simpler.

    ReplyDelete
  6. "(aside from very few practically irrelevant problems)."

    I'd hardly call the fact that it's essentially impossible to write any program of any reasonable complexity without explicit type checks and type casts "irrelevant."

    ReplyDelete
  7. Great post. Can you elaborate on why reified generics are a bad idea (besides Java interop problems).

    ReplyDelete
  8. Hi,

    Isn't xtext targeted for niche/business specific DSL's and not so much for large codebases of a generic language ?

    /max

    ReplyDelete
  9. Loved the article, but I was a little confused about the point you are making with generics. It hasn't been my experience that all nine million Java programmers have a solid understanding of generics. Sure, anyone can deal with array lists of Foo, but when it comes to wildcards or erasure, I think most people just close their eyes and hope that the moment will pass. Which it usually does--except for library writers, of course, who will need to knuckle under and master this stuff. So, to get to my question, how is different from Scala? In Scala, you don't worry about variance unless you write your own library with generics. Most people just use collections of Foo and the like. I don't see how for application programmers the difference between wildcards and variance declarations is even going to register.

    ReplyDelete
  10. Enjoyed this blog post immensely (I disagree with a couple of the arguments made though)

    ReplyDelete
  11. thanks for write up
    I would like to add that apart from ide/tools consideration the language shoul be easy to type also. for example i hate to type colon :

    moreover creating a new language should be an community process at least to define feature set (specs) and implementation. it should be developed bit by bit.

    new language must not be developed in isolation otherwise it will fail.

    ReplyDelete
  12. I've responded to this post wrt Gosu here:

    http://guidewiredevelopment.wordpress.com/2011/04/16/re-dear-java-killers/

    Cheers,
    Carson

    ReplyDelete
  13. What is irregular about the concept of "extending" the contract of an interface with a specialised interface, and "implementing" said contract with a class? Java is extremely regular in that respect.

    ReplyDelete
  14. > Eric Parnell said...
    >I would add that you need some level of functional >programming ability such as maps, folds, pattern >matching, currying, and functions as first-class objects.

    I have omitted libraries in my most just to make it shorter, but integrating with the JDK and at the same time being able to add features to existing types like collections is very important. The higher-order functions you mentioned should ideally be made available on java.lang.Iterable using a library approach.

    Pattern matching is nice but I think a type guarded switch statement is sufficient and much easier to understand (and to implement).

    Currying can be done via library and functions as objects IMHO isn't a good idea since objects of types with just one method fits much better with the existing JVM concepts and libraries.

    ReplyDelete
  15. > Anonymous said...
    > The example used for your fifth point is inconsistent
    > with your fourth. If you're using type inference then
    > the IDE won't know the type until your past the
    > assignment operator i.e. val myVal = ...

    That is only if you initialize the field.
    My example also works for function/method parameter declarations in Scala vs. Java.

    ReplyDelete
  16. Couple of things:

    3)
    Type erasure was the worst idea ever and as far I know it was done as a compromise for compatibility reasons. If this would be working, we would be much more expressive + we would not need to have 100 mil "advice" files spread across enterprise projects (see projects like Dozer,...)

    Type erasure also heavily contributed to all the edge cases, exceptions and complexity of generics in general.

    7)
    Fundamentals like bit operations are required: hashing, libraries that deal with binary and hi performance stuff and others rely on this... (unsigned data types are completely missing... leading to inefficient hacks on java side)

    I would like to know the reaction of ANTLR guys to fall trough switch removal proposals... :)

    Good post.

    Thanks

    ReplyDelete
  17. "Given that over nine million people are used to it and absolutely understand the semantics, I'd consider this an unfortunate decision."

    Just do as in C#, whether impl inheritance or interface inheritance, it's the same keyword (or character).

    "I wouldn't try to come up with a new statically-typed Ruby, since that doesn't make much sense."

    Well that's what Mirah is though.

    "Also note that reified generics are a bad idea."

    Beg to differ. Try to explain to a Java generics newbie, why he is not allowed to implement Comparable[T] and Comparable[K]. That's a duality of the type-system we just don't need.

    ReplyDelete
  18. > Dennis Doubleday said...
    > Java 7 does have some movement toward type
    > inference in Project COIN.

    Yes, it's slowly getting better. Very slowly.

    >
    > "There is no way your language can be successful
    > without closures." Seems a bit extreme--after all, 9
    > million Java developers disagree :-)

    I don't think they disagree.
    They might be a huge percentage of developers who don't know what closures are or at least don't know why they are so important.

    The rest simply agrees with me :-)

    ReplyDelete
  19. > Adam Rabung said...
    > Great post. Can you elaborate on why reified generics
    > are a bad idea (besides Java interop problems)

    It is mostly the interop problem.
    But they would also have a huge effect on a lot of other language concepts. You should for instance change the instanceof operator to allow generified types. And what about overloaded methods such as foo(List) {} foo(List) {} should that be allowed than?

    ReplyDelete
  20. > Isn't Xtext targeted for niche/business specific DSL's
    > and not so much for large codebases of a generic
    > language ?

    DSLs are small programming languages and Xtext is a framework for programming language development.

    It is mostly used for smaller languages but that is just because the market for general purpose languages is not that big. Xtext is used for grown-up language IDEs (also commercial ones).

    This one for instance is built with Xtext : http://goo.gl/Hsz79

    ReplyDelete
  21. > Cay Horstmann said...
    > Loved the article, but I was a little confused about
    > the point you are making with generics. It hasn't
    > been my experience that all nine million Java
    > programmers have a solid understanding of
    > generics. Sure, anyone can deal with array lists of
    > Foo, but when it comes to wildcards or erasure, I
    > think most people just close their eyes and hope
    > that the moment will pass. Which it usually does-

    Yes, they are somehow used to it.

    > -except for library writers, of course, who will
    > need to knuckle under and master this stuff. So, to
    > get to my question, how is different from Scala?

    Even if Scala's type system is maybe even better from a neutral point of view it is very different and that scare a lot of people off. Even though they don't fully understand the type system they use (Java's) they are used to it.

    I don't think that there are library users and library writers. Every programmer is both. It's just that some libraries are bad and some are good and some have few clients some have many.

    ReplyDelete
  22. > Anonymous said...
    > moreover creating a new language should be an
    > community process at least to define feature set
    > (specs) and implementation. it should be developed
    > bit by bit.
    >
    > new language must not be developed in isolation
    > otherwise it will fail.

    A programming language needs to be concise and coherent. It needs to be scoped, and should have a certain flavor and *feel*. A lot of very good languages have been initially designed by just one person (e.g. Scala, Java, Ruby, Python).

    ReplyDelete
  23. > Dusan said...
    > Couple of things:
    >
    > 3)
    > Type erasure was the worst idea ever and as far I
    > know it was done as a compromise for
    > compatibility reasons. If this would be working, we
    > would be much more expressive

    In what ways?

    > + we would not
    > need to have 100 mil "advice" files spread across
    > enterprise projects (see projects like Dozer,...)

    What are you doing there?

    >
    > Type erasure also heavily contributed to all the
    > edge cases, exceptions and complexity of generics
    > in general.

    Any examples?

    >
    > 7)
    > Fundamentals like bit operations are required:
    > hashing, libraries that deal with binary and hi
    > performance stuff and others rely on this...
    > (unsigned data types are completely missing...
    > leading to inefficient hacks on java side)
    >
    > I would like to know the reaction of ANTLR guys to
    > fall trough switch removal proposals... :)

    My last point should have been:
    Don't be everything to everyone. ;-)

    ReplyDelete
  24. Completely agree "timate thing for a developer to write programs in a language you've built your own", on that situation you can't blame other for any shortcomings of language though :)

    Javin
    Top 10 Java Serialization Interview Question

    ReplyDelete
  25. Q1: In what ways?

    Logically, you should be able to do more with more information available; things like

    - do X based on incoming not erased type; X={routing, (un)marshalling, ...}
    - new T (generic argument injected from outside)
    - remove useless generic arguments and casting from codebases with stuff similar to this
    http://www.vshank77.com/2009/12/builder-pattern-abused.html
    - ...

    Q2: What are you doing there?

    With 100 mil. "advice" files I meant many files across enterprise projects in whole industry, not in my project :)

    Q3: Any examples?

    Let's assume you have classes Animal, Dog & Cat...

    Would code bellow be a issue, or it is just a special case because of type erasure?

    public void go(Collection c) {
    c.add(new Cat());
    }

    ReplyDelete
  26. Blogger clearly prefers to erase the types as well while processing the entries :)

    c in last snippet is collection of animals.

    ReplyDelete
  27. Mmm, I tried to post my comment some days ago (would have been the first one) but Blogger rejected it. Looks like it is the OpenID identification which is broken... [EDIT: Google account doesn't work either. Will try just Name/URL...]

    I will just paste the comments below, even if some points have been already addressed by some comments above.


    "Don't make unimportant changes"
    I agree, I found that the search of originality in Ceylon was sounding strange. If Java was irregular, it was just a matter of making it more regular with existing vocabulary.

    "Static typing"
    I won't argue there! :-) But "I wouldn't try to come up with a new statically-typed Ruby, since that doesn't make much sense.": you don't know Mirah programming language, don't you? ;-)

    "I don't think that more than 1% of all Java developers have really understood generics. [...] most Java developers think they know generics and feel quite comfortable with it"
    So there is a difference between self perception of knowledge and real knowledge? :-)
    "After all generics in Java are rather sound"
    Indeed, they have been designed by the creator of Scala... :-) who has been frustrated by the limitations of the JVM.
    I think I disagree with your view about reified generics, but I am not competent enough to argue...

    "Use Type Inference"
    Not arguing there! ^_^'

    "Care about Tool Support"
    Idem, and the Scala community is aware of that as well. I just disagree with your analysis of the name proposal. Actually, I didn't even know the feature existed, and I doubt I will use it... Yes, I like the varName: typeName syntax, I find it clearer/more regular than the Java syntax and that's one point that annoys me in Ceylon.

    "Closures"
    Yes, we need them.

    "Get rid of old unused concepts"
    I agree, except we still do lot of bit shifting and masking, eg. in image processing, I don't see them superfluous.
    OK, maybe we don't need them at the syntax level, they can be hidden in a class, speed is rarely a concern there.

    ReplyDelete
  28. With respect to reified generics, the essential problem with them is that they encourage a style of programming that is not scalable: they encourage explicit type checks. One of the fundamentals of object-oriented programming is that it is necessary to separate interface from implementation to keep concerns properly decoupled, but as soon as you start doing explicit dispatch based upon concrete type, you couple whatever is doing the dispatching to the implementation of those types.

    There are, of course, situations (serialization being the prime example) where having the extra type information would be nice; here, Scala provides such a mechanism with implicit manifests. Local and optional reification can be useful, but I don't believe that it should be the default, as most of the time it would cause more problems than it solves.

    ReplyDelete
  29. > closures

    Java has closures since ... i don't know when -- inner classes create closures. What it's lacking is clean syntax for lambdas.

    ReplyDelete
  30. Wanderson Santos4/19/11, 8:44 PM

    With Groovy, you can have the benefits of static checking with IDEA joint-compiler. It's like bring the best of both worlds.

    ReplyDelete
  31. Unreified generics get a bad rap, but maybe undeservedly so. One thing that's important to note is that the only widely-used languages that do reify generic are the .net languages and C++. Do ML, OCaml, or Haskell have reified generics? Nope. And, yet, they are often seen as ideal implementations of universal polymorphism.

    In my opinion the shortcomings of Java's generics lie elsewhere: No coverage of primitive types, wildcards making life complicated for library users (as opposed to library designers), arrays sticking out like a sore thumb, to name just three. Neither of these problems is found in .net (nor, I should say, Scala), and that's why .net and Scala generics are generally perceived to be superior.

    ReplyDelete
  32. I agree with most of your post, which is why I don't understand the decision to add a "def" to function definitions in Xtend which only adds unnecessary verbosity.

    ReplyDelete
  33. Excellent points, most of them. I'd agree with Dusan about what you're calling "obsolete" features, though.

    As a guy who worked on ANTLR, and other code-generators, constructs like break, continue, case-fallthru make code generation much simpler.

    In day-to-day programming, I also regularly use break/continue to make loop processing much more clear, eliminating yet-another-boolean-flag-in-a-loop-condition growth.

    Keep up the good work on xtext! Awesome tool!

    ReplyDelete