Wednesday, September 04, 2013

Better I18n in Java

Internationalization is not the most loved topic for Java developers. This is merely because it makes code less readable and introduces the need to work with property files and constants. As a result there is no static typing in that area and we are in need to get supported by tools, such as IDEs. In this blog post I want to show how this can be elegantly solved with Xtend.

I18n - The JDK Way

The JDK offers well working facilities for externalizing messages and formatting values such as dates and currencies in a proper localized way. For externalizing messages the class ResourceBundle is your friend, as it not only reads and parses property files but also supports flexible way of how property values are looked up based on Locales. The other interesting bit is the MessageFormat hierarchy which contains strategies to turn data types into localized strings.

The provided functionality is very powerful but enforces a cumbersome programming model. The standard way is to have a bunch of property files and whenever you need to get the contained values you use a ResourceBundle which has to be created like this:

ResourceBundle messages = ResourceBundle.getBundle("MyMessages", localeToUse);

Now you obtain the messages using the property keys, which are just strings. To minimize very likely spelling problems and it is good practice to have a declaring all the keys as constants. Then you only have to declare them once in the code and once for each property file.

public class MyMessagesKeys {
  public final static GREETING = "greeting";
  ...
}

Clients can access the values using the constants :

messages.getString(MyMessagesKeys.MY_KEY);

A common requirement and feature is to have variables in the messages which should provided by the program. These arguments can even be of different types, where the MessageFormat types come in. But again the Java compiler doesn't know if there are any arguments, nor how many and of which types they are. So you are back in untyped land and will learn about your errors only at runtime.

Let's see how we can improve the API for the JDK classes, such that we get rid of all the duplication and weakly typed access.

DRY With Xtend

Xtend is not a language which is meant to be used instead of Java, but is an extension to Java. You should use it in places where Java just doesn't cut it, like in this example.

In the following we're going to built a statically typed API to be used from Java by means of an active annotation. The basic idea is, that we make the compiler turn a constant containing a message into a Java facade and a corresponding properties file. Consider the following declaration of messages:

@Externalized class MyMessages {
  val GREETING = "Hello {0}!"
  val DATE_AND_LOCATION = "Today is {0,date} and you are in {1}"
}

This is all the developer should need to declare. Now we implement an active annotation, that turns the field declarations into static methods. It even adds typed arguments for the placeholders in the messages. That is the generated Java provides the following sigatures:

// Generated Java signatures

/**
 * Hello {0}!
 */
public static String GREETING(String arg0) { ... }

/**
 * Today is {0,date} and you are in {1}
 */
public static String DATE_AND_LOCATION(Date arg0, String arg1) { ... }

The actual values are written to a *.properties file and the implementation of the static methods use a ResourceBundler to look them up. So in case you need a locale-specific translation, you only need to add an additional *.properties file. Even that could be easily validated for miss-spelled or non declared keys. But I'll leave this feature as an exercise for other people :-).

Java clients can now access the messages in a statically typed way, and if you are working within Eclipse you even get the default message displayed when hovering over a property.

Building The @Externalized Annotation

Developing the active annotation is relatively easy. First you need to create a separate Java project. Although an active annotation is just library, it has to sit in an upstream project or jar, i.e. cannot be used within the same project. We would run into chicken 'n egg problems during compilation if that was allowed.

In that project create a new Xtend file and name it Externalized.xtend. We first declare the annotation and annotate it with @Active so the compiler is aware of it being an active annotation. We also have to provide a processor class which is executed by the compiler:

@Active(ExternalizedProcessor) annotation Externalized {}

class ExternalizedProcessor extends AbstractClassProcessor {

  override doTransform(MutableClassDeclaration clazz, extension TransformationContext ctx) {
    // To be implemented ...
  }

  override doGenerateCode(ClassDeclaration clazz, extension CodeGenerationContext ctx) {
    // To be implemented ...
  }

}

As you can see two callback methods have been overridden, which correspond to certain phases in the compiler. The first phase we participate in is doTransform in which the annotated classes can be mutated. The second phase doGenerateCode allows us to write to the file system. We will later write the properties files during this phase. The doGenerateCode phase has been introduced in version 2.4.3, so make sure you have updated accordingly if you want to use this.

Step 1: Transforming The Fields Into Methods

We want to turn all field declarations into static methods, so we first iterate over the declared fields of the given class.

override doTransform(MutableClassDeclaration clazz, extension TransformationContext ctx) {

  for (field : clazz.declaredFields) {

    // get the actual value of the field
    val initializer = field.initializerAsString

    // create a message format object for that value
    val msgFormat = try {
      new MessageFormat(initializer)
    } catch(IllegalArgumentException e) {
      field.initializer.addError("invalid format : " + e.message)
      new MessageFormat("")
    }

    // check the syntax and report back in case of problems
    val formats = msgFormat.formatsByArgumentIndex
    if(msgFormat.formats.length != formats.length) {
      field.initializer.addWarning('Unused placeholders. They should start at index 0.')
    }

    // add a method using the field's name
    clazz.addMethod(field.simpleName) [

      // return type is always string and the method is static
      returnType = string
      static = true

      // add parameters for the given arguments in the message format object
      formats.forEach [ format, idx |
        addParameter("arg" + idx,
          switch format {
            NumberFormat: primitiveInt
            DateFormat: Date.newTypeReference()
            default: string
          })
      ]
      
      // add the value as a comment, for documentation purpose
      docComment = initializer
      
      // add the actual body. It's generated Java code.
      val params = parameters
      body = [
        '''
          try {
            String msg = RESOURCE_BUNDLE.getString("«field.simpleName»");
            «IF formats.length > 0»
              msg = «toJavaCode(MessageFormat.newTypeReference)»
                    .format(msg,«params.map[simpleName].join(",")»);
            «ENDIF»
            return msg;
          } catch («toJavaCode(MissingResourceException.newTypeReference)» e) {
            // TODO error logging
            return "«initializer»";
          }
        ''']
    ]
  }

  // now that we have the methods we can just remove the fields, 
  // as they are no longer needed
  clazz.declaredFields.forEach[remove]

  // add a static ResourceBundle to be used from the methods we just created.
  clazz.addField("RESOURCE_BUNDLE") [
    static = true
    final = true
    type = ResourceBundle.newTypeReference
    initializer = ['''ResourceBundle.getBundle("«clazz.qualifiedName»")''']
  ]
}

Now we have successfully transformed an @Externalized class written in Xtend into a Java class containing corresponding static methods.

Step 2: Generating The Properties File

The last thing we need to do now is to write the values of the fields into the properties file. For this we participate in the code generation phase by overriding doGenerateCode. While during doTransform we got a mutable Java representation of the original sources, in this phase we get the unmodifiable original source passed in.

override doGenerateCode(ClassDeclaration clazz, extension CodeGenerationContext ctx) {
    
  // obtain the target folder for the given compilation unit
  val targetFolder = clazz.compilationUnit.filePath.targetFolder
    
  // compute the path for the properties file
  val file = targetFolder.append(clazz.qualifiedName.replace('.', '/') + ".properties")

  // write the contents to the file
  file.contents = '''
    «FOR field : clazz.declaredFields»
      «field.simpleName» = «field.initializerAsString»
    «ENDFOR»
  '''
}

Check Out The New Release

The described example is included in the "active annotations examples" coming with latest release.

7 comments:

  1. Hi Sven,

    nice idea, but I think your transformation example is missing a concept you have mentioned at the beginning of this blog. Because as far as I can see your generated code is missing the "localeToUse" and it's not really helpful on a web-server where the static fields (=your RESOURCE_BUNDLE) represent a problem for handling different languages.

    But I think this could be achieved as a kind of homework for developers ;-)

    ReplyDelete
  2. if an arg is more the one time used:
    if(msgFormat.formats.length != formats.length) { ...
    ->
    if(msgFormat.formats.length < formats.length) { ...

    ReplyDelete
  3. Hi Sven, I'm using Xtend for a while for explicit code generation and found it very exciting (BTW Xtext too!). It saves a lot of my time during development replacing another home brewed generator technology. However, I never tried to use its active annotations.
    Today I played around with the example coming from your article in the JavaMagazin (December 2013). The concept of active annotations works quite well for Xtend classes. Currently we are almost implementing in Java and I would like to provide this possibility of code generation via annotations to my colleagues. So, I'm wondering if the active annotations can also be used for Java code. I tried to apply it - quite naively - to a Java class put into the the xtend client example coming with the Xtend plugin, but it did not work. Does it not work at all or does is need a specific set-up, maybe?
    Denis - blog.grammarcraft.de

    ReplyDelete
    Replies
    1. Hi Denis, active annotations are a compiler feature of the Xtend compiler. The Java compiler doesn't know wabout it, therefore you get the effect only when annotating Xtend members.

      Delete
    2. Ok. I've understood. So, I will implement an ordinary Java annotation applying an Xtend code generation inside. When configuring this annotation for the Eclipse annotation processor it should work similar, generating additional code for an annotated class, isn't it?
      Of course, not so smooth and on the fly as with Xtend active annotations. :-)
      Neverthless, maybe the first step into Xtend...
      Happy New Year!

      Delete
  4. I recommend exploring the possibilities of https://poeditor.com as internationalization tool. It is very useful and open source-friendly. It has github integration, automatic translation and many other useful features.

    ReplyDelete
  5. I think it would. But I'm also just maybe Realizing That I Already have enough fun in my life! hip hop shop

    ReplyDelete