Mainly an internal DSL is about pushing the syntactic flexibility of a language to it's limits in order to write against an API in a domain-specific manner. What that means depends on the DSL but in most cases it means simplification of client code by implying context, allowing declarativity and avoiding DRY-violation.
Many people think, that it is the dynamic nature of a language which makes it suitable for internal DSLs. That is only partly true. More important than the possibility to do meta programming is to have a flexible syntax allowing to invoke functions with different syntaxes (omitting paranthesis, semicolons, etc.). Ruby and Scala are both very flexible in this sense.
Working on the new version of Xtext, I had an idea for what I think would be a cool language feature.
Introducing new syntax via libraries
When declaring a function, one typically only declares a name and what kind of parameters need to be passed in. Programming languages themselfs then have defined a (more or less flexible) generic way of how such functions can be invoked.
Example:
def foo(String x) : "foo"+x;which is invoked like this :
foo("bar")Imagine a language which alternatively allows to specify the concrete syntax of how such a function is invoked.:def myFunc : 'foo' x=ID : "foo"+x;Where the part between the first and the second colon introduces a new syntax (I've reused the Xtext syntax here) which can be used to invoke the function, so one could simply write:
foo barWhere 'foo' is a newly introduced keyword (limited to the static scope where the definition is visible of course) and bar is an identifier which is a builtin lexer token of type String.
Another example:
def Person :which would allow for expressions like:
('girl'|male?='boy') name=ID (lastName=ID)? ('from' city=ID)? :
new Person(male,name,lastName,city);
def greet :
p1=Person 'greets' 'the'? p2=Person :
p2.greetedBy = p1;
boy Sven from Kiel greets the girl Scarlett Johanssonand would effectively construct two instances of person and link them. :-)
Obviously this would be a tough thing to implement (parsers which dynamically change their behaviour) and it would also be problematic to avoid ambiguities with syntaxes imported by other libraries or introduced by the host language. I haven't spent much time on thinking it out yet. So maybe there are obvious show stoppers I overlooked or one of your neighbors has already developed such a language a decade ago?
11 Kommentare:
Smalltalk has a method syntax that was keyword: value pairs which could produce
Boy named: "Sven" from: "Kiel" greets: (Girl named: "Scarlett" lastName: "Johansson")
Where Boy/Girl are classes that derive from Person and have different defaults for "male, and there has a class method
named: x
which contructs a named Person, and allowing Person to have the methods:
lastName: x
from: x
each returning a "Person" and
greets: x
Returing a greeting. However, I don't think you can get rid of the parens because its not clear from the parsers standpoint whether the second 'named:' should bind to 'Girl' or the result of 'greets: Girl' without some analysis that would determine that a Greeting can't be 'named:'.
This is a neat language feature!
I think that Katahdin contains a fairly good implementation of mutable runtime syntax (and semantics):
http://www.chrisseaton.com/katahdin/
If I recall correctly, Katahdin uses a non-standard parsing technique (packrat parsing) to enable this.
You may be interested in term rewriting lanuages' implementations due to their very flexible and customizable syntax:
see Maude (http://maude.cs.uiuc.edu/) for example
Sven, that's a really cool idea!
I believe you are looking for Lisp's macros. The PLT Scheme community in particular has done much with syntax, e.g. Scheme with Classes, Mixins, and Traits.
If you're a Mac software developer or a die-hard fan, you probably know that WWDC is Apple's World Wide Developer Conference. Apple hosts sessions to teach about development tools and related topics and they also give labs to sit and talk to the people that designed the Apple's systems. http://www.infysolutions.com
You should really stop reinventing the wheel for DSL's and have a look at dynamic object systems combined with dynamic languages. Just do what I did: have a look at Smalltalk.
Dont be afraid of its age - since it is implemented using two simple concepts (objects and messages) its very powerfull and expressive.
I think this blog post from another user is a good summary...
Dave
thanks for the pointer. I know (and like) smalltalk but am not aware of a feature which let's you freely define introduce new syntax for expressions.
Did I miss it? Or are you referring to the usual way you send messages which has already mentioned by the first commenter? This is nice but different.
I agree that this idea is promising.
Has anything happened to this recently?
I noticed this post a bit late, by researching on ways to combine the power of XText with the flexibility of my internal DSL (using DLTK Ruby).
Do you think it would be possible to define a XText grammar for the basic, stable part of an internal DSL (+ get the great editor support&co), and still be able extend/experiment with the language for in the dynamic Ruby way that XText would "tolerate" (even if not fully support it).
Or are there ways to dynamically build up a grammar (which I could build up from JRuby)?
Cheers, Gergo
@Gergo:
As long as you have some syntax which separates DSL-code from Ruby-code, it should work. Although I haven't heard of this combination before.
Dynamic grammar creation is possible, but you need some code generation (e.g. to generate the parser). You could load those java classes dynamically using a URLClassLoader, but I'm not sure how beneficial all this would be in the end.
@Sven: Thanks for the reply.
I guess one of the problems is a general one of nesting multiple parsers- if the would allow to plug in another parser at a particular level of the "host" grammar, that would be great. Do you think Xtext could help here at all? I'd think its and flexible architecture might make this possible(not for free of course).
By dynamic grammar generation I was more thinking along the lines of dynamic EMF models(no code generation), which could also be used by the parser architecture.
I'd rather not include a Java compiler, and complex classloading stuff in my RCP - but, in the worst case I could accept that.
The benefit would be just the combination of the best of internal DSLs (easy to extend,"scaffold", get-out-of-the-jail, tons of libraries in the real language), and the external DSL (easy to learn - partly due to the focused tool support, good error checking).
Anyway, what's the story with your idea in the post? Is Xtext any closer to that now?
Thanks again,Gergo
Post a Comment