This is part of a series
I started in March 2008 - you may want to go back and look at older parts if you're
new to this series.

From now on I'm not going to cover every little change - too much of it is repetitive, but I'll keep posting parts of this series to cover major changes. If you want to keep track of the changes "blow by blow", follow the project on GitHub. If you find something I've skipped over that you'd like to know more about, I'd be happy to expand on it in the comments, or perhaps cover it in the next part.

Extending the parser: The first bits of "real" syntax.

We'll keep the s-expression grammar separate from the grammar for the rest of the language, so in the BNF grammar below all productions refers to other parts of this grammar other than "sexp" which refer to the start production of the s-expression grammar. This grammar covers the first set of extension for this part:

Not exactly a tremendous difference, but it's a start, and it puts the basics in place to chip away at the parser bit by bit, so lets look at some code. It's fairly straightforward, so I'll just go through the "def" production - you can see the rest of this version here.

As for the s-expression parser it's a fairly straightforward translation of the BNF into a recursive descent parser, with some minor additional error handling. Note how optional rules simply translate into not checking if it returns nil or not, and "zero or more" rules translate into parsing into an array.

This isn't terribly exciting (and you can probably see now why I won't be writing up every single commit), so lets expand the grammar a bit more:

The new thing in the updated grammar is redefining "args" and adding "arglist". Those rules effectively says:

A set of args consist of optional whitespace excluding line feed (and carriage return) followed by either an arglist, or an arglist in parentheses. An arglist consists of of an optional "*" ("splat",equivalent to our :rest flag) followed by a name, whitespace excluding lf, and optionally ",", whitespace and arglist. In other words we use recursion allow a list of arguments (and just while writing this I realized that grammar is stupid, as it allows "*" in front of all arguments, which makes no sense - to be fixed...)

I'm a Norwegian technologist who has been living in London since 2000. I have a son, Tristan, born in 2009.

I've got development and management experience from a number of startups as well as more established companies, after co-founding my first company at age 19. I currently split my time between being the Technical Director of Nudge CRM Ltd, and running my consulting business, primarily related to development and devops. (More...)