On rewriting from scratch
As one might surmise from reading my recent posts, as well as watching the experimental branch of Arcueid on github, I’m in the process of furiously rewriting most of Arcueid from scratch in order to address some of the most serious architectural deficiencies that I encountered when making the 0.0.x series. Joel Spolsky famously warned us that rewriting from scratch is something one should never do, but I do think that there are good reasons to do it. A page I found gives some good and bad reasons for the rewriting from scratch, and I’d like to think I’ve embarked on this task of rewriting nearly from scratch for many of the good reasons and not too many of the bad. The original 0.0.x code base suffered from several rather serious architectural problems that were fundamental, as I tried to get something that would more or less work:
- To avoid using the oddball foreign function interface that I wound up inventing that made use of ideas from Simon Tatham’s C coroutines, there was a fair amount of code that wound up making use of an alternate mechanism for continuations that involved copying the entire C stack before doing a setjmp and then making a longjmp, and this wound up getting used throughout most of the I/O functions. This did not play well with normal continuations and things got hairy. I was actually thinking of using this copying technique (along with local variables produced by alloca), but I had no idea how this would work with garbage collection, and so ran with a more streamlined version of the C coroutines trick.
- Boxed types used unions that grew in size as we layered stuff on top of it. This was a bad idea in retrospect but it made things simple in the beginning.
- There was no real tail call optimisation done at all. This is not absolutely required by Arc, but given how most Arc code is like Scheme in that it implements iteration by tail recursion, it is rather important.
There are several more issues with the current code like this that fixing them would involve rewriting nearly the entire code base. There are some places where using the old code was possible, and being lazy I tried to do that as much as I could, some other places where the original algorithms could be adapted to the underlying framework (semi-rewrite from scratch), and other places where this simply wasn’t possible and completely new code had to be written.
Anyhow, I think the core interpreter and the byte compiler are now complete, although the core built-in functions still aren’t complete and as such it isn’t yet ready for use (I knew that coerce was a complicated function, and rewriting it along with a complete suite of unit tests is proving to be a real chore). I hope we’ll have a new 0.1.0 release incorporating the new code before the end of the month.