C Extensions and the Garbage Collector
Life really has a way of catching up with you when you least expect it. Just last week I and my whole family were very nearly killed in a car crash while heading home from a vacation. Luckily though, all that happened was that my car is a total wreck, but I, my wife, my son, and my mother all walked away from the crash completely unscathed. The huge truck that hit us only managed to destroy my car. Also, the police basically gave a report that blames the entire incident on the truck driver, so with any luck the trucking company’s insurance will cover the repairs to my car.
Well, now that the shock of that event has sort of died down for me, I should get back to work (and well, they all say that trying to get back to normal life is a good way of getting over such an event), and so here I am writing where I left off last time, and figuring out that another interesting problem has come up with the plan I came up with for dealing with C extensions and giving them the power to call Arcueid bytecode while at the same time preserving the multitasking semantics of the Arcueid thread scheduler. And so here I am, tumbling down the rabbit hole, which winds up going much deeper than I anticipated. Several requirements, each of them seemingly very simple by themselves, have combined to increase the complexity of the Arcueid interpreter further than I expected:
- It should be possible to call Arcueid bytecode from within C code,
- Pre-emptive multitasking should be possible by means of a green thread scheduler: execution of bytecode invoked by a C extension should also be subject to this scheduler, and
- The garbage collector is a concurrent garbage collector scheduled itself as though it were a thread in the green thread scheduler.
The first requirement means that we have to have continuations that can restore execution to some point in C code. The second requirement means that these continuations must have essentially indefinite extent, and the third requirement means that any Arcueid values which are local to the C function and any C function that might have called it must also be examined by the garbage collector if the continuation it generated is seen in a garbage collection cycle.
Which of course makes this whole business messy in the extreme. Any continuations that contain saved stacks which are seen from the root set must now be inspected for Arcueid values, and should be marked accordingly by the garbage collector. This means that Arcueid values must then be recognizable as such from within a saved stack frame. I don’t know if this comes up as a problem in Guile, which also uses the stack copying technique I’m trying to implement, but trying to study the code of their interpreter is probably not such a good use of my time.
Well, the simplest way to do this would be to prefix a block of all Arcueid values within every C function with some fixed magic number which should be easy for my code to recognize. Follow it by the number of values within the block, and then end it with yet another magic number just to be sure. The garbage collector looks for the pattern of opening magic number, number of values, actual values, then closing magic number. We can set this up by defining a bunch of macros that must be used every time C code has at least one value in its local variables. Of course, not using these macros will cause problems if they are not used, and there’s a lot of old C code in the interpreter that needs to be amended. I’ll basically have to go over every function I ever wrote, but I guess that this was bound to happen sooner or later. Obviously this should be done for every function except those in the garbage collector itself.
Another possible way would be to save the difference between the address of a value and the bottom of the stack in a variable local to a thread. When a continuation is made from within C code, this variable gets cleared. Only trouble is, when a function returns, it must also remove the values it added from the list, and I don’t think I can write macros that do it efficiently.
Anyone out there with better ideas, or perhaps even thoughts on how better to approach this problem than this way, I’m listening.
Arcueid 弧 