I've ported events, sequencers and event-based communication primitives from Nemesis. It's a little bit messy at the moment (mostly because of mixing C and C++ concepts in one place), but I'm going to spend the autumn time on cleaning it up and finishing the dreaded needs_boot.dot dependencies to finally bootstrap some domains and perform communication between them. Obviously, the shortest term plan is timer interrupt, primitive kernel scheduler which activates domains and events to move domains between blocked and runnable queues.

There's some interesting theory behind using events as the main synchronization mechanism, described here in more detail.

For the vacation time I've printed some ANSA documents, which define architectural specifications for distributed computation systems and is very invaluable source of information for designing such systems. The full list of available ANSA documents can be found here. A System Designer's Introduction to ANSA is available here. Good reading.

I needed to quickly check how much of Nemesis support has to be ported over before I can start launching some basic domains.

I used a simple shell one-liner to extract NEEDS dependencies from the interface files. It's easy to do in Nemesis because of explicit NEEDS clause in each interface (would be nice to add this functionality to meddler, it also has the dependency information available).

Here's the shell one-liner:

echo "digraph {"; find . -name *.if -exec grep -H NEEDS {} \; | grep -v "\-\-" | 
  sed s/ *NEEDS //g | sed s@^\./@@ | sed s/\.if// | awk -F: {print $1, "->", $2}; echo "}"

This generated a huge graphic with all dependencies, which I then filtered a bit by removing unreferenced entities and culling iteration after iteration.

The resulting graphic is much smaller and additionally has a hand-crafted legend (green - leaf nodes, yellow - direct dependencies of DomainMgr and VP, my two interfaces of interest). This shows I need to work on about 10-12 interface implementations to be able to run domains.

And my ticket tracker of choice, bugs-everywhere now has an entry 7df/0fe 'Generate dot files with dependency information in meddler'. Time to sleep.

I've been working on toolchain building script, now at least on Macs it's possible to build a standalone toolchain for building Metta and you can download it and try to build it yourself. All necessary details are descibed on SourceCheckout wiki page. There is followup work to remove dependency on binutils and gcc (gcc will probably go first, then once lld is mature enough I could get rid of ld/gold).

Another update is about type system. The operations on type system are implemented now, I can successfully register type information and query it - some examples of that are in the recently released iso image R925. Next up is fixing some of naming context operations so I can actually create and operate hierarchical naming contexts.

Since I've decided to approach the system development from both low-level and high-level perspectives, one of the applications I have in mind for demo purposes is a little console tool which lets you activate various parts of the system, list available services and call operations on available interfaces.

Imagine a little tool that allows you to pick a video file, seek it to a particular time and play it frame by frame, then run face recognition on each frame and make a database of recognized faces. Being able to make such applications "mashup style" by just fiddling with text and pictures in the command line should enable people to create more and more interesting tools from the basic building blocks presented by the system.

This tool would need to inspect installed interfaces and types of the running components and be able to construct calls to these components directly from the command line. This requires introspection, or the ability to describe structure of objects in the system.

At the moment I'm working on the extension of meddler that allows to generate introspection data from the interface IDL. It is generally simple and then the next step would be to somehow register this information in the system when a new interface type is introduced. This is harder and requires some design effort. In the first approach of course only boot image is loaded, so registering types is very simple.

Next up is actual introspection interface - how to know what format a particular data type is and how to marshal/unmarshal it for the purpose of interchange and operations calls on interfaces.

See this little script for the possible demo storyboard.

A little sidetrack into the world of PCI probing and NE2000 network card emulation.

Wanted to have a taste of sending and receiving network packets inside my little OS, so I went and implemented PCI scanning (extremely simple) and NE2000 card driver (fairly simple too, their doc is quite good although misses some crucial points).

So, after some fiddling I was able to send a packet and receive it through the bochs virtual network card. I've then connected bochs to the host network card and stared at network packets for a while. Cool stuff.

Here's the screen dump of the sent and then received broadcast packet.

IRQ11 enabled.
Finished initializing NE2000 with MAC b0:c4:20:00:00:00.
Received irq: 0x0000000b
Packet transmitted.
Packet received.
Received packet with status 33 of length 68, next packet at 82

0x004f0064  ff ff ff ff ff ff 28 cf  da 00 99 f5 00 10 48 65  ......(.......He
0x004f0074  6c 6c 6f 20 6e 65 74 20  77 6f 72 6c 64 21 00 00  llo net world!..
0x004f0094  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................

Of course, the clang's implementation of setjmp is very generic and uses quite an abstraction of program state, which makes it hardly suitable for the ad-hoc local exception support I'm using. Since my requirements for setjmp were quite simple (just give me back my damn registers and stack frame), I went and implemented a very custom-tailored versions __sjljeh_setjmp and __sjljeh_longjmp which do just what I need.

With that stuff out of the way, my entire boot sequence now works and I can finally fiddle with more interesting stuff. Type system and introspection, here we go.

The reason for not booting was simple - Clang, seeing that target is Pentium 4 and above, optimized some memmoves into SSE operations. Bochs didn't expect that.

Now everything boots up until exceptions, at which point I believe the __builtin_longjmp primitive fails. Debugging it.

Anyway, I've done doing the craziest port of recent times - at the same time GCC to Clang and from waf 1.5 to 1.6.

Quite a bit of quirks to work around.

waf has changed a lot internally, and from occasional backtraces I still see that I'm using compatibility mode somewhere. Oh well, one day when I'm bored…

Clang is also full of quirks. First, I had to build a cross-gcc for it anyway, because otherwise it totally refuses to link or assemble anything. Second, the freestanding standard C headers are not quite finished it seems - stdint.h for example spits out about 20-30 warnings about redefined macros, so I had to disable -Werror for now just to get it to compile. Third, the generated code, obviously, doesn't run. I got only first couple functions of kickstart bootloader to work in bochs, after that it just GPFs. Now if it keeps raining tomorrow like today, I'll certainly will go and look what happens there, otherwise it might have to wait until next weekend (actually, in two weeks).

MMU initialisation

First off, we start by loading the needed modules - mmu_mod, frames_mod and heap_mod.

mmu_mod will allocate and map the first chunk of memory to use. It does so without knowing much about memory allocation - just takes a first fit chunk from the physical memory map, passed to us by the bootloader. Parts of this memory will be used by frames_mod and heap_mod as well, so we need a way to know how much memory they would need.

Read more

OMAP3 beagleboard.org # mmcinit
OMAP3 beagleboard.org # fatload mmc 0 0x82000000 kernel.bin
reading kernel.bin
640 bytes read
OMAP3 beagleboard.org # go 0x8200024c
## Starting application at 0x8200024C ...
hello, world!
## Application terminated, rc = 0x0