05 April 2010

Implications of ECMA-335 and Partition I, 12.1.2 and 12.

As I’ve been spending quite some time this weekend to refactor parts of the MOSA compiler and fixing things small and large. I’ve stumbled once again over our memory model. I was refactoring our internal representation in order to make load and store operations explicit and broke almost all of our tests at once. Fixing them was pretty easy, except for the smaller types... Section 12.1.2 states:

„Loading from 1- or 2-byte locations (arguments, locals, fields, statics, pointers) expands to 4-byte values.“

Ouch. We’ve gone through a lot of trouble to ensure correct arithmetics on all types and have been badly missing the point: All smaller integral types are handled at 4 bytes in size on the evaluation stack.

Next step was to change the CIL load instructions to correctly reflect this fact and fortunately we already had the appropriate instructions in the IR. So the current state of work is that most of our tests are passing again, but not all yet. Then I started wondering about the floating point specification. Looking at the section for floating point values (12.1.3), it states:

„The supported storage sizes are float32 and float64. Everywhere else (on the evaluation stack, as arguments, as return types, and as local variables) floating-point numbers are represented using an internal floating-point type. In each such instance, the nominal type of the variable or expression is either float32or float64, but its value can be represented internally with additional range and/or precision. The size of the internal floating- point representation is implementation-dependent, can vary, and shall have precision at least as great as that of the variable or expression being represented. An implicit widening conversion to the internal representation from float32 or float64 is performed when those types are loaded from storage. The internal representation is typically the native size for the hardware, or as required for efficient implementation of an operation.“

So for floating point types we have exactly one stack type F, but the implementation is free to choose the precision of its operations as long as it is at least as large as the storage size of the floating point type. Since we’ve spent a great deal of time on single precision arithmetics, I’m inclined to keep the reduced precision operations there. Any opinions?

I’ll continue fixing this in the next couple of days.

04 April 2010

Why the Switch Fixture has crashed on I2/U2 and others...

The tests are run by .NET calling through a function pointer delegate using the stdcall calling convention. This calling convention is similar to cdecl. One of the similarities is that the EBX register must be saved by the callee and restored before it returns to the caller. We didn’t do that and thus corrupted the state of the .NET runtime on Windows. Bug fixed and commit following soon.

27 March 2010

The next step: Virtual methods and method tables

After the static object allocation, I’ve finished the next step for MOSA. The compiler now emits mtable (virtual method tables) records for compiled types and is able to properly call virtual functions. The test to check these is in CallVirtFixture. I’ll add a couple more tests there to check for proper hiding, base class calls and other things - hopefully the current code should handle all of those cases sufficiently well.

In order to accomplish this I’ve had to add a fake System.Object implementation to the existing tests, as those classes wouldn’t compile anymore - the linker couldn’t create the vtable for them due to the 4 virtual methods every object inherits from System.Object: ToString, GetHashCode, Equals and Finalize.

The good thing about this is of course we can now use virtual functions and use overrides to do OO-kernels, the down side of course is: Every kernel has to provide at least a fake implementation of System.Object.

I’ve added a fake System.Object to the existing HelloWorld kernel.

Let’s see what Phil and Simon can come up with in Hello World, now that this is out of the way.

15 March 2010

Making good on a promise I've made a long time ago.

After improving our test situation on the weekend, I’ve started on making good a promise I’ve given a long time ago. The promise was: Allocate static objects at compile time for core kernel services.

The issue with writing a managed operating system or any operating system is memory management and moving to the OS way of working at boot time. The core problem is that there are assumptions about objects, which can’t be met easily as memory management is being initialized on the CPU(s) the OS will run on. MOSA is facing this problem too, with one addition: Writing non-OO code in an OO language feels broken.

Classical operating systems solve this by having a reduced set of services while booting and initializing the OS services later in the OS specific fashion. We could’ve done this too, but why go for common ground if there’s new to explore.

So what does this feature do? This feature detects all dynamic memory allocations happening in static constructors and allocates memory for the allocated objects at compile time in the bss segment of an executable. It replaces the call to new with a load of the address of the data segment location, making the position of the object fixed in memory relative to its load address. This allows core OS services to be written using C# classes right from the start and allows them to be used as such.

There are some limitations though: The allocated object must be fixed in size, it must not have a complex structure and the field used to store the object must have the exact same type as used for the new operator. No casts allowed.

There’re probably further limits to this feature, but I haven’t figured them out yet. It will certainly be interesting to explore our HelloWorld kernel with OO-features using the new MOSA compiler.

Oh and before I forget: The feature must be turned on explicitly on the command line. Use --enable-static-alloc (or the shorter --sa) to enable it.

I’ve update the HelloWorld projects CMOS and Boot classes to take advantage of this mechanism.

11 March 2010

Changes I've made to MOSA to support generics, Part I

Over the past couple of days I’ve made changes to MOSA to support generics. The first and primary change was to support a scheduled compilation model.

Let me explain this: Up to these changes the MOSA compiler would just scan an assembly, locate all types and compile each method contained in them. The only exceptions to this rule were native methods, generic types and generic methods.

The revised compilation scheduler stage

However skipping generics doesn’t work anymore - you want those generic types and methods compiled too. The issue you face is how to compile these methods and types without knowing the usage. So I basically kept the current compilation scheduler, but added the capability for the pipeline to schedule additional types and methods in the scheduler. In order to do this, there’s a new assembly compilation stage: The ICompilationSchedulerStage. This stage performs the type lookups that used to be done by the MethodCompilerBuilderStage.

The compilation scheduler now maintains a schedule of methods and types to compile and executes these in order. Once all scheduled methods and types are compiled, the entire assembly including all of its generic usages has been compiled.

MOSA: Generic method compilation, first test succeeded

After about 3 hours patching and fixing the last issues I finally completed the first successful test of generics in MOSA. The following C# fragment compiles successfully and passes all tests:

static class Test
        private static T GenericMethod<T>(T value)
                return value;

public static bool TestCallGenericMethodWith(int value)
                return value == GenericMethod(value);

It may not look like much, but this test is the ground for all other generic arguments and has moved forward not only the compiler, but also the assembly loader, the MOSA runtime and some other additions like cleaner error messages from mosacl.

I’ll finish this test case tomorrow and push my changes to Github, I’ll probably need some help with that from fellow MOSA contributors.

Good night.

09 March 2010

mosacl is compiling very simple generic types

Let mosacl speak for itself:

Mini:Bin Michael$ mono --debug mosacl.exe -a x86 -o MQT.exe -f PE Mosa.QuickTest.exe
MOSA AOT Compiler, Version 0.6 'Tanigawa'
Copyright 2009 by the MOSA Project. Licensed under the New BSD License.
Copyright 2008 by Novell. NDesk.Options is released under the MIT/X11 license.

Output file: MQT.exe
Input file(s): /Users/Michael/Projects/MOSA/MOSA-Project/Bin/Mosa.QuickTest.exe
Architecture: Mosa.Platforms.x86.Architecture
Binary format: Portable Executable File Linker
Boot format: Not bootable
Is executable: True

Compiling ...
Scheduling type Mosa.QuickTest.App for compilation.
Compiling type Mosa.QuickTest.App
Compiling method Main
Loaded generic type Mosa.QuickTest.Generic`1
Scheduling type Mosa.QuickTest.Generic`1<System.Int32> for compilation.
Compiling type Mosa.QuickTest.Generic`1<System.Int32>
Compiling method .ctor
Compiling method get_Value

Unresolved symbols.

Mini:Bin Michael$

07 March 2010

Signs of generic types

After hacking MOSA two days again and trying to figure out what I was thinking way back, here’s one of the first runs with generic types:

        mini:Bin Michael$ mono --debug mosacl.exe -o MQT.exe --Architecture=x86 --format=PE Mosa.QuickTest.exe
        MOSA AOT Compiler, Version 0.6 'Tanigawa'
        Copyright 2009 by the MOSA Project. Licensed under the New BSD License.
        Copyright 2008 by Novell. NDesk.Options is released under the MIT/X11 license.

        Output file: MQT.exe
        Input file(s): /Users/Michael/Projects/MOSA/MOSA-Project/Bin/Mosa.QuickTest.exe
        Architecture: Mosa.Platforms.x86.Architecture
        Binary format: Portable Executable File Linker
        Boot format: Not bootable
        Is executable: True

        Compiling ...
        Scheduling type Mosa.QuickTest.App for compilation.
        Compiling type Mosa.QuickTest.App
        Compiling method Main
        Scheduling type System.Collections.Generic.List`1<System.Int32> for compilation.
        Scheduling type Mosa.QuickTest.Generic`1<System.String> for compilation.
        Compiling type System.Collections.Generic.List`1<System.Int32>
        Compiling method .ctor

Unfortunately this is as far as it gets right now, but I’m sure I’ll find the reason for this soon.

I had to rewrite a whole bunch of stuff to get this far, including the way types are scheduled for compilation by mosacl. Surprising is that generics can be AOT’d under certain circumstances.

06 March 2010

Solving the pkg-config and Mono 3.5 Profile mysteries on MacOS X 10.6 (Mono 2.6.1)

Yesterday I’ve tried to install Mono 2.6.1 on my Mac mini to do some work on my pet project MOSA. Unfortunately things didn’t work out as smoothly as I thought as nant greeted me with the following error message:

        Failed to initialize the 'Mono 3.5 Profile' (mono-3.5) target framework.

Furthermore it stated:

        'pkg-config' failed to start.

I tried all sorts of things from reinstalling, to cleaning, to trying to pull mono from Macports and even compiling Mono myself. Nothing has helped, every attempt was stopped by some error at some point.

The I started to do some digging and tried running pkg-config from Terminal - sure enough it wasn’t there. It was however in my /opt/local/bin folder, so running

        /opt/local/bin/pkg-config --modversion mono

resulted in:

        Package mono was not found in the pkg-config search path.
        Perhaps you should add the directory containing `mono.pc'
        to the PKG_CONFIG_PATH environment variable
        No package 'mono' found

Ok, so again I stumbled - looking further I found another pkg-config installed by Mono itself. So running it with the following two commands in Terminal:

        export PATH=/Library/Frameworks/Mono.framework/Versions/2.6.1/bin:$PATH
        pkg-config --modversion mono


        Package glib-2.0 was not found in the pkg-config search path.
        Perhaps you should add the directory containing `glib-2.0.pc'
        to the PKG_CONFIG_PATH environment variable
        Package 'glib-2.0', required by 'Mono', not found

Again I stumbled. Looking for glib-2.0.pc I found it in /opt/local/lib. So I added it to the PKG_CONFIG_PATH and ran it again:

        export PKG_CONFIG_PATH=/opt/local/lib/pkgconfig:$PKG_CONFIG_PATH
        pkg-config --modversion mono

Now I finally got the response:


So I’ve finally got pkg-config to work and running nant worked too - I can now start contributing some code again...