C++/CX gotchas #3: Private interfaces (continued)

As an addendum to the last post, I've realized that there is a lot of sense in the way Microsoft themselves seem to implement "private interfaces". We only need to look to IBuffer to see how, since an IBuffer implementation is required to also implement the (pure COM) interface IBufferByteAccess.

Sadly, this means C++/CX is unsuitable since it does not support mixed WinRT/COM classes. The most straight forward way to use private interfaces is therefore to switch to WRL instead.

Even better, you can consume and author these private interfaces in the .NET projection of WinRT, thanks to the very good COM interop support in the CLR.

All you need to do is to make sure the method order and signatures match, and that the interface declarations have the same IID. Let's see how, using the IBufferByteAccess interface on the C++ side (this is from robuffer.h):

    [uuid(905a0fef-bc53-11df-8c49-001e4fc686da)]

    struct IBufferByteAccess : public IUnknown

    {

        STDMETHOD(Buffer)(_Outptr_result_buffer_(_Inexpressible_("size given by different API")) byte **value) = 0;

    };

The C# counterpart of this declaration can be: 

    [ComImport]

    [Guid("905a0fef-bc53-11df-8c49-001e4fc686da")]

    [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]

    interface IBufferByteAccess

    {

        void Buffer([Out] out IntPtr value);

    }

You can in fact cast an IBuffer directly to this interface. No unsafe code needed.

Also interesting is that the C# method signatures are somewhat flexible. The following is also valid:

    [ComImport]

    [Guid("905a0fef-bc53-11df-8c49-001e4fc686da")]

    [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]

    interface IBufferByteAccess

    {

        IntPtr Buffer { get; }

    }

In fact, looking at Microsofts own System.Runtime.InteropServices.WindowsRuntime.WindowsRuntimeBuffer class in DotPeek shows that they use this form. 

Now, why does this even work? Not sure, but I suspect it's NOT to do with the COM interop functionality being clever - since there IS no metadata beyond the GUID here. No TLB or winmd is involved. Rather, I think it's because such a property getter has the exact same signature as the Buffer method with an [Out] parameter does.

C++/CX gotchas #2: Private interfaces

Microsoft really locked down what you can express in the Windows Runtime ABI.

How do you access implementation specific stuff in a public ref class in C++/CX, assuming you know the underlying implementation is yours?

Here a ficticious example:

public interface class IFoo
{
    IAsyncAction^ SomethingAsync();

    void SomethingElse(IVector<int>^ y);    
};

public ref class Foo sealed : IFoo
{
public:
    Foo(int x);

    IAsyncAction^ SomethingAsync();

    void SomethingElse(IVector<int>^ y);    
};

Here, what you see is what you get. There is a class Foo which implements IFoo. That's a great "outer face", but not a good "inner face".

Let's assume I have a larger system which deals with IFoo objects. How do I take an IFoo^ reference and turn it into something where I can access internals and implementation details?

I could of course do dynamic_cast<Foo^>(someIFoo). This would either get me a valid Foo^, or nullptr. Great. But not very scalable. I'd have to treat every concrete implementation separately. It gets ugly fast.

Sadly there is no way to let this public ref class have a base class of any sort in the current Windows Runtime environment. Published/public ref classes must be sealed, and standard C++ classes can't be bases of ref classes at all. I can stuff all kinds of internal and private code in Foo, but there's no obvious way to access it once the object has been cast to IFoo^ and been across the ABI border.

What I really want is a private interface, but there is no support for that. Frustrating! Then I came up with a soluation, of sorts... by staring at the interface declaration and trying to think like a burglar.

I made a class called AsyncActionWrapper<T>, which implements IAsyncAction and holds two things internally:
- A real IAsyncAction^.
- A hidden "payload".

The wrapper implements all the IAsyncAction methods, and simply passes them on to the inner IAsyncAction. Thus the object appears perfectly normal, but there is an extra method there, if you know about it:

T GetPayload() { return m_payload; }

I make sure that some, or all, implementations of the method SomethingAsync wrap their result in this type of object, and hide a useful payload.

Thus I can have IFoo objects, implemented by me, or implemented by the user, and then when they are passed back in to my API, I call SomethingAsync... and then immediately cast the returned IAsyncAction:

auto asyncAction = someFoo->SomethingAsync();
auto wrapper = dynamic_cast<AsyncActionWrapper<IHiddenInterface^>^>(asyncAction);

if(wrapper)
{
    IHiddenInterface^ payload = wrapper->GetPayload();
    payload->SecretMethod("Win!");
}

Voila, private interface.

But there are drawbacks naturally. I have to either be sure that SomethingAsync does something very benign, since I'm calling it with ulterior motives (but do have to call it), or that it starts work I'd have wanted to start anyway.

There are countless variations of the trick, and obviously it doesn't have to involve async actions at all. It just happened to fit this occasion. You could just as well wrap an IVector or something else, any interface reference should do just as well.

C++/CX gotchas #1: Careful in the ctor

I've been working within the relatively new world of Windows Runtime/RT, C++/CX, WRL etc lately. It's a somewhat treacherous sea and the maps are sketchy yet (there's not much to find on Google), so I think I'll publish some tips, tricks and gotchas based on what I've found.

In no particular order, here's point number one:

Don't assume too much in the constructor.

Or rather, "in the initializer list".

One of the benefits of C++/CX is that it hides away much of the COM/WinRT plumbing. Your class gets IUnknown, IInspectable implemented automatically, and you don't normally see the intermediate code that's generated.

In this case, I needed weak references to my objects, and as it turns out, there is something called IWeakReferenceSource which is automatically implemented by all ref classes. From what I can tell, you won't get weak reference support unless your object has that plumbing.

In this case, I had set up a sort of fake aggregated "base class" (due to the Windows Runtime limitations) called Base in order to share code between my concrete classes. Each concrete class, e.g. Foo, had a reference to its Base member, and there was also the need for Base to refer back to the Foo. In the Windows Runtime world of reference counting, that means neither object will get destroyed and just lead to a memory leak (or worse, if they aren't just sitting idle).

This is well known of course, and Base needed a weak reference to Foo:

private ref class Base sealed
{
public:
    Base(Foo^ foo) : 
        m_weakFoo(foo)
    {
    }

private:
    Platform::WeakReference m_weakFoo;
};

public ref class Foo sealed
{
public:
    Foo() : 
        m_base(ref new Base(this))
    {
    }

private:
    Base^ m_base;
};

This failed. From Base's perspective, it just got nullptr back from m_weakFoo.Resolve<Foo>().
Head-scratching time ensued.

Observing it in the debugger showed something interesting. Turns out the initializer list is a bit too early to rely on the ref class to be properly set up. Especially the IWeakReferenceSource related members of the class looked quite uninitialized in the debugger. Oops.

And it seems obvious that order of initialization matters... but I've forgotten enough C++/COM to not remember the rules of thumb. I had just wrongly assumed that C++/CX would hold my hand all the way.

So with that lesson learned, I moved the m_base initialization into the constructor body instead:

  Foo() 
  {
      m_base = ref new Base(this);
  }

...and it worked perfectly. (Still does in fact.)

I now stay away from doing any kind of complex logic in the initializer list, at least on my ref classes. 

Simple value initialization should be fine, and really, anything which is pure C++ should work as well, but I figure its better to be safe than sorry.

Feedback welcome, as always.

SOLVED: Aspire One MTRR performance warning

Finally I managed to get rid of the i915/drm/MTRR issue on the Acer Aspire One (AOA150) as mentioned in the previous post. Basically, Write Combining seems to be disabled for VRAM because the drm module finds no spare room in the (hardware) MTRR list.

I found no clear solution online, but the general consensus is that the BIOS is needlessly filling all eight entries in the variable MTRR list. It's quite easy to rearrange them by writing to /proc/mtrr, and in the case of my particular machine (probably all AOA150's) entry #1 in particular is useless. To get rid of this, you just do:

# echo "disable=1" > /proc/mtrr

The problem is that the i915/drm modules have already tried to do their stuff at this point. In fact, it happens in the initramfs image, which is well before rc.local, for instance.

So I needed to do it as early as possible, and someone on Superuser suggested he'd edited his initramfs image to do it. Great, and it turns out you can script it.

/usr/share/initramfs-tools/scripts/init-top/ contains "early" scripts, and some clues were scripts called 'udev' and 'framebuffer'. Looks like the right spot. They are not ordered explicitly, but specify prerequisites (so the 'framebuffer' script requires 'udev' to be run etc).

I simply added a new file called 'aoa150_mtrr' and based it on the 'udev' one. It now looks like this:

#!/bin/sh -e
# initramfs init-top script for AOA150 mtrr fixups


PREREQ=""


# Output pre-requisites
prereqs()
{
echo "$PREREQ"
}


case "$1" in
    prereqs)
prereqs
exit 0
;;
esac


# Remove one unnecessary mtrr entry. 
echo "disable=1" > /proc/mtrr

Really, only the last line is interesting, the rest is scaffolding. So what it does is to clear the #1 MTRR entry. To ensure that the script is run before the graphics initialize, I added a reference to PREREQ in the 'framebuffer' script:

PREREQ="udev aoa150_mtrr"

Then,

# update-initramfs -u -k 3.2.0-17-generic

Obviously replace the version with whatever kernel you're running. 

Reboot and... no more MTRR error message! The drm module no longer complains, and cat /proc/mtrr clearly shows that it has succeeded in setting up write combining:

reg00: base=0x0fffe0000 ( 4095MB), size=  128KB, count=1: write-protect
reg01: base=0x040000000 ( 1024MB), size=  256MB, count=1: write-combining
reg02: base=0x000000000 (    0MB), size=  512MB, count=1: write-back
reg03: base=0x020000000 (  512MB), size=  512MB, count=1: write-back
reg04: base=0x03f800000 ( 1016MB), size=    8MB, count=1: uncachable
reg05: base=0x03f600000 ( 1014MB), size=    2MB, count=1: uncachable
reg06: base=0x03f500000 ( 1013MB), size=    1MB, count=1: uncachable
reg07: base=0x000000000 (    0MB), size=  128KB, count=1: uncachable

Yay! However... I've yet to measure any performance difference. :/ 

gtkperf hovers around 22.50 (YMMV) and glxgears gives me about 350 fps.

I'll maybe look for a more direct benchmark for CPU->VRAM writes which is what this fix should help with.  

Troubleshooting #1: If you're using a custom kernel, DO ENSURE that i915, drm etc are modules and not compiled directly into the kernel, because otherwise they will initialize way too early! I have no clue how to get around that, so try lsmod | grep i915 to confirm that it's a module.

How to compile mtrr-uncover for recent Linux versions

While trying to fix the degraded graphics performance with Ubuntu 11.10 (Oneiric Ocelot) on my Acer Aspire One (AOA150 ZG5 with 945GME/GMA 950), I found the following in dmesg:

[   25.513451] mtrr: no more MTRRs available
[   25.513463] [drm] MTRR allocation failed.  Graphics performance may suffer.

There is a tool called mtrr-uncover that supposedly helps with this, but the latest version I could find was from 2009: ftp://ftp.cs.utoronto.ca/pub/hugh/

It doesn't compile here, I'm on kernel 3.2.x and it seems they added a mtrr_type typedef to the kernel headers since mtrr-uncover was last updated. So to get mtrr-uncover to compile, you just have to rename the mtrr_type struct that's in mtrr-uncover.c.

Patch: 

http://pastebin.com/ZKKkdjLe

Now it at least compiles, but I've yet to test whether it's useful for the original problem. :)

Small footprint Mono #1

Tiplet #1: Extension to make dictionary lookups less clunky

For TValues that are of reference type, looking up values can be made less clunky by the following extension method:

public static TValue TryGetValue<TKey, TValue>(this Dictionary<TKey, TValue> dictionary, TKey key)
        where TValue : class
{
        TValue val;
        dictionary.TryGetValue(key, out val);
        return val;
}

The return value will be null if the key doesn't exist. This will also be the case if there is a key with a null value, but often you don't care about the distinction.

Since the hateful "out" is gone, you can now use the method directly in expressions.

Value val = dictionary.TryGetValue(key);

return dictionary.TryGetValue(key);

other.Foo = replacementFoos.TryGetValue(fooKey) ?? other.Foo;