Reflection in C++ - a teaser

Whether we like it or no - C++ is an ancient language. It doesn’t have many features that are considered normal in some more modern languages. One of such missing features is reflection. It also never been lucky enough to get any kind of standard implementation. Over the years programmers, both main-stream & gamedev - have been patching their own systems. Today, I’d like to present yet another approach that I’ve developed.

The ‘classic’ way, used in majority of engines out there (including some that I worked with) is to describe reflection using preprocessor/template magic. Nice example of this approach can be found in Game Programming Gems 5 article - ‘Using Templates for Reflection in C++’. Amount of additional code that we have to type is almost directly related to amount of data that we need. If all we want is to create some objects using type-name - it’s just a few lines here and there. Things get more complicated if we want to reflect fields/functions as well. In many cases we’re also limited by having single-root class hierarchy (ie, for the type to support reflection it has to derive from some kind of base ‘Object’ type providing basic functionality). Here’s how it may look in a hypothetical preprocessor based solution:

struct Vector3
{
    float x, y, z;
    BEGIN_REFLECTION('Vector3', Vector3, NULL);
        ADD_FIELD('x', x, float);
        ADD_FIELD('y', y, float);
        ADD_FIELD('z', z, float);
    END_REFLECTION
};

Problem with all those solution is that they require a fair bit of maintenance. You need to remember about setting it up & keeping up to date. It can be tedious and bug prone.

I’ve been experimenting with various approaches for a while, but still wasn’t 100% satisfied. I wanted my ideal solution to have the following properties:

• require minimal (or none) amount of additional code when adding new types/fields,
• changes to existing types should be as easy as possible, ideally without any code modifications,
• parts of type-info should be optional. Editor may need much more informations than game for example. In some cases game may not need it at all, why pay for something that’s not being used?

It’s visible that those goals are hard to achieve using any kind of macro/template technique. Ideally, the reflection system would be totally detached from rest of the codebase and all the information loaded optionally. In order to achieve this, we can try to generate reflection database offline. Initially, I experimented with parsing C++ code with GCCXML. It’s possible to make it work, but resulting data would have to be re-parsed again (admittedly, that’s not so big problem, but being lazy bum I am, still wanted to avoid it). Later, I recalled that I’ve been using MSDIA quite extensively in my other projects, why not try it again? My rough plan was:

• extract reflection info from PDB files using MSDIA SDK,
• convert it to my internal format and save to file,
• load info in game/editor when requested,
• profit!

Let’s go through these points one by one.

Extract reflection info from PDB files using MSDIA SDKA

Done using external application named - you guessed it - Reflector. It operates on specified PDB file and list of types it should reflect. Then processes all UDTs & enums in PDB, when type from the list is encountered - it tries to dig out all the necessary information for this type and its fields. Process is semi-automatic, ie. if type Foo has field of type Bar, it’ll generate reflection info for Bar even if it’s not on the to-reflect-list (same applies to base classes).

Convert it to my internal format and save to file

Result of the program is reflection info database saved to binary file. It contains all necessary info like type name, size, list of fields for class types, number of elements for array types, list of constants for enums and so on. For class types it will also try to find a few crucial functions, I’ll write more about it later.

Load info in game/editor when requested

That’s the easy part, format is rather simple so it’s a matter of loading type info and creating internal description structures.

Profit

Here’s fragment of my test application, contains no preprocessor magic at all, reflection info is 90% independent from the rest of the code (100% if you don’t care about load-in-place and create by name). Application has Reflector set as a post-build step, so that reflection info is always up to date.

enum EInitVTable
{
    INIT_VTABLE
};
class Vector3
{
    float x, y, z;
};
struct Color
{
    float r, g, b;
};
struct SuperBar
{
    SuperBar(): p(0), psb(0) {}
    explicit SuperBar(EInitVTable): v(rde::noinitialize) {}
    virtual ~SuperBar() {};

    virtual int VirtualTest()
    {
        return 5;
    }

    static void* Reflection_CreateInstance()
    {
        return new SuperBar();
    }
    static void* Reflection_InitVTable(void* mem)
    {
        return new (mem) SuperBar(INIT_VTABLE);
    }
    unsigned long i;
    float*p;
    bool  b;
    signed chars;
    Color color;
    SuperBar* psb;
    typedef rde::vector tVec;
    tVec  v;
};

struct Bar : public SuperBar
{
    enum
    {
        ARR_MAX = 10
    };
    enum TestEnum
    {
        FIRST = 0,
        SECOND,
        LAST = 10
    };
    virtual ~Bar() {}

    float   f;
    char    c;
    short   shortArray[ARR_MAX];
    Vector3  position;
    Vector3*   porient;
    SuperBar   sb;
    SuperBar** pb;
    TestEnum   en;
};

rde::TypeRegistry typeRegistry;
if (!LoadReflectionInfo("reflectiontest.ref", typeRegistry))
{
    printf("Couldn't load reflection info.\n");
    return 1;
}

Bar bar;
const rde::TypeClass* barType = 
    rde::SafeReflectionCast(typeRegistry.FindType("Bar"));
rde::FieldAccessor accessor_f(&bar;, barType, "f");
accessor_f.Set(5.f);
RDE_ASSERT(bar.f == 5.f);

const rde::Field* field = barType->FindField("i");
RDE_ASSERT(field->m_type->m_reflectionType == 
    rde::ReflectionType::FUNDAMENTAL);
RDE_ASSERT(field->m_type == rde::TypeOf());
field->Set(&bar;, barType, 10);
RDE_ASSERT(bar.i == 10);

rde::FieldAccessor accessorArray(&bar;, barType, "shortArray");
short* pArray = (short*)accessorArray.GetRawPointer();
pArray[0] = 0;
pArray[1] = 1;
pArray[2] = 2;
RDE_ASSERT(bar.shortArray[0] == 0 && bar.shortArray[1] == 1 
    && bar.shortArray[2] == 2);
const rde::Field* fieldShortArray = barType->FindField("shortArray");
const rde::TypeArray* typeShortArray = 
    rde::SafeReflectionCast(fieldShortArray->m_type);
RDE_ASSERT(typeShortArray->m_numElements == Bar::ARR_MAX);

// Print all pointers of Bar.
barType->EnumerateFields(PrintField, rde::ReflectionType::POINTER);

// Print all enum constants from TestEnum.
const rde::TypeEnum* enumType = 
    rde::SafeReflectionCast(typeRegistry.FindType("Bar::TestEnum"));
printf("enum %s\n", enumType->m_name.GetStr());
enumType->EnumerateConstants(PrintEnumConstant);

SuperBar* psb = typeRegistry.CreateInstance("SuperBar");
RDE_ASSERT(psb->VirtualTest() == 5);
delete psb;

What’s next

In the next installment I’ll write a little bit about simple load-in-place system I slapped on top of this code and publish the source, I have to clean it up first (another one of those quick experiments, so it’s quite messy).

Old comments

Z 2009-11-25 08:37:22

This sounds awesome! Looking forward to your next post :)

mINA87 2009-11-26 08:58:46

Great job :) I also thought about this approach. Are You going to implement calling conventions and provide reflection layer with ability to call methods/functions?

mINA87 2009-11-26 09:05:26

Just one more question - did anybody tried ever to hack a little bit IDebugExpressionEvaluator?
http://msdn.microsoft.com/en-us/library/bb144988.aspx
It looks promising but I didn’t have enough time to investigate it further.

admin 2009-11-26 09:11:39

@mINA87: Re-functions – probably not. I made some experiments and it is possible (I do call some ‘special’ functions, but that’s a little bit different story), but I wouldn’t like to make it less portable than it already is :)

No 2009-11-27 09:15:15

In “I wanted my ideal solution to have the following properties:”
you forgott to mentioned:
- platform independent!

admin 2009-11-27 09:38:37

@No: I didn’t. Wasn’t my priority. I’m fine with it working out-of-the-box on PC & X360 and on PS3 with some effort.

No 2009-11-27 11:09:29

Ok. But I’m looking for a nice platfrom independent reflection solution for c++.

gregory 2009-12-07 19:35:23

did you see this reflection system made available by avi?
http://www.realityprime.com/articles/c-runtime-type-system

admin 2009-12-07 22:08:37

Nope, haven’t seen this one before, looks interesting.

Ion Todirel 2011-10-04 02:20:47

@gregory, that requires a manual step, so no-op really