Issue with bitfields

On my rendering system, I’m using something I call a “render environment” to compile the right version of the shader. This has data like how many lights are active, what types, what components are enabled on the mesh (normal, how many texture coordinates), what render pass are we doing, etc.

This is stored on a structure that can easily be converted to a number for fast comparisons on a hash map:

    struct Light
    {
        unsigned char    _type:2;
    };
    struct Props
    {
        // View (2 bits)
        unsigned char    _per_pixel:1;
        unsigned char    _fog_linear:1;
        // Light (20 bits)
        unsigned char    _light_enable:1;
        unsigned char    _light_count:3;
        Light            _lights[MAX_LIGHT];
        // Material (9 bits)
        unsigned char    _map[MAX_TEX/8];
        unsigned char    _alpha_test_enabled:1;
        // Mesh (13 bits)
        unsigned char    _normal:1;
        unsigned char    _psize:1;
        unsigned char    _color0:1;
        unsigned char    _color1:1;
        unsigned char    _tex_coord_count;
        unsigned char    _reserved:1;
        // Other
        unsigned char    _pass:4;
        unsigned char    _reserved_other[2];
    };

I use bitfields so that the structure becomes smaller. So, on the example above, the structure should fill neatly 64 bits (8 bytes), so I could just use a 64-bit number to encode this.

The problem with this is that alignment kicks in. For example, the _lights array should only have 16 bits (MAX_LIGHTS is 8, times 2 bits of the type), but each element of the array occupies a byte, so that structure will really occupy 8 bytes instead of 2.

Ok, I can work that out and use a 16-bit number to store and take care of the types myself:

    struct Props
    {
        // View (2 bits)
        unsigned char    _per_pixel:1;
        unsigned char    _fog_linear:1;
        // Light (20 bits)
        unsigned char    _light_enable:1;
        unsigned char    _light_count:3;
        unsigned short    _light_type;
        // Material (9 bits)
        unsigned char    _map[8/8];
        unsigned char    _alpha_test_enabled:1;
        // Mesh (13 bits)
        unsigned char    _normal:1;
        unsigned char    _psize:1;
        unsigned char    _color0:1;
        unsigned char    _color1:1;
        unsigned char    _tex_coord_count;
        unsigned char    _reserved:1;
        // Other
        unsigned char    _pass:4;
        unsigned char    _reserved_other[2];
    };

Still, this structure has 10 bytes instead of 8, because of alignment issues. So, we have 6 bits before the array, so there’s 2 bits unused before the array… So the next changes to fit in the bits will cause the system to structure to look chaotic, instead of organized by function. From a compiler perspective, it’s the same, but for me it’s a bit more desorganized…

Still, there’s really no way to go around this, so the structure becomes:

    struct Props
    {
        unsigned char    _per_pixel:1;
        unsigned char    _fog_linear:1;
        unsigned char    _light_enable:1;
        unsigned char    _light_count:3;
        unsigned char    _alpha_test_enabled:1;
        unsigned char    _normal:1;
        unsigned short    _light_type;
        unsigned char    _map[8/8];
        unsigned char    _psize:1;
        unsigned char    _color0:1;
        unsigned char    _color1:1;
        unsigned char    _reserved:1;
        unsigned char    _pass:4;
        unsigned char    _tex_coord_count;
        // Other
        unsigned char    _reserved_other[2];
    };

So this should fix it, right?

Not yet… There’s more alignment issues involved, even if there are no “gaps” in the memory layout…

The 16-bit “_light_type” field wants to be 16-bit aligned (due to compiler optimizations, etc), so it will waste another byte there and throw the rest of the alignments out of whack, so I have to organize the code like this:

    struct Props
    {
        unsigned char    _per_pixel:1;
        unsigned char    _fog_linear:1;
        unsigned char    _light_enable:1;
        unsigned char    _light_count:3;
        unsigned char    _alpha_test_enabled:1;
        unsigned char    _normal:1;
        unsigned char    _map[8/8];
        unsigned short    _light_type;
        unsigned char    _psize:1;
        unsigned char    _color0:1;
        unsigned char    _color1:1;
        unsigned char    _reserved:1;
        unsigned char    _pass:4;
        unsigned char    _tex_coord_count;
        unsigned char    _reserved_other[2];
    };

And this has 8 bytes, even if I’m not storing anymore data… And I’ll have to look into this again when I port the game to Mac and Linux, since their compilers might have different rules…

This is one of the cases that building portable code is a bit hard, when you’re really interested in optimization…

The interesting part is that this error was in for ages in the system, and basically the rendering worked by chance!

Now listening to “Rethroned” by “Northern Kings”