Few weeks ago I’ve been watching The Curious Case of Benjamin Button. It is a nice movie, but also painfully long – almost 3 hours. It got me thinking – are movies getting longer & longer? I could have sworn that I have remembered them to be shorter, usually around 1.5h, now it feels like it’s closer to 2h. Obviously, this was only my impression, I had no hard data to back it up. This weekend I decided to play with Ruby a little bit and it gave me a chance to verify my theory.
Another interesting photo technique (named after Michael Orton). To be perfectly honest, I’m not that big fan, it makes images a little bit ‘cheesy’. However, it should be interesting for every graphics programmer, as the idea behind it is almost exactly the same as behind fullscreen glow/bloom effect. Basically, it’s a blend of two images, the original one & overexposed/blurred layer. Sample result:
Been playing a little bit with tilt shift photography today. My first try here (Stockholm). Not the best source image (sky especially, technique tends to look better with whole picture ‘busy’) and perhaps a little bit too strong blur, but still kinda captures the ‘fake miniature’ look:
Today I’d like to describe a relatively unknown feature of Visual Studio that can be extremely useful in certain situations – remote debugging. Remember all those situations when application work perfectly on your machine, but crashes on tester’s? “Old school” way of trying to debug this problem was to add a very detailed logging, then analyze it. Fortunately, there is a more convenient method – we can simply connect to his machine and attach debugger to running application. Setting it up is relatively simple and boils down to the following steps (machine A = tester’s machine with application running, machine B = developer’s machine with Visual Studio):
“install” Remote Debugger on machine A. It should be located in in Visual Studio dir/Common7/IDE/Remote Debugger/[architecture type], just copy the whole folder,
run msvsmon.exe on machine A,
set debugging permissions. In a typical scenario of both computers in same (safe) company network just select the following from Options menu: No special permission needed, Allow any user to debug,
msvsmon log window should read something along the lines of: “Msvsmon started a new server named ‘XXX:port’. Waiting for new connections”.
machine B – run Visual Studio, select Debug –> Attach to Process. Transport: Remote, then enter machine A’s address/listening port in Qualifier. If done correctly, it should display a list of running processes. Select your application and tap ‘Attach’.
That should be all that’s needed. Just little note about symbols: when debugging native code, up-to-date symbol files (*.PDB) should be present on machine B, otherwise you’ll have problems with source level debugging. Remote debugging may be not as convenient as local debugging as in many cases it means analyzing optimized code (for bugs that only happen in final builds), but it surely can be a life saver.
(or rather: how & when to assert). Assertion is probably one of my favourite programming tools, however there are still few areas that I’m not entirely sure how to solve in an optimal way. First, let’s start with things that I’m rather convinced about, few simple guidelines for my ideal assert macro (funnily enough, during my career I’ve never seen implementation respecting all those rules at once):
when it breaks into debugger make sure it stops in the same line assertion actually failed, not two hierarchy levels down (so every time it’s triggered one has to rewind first to see what’s wrong exactly)! Simple rule, yet I’d say it’s only 50% chance in a typical codebase to be implemented this way (I think the record I’ve seen is 3 callstack levels away, that’s also how default MSVC assert behaves). Easiest way to do it:
// (assertion macro)
do
{
if (!(expr))
{
if (!AssertionFailureHandler(#expr, __FILE__, __LINE__))
BREAK_TO_DEBUGGER;
}
} while (false)
All your fancy code (logging etc) goes into handler, then return false (true/code/whatever you like) to indicate we should break to debugger.
Use per-module/per-level assertions. The way I like to do it is to have a ASSERT_ALWAYS macro for generic assert implementation. Then, global ASSERT macro that can be disabled in some builds/enabled in other, ASSERT_PEDANTIC that’s only enabled in “very debug” builds, then per-module assertions, which can be enabled/disabled independently from global settings. This way I can quickly build, say, only math library with assertions enabled (all those: ASSERT(x >= 0) in sqrt wrappers etc) while hunting for obscure calculation bugs etc.
Assertion overhead should be as little as possible. Sure, it is a debugging tool, but at the same time we should try to avoid bringing whole application to crawl whenever we enable it. Usually, it’s platform specific, so refer to documentation/tutorials. For X360, there’s publicly available Gamefest 2008 presentation by Bruce Dawson himself – “Xbox360 Compiler and PgoLite Update. Making your code faster, with little effort”. Microsoft tends to change document locations all the time, so I don’t dare to put links here, Google should help.
Now, assuming we have our perfect assertion system, there are still some problems left to solve. Things get more complicated here, personally, I don’t have definite answers for the following issues:
when should assertions be enabled? There are various ‘schools’ here. I’ve seen the following solutions so far: only in debug builds, in debug + QA builds and always. Problem with the first one is obvious – usually only programmers run debug builds, so only they would benefit from assertions. Actually, there’s another problem – it’s quite often that true, 100% debug builds (with no optimizations and all assertions enabled) run so slow they are virtually unplayable with bigger levels. The ‘solution’ usually is to compile some libraries (third-party or rarely modified and throughly tested) with optimizations enabled/asserts disabled (depending on policy). Having at least some assertions enabled in QA builds (builds that are used by the team during development) is way to go, IMHO. I know there are some titles shipped with (selected) assertions enabled in retail builds, but that goes against basic idea that final program should work without assertions in exactly same way as with them.
assuming we do have assertions enabled in builds that are distributed amongst the designers/graphicians, people running it without debuggers, next question is: what should assertion handler do?
Should assertion be treated as fatal error, report, close the game, exit or should it be skippable? In an ideal world, assertions are used to handle code errors only, so terminating sounds like a sane idea. After all, we found ourselves in a situation that’s most probably not recoverable. Sadly, there are places, especially in the lower level libraries where you just can’t guarantee that input won’t change depending on the data. Basic example – out of range indexing in your vector class:
Imagine we use this structure to store vertices, then index it using triangle data loaded from disk. If mesh is corrupted, we get out-of-bounds access & fatal error. This also means that 100 people can’t work now, because level crashes at the start. That’s simplified example, but you get the picture. That’s another subject on its own actually – should game care about invalid data?
Should we have special code paths for final builds dealing with fatal errors? Consider the following snippet:
Object* obj = ObjectForId(m_idOpponent);
ASSERT(obj);
if (obj) // should this test be here or not?
obj->DoSomething();
Here, I’m certain: there should be no special test. If opponent ID can be invalid, remove the assert. If you’re sure it’s always valid – put the assert, remove the test, let it crash.
That’s actually one of places where I *hate* to see people “fixing” problems like that with cheap workarounds. Too many times I have seen a situation when programmer “fixed” the crash by adding if (ptr) and happily moved to other tasks. NEVER DO THIS, that’s the worst thing you can do. You don’t fix problem this way, you merely sweep the dirt under the carpet and make the real root of the bug even harder to track down (as now, it has a big chance to crash many calls later). If you are absolutely sure ptr can’t be NULL there – don’t try to work around it, it’s a fatal error and should be treated as such. Sure, you may have some higher level architecture preventing application from a hard crash (like SEH), but log it, do NOT exit quietly. Of course, try to find why ptr was NULL in the first place. Is it a bug or a valid code path? Generally, I’d say we should try to minimize number of functions that care about NULL pointers, do tests at the highest level possible. I treat if (ptr) explosion that’s sometimes visible in later stages of the project as sign of laziness and lack of understanding of program architecture. “The best” of this category that I’ve actually seen:
if (ptr == (Foo*)0xcdcdcdcd || ptr == (Foo*)0xbaadf00d || ptr == (Foo*)0xfeeefeee)
return; // YAY, I'm so smart!
My (rough) idea of good assertion/warning/error system:
distinguish between tests of internal program state (assertions) & general error handling (file not found, networking, user input). There are games out there that use assertions as error handling mechanisms. Let me tell you, it’s not pretty. Game without bugs should work with the first system disabled (second is still there in final build). That’s another way to look at it – ‘proper’ error handling for situations that can happen in retail builds, asserts for those that can not.
deal with missing resources by using generated placeholders (pink cubes etc), not needed in final builds.
game can assume that provided data format is always valid, no special error handling here. It’s tool/exporter/data cooker responsibility to guarantee it. Invalid data does not get exported, period. In practice it is impossible to keep it that way 100% of the time and the game will crash on invalid data from time to time, but in my experience it’s pretty rare and usually happens more often at the beginning of the production process. With mature tools it’s a reasonable goal. Assertions still may be a good idea, to provide more information in case of crash. Nothing too fancy, though, just basic stuff, like ranges etc, no tests if shadow geometry is valid. That’s exporter responsibility, again, here’s it’s a different story, tests should be as strict as possible. It’s probably lesser evil to have two artists unable to export their mesh because limits are too strict than to have 70 people stuck because invalid data has been exported.
asserts are fatal, game terminates, every failed assert should be logged, at least to local file (that’s later attached to every crash report). I’d definitelly would like to experiment with central assert database (see Mick West’s article). Later it can searched for recent asserts, see which fail most often etc. There is no good way to work around failed assert (it’s too general & too low level, how do you handle out-of-range indexing for example?).
if there are situations that can cause problems, but it’s possible to continue (usually data/input related) – make it a warning, not an assert. Ideally, there should be only few warning checks in game and hundreds in tools. Do not spam screen with warnings or people will stop caring. See Scott Bilas’ “Optimizing the Development Pipeline” presentation for good ideas. I especially like the big warning counter. Warnings could go to database as well, this way producers could see who’s responsible for the oldest active ones and make victim life miserable until he fixes it.
auto-test integration. Every time a new build is one, build machine tries to run the game for few minutes with random/recorded input. Any assert/crash causes the build to be considered broken, so that people don’t sync to it. This should deal with 90% of simple data problems/oversensitive assert conditions.
I played a little bit with my Load-In-Place system. It used to only support vectors of PODs (stored by value). Now it handles also vectors of pointers & vectors of classes. Sample structure that’ll be saved/loaded automatically:
struct IntContainer
{
int* pInt;
};
struct SuperBar
{
unsigned long i;
// [Hidden]
float* p;
bool b;
signed char s;
Color color;
SuperBar* psb;
typedef rde::vector<int> tVec;
tVec v;
rde::vector<Color*> someColors;
rde::vector<SuperBar*> superBars;
rde::vector<IntContainer> containers;
IntContainer ic;
};
[...]
// (fill sb)
// Code to save this.
SaveObject(sb, ofstream, typeRegistry);
// Code to load:
SuperBar* psb = LoadObject<SuperBar>(ifstream, typeRegistry);
Cool thing is, fixup cost for loading is roughly linear (it’s constant per-pointer, no recursion), only saving gets more complicated as structure complexity increases (so it’s still good to be careful, of course). I’ve updated download package.
In a previous note I wrote a little bit about my experimental reflection system for C++. Today, I’d like to describe a simple load-in-place mechanism I built on top of that.
Basic idea behind LIP is to minimize the overhead of any processing after data is loaded from a file. Ideally, all that needs to be done is call to ‘fread’ to a memory buffer. For more detailed description of such system see Fast File Loading article by Ent/Incognita.
Whether we like it or no – C++ is an ancient language. It doesn’t have many features that are considered normal in some more modern languages. One of such missing features is reflection. It also never been lucky enough to get any kind of standard implementation. Over the years programmers, both main-stream & gamedev – have been patching their own systems. Today, I’d like to present yet another approach that I’ve developed.
1996 was one of the best PC scene years in my opinion. It was also the year when some legendary Amiga/Atari ST groups released their first PC productions. It started at Saturne’96 party. It wasn’t one of the great 3 events (The Gathering/Assembly/The Party), it was local party, with French groups mostly. In 1996 however, the competition level was just crazily high. To this day I’m still not sure which one of the first two productions I like more. Winner was Contrast by Oxygene:
Second place went to Impact Studios for Bomb demo (Gengis/Made/Titan/Clawz (again)). Sadly, couldn’t find it at Youtube, so here’s Pouet link. Those were two highlights of the competition, but whole top 5 is actually very good and any of those could easily won any other party.
Oxygene later made one more PC production – Solex (same team as with Contrast):
As it usually happens, lots of Oxygene/Bomb members are/were involved in the gamedev industry. Impact Studios (known as Bomb at Amiga scene) created Fears – Amiga’s response to Doom.
Even more interesting are later adventures of Oxbab (Oxygene coder). He went on to to work at Red Storm, later SCEA, then Naughty Dog and now he’s co-president of ND and, knowing his skills, probably one of the reasons why Uncharted is so damn great. To end with a nice accent, one of Oxygene’ Amiga demos — Control:
I’ve read this note on Rachel’s blog and found out there’s tool similar to Cruncher, just for ELF, named pahole. It has a nifty feature of showing cacheline boundaries. Had a little bit of spare time this weekend, so decided to add it to Cruncher# as well. By default, it assumes that object’s start offset corresponds with cacheline start, but you can set ‘prefetch’ start offset from context RMB menu. Screenshot:
