GDC 2010 proceedings
March 9, 2010 – 11:49 amI’ll keep updating this post with links as I find them.
Data breakpoints
February 27, 2010 – 2:15 pmData breakpoints are one of the most helpful debugger features when trying to hunt for memory overwrites/ninja variable modifications. In majority of cases it’s enough to set them up from debugger, however, there are situations when it’s not possible. Sometimes breaking into debugger changes program behavior (I had this problem just yesterday), sometimes we don’t want to catch every variable access, just some of them (as others are legal). In situations like that we need to set data breakpoints from code. Read the rest of this entry »
Adding layers
February 20, 2010 – 6:02 pmSimple technique that can help in making code more future-proof and easier to modify. Applicable in most situations where two subsystems need to exchange informations. Let’s imagine the following scenario – for some reason we need to introduce “unlimited mana” feature in our game. If hero is in berserker mode, his attributes increase, time slows down, he is able to cast spells even if he’s out of mana and his mana pool doesn’t deplete at all.
The simplest way to implement it is to test ‘am I in berserker mode’ condition in every place when we modify mana amount. Minimal example:
if (m_hero->IsInBerserkerMode() || mana > spellCost)
{
CastSpell();
if (!m_hero->IsInBerserkerMode())
mana –= spellCost;
}
It will probably work OK, however it strengthens the coupling between Spell & Hero components. Next week a designer stops by our desk and tells us about his new cool idea – mana shouldn’t be reduced if player is in berserker mode OR if he has Jewel of Mana in his inventory. We need to modify both conditions and bind both components even tighter. This will repeat itself every time we need to modify the system. How to improve it?
One way is to introduce intermediate layer between both components. We can add ‘unlimited mana’ mode to spell component, which is enabled/disabled according to specified conditions (quite rarely, probably). Now, it doesn’t need to ask about it every time. Our code snippet becomes:
if (IsManaUnlimited() || mana > spellCost)
{
CastSpell();
if (!IsManaUnlimited())
mana –= spellCost;
}
Now, every time we introduce a new condition we only need to modify higher level code, but the implementation responsible for casting spells stays without changes. Yes, we introduce additional variables, which is the hidden cost of this approach. It shouldn’t be choice that’s taken blindly, but in most situations shouldn’t be a problem (can be just one more flag).
Playing with bits
February 9, 2010 – 8:39 pmLittle challenge posted by a workmate. Good for those 5 minute breaks at work when you need to force your brain to think about something else than your current task.
Recode the following piece of code so that it doesn’t use branches/multiplies:
unsigned char* oldStart = 0;
if (m_start + size < m_bottom)
{
oldStart = m_start;
m_start += size;
}
return oldStart;
Don’t read below if you want to try it yourself.
Are Movies Getting Longer?
January 31, 2010 – 7:16 pmFew weeks ago I’ve been watching The Curious Case of Benjamin Button. It is a nice movie, but also painfully long – almost 3 hours. It got me thinking – are movies getting longer & longer? I could have sworn that I have remembered them to be shorter, usually around 1.5h, now it feels like it’s closer to 2h. Obviously, this was only my impression, I had no hard data to back it up. This weekend I decided to play with Ruby a little bit and it gave me a chance to verify my theory.
The Orton Technique
January 27, 2010 – 9:36 pmAnother interesting photo technique (named after Michael Orton). To be perfectly honest, I’m not that big fan, it makes images a little bit ‘cheesy’. However, it should be interesting for every graphics programmer, as the idea behind it is almost exactly the same as behind fullscreen glow/bloom effect. Basically, it’s a blend of two images, the original one & overexposed/blurred layer. Sample result:
Tilt shift photography
January 24, 2010 – 10:17 pmBeen playing a little bit with tilt shift photography today. My first try here (Stockholm). Not the best source image (sky especially, technique tends to look better with whole picture ‘busy’) and perhaps a little bit too strong blur, but still kinda captures the ‘fake miniature’ look:
Remote debugging in Visual Studio
January 23, 2010 – 10:47 amToday I’d like to describe a relatively unknown feature of Visual Studio that can be extremely useful in certain situations – remote debugging. Remember all those situations when application work perfectly on your machine, but crashes on tester’s? “Old school” way of trying to debug this problem was to add a very detailed logging, then analyze it. Fortunately, there is a more convenient method – we can simply connect to his machine and attach debugger to running application. Setting it up is relatively simple and boils down to the following steps (machine A = tester’s machine with application running, machine B = developer’s machine with Visual Studio):
- “install” Remote Debugger on machine A. It should be located in in Visual Studio dir/Common7/IDE/Remote Debugger/[architecture type], just copy the whole folder,
- run msvsmon.exe on machine A,
- set debugging permissions. In a typical scenario of both computers in same (safe) company network just select the following from Options menu: No special permission needed, Allow any user to debug,
- msvsmon log window should read something along the lines of: “Msvsmon started a new server named ‘XXX:port’. Waiting for new connections”.
- machine B – run Visual Studio, select Debug –> Attach to Process. Transport: Remote, then enter machine A’s address/listening port in Qualifier. If done correctly, it should display a list of running processes. Select your application and tap ‘Attach’.
That should be all that’s needed. Just little note about symbols: when debugging native code, up-to-date symbol files (*.PDB) should be present on machine B, otherwise you’ll have problems with source level debugging. Remote debugging may be not as convenient as local debugging as in many cases it means analyzing optimized code (for bugs that only happen in final builds), but it surely can be a life saver.
To assert or not to assert
January 6, 2010 – 10:49 pm(or rather: how & when to assert). Assertion is probably one of my favourite programming tools, however there are still few areas that I’m not entirely sure how to solve in an optimal way. First, let’s start with things that I’m rather convinced about, few simple guidelines for my ideal assert macro (funnily enough, during my career I’ve never seen implementation respecting all those rules at once):
- when it breaks into debugger make sure it stops in the same line assertion actually failed, not two hierarchy levels down (so every time it’s triggered one has to rewind first to see what’s wrong exactly)! Simple rule, yet I’d say it’s only 50% chance in a typical codebase to be implemented this way (I think the record I’ve seen is 3 callstack levels away, that’s also how default MSVC assert behaves). Easiest way to do it:
// (assertion macro) do { if (!(expr)) { if (!AssertionFailureHandler(#expr, __FILE__, __LINE__)) BREAK_TO_DEBUGGER; } } while (false)All your fancy code (logging etc) goes into handler, then return false (true/code/whatever you like) to indicate we should break to debugger.
- Use per-module/per-level assertions. The way I like to do it is to have a ASSERT_ALWAYS macro for generic assert implementation. Then, global ASSERT macro that can be disabled in some builds/enabled in other, ASSERT_PEDANTIC that’s only enabled in “very debug” builds, then per-module assertions, which can be enabled/disabled independently from global settings. This way I can quickly build, say, only math library with assertions enabled (all those: ASSERT(x >= 0) in sqrt wrappers etc) while hunting for obscure calculation bugs etc.
- Assertion overhead should be as little as possible. Sure, it is a debugging tool, but at the same time we should try to avoid bringing whole application to crawl whenever we enable it. Usually, it’s platform specific, so refer to documentation/tutorials. For X360, there’s publicly available Gamefest 2008 presentation by Bruce Dawson himself – “Xbox360 Compiler and PgoLite Update. Making your code faster, with little effort”. Microsoft tends to change document locations all the time, so I don’t dare to put links here, Google should help.
Now, assuming we have our perfect assertion system, there are still some problems left to solve. Things get more complicated here, personally, I don’t have definite answers for the following issues:
- when should assertions be enabled? There are various ‘schools’ here. I’ve seen the following solutions so far: only in debug builds, in debug + QA builds and always. Problem with the first one is obvious – usually only programmers run debug builds, so only they would benefit from assertions. Actually, there’s another problem – it’s quite often that true, 100% debug builds (with no optimizations and all assertions enabled) run so slow they are virtually unplayable with bigger levels. The ‘solution’ usually is to compile some libraries (third-party or rarely modified and throughly tested) with optimizations enabled/asserts disabled (depending on policy). Having at least some assertions enabled in QA builds (builds that are used by the team during development) is way to go, IMHO. I know there are some titles shipped with (selected) assertions enabled in retail builds, but that goes against basic idea that final program should work without assertions in exactly same way as with them.
- assuming we do have assertions enabled in builds that are distributed amongst the designers/graphicians, people running it without debuggers, next question is: what should assertion handler do?
Should assertion be treated as fatal error, report, close the game, exit or should it be skippable? In an ideal world, assertions are used to handle code errors only, so terminating sounds like a sane idea. After all, we found ourselves in a situation that’s most probably not recoverable. Sadly, there are places, especially in the lower level libraries where you just can’t guarantee that input won’t change depending on the data. Basic example – out of range indexing in your vector class:T& operator[](int i) const { ASSERT(i < Size()); …Imagine we use this structure to store vertices, then index it using triangle data loaded from disk. If mesh is corrupted, we get out-of-bounds access & fatal error. This also means that 100 people can’t work now, because level crashes at the start. That’s simplified example, but you get the picture. That’s another subject on its own actually – should game care about invalid data?
- Should we have special code paths for final builds dealing with fatal errors? Consider the following snippet:
Object* obj = ObjectForId(m_idOpponent); ASSERT(obj); if (obj) // should this test be here or not? obj->DoSomething();Here, I’m certain: there should be no special test. If opponent ID can be invalid, remove the assert. If you’re sure it’s always valid – put the assert, remove the test, let it crash.
That’s actually one of places where I *hate* to see people “fixing” problems like that with cheap workarounds. Too many times I have seen a situation when programmer “fixed” the crash by adding if (ptr) and happily moved to other tasks. NEVER DO THIS, that’s the worst thing you can do. You don’t fix problem this way, you merely sweep the dirt under the carpet and make the real root of the bug even harder to track down (as now, it has a big chance to crash many calls later). If you are absolutely sure ptr can’t be NULL there – don’t try to work around it, it’s a fatal error and should be treated as such. Sure, you may have some higher level architecture preventing application from a hard crash (like SEH), but log it, do NOT exit quietly. Of course, try to find why ptr was NULL in the first place. Is it a bug or a valid code path? Generally, I’d say we should try to minimize number of functions that care about NULL pointers, do tests at the highest level possible. I treat if (ptr) explosion that’s sometimes visible in later stages of the project as sign of laziness and lack of understanding of program architecture. “The best” of this category that I’ve actually seen:if (ptr == (Foo*)0xcdcdcdcd || ptr == (Foo*)0xbaadf00d || ptr == (Foo*)0xfeeefeee) return; // YAY, I'm so smart!
My (rough) idea of good assertion/warning/error system:
- distinguish between tests of internal program state (assertions) & general error handling (file not found, networking, user input). There are games out there that use assertions as error handling mechanisms. Let me tell you, it’s not pretty. Game without bugs should work with the first system disabled (second is still there in final build). That’s another way to look at it – ‘proper’ error handling for situations that can happen in retail builds, asserts for those that can not.
- deal with missing resources by using generated placeholders (pink cubes etc), not needed in final builds.
- game can assume that provided data format is always valid, no special error handling here. It’s tool/exporter/data cooker responsibility to guarantee it. Invalid data does not get exported, period. In practice it is impossible to keep it that way 100% of the time and the game will crash on invalid data from time to time, but in my experience it’s pretty rare and usually happens more often at the beginning of the production process. With mature tools it’s a reasonable goal. Assertions still may be a good idea, to provide more information in case of crash. Nothing too fancy, though, just basic stuff, like ranges etc, no tests if shadow geometry is valid. That’s exporter responsibility, again, here’s it’s a different story, tests should be as strict as possible. It’s probably lesser evil to have two artists unable to export their mesh because limits are too strict than to have 70 people stuck because invalid data has been exported.
- asserts are fatal, game terminates, every failed assert should be logged, at least to local file (that’s later attached to every crash report). I’d definitelly would like to experiment with central assert database (see Mick West’s article). Later it can searched for recent asserts, see which fail most often etc. There is no good way to work around failed assert (it’s too general & too low level, how do you handle out-of-range indexing for example?).
- if there are situations that can cause problems, but it’s possible to continue (usually data/input related) – make it a warning, not an assert. Ideally, there should be only few warning checks in game and hundreds in tools. Do not spam screen with warnings or people will stop caring. See Scott Bilas’ “Optimizing the Development Pipeline” presentation for good ideas. I especially like the big warning counter. Warnings could go to database as well, this way producers could see who’s responsible for the oldest active ones and make victim life miserable until he fixes it.
- auto-test integration. Every time a new build is one, build machine tries to run the game for few minutes with random/recorded input. Any assert/crash causes the build to be considered broken, so that people don’t sync to it. This should deal with 90% of simple data problems/oversensitive assert conditions.
Your thoughts?
Reflection in C++ – Load In Place
December 6, 2009 – 10:13 pmI played a little bit with my Load-In-Place system. It used to only support vectors of PODs (stored by value). Now it handles also vectors of pointers & vectors of classes. Sample structure that’ll be saved/loaded automatically:
struct IntContainer
{
int* pInt;
};
struct SuperBar
{
unsigned long i;
// [Hidden]
float* p;
bool b;
signed char s;
Color color;
SuperBar* psb;
typedef rde::vector<int> tVec;
tVec v;
rde::vector<Color*> someColors;
rde::vector<SuperBar*> superBars;
rde::vector<IntContainer> containers;
IntContainer ic;
};
[...]
// (fill sb)
// Code to save this.
SaveObject(sb, ofstream, typeRegistry);
// Code to load:
SuperBar* psb = LoadObject<SuperBar>(ifstream, typeRegistry);
Cool thing is, fixup cost for loading is roughly linear (it’s constant per-pointer, no recursion), only saving gets more complicated as structure complexity increases (so it’s still good to be careful, of course). I’ve updated download package.
