A Decade

May 10, 2012 – 5:48 am

I can’t remember the exact date, but I know it was early May or the end of April, 2002. I’ve just realized I had started my gamedev adventure almost exactly 10 years ago. It seemed like a logical step, never really considered any other career choice, it was my dream job and quite natural progression after few years of demo coding. I responded to a job advertisement at the gbadev mailing list (wow, I love the Internet, I actually found this post), flew to Palermo for a weekend, survived my first job interview (even though I was still slightly hung over from the party 7th Sense guys took me the day before…) and before I knew it – I was flying to Italy again, this time for good.

Read the rest of this entry »

Null references – addendum

April 6, 2012 – 5:11 am

My recent article turned out to be quite popular, however it seems that people focus on the problem of undefined behavior rather than optimization itself. Quick clarification then, so that we’re all on the same page. I do not advocate relying on undefined behaviors. They are… undefined (duh). I merely noticed in practice you can have ‘null references’, just as you can have null pointers. I also would be very surprised if there was any compiler that actually generates code testing against null pointer dereference. It’d completely kill the performance of compiled application.

 

Compressing integers

April 2, 2012 – 4:31 am

Programming games is an uphill battle. We always try to fight for more, but our resources are limited. There’s never enough RAM, the CPUs are never too powerful and obviously same thing applies to bandwidth – we could always use some more. The eternal problem of mutiplayer game is trying to send as much information as possible using as little memory as possible. One of the weapons in our arsenal is compression. I will focus on integers today as there seems to be enough information about floats/vectors out there.

Let’s start with easy cases first (I’ll link to Shawn Hargreaves blog, good descriptions there):

  • “enum” type integers with limited range of possible values – use bitfields. If you know you only have 4 types of grenades, no need to waste 8 bits for grenade type, you only need 2 bits, obviously
  • arithmetic encoding for even more effective bitpacking

That’s all nice and good, but things aren’t always that straightforward. Imagine 32-bit variable that can take the complete range of values. Typical example is health/damage. Some level 1 creature might have 50hp, but your endgame boss sits on 250k. Same thing with damage, there’s a difference between poking someone with a stick and dropping a nuke from the orbit. Our previous approaches won’t help us much here, because even though we know the range  — it’s huge. We could give up and always send 32-bits, but it seems wasteful, after all in majority of cases we require much less memory (huge damage spikes are rare).

At first, co-worker directed me towards Google’s Protocol Buffers (or rather the encoding scheme they used). They were created with more complicated applications in mind, but it seems like they’d work for us quite nicely. You’ll want to read the linked document for details, but the general idea in a nutshell is – we employ base 128 varints. We only use 7 bits in a byte for our data and the remaining bit is used to signalize if there are more bytes to consider. Some examples:

  • 1 – 1 byte, 0000 0001 (MSB not set),
  • 300 – 2 bytes, 1010 1100 0000 0010 (MSB set in the first byte, so there’s more data, MSB not set in the second (last) one)

Sounds better, but is it the best we can do? That depends on the typical distribution of values for our transmitted variable. For my purposes (damage/health) it still wasn’t perfect. My data was heavily skewed towards zero. Most typically values were in 0-230 range. Problem with 128 base varints is that values greater than 127 will not fit in a single byte (as we only have 7 bits)…

I decided to go with simpler solution that fit my data better – every ‘varint’ data starts with 2 bits telling us how many bytes follow.
Examples:

  • 1 = 10 bits, 00 0000001
  • 128 = 10 bits, 00 10000000
  • 300 = 18 bits, 01 100101100

It costs us additional 2 bits in every case, so in the worst situation we’re sending 34 bits, but this should happen once a blue moon. For
values from [0,127] range it’s a little bit worse than base 128 as it requires 10 bits instead of 8. However, for the [128-255] it still
only needs 10 bits (not 16). There’s no clear winner here, it all depends on your dataset, for me solution #2 was performing better (sending 10 bits in the overhelming majority of cases).

Null references

March 25, 2012 – 7:35 pm

One of the most popular questions that fresh C++ programmers ask is about differences between pointers and references and which one to use. One of the differences people cite is “references can never be NULL”. That’s true in theory and according to the standard, but in practice, especially when mixing pointers and references there’s nothing preventing you from doing this:

void Bar(Foo& f)
{
    f.x = 5;
}
...
Foo* pf = NULL;
Bar(*pf);

Now, technically, this code doesn’t conform to the standard and dereferencing a NULL pointer is an undefined behaviour (line 7). In practice, I have yet to see a compiler that gives a damn. Program will obviously crash later, when trying to write to x (line 3).

What’s interesting however, some compilers (I know of GCC/SNC, but there might be more) take advantage of the fact that standard conforming code cannot contain null references.

Read the rest of this entry »

GDC 2012/Mexico

March 13, 2012 – 5:17 am

Usually, around this time, I publish my collection of links to GDC presentations. This year however  haven’t been really paying too much attention, took a week off and went to Mexico. I’ve only started to catch up… Do not despair, though, Jare has done a great job and you can find links at his blog.

The trip itself was fun, although very different from our previous adventures as it was the first time we took our 10 months old daughter with us. She’s a natural born traveller though, didn’t complain even when our flight has been cancelled (stairs truck hit the plane just before the take off…) and we had to stay an extra night at the hotel. Some photos below, mostly hotel area, as I mentioned – we’ve been way less mobile than usual. BTW, if you’re looking for a sturdy camera, look no further – Panasonic DMC-GF1 is what you want. I managed to drop mine on a concrete floor, so hard that the lens unattached. I attached it back and it still works like before.

 

Darkness 2

February 12, 2012 – 9:14 pm

Darkness 2 poster Few days ago, we released Darkness 2, the game I’ve been working on for the past 18 months. I won’t write too much about the game itself, if you’re interested, you probably know about it already, if not – check out the demo. Usually, once the game is on the market, devs will keep on stressing how they were killing themselves in order to make the best product possible. It’s not fashionable to admit you were not working every weekend for the last 6 months. For some reason, the fact you can’t plan for shit has become the badge of honour. I feel we should talk more often about projects that didn’t require the team to live in the office.
Today, encouraged by an excellent Ted Price’s talk, I will do just that. First things first, though. I will write mostly about me and a little bit about our programmers, I didn’t monitor everyone’s working hours. This is definitelly not an official company statement, just my observations.
Let’s open with a bold statement: Darkness 2 has been the best managed project I have seen in my 10 years in the industry. It’s a 3 platform (simultaneous launch) AAA game with cooperative multiplayer, it uses our own in-house engine. We did not have a 200 person team. Admittedly, it’s been delayed by few months, but it was truly to polish the game some more, not to actually finish it. I’m not saying it was perfect or there was no crunch at all. Last ~2 months were quite tough, especially for the MP team, but it definitelly wasn’t the usual level of craziness. It varied from person to person, but one important thing is — no one worked harder than our leads (that’s not always the case) and it was truly voluntary. In my opinion main reasons we managed to keep it sane were:

  • the genuine desire to avoid crunch. This one should go without saying, most studios claim they try, but in many cases it’s still treated as necessary evil. Not to mention those studio heads who admit themselves they don’t believe you can make good game without crunch
  • game/bug state monitored at all times. When we entered the final bug fixing stage, we knew our pace, we knew how many bugs we had, we knew how many we had to fix roughly to hit zero. We had this little web site showing all this info (Dr Mario style, too!). We started doing this early enough to distribute the workload a little bit and ensure that daily bug quotas were reasonable.
  • our testing framework/build system. That’s a big one. As the game comes closer to completion it gets more and more risky and time consuming to change things. Every modification can potentially introduce a new bug. It’s quite often you fix one thing and break another. We have a nice safety net of smoke/soak tests that makes it much easier to verify our modifications. It’s not perfect (mostly because we don’t have tests for everything), but it’s much more extensive than I’ve seen in the past and it really helped.

It’s hardly a rocket science, I’m guessing most companies know this stuff already. It’s mostly about committing to point #1 and consequence. As I mentioned, there still was some chaos at the very end, we discovered some issues that probably should have been fixed sooner, but I’m confident we’ll use this experience to make sure our next project goes even smoother. I also hope that we’ll slowly see the change of attitude and working crazy hours will no longer be the point of pride in our industry. Working hard is important, working smart probably even more so.

Resolving PS3 callstacks

October 29, 2011 – 6:06 pm

Hopefully, your engine includes a crash handler. If it does not, stop reading, add one (here’s simple example) and then continue. In most cases, when something goes wrong, log files contains fully resolved callstack. However, sometimes, especially at the last stage and (semi) final builds, crash reporter might be disabled and now all we have is a list of addresses. Luckily, it’s still possible to resolve those, using Sony’s tools (that’s also what I use in PS3 version of MemTracer). It’s a little bit cumbersome, as you need to run commandline utility for every address. I had to do this recently and it was annoying enough to make me hack a quick Perl script that automates the whole process. It’s really simple, just copy your stack trace from log to text file and run ps3bin.pl self_file txt_file_with_trace . It’ll try to resolve any hexadecimal numbers it’ll find in the text file. Script can be found here.

The Broken Windows Theory

October 3, 2011 – 12:43 am

The broken windows theory is a criminological theory first introduced in 1982 article by James Q. Wilson and George L. Kelling. The gist of it is given in this example: “Consider a building with a few broken windows. If the windows are not repaired, the tendency is for vandals to break a few more windows. Eventually, they may even break into the building, and if it’s unoccupied, perhaps become squatters or light fires inside.”

I have first found out about it when reading about zero tolerance policy in NY in early nineties. Some studies claim it was one of main reason for sudden drop of crime in NYC during Bratton/Giuliani era (if you’re interested in one of alternative theories – read Freakonomics, not sure if it makes more sense, but surely is more entertaining).

If you think about it, similar analogy can be applied to software. We start with a clean plate and build something from nothing. Everyone has seen examples of mature code bases. Do you think their current shape matches the initial plan?

Read the rest of this entry »

Trailer time!

September 30, 2011 – 4:49 am

Two trailers released recently. Starbreeze showed first footage of their Syndicate remake:

I mostly worked on cancelled Bourne project when I was there, but lots of friends still at SBZ, so I’m glad to see their hard work is coming to fruition.

Few days before, 2K released trailer for the game I’ve been working on for the last 15 months – Darkness 2:

I really like it, mostly because it explains the setting/first game events a little bit. There’s still lots of work ahead of us, but we’re on the homestretch.

Optimizing without fear

August 21, 2011 – 5:59 pm

As you can probably tell from this blog, I spent fair chunk of my time optimizing code/thinking about optimizations. One of the main problems with optimizing code is making sure we didn’t break anything in the process. It might be tricky, especially when modifying algorithms with many edge cases, but even simple changes are dangerous, probably because people don’t test them so thoroughly. I still feel a little bit nervous upon seeing “Little optimization in XXX” in P4 changelist description (heck, I’ll be the first one to admit, I broke code as recently as last week…). What can we do to make optimization process a little bit less scary?

Read the rest of this entry »