Some more benchmarks

October 26, 2010 – 12:30 am

Gave tr1::unordered_map a try, it seems to do perfectly average, nothing spectacular, nothing awful, except for predict_grow with 4 byte objects, where it’s almost 10x slower than the next implementation (that’s not an anomaly, it happens every time).

I also found a bug in RDE where I overoptimized it a little bit – could result in a crash when searching for item that’s just been removed. It slowed fetch by about 3%, however later I found a way to remove some branches and sped it up by ~6% again, so final version is even faster. It made fetch_empty much slower (no early exit), but I don’t care about this scenario too much (how often do you actually care about quick lookup in an empty map?) and am willing to trade it for higher performance in 99% of cases. x86 (desktop machine) graphs below (two versions for 4 byte objects, as TR1 skewed the results quite a bit):

graph4

graph4

graph16

graph256

I also ran tests on X360 (only dense_hash_map, EASTL & RDE). It should be said, they’re not terribly realistic, especially ‘fetch’, after all it’s not normal to just sequentially fetch 50k objects one by one. To make it at least a tiny bit more interesting I added fetch2 test, which does the following:

  • objects are fetched in a random order, not sequentially,
  • every 64-th iteration, it tries to find item that’s not present in the map.

Xenon results (nice thing about them is they’re much more consistent, differ by maybe 1-3ns from run to run):

graph4

graph16

graph256

Obviously, it’s still quite far from real-life applications, so it’s best to just plug in some of those implementations and give them a test drive.

  1. 4 Responses to “Some more benchmarks”

  2. very nice! looking forward to “move semantics” in RDESTL.

    By xp on Oct 26, 2010

  3. Could you please provide a source code of your test suite? I find the results you provided somehow confusing, because it contradicts the results of my company’s internal benchmarks for stl containers. Maybe you’re using the custom hash function for std::hash_map\unordered_map or something like that..

    By Mike on Jan 18, 2011

  4. Sure, but it’s not exactly ‘my’ test suite. I use Google’s sparsehash – http://code.google.com/p/google-sparsehash/ (time_hash_map.cc).

    By admin on Jan 19, 2011

  1. 1 Trackback(s)

  2. Oct 27, 2010: Tweets that mention More benchmarks | .mischief.mayhem.soap. -- Topsy.com

Post a Comment