Systematic Gaming

January 13, 2009

Performance: C# vs C++ – Part 2

Filed under: c++ vs c#, optimization, profiling — Tags: , , — systematicgaming @ 1:01 pm

I was pretty content with my previous tests on C# sorting performance, which were pretty disappointing.  Normally I’d be happy to move on, but one commenter did pique my curiosity pointing to a MSDN article about how lousy the 2.0 CLR is with structs.  So I downloaded the Visual Studio 2008 Express Edition to give it a try.  I admit, I was a bit surprised and impressed with the differences.

First things first, here’s the code I ran – it’s pretty much the same as the last test with a few fixes I noted in my “Revisited” post, like fixing the byte/char mistake and even getting rid of that pesky fixed keyword that had so many people worked up.

Here’s the direct results of 2005 vs 2008 (or perhaps better said CLR 2.0 vs 3.5), with lower being faster:

cs_2008_20052

Ok, so the chart itself is a little crowded, but the trend is clear 2008 is a clear improvement over 2005.  Just how much is easier to see in this chart which show the percentage that 2008 has improved, higher being faster:

cs_2008_2005_percent

Now we’re getting decent improvement all around, with most of the generic sort versions doubling in speed.  The fact that we’re seeing a fairly constant improvement with respect to data size leads me to believe the improvement is entirely in the CLR rather than any algorithmic improvement in sorting.  Which is really no surprise, it’s not like CLR 2.0 was using bubble sort.

What else can we learn from these graphs?  Well for one, something is a little off with the Sort<Key, Items>.  The the percentage comparison the 1024-length array isn’t seeing the same improvements that the other lengths are seeing, also the first graph shows a hiccup.  If you look back at my original C++ article, you can see that the indirect sort needs a second array to copy into, so it’s pretty likely the C# sort is allocating some temporary memory behind our backs.

In the interest of a fair comparison to C++, I reimplemented my C++ indirect in C# to do a direct comparison and remove any “hidden” work that may be done in the Array.Sort<Key, Items> method.  (It’s the SortTestIndirect2 method in the code linked at the start, if you’re interested).  Let’s see how this hand rolled indirect sort compares to the one supplied in the Array class:

cs_2008_2005_indirect

You can see that the custom indirect sort has a much straighter curve, especially on small sizes.  Since the graphs do converge at longer arrays, we’re mostly seeing a constant or near-constant overhead in the Array.Sort method.

Now for what every one’s been waiting for: C++ vs C# 2008.  This is the same C++ code ran previously, re-compiled with 2008 (with no appreciable difference from 2005) compared against the new C# indirect sort.

cs_cpp_vs2008

The end result is we’re seeing about 2.5 times difference between C# and C++.  That’s certainly better than the 10 times we saw against 2005 in the last comparison (admittedly comparing two different sort algorithms).

A two times speed difference is still significant, and will impact the maximum performance you’ll be able to squeeze out of C#, especially in intensive apps.  Which is a shame for people using XNA – because they’ll hit a performance wall much sooner than if they’d have used C++.  But still credit should be given where credit is due, the CLR folks over at Microsoft have been putting notable effort into optimizing C# since 2005.  Hopefully Visual Studio 2010 brings similar improvments.

Advertisements

4 Comments »

  1. What strikes the fear of god into me is the thought that in a very processor intensive app – a game, or image processing – you might have tens of operations similar to the sort tested above all nested in a complex way, running over and over, updating a UI. The slowdowns in this case are going to multiply, so you may end up with an app that is 5,10,50 times slower than the equivalent coded in C++.

    Of course, soon we’ll have double the number of cores to use, and the CCR to make it easy to coordinate all that parallelism.

    Comment by Ben0 — January 29, 2009 @ 5:55 pm

  2. > Which is a shame for people using XNA – because they’ll hit a performance wall much sooner than if they’d have used C++.

    Not really. Because they’ll be getting more done in a more expressive language.

    Granted, AAA, $20M budget titles made by 100+ person teams will get more performance out of C++ than XNA. Not going to argue with that. A or B titles made by teams of 1-10 people in their spare time have got a much better chance of actually, y’know, finishing the game than if they tried to use C++.

    > The slowdowns in this case are going to multiply, so you may end up with an app that is 5,10,50 times slower than the equivalent coded in C++.

    And yet people are making games in XNA. AAA chart-topping world beaters, perhaps not. But they are doing it. How it that even possible??!??!?!?!?!!!1!!!1?!one!!?

    Comment by bittermanandy — July 17, 2009 @ 1:16 pm

    • As someone who regularly uses both C++ and C#, I’m not sure I agree that C# is really any more expressive than C++. Sure it has some nice features like reflection, and I’ll agree it’s a bit more forgiving language and probably easier to learn, but not more expressive. I’ve often wished for a more flexible generics system, the C# is often too restrictive compared to C++’s templates.

      It’s mostly the sane and thorough libraries that makes C# productive and nice to use compared to C++.

      Comment by systematicgaming — July 18, 2009 @ 12:32 am

  3. its a simple thing c# is better than c++ but truly speaking they are all the same

    Comment by NIIT — July 12, 2011 @ 8:00 am


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: