Systematic Gaming

January 3, 2009

Performance: C# vs C++

Filed under: c++ vs c#, optimization, profiling — Tags: , , — systematicgaming @ 6:02 am

In my last post I discussed cache pressure and how important the cache is for performance.  Out of mild curiosity and a bit of boredom, I decided to see how the same code (or near enough) performed in C#.  Why C#?  A lot of tools are written in C# now, mostly because it’s a nice language with fantastic library support and solid GUI integration.  It’s a more productive environment for all those programs that don’t have to run at 60 fps.

Now given how straightforward the test is – sorting an array – I’d expected performance to not differ by much compared to the C++ version. Before I ran this code I was guessing that C# would be maybe 50% slower and at most half the speed than the C++ version.  Especially since the sorting should be mostly algorithmically and memory bandwidth bound.

You can get both versions of the code here: C++, C# to test for yourself. [EDIT: fixed link to C# code]

Now for the results.

[EDIT: a number of issues were raised about this test in the comments and elsewhere - here's my response]

The first issue was to figure out how to actually sort my array in C# – there are a variety of ways to store and sort objects in C# so I first tried the various combinations of Sort & ICompare,  Sort<T> & ICompare<T>, using a Comparison function pointer and even C#’s built-in indirect sort Sort(Keys, Items).  Let’s set the raw results:

cs_cpp_various

There’s nothing too surprising really – the generic version of Sort<T> is faster than the non-generic one, probably because the CLR can remove dynamic casts or something.  Also no surprise that the indirect Sort<Key, Items> version is much, much faster than all the other version – this is what we looked into in my original Cache Pressure post.  The only surprise to me is that the IComparer version of the sort is faster than the Comparison function pointer version – I have no idea why, I’d expect both to be nearly identical.

Anyways, we’re here to compare C++ to C#, so lets put the results from our Cache Pressure test side by side with these C# tests:

cs_cpp_vs

So here’s the graph of the C++ direct sort vs the generic Sort<T>, IComparer<T> version.  In case you can’t read that properly, the C++ version is 10 times faster than the C# version.  You can also see that the lines are nice and parallel, so that’s no algorithmic difference in sorting routines, both implementations have the same runtime complexity.  We’re seeing 10 times overhead in execution.

Damn!  That’s pretty pathetic C#!

And you know what’s even sadder?  Here’s a chart of the slow direct C++ sort versus the fastest indirect C# sort:

cs_cpp_vs2

So C++ still comes out  a head, over twice as fast in fact.  Considering this is the “fast” C# implementation and the “slow” C++ implementation beats is hands down, you can’t help but be disappointed by C# here.  I won’t even bother comparing C#’s sort to C++’s faster indirect sort, we already know the outcome.

Conclusion

So mostly I’ll let the number speak for themselves, the programs I ran are freely available for you to confirm for yourself.  I don’t pretend to be any type of C# expert, but my code is pretty much a textbook (or MSDN example) implementation.  I even use the exact timing method as the C++ version.

I was very surprised by how much faster C++ ran.  I never expected a 10 times difference in performance.  This doesn’t mean I’ll stop using C# – it’s still a very productive development environment, especially for GUI tools, which is still often a more important factor than exectuion time.  It does mean that I’ll probably think twice about implementing high-performance code in C#.

35 Comments »

  1. It’s even worse on the 360. C# oan the 360 can be 10x slower than C# (nevermind c++) on Windows.

    Comment by Dan — January 3, 2009 @ 2:47 pm

  2. A couple of quick questions:

    What were compilers? Where they both MS products?

    Also, have you tried other C++ compiler like MING or GCC?

    Was the C# compiled on .NET 2008 or MONO?

    Comment by Ervin Kosch — January 3, 2009 @ 4:06 pm

    • I used Visual Studio 2005 SP1 for both C++ and C# tests on Windows XP.
      Not the latest and greatest, but it’s what I had installed.

      Both tests were done using the default “release” compiler settings.
      I haven’t tried other compliers, I’d have to find an appropriate high-resolution timer function to port the tests.

      Comment by systematicgaming — January 3, 2009 @ 4:23 pm

  3. [...] you were on reddit today, you probably saw this article, damning C#’s performance as being ten times worse than C++’s. Holy shit balls, [...]

    Pingback by journal.stuffwithstuff.com » Blog Archive » Debunking C# vs C++ Performance — January 3, 2009 @ 9:28 pm

  4. Using a struct with a fixed array for your C# data type seems a bit… odd… to me. So I did some comparison using a regular C# class. Results are here:

    http://journal.stuffwithstuff.com/2009/01/03/debunking-c-vs-c-performance/

    Comment by bob — January 3, 2009 @ 9:35 pm

  5. unsafe struct Data
    {
    public int key;
    public fixed char data[128];
    }

    That’s some pretty lousy C# code, not wonder you are seeing performance degradtion.

    Comment by Jonathan Allen — January 3, 2009 @ 10:06 pm

  6. … “…C# and C++ performance are neck and neck, if not a bit faster in C#”

    Comment by Anonymouse — January 3, 2009 @ 10:10 pm

  7. Please see… http://journal.stuffwithstuff.com/2009/01/03/debunking-c-vs-c-performance/

    Comment by anonymous — January 3, 2009 @ 11:03 pm

  8. This debunks nothing.

    In fact it misses the point entirely that was made in the original article about cache pressure.

    By making the data a pointer and not a fixed block of memory you drastically reduce the cost of copying the structure.

    Comment by systematicgaming — January 4, 2009 @ 1:08 am

  9. I would disagree that it doesn’t debunk your idea. You are essentially forcing the language to fail by using it in a way that is not normal.

    Comment by John — January 4, 2009 @ 1:35 am

  10. I find this rather disturbing if C# were used to develop any form of number crunching application (like gaming). Then again, it is probably wise to develop those subroutines as a separate engine in C or C++ and call it from C#.

    In any case, great article!

    Comment by aeka78 — January 4, 2009 @ 3:13 am

    • May be you should have a look on sdrsharp.com. I’s one of the rare examples where C# compares very well to C++ in number crunching even with intrinsics.

      Comment by Anonymous — October 12, 2012 @ 5:48 pm

  11. unsafe struct Data
    {
    public int key;
    public fixed char data[128];
    }

    Only a moron would use that structure in C#. You are trying to fix the results by comparing something that nobody in their right mind would do.

    The whole point of garbage collection in C# is that you do not have to do crap like you have.

    You can make any language look bad when you have no idea how to use it

    unsafe – says it all lol

    Comment by Woz — January 4, 2009 @ 5:23 am

  12. “This debunks nothing.

    In fact it misses the point entirely that was made in the original article about cache pressure.

    By making the data a pointer and not a fixed block of memory you drastically reduce the cost of copying the structure.”

    lol, why would you NOT want to reduce the cost of copying the structure? If that is your view why not add more crap into the unrealistic structure to make the language look even worse.

    It’s not C#’s fault C++ has no GC.

    Use the language how its meant to be used. The GC is a wonderful tool, since leaving C++ behind I have never looked back :)

    Comment by Woz — January 4, 2009 @ 5:29 am

  13. This test has nothing to do with garbage collection, simply the time taken to sort identical arrays in two languages.

    Comment by systematicgaming — January 4, 2009 @ 6:01 am

  14. My advise is to learn C# before you put it to the test. The test here is…. well…. nothing else than a confirmation of the lack of C# experience the tester has. NO ONE using C# even thinks in such constructs as shown here unless they have a gun pointed to their head (or insane legacy concerns to cope with)…

    Comment by jason — January 4, 2009 @ 4:19 pm

  15. Quite frankly, even in C++, if I were concerned about performance and cache misses, I wouldn’t be sorting such a giant data structure. I would create references to the data structure and sort those. In my opinion, this mostly proves that one language performs more poorly on bad code than another. However, I guess to those that try to do such things in their code, this is useful information.

    Comment by Deon — January 5, 2009 @ 6:26 pm

  16. [...] Performance: C# vs C++ In my last post I discussed cache pressure and how important the cache is for performance.  Out of mild curiosity and [...] [...]

    Pingback by Top Posts « WordPress.com — January 6, 2009 @ 12:10 am

  17. Thats true no doubt that C++ is way faster than C#. I’ve seen some foolish people doing propaganda against C++ is slower than C# and Java. Thats why I decided to do benchmarking on my own. I peformed FFT and did it several times. C++ was always more than 3 times faster than C#.

    Comment by Hamid Mushtaq — January 31, 2009 @ 3:32 pm

    • Well, I personally blame your code :)
      I’ll write an *optimized* FFT in C# that’s just as fast as your C++ code anyday.

      It’s all about knowing the language – one can write slow C++ code just as well as one can write slow C# code.

      Comment by Led — December 9, 2009 @ 9:51 am

      • If you can write a faster sort, please do so and post the results. My code is available for optimization, please create the faster C# sort so we may all benefit.

        Comment by systematicgaming — December 9, 2009 @ 11:11 am

  18. I just pissed off the one of the Mods and GameDev.net by claiming that on average C++ is 3X to 10X faster than C#. I have done many tests over the years and I tend to get these results. However if I say anything, C# guys jump all over me. It’s not even hard to see why this is true. C++ goes though the optimizer and you can control memory allocation yourself. This can mean nothing more than not being forced to use pointers everywhere. C# has them everywhere and it hides them from you. I’m sorry but I’ve been programming for well over 27 years and I have never seen C# beat C++ on anything worth running. Every time someone challenges me I can always rewrite their C# code in C++ and it runs several X faster. But I still have to put up with the BS excuses of cache this, and GC that, and this algorithm is not suited to C# bla bla bla … it’s endless. If you want to claim C# is easier to code I won’t argue but C++ crushes it for performance.

    Comment by Polypters — April 17, 2009 @ 2:50 am

    • authority-arguments are invalid.
      If you know the language it’s perfectly feasible to use un-GC’ed memory, unsafe pointer-access etc. – which makes it just as fast as C++.

      Just because you don’t *have* to doesn’t mean you *can’t* write pedal-to-the-metal fast code in C# ;)

      The thing about C# is that for 95% of the time you have to worry about a lot less stuff, and the other 5% it enables you to write fast code just as well as C++ does (think of the keywords fixed, unchecked, unsafe, stackalloc etc.)

      Comment by Led — December 9, 2009 @ 9:55 am

  19. Did your benchmark suite make sure that the C# code had been run once before the test pass to JIT compile the code? If not then your benchmarks will be off. When benchmarking .NET code you want to use a two pass approach to make sure that the code is loaded and the JIT compiler has run.

    Ade

    Comment by Ade — May 21, 2009 @ 8:17 pm

    • Well you can see the code for yourself. The sort is run multiple times and averaged, so the initial JIT time would be amortized across the runs, but not removed entirely. Its arguable whether the JIT cost should be included as part of the timings or not. Either way, it’s highly unlikely the JIT cost would explain the order of magnitude difference between C# and C++.

      Comment by systematicgaming — May 22, 2009 @ 12:09 am

  20. Hmm, you should really be comparing Java vs C#…

    Comment by David — November 10, 2009 @ 3:14 pm

    • Why? I’m not interested in Java. I program in C++ and C#, and therefore desire a better understanding of their relative performances. Knowing Java’s performance characteristics is of no interest to me.

      Comment by systematicgaming — December 9, 2009 @ 11:04 am

  21. I bet an experienced C# programmer without any C++ experience can write code that proves exactly the opposite by writing bad C++ code :)

    Basicly he would do to the C++ code what you did to the C# code : try to port it directly without an understanding of the language and the way it’s supposed to be used.
    Which means he would first write a garbagecollector in C++ etc, and then shout it wouldn’t perform up to par with C# code…. :)

    Comment by Led — December 9, 2009 @ 9:59 am

  22. run all tests
    best results for size = 524288, sort type – indirect
    original C# – 1417
    debunking-c-vs-c-performance C# 462
    original C++ – 77
    462 / 77 = 6
    C++ 6 times faster than C#

    Comment by 15 — February 17, 2010 @ 5:39 am

  23. C# can actually have better optimizations for gamedev than C++, for example, look at Mono.SIMD, check this out: http://tirania.org/tmp/xpqetp.png

    You better don’t complain and help mono do better optimizations :\

    Mono looks like a promising platform for gamedev, at least you can do 1% bottleneck in c (like sorting 1 million objects, lol), and other 99% stuff – in C#

    Comment by cas — May 21, 2010 @ 9:35 pm

  24. The poster is absolutely correct, if you compare brute force, C++ is ALOT faster than C#.

    The thing is that C# in this case allows for optimization not available in C++, wich makes the algorithm run as fast as a C++ version.

    Does this prove that C# is slow? not really. It shows that C# is slower at doing things C++ is good at, but with the proper C# implementation of the task at hand, it is as fast as C++.

    Of couse you say it is unfair because C# doesent have to copy as much data and so forth, but as C# allows you to do basicly the same thing WITHOUT copying data, why should you not use it?

    In my experience, C++-programmers complaining about the slowness of C# are just plain lousy C#-programemrs that try to program C# like C++, wich results in VERY inefficient code.

    Comment by Stig-Rune Skansgård — May 26, 2010 @ 10:57 am

    • Ok please do this again .. Write a C++ sort that does the same as C# here…

      It has nothign to do with cache pressure.

      It has to do with the C# version creating objects and passing more objects in the paramaters and using a write barrier. Because of the GC write barrier safe writes are slower as it updates a memory used table on each write.

      You parse in a new object (IComparer) which can modify the search on any part of the object. It is not a function pointer – it is a reference to an object. Later .NET version do have some new methods that take a delegate as a paramter.

      This is not an apples to apples comparison..

      In fact C# doesnt have a simple sort ( the last one calls the default comparer for int) and using one would IMHO be considered premature optomization.

      If really needed its easy enough to write but only if performance of that part is critical and you would use unsafe to avoid the write barrier.

      But people who write C# vs C++ dont really understand C#.

      Comment by ben (Shanghai Software) — July 18, 2010 @ 3:50 am


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Silver is the New Black Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: