Each of these questions targets a specific part of the asset pipeline that deserve consideration.  By reflecting on these points you can see ways to improve your own asset pipeline and workflow.

Runtime

Are shared assets duplicated unnecessarily?

Are instanced assets handled efficiently?  With minimal overhead and waste?

Do you support generated (procedural) assets seamlessly?

Are you wasting memory on strings or filenames?

Do you avoid unnecessary loading or reloading between levels?

Processing

Are you processing data to make runtime asset management more efficient?

Do you pack referenced assets together for faster loading?

Do you optimize disc layout based on asset usage?  (Duplicate data, sort of dependency graph, etc)

Content & Workflow

Is your source data versioned?

Can you re-create a previous version of your data?

Can you identify who last changed an asset? And why?

Are your tools integrated with your asset management system?

How many steps are required by a user to add a new asset?  How many to update or tweak an asset?

How long does it take to view a data change in game?

This is the layer where the content producers – artists, level designers, sound artists,  anyone who creates game content truly interfaces with the game engine. At this level we’re not just dealing with runtime data – we need to manage digital content creation (DCC) tool data, such as PhotoShop PSD files, Maya files, etc.  In addition to custom data types produced from in-house tools or game engines.

This layer is all about how content creators use and interact with data and the game engine.  The key components of this layer are:

• Asset tracking and versioning
• Tool integration and workflow

Tracking and Versioning

The central issue of asset management is not loading, processing or low-level data manipulation.  At its core asset management is all about identifying and tracking data.  We need to know where data comes from and what state it’s in.  This information is important for a number of reasons:

• It lets us track project progress – if we know only 3 of 10 levels have been built we know we’re at most only 30% done
• It lets us track bugs – if our licensed logo needs to be fixed because the TM is slightly off center we need to be able to find and fix all instances of that logo in our game
• Proper tracking can be used to identify problem areas – why do we have two brick wall textures?
• Tracking will identify how modifying an asset will affect the game – if you update the brick wall texture what levels are affected?

The bottom line is that asset tracking and versioning lets us know what data is in our game.  This may sound simple, but I’ve worked on projects where source data only existed on some artists desktop folder.  Data was put in the game simply by copying it to some network drive.

This creates huge problems when the artist leaves and their hard drive is formatted, or you try and port to a new system and can’t find the “c:\bobs_work\head.tga” file.  In a complete asset management system these situations shouldn’t be possible.  All game data would be properly tracked.

What do we mean by tracking data?  The idea isn’t really that different from managing code – any competent programming team is using source control and some form of bug tracking.  We just need to create a parallel for managing data.  The key features we want are:

• Version control - there is a need to see who changed what, when it was changed and the reason for change.  We also have need to revert to older versions of data for testing and bug fixing purposes
• Authoritative version – we need some way of identifying the “official”  version of our data
• Asset tracking – we need to know the path data takes to get into the game (e.g. head.psd is exported to head.tga which builds the texture “head”)

Version control can be done a multitude of ways with the simplest being using an existing source control system, like Perforce or Subversion.  Some, such as AlienBrain are designed to work with source art data.  Some systems are faster than others, some cheaper than others, but as long as you can add/remove/update/restore versions of your data your set to go.

Authoritative versions of data require you have some means of identifying (or re-creating) the exact version of data you intend to build.  It is important to know exactly what data is on the DVD you’re creating.  There has been more than one game that shipped with inappropriate “placeholder” assets still on the disc – later discovered by curious gamers.  Any proper version control system will let you do this easily – usually through a  tagging or branching mechanism.

Asset tracking can be difficult to solve with technology – there’s so many possible paths for data.  At the very least you need to be able to clearly associate your runtime data with the source data that generated it.  Whether this is done by some tracking software or simply file convention and documentation (e.g. runtime asset name = source file name), it’s important that the relation is well understood by both the game engine team and the content producers.

Just like with programming you manage your source information, not your output.  Like in programming, proper asset management requires version control.  Version control lets us know exactly what data is being used in our game.  This can be very important if you have to change something like a licensed logo that needs to be fixed or removed. It also lets us ensure we’ve backed up our data properly.  Another common reason for controlling source data is for sequels and ports – for good or bad outsourcing such products is very common, having versioned and controlled access to source assets makes this a more efficient  process.

Tool Integration and Workflow

Tools and workflow are important areas of game development. Workflow is especially important – with today’s content heavy games you need to be able to add, adjust and improve content efficiently.  Asset management plays a large part in a smooth workflow.

Game content is often authored by different people using a variety of tools.  Each step in this process – called the pipeline - may require a separate specialized tool.  This series of tools results in what we call a tool chain – basically a set of tools that we need work with our data.

To meet our goals of tracking and versioning assets, we should be managing the assets as they pass through the entire development pipeline.  It’s clearly desirable that asset management be integrated closely with our entire tool chain.  Close integration will then allow us to track and version data at each and every stage of our pipeline.

Tool integration means our tools understand our underlying asset management system.  This may simply mean that a tool can load/edit/commit changes to our source control.  It can be expanded to include automatically build runtime data whenever we make changes to source assets.  This tightly-coupled integration between tools, pipeline and runtime provides an efficient workflow for content creators.

How we integrate with tools varies a lot from tool to tool.  In-house developed tools can be most easily integrated since we have complete control over them.  External tools – such as Maya and Photoshop – are a bit more difficult.  Some, like Maya, allow for extensive customization and have their own scripting environments.  With effort we can mold them to integrate with our asset management systems.  Some tools have only minor customization – via plugins – or none at all.  This means we have a less than seamless integration, any may require some manual step to import/export data.   Alternatively we may choose to develop an external management system, such as a custom GUI asset management program, a Windows Explorer extension or even a lowly set of batch files from which to manage our assets.

Ultimately we need to make the experience of managing assets simple, seamless and error free as possible.  Remember – the target audience of these tools aren’t usually programmers and generally won’t fully understand the end-to-end pipeline.  So we should try to automate as much as possible with out removing control when needed.  There will always be a balance between simplicity and control, you’ll have to find your own balance to meet your project’s needs.

Workflow is what people do on a day to day basis.  The key there is people.  Workflow is all about how people operate and work with our game engine and tools.  Where the pipeline is the path that data travels to get into our game, the workflow is the collection of processes people follow to create and manipulate our assets.  The distinction is important because we can implement every technical requirement I’ve mentioned: versioning, optimized data packing, runtime ready data, and still have a poor workflow.

Where asset management comes in is with a managed workflow – which ties the human process of content creation to the underlying asset management system.  The exact details of a managed workflow are very task specific – each type of content, each stage of a process may have very different requirments.  However, a few common high-level issues that asset management can improve are:

• Locating - we have thousands of assets in our game, how do we find the one we need right now?
• Sharing - content may pass through many hands and tools, how do we smoothly pass data between different people?
• Validating - how do we know if our data is correct? Functional?  High quality?

Locating assets is actually a very large problem, difficult to locate data makes creating composite assets time consuming.  The simplest methods are simply searching by asset name – but this requires consistent and logical naming.  Creating a consistent and logical naming convention for assets is more difficult than it first sounds, and often leads to ugly names – e.g. dg043_bark.wav, hd_093_spec_03.tga.  Which may be perfectly logical – but not very easy to remember.

We should attempt to match the way we locate data with the way to use data.  This can be as simple as adding categories to each asset (tag-like metadata).  Perhaps it’s better to visually represent data if possible – a thumbnail gallery for textures is one possibility.  Even better still to allow both – categorized galleries.  The ultimate may be a context senstive editor that automatically displays appropriate assets – rock and tree textures when editing a forest level, bricks and rusty metal for your urban scenes.  However, the needs of your animator may be completely different.

Sharing assets is more and more important as teams grow and roles are specialized.  Once upon a time a single game artist may do it all – modeling, texturing, rigging and animation.  Now it likely that separate people – even separate teams – are resonsible for each of the above steps.  You need to optimize this workflow by providing an efficient way of passing assets through this pipeline.  Maybe just adding a “status” tag to each asset is enough, maybe you need to add some notification to the next person along the pipeline when a stage is completed.  Whatever you do you need to ensure that you manage these transitions when tracking your assets.

Validating assets involves testing the data in the actual game before releasing it to the wider team.  Each stage of a pipeline may need to be reviewed before passing along to the next.  In level design often a “level designer” will create a basic block level, pass it along to an artist to texture and polish, then pass it to a lighter who’ll add lighting to the scene.  At each stage we may need an art director or gameplay lead to review the work before passing along to the next person in the pipeline.  Asset management needs to accomidate these reviews, and similar to the sharing, allow the status of assets to be know.

Summary

The highest level of asset management you’ll notice there aren’t as many clear answers or technical solutions to the issues.  This is because content creation – where we design and compose our assets – is not about implementing technolgy to solve our problems.  Its about finding creative people and helping them create assets.

Hopefully you can now see how asset management is involved with every stage of the data pipeline and how we can improve our workflow and game engines by leveraging proper asset management.

   Vector3 position;
   Vector3 rotation;
   Vector3 scale;

• Asset referencing
• Building (or baking)
• Packing

Asset Referencing

Asset referencing is the act of having one asset use another.  A material using a texture for example.  We mentioned composite assets in the runtime layer but didn’t go into detail on how we should actually maintain those references, or links.

Some links are implicit and some are explicit.  Implicit links are ones that are assumed by the engine, for example a model named “monster3″ might always use the material named “monster3″.  Implicit links are based on game or engine specific rules and are quite common.  Explicit links are specified by the asset itself and are the general way assets reference each other.

The naive approach for supporting references is to store asset file names for each link.  This is quite straightforward to implement, but has a number of drawbacks.  File names require us to store and manipulate strings, which is waste of CPU time and memory. It also means assets are tightly coupled to our file system, which doesn’t always make sense for generated assets.

One way is to improve this is to assign each asset a unique id and reference the id instead of a file name.  This means we can save memory and processing time since we only need to store the asset id for each reference.  This id can be be assigned externally (from some central database) or simply be a hash of the asset name.

For example we could uses a simple CRC hash such as CRC(“Texture:Monster3Head”).  Hashes are nice because we can generate them on the fly if needed.   Alternatively, with assigned asset ids, asset references can remain stable even if the linked asset’s name changes.  In addition we don’t have to worry about hash collisions.  However, we do need a way to assign unique ids.

Either method means that we can store references compactly and uniquely identify each asset.  We’ll go into more depth about asset ids when we talk about the higher levels of asset management.

Data Building

Data building is actually a very involved process, involving converting data from artist created formats to runtime ready data.  A detailed description of the actual data conversion process, sometimes called baking, is outside the scope of this article.  However there are some important optimizations and considerations that are influenced by asset management.

One important optimization in building data is the idea of a load and go format – creating data than can be loaded and used right away with no post-load processing.  This can really help reduce memory fragmentation and the CPU cost of loading data. However the type of asset we are building will influence our data layout.

Let’s take an animated model as our example.  Our model will consist of a hierarchy of animated joints (Node) each with geometry (Mesh) attached.  One way of organizing this data could be:

struct Model
{
   int numNodes;
   Node nodes[numNodes];
};

struct Node
{
   Transform transform;
   Mesh mesh;
};

struct Transform
{
   Vector3 position;
   Vector3 rotation;
   Vector3 scale;
}

struct Mesh { ... };

This is a fairly straightforward way of processing our model.  However, since our model is animated we will need a per-instance copy of each Node’s transforms.  We could walk through the array of nodes and make a copy of each transform when we create a new instance.  But it would make updating the transforms when we draw difficult and messy as well a causing us to walk a relatively large structure.

By simply reorganizing the model’s data structures we can make a data layout that’s more friendly to asset management:

struct Model
{
   int numNodes;
   Transform transforms[numNodes];
   Mesh meshes[numNodes];
};

All we’ve done is separated the transform and mesh data into separate arrays, but now the data we want to copy per-instance is cleanly separated.  Also, it makes instanced models easier to handle at runtime, because we’ve separated mesh data from transform data we can more easily structure our code to handle transforms and meshes separately.

This is just one example of how we need to build our data with asset management in mind.

Data Packing

In my series on load times I mentioned the importance of packing files together to reduce load times.  The processing layer of asset management is where we perform this optimization and more.

Proper asset management allows us to track which assets reference each other. This means we can easily group these assets together on disc or within the same packfile.  In some cases we can merge assets into a single data set.

This allows us to intelligently duplicate data on the disc if needed. If an asset, such as a texture, is shared by multiple assets it may make sense to place multiple copies of the shared asset near assets that reference it.  We would then load the closest copy when the shared asset is needed.

In a open world game, where assets are streamed continuously as the player moves, we can use our asset management system to layout data such that data is organized by in-game proximity.  Since we know the world layout and the position of assets in the world, we can put assets that are close in game, close on disc.  Reducing seeks and allowing more efficient streaming.  We can also use our knowledge of asset references to determine how much memory is required to load a section of the game.

Clearly asset management, especially asset references, has many useful applications we can apply during the data processing phase.

Next article we’ll examine the highest level of asset management – where tools and designers interact with our engine and pipelines.

1. Raw data
2. Instanced data
3. Procedural assets
4. Composite data

Raw data refers to data that is static, meaning it doesn’t change during the game’s execution, and loaded off disc when needed.  Textures, sound, music, mesh data, character dialog, and many other types of content would be considered raw data assets.  I’d say that most data – when measured by bytes – falls into this category.  Raw data is the simplest type of asset to manage – it’s either in memory or not, and since it’s static you know it won’t be changed during gameplay.  This means raw data only needs to be loaded once and can be shared trivially.

Instanced data is similar to raw data, in that it represents a single piece of content.  The key difference is that some (or even all) of the data will be modified during game.  This means we can’t simply share this asset; some part of the data will be different each time we use this asset.  Which is where the term instanced data comes from: each instance of this data is slightly different.

A common example of instanced data is an animated model: the polygon data can be shared, but the animated pose of the  joints would be different for each character using the model.

Generated data is an asset that is generated by the game engine.  This type of data includes everything from 100% procedurally generated content to customized data, such as a texture that is re-coloured after loading.  Pretty much all user-generated content can be considered generated data.  Engine-generated data, such as shadow maps or particle vertex buffers, can be considered as generated assets.  The key point is that not all assets are necessarily loaded from disc, where raw and instanced data are usually represented by file, generated data can be more abstract.

It should be mentioned that generated data can also be static like raw data, or act like instanced data or be completely dynamic – as the case is for shadow buffers.

Composite data is an asset that references other assets.  Many assets are almost always build from smaller pieces of data.  Take a simple level from most games: it’s composed of sets of geometry, materials, textures, ambient sounds even gameplay scripts.  A level asset is really just composite data that tells the engine what other assets are needed.

In fact, most game content is composite data in some form, and proper management of these references that forms the goal of any asset management system.

An important point to note is that these asset classifications aren’t mutually exclusive.  You can have instanced generated assets, composite instanced assets, or just about any combination that makes sense.  It’s probably best to think of the above categories as asset properties.

For example you could have a composite generated asset.  It’s even possible to consider instanced assets as a combination of raw data – for the static portion- and a generated asset for the instanced portion.

Low-level Management

Now that we know the basic types of assets, we can figure out how to handle them in our runtime. Our goals are simple: we want to minimize resources (memory) while supporting the various asset types described above.

If we look at our description of a composite asset above, we can best picture it as an acyclic directed graph of data usage – one asset references one or more other assets.  The leaf nodes of our graph are the raw and instanced assets.

It’s now possible to design out system as a simple referenced counted graph.  When an asset is needed we load the referenced data and increment the count.  When we delete an asset, we decrement its references and unload data with a zero count.

This gives us nice and simple system.  To load an assets you just have to load or generate the associated assets and increment the reference count for each shared asset.  You continue recursively until you have no links and only raw data.  Each reference will performs the appropriate action for instanced assets, copying data as necessary.

An asset dependency graph doesn’t just allow us to track which assets need to be loaded, it allows us to reduce load times between levels.  Assuming that two levels have one or more assets in common, we can keep them in memory by loading intelligently:

1. Increment the refcount of assets in the next level
2. Decrement the refcount of assets in the current level
3. Remove all assets with refcount = 0
4. Load assets from the next level

That simple procedure will keep the shared assets in memory between levels.  The order is important – we must unload the unneeded assets from the current level or we may not have enough memory for the assets in the next level.  Also, loading and unloading assets this way can cause memory fragmentation, so we may choose to handle assets in groups or if possible defragment memory after step 3.  This also requires we have some way of knowing which assets are needed before we begin loading the next level.  We’ll look at these issues in the next article.

As we can see, basic refcounting provides a very simple, yet powerful, way of sharing assets.  Combined with proper data instancing we can manage our assets at runtime with a minumum of overhead and waste.

What exactly is an asset?

For the purpose of this series we’ll define an asset as a set of piece of data used by the game.  This is very inclusive, and should help illustrate the scope of the problem.  A texture, a model, an sound, an AI script, really just about any data used can be considered an asset.

What is asset management?

Its the process of tracking data used by the game: when it is used, how it is used and how individual pieces of data relate to each other.  For example, to load a character we need to need: a model, the material and shaders, the textures, the animation data, and possibly more (such as AI or motion graphs).  Asset management is an integral part of workflow, data processing and runtime optimizations.

Proper asset management is a major issue in game development.  Solid asset management is required to handle the amount of content modern games use.

In this series we’ll take a closer look at the major layers of asset management:

• Runtime Layer – Where the game engine deals with assets
• Processing Layer – Where assets are processed for the engine
• Content Layer – Where assets are created and used by designers

Each layer has its own issues to handle, and we’ll investigate these issues and look into possible management solutions.

Part 1: The Runtime Layer

Part 2: The Processing Layer

Part 3: The Content Layer

Finally we have a checklist of important points to consider when building and maintaining your asset management system.

So what does this have to do with game systems programming?

The more content you game has the better your tools have to be.  The best tools are robust, efficient and as seamless as possible.  The key to a seamless workflow is close integration with your tools and game engine.

There are basically two ways to accomplish this:

• Direct integration of the game engine with the tool
• Remote integration via a communication layer

Direct Integration

Essentially, direct integration is embedding you game engine into a tool.  How much you embed – just the rendering engine or the entire game – depends on the tool and the level of integration you need. Direct integration can provide the most seamless experience, since the tool is essentially running the game.  Changes in the tool can quite easily be immediately updated in the engine.  However this often incurs a great deal of engineering time. Consider the amount of effort required to implement a full-blown 3D editing package on top an entire game engine.

Often direct integration is reserved for custom tools that create and edit custom data.  For example it’s very common to have a particle effect tool that uses the game’s rendering engine and allows effect artists to view their changes in real-time.  However, embedding a game engine with a 3D package like Maya is a nearly impossible engineering feat.

Direct integration is often only really useful on games with a working PC version.  Many console-only games have some running PC version – often not near shipping quality, but functional.  The GUI limitations of console platforms means that they don’t make very good editing environments.  Real-time editing is usually reserved for some basic debug menu functionality.  Also screen space is limited on TVs, a problem PCs don’t suffer from.

Remote Integration

Remote integration refers to having a tool communicate with another process which may or may not be running on another device like a console.  The key point is that the tool and game engine are in separate processes, meaning you can’t just start poking at memory to change stuff.  You need a method of communication and a protocol for communicating.

The simplest and most common way is a simple IP connection between the processes.  This works well because TCP is well supported on most platforms so it’s fairly portable.  Not all consoles have a network interface suitable for development usage, usually they do support some sort of data connectivity and you can often implement a IP-like layer yourself.

Remote integration requires effort to expose the engine’s internals to the communication layer.  Each feature you want to integrate will usually need some sort of communication code.  The amount of effort directly depends on the level of integration desired.  Setting a specific variable can be done easily by sending a simple message (e.g. “SetColorRGB(1, 0.5, 0.3)”). If the engine has a scripting system already it may be best to simply integration your communication layer on top of the scripting engine, allowing you to script the game remotely.

It gets more complicated when you want real time data from the game displayed in the tool.  Networking often doesn’t have the bandwidth or latency to keep up with a 60fps engine.

Direct or Remote?

So how do you choose to integrate a tool direction or remotely?

Often you don’t have a choice – the tool may not support the level of customization required, this is common for 3rd party tools.  Even if you can integrate directly, you may choose not to because it’s just too much work.  Unfortunately some 3rd party tools don’t really support any customization, so integration of any kind is a moot point.

For first party tools you of course have the freedom do what ever you want.  So how do you decide?  Here are some of the main factors I take into consideration:

• What data does the tool work with?
• Who is the main user of the tool?
• What platforms are you targeting?
• How much effort is required?

The data a tool operates on is important – tools that work with large data sets like textures and models can be difficult to use through remote communication efficiently.  If a tool works on small data sets, like character attributes (hit points, damage), a remove communication can work quite effectively.  Where a directly integrated engine can update textures directly, a remove tool may have to settle for sending a “reload” message to the engine, and have the engine do the heavy lifting.

The user of a tool is also very important.  Some users require faster feedback than others.  Designers who are trying to balance your game will probably want real-time tuning of data.  However modelers don’t usually need that kind of instant feedback.  Not that it wouldn’t be useful, just not really needed.

The target environment is important, if you’re making a console-only game then you don’t care too much about tuning your windows build.  Sure, it should be a reasonable representation of your game, but what really matters is your final target platform.  Generally if you’re focused on console development, you want to manipulate your core data on the PC and be able to tune and adjust values on the console.

Of course effort of integration is very important.  A complex feature may need a very complex communication protocol to allow remote integration.  A old, or 3rd party tool may require a great deal of effort to directly integrate with a newer game engine.

In general – you want direct integration when dealing with large data sets or intensive operations.  For example a custom map editor works well with direct engine integration.   You can see changes to levels in real-time and with may game engine features instantly available.  For tuning simple values – like AI parameters or material and simple graphical settings – remote integration works well, and is very console friendly.

In the end…

The only thing that really matters is you have a solid, seamless way of interfacing your tools to your game.  Direct and remote integration and just the two basic approaches.  It’s quite feasable and common to have some support for both – like a PC tool that’s directly integrated with your engine and able to communicate remotely with your console version. Both your tools and workflow must be constantly evolving and improving and game-tool integration is a key component.

First things first, here’s the code I ran – it’s pretty much the same as the last test with a few fixes I noted in my “Revisited” post, like fixing the byte/char mistake and even getting rid of that pesky fixed keyword that had so many people worked up.

Here’s the direct results of 2005 vs 2008 (or perhaps better said CLR 2.0 vs 3.5), with lower being faster:

Ok, so the chart itself is a little crowded, but the trend is clear 2008 is a clear improvement over 2005.  Just how much is easier to see in this chart which show the percentage that 2008 has improved, higher being faster:

Now we’re getting decent improvement all around, with most of the generic sort versions doubling in speed.  The fact that we’re seeing a fairly constant improvement with respect to data size leads me to believe the improvement is entirely in the CLR rather than any algorithmic improvement in sorting.  Which is really no surprise, it’s not like CLR 2.0 was using bubble sort.

What else can we learn from these graphs?  Well for one, something is a little off with the Sort<Key, Items>.  The the percentage comparison the 1024-length array isn’t seeing the same improvements that the other lengths are seeing, also the first graph shows a hiccup.  If you look back at my original C++ article, you can see that the indirect sort needs a second array to copy into, so it’s pretty likely the C# sort is allocating some temporary memory behind our backs.

In the interest of a fair comparison to C++, I reimplemented my C++ indirect in C# to do a direct comparison and remove any “hidden” work that may be done in the Array.Sort<Key, Items> method.  (It’s the SortTestIndirect2 method in the code linked at the start, if you’re interested).  Let’s see how this hand rolled indirect sort compares to the one supplied in the Array class:

You can see that the custom indirect sort has a much straighter curve, especially on small sizes.  Since the graphs do converge at longer arrays, we’re mostly seeing a constant or near-constant overhead in the Array.Sort method.

Now for what every one’s been waiting for: C++ vs C# 2008.  This is the same C++ code ran previously, re-compiled with 2008 (with no appreciable difference from 2005) compared against the new C# indirect sort.

The end result is we’re seeing about 2.5 times difference between C# and C++.  That’s certainly better than the 10 times we saw against 2005 in the last comparison (admittedly comparing two different sort algorithms).

A two times speed difference is still significant, and will impact the maximum performance you’ll be able to squeeze out of C#, especially in intensive apps.  Which is a shame for people using XNA – because they’ll hit a performance wall much sooner than if they’d have used C++.  But still credit should be given where credit is due, the CLR folks over at Microsoft have been putting notable effort into optimizing C# since 2005.  Hopefully Visual Studio 2010 brings similar improvments.

That said, I’d like to address out the following issues in my testing:

1. I used a char array in C# to simulate a similar char array in C++.  This incorrect since C# chars are 16 bits while C++ they are only 8 bits.  I admit this was pretty sloppy.  Replacing char with byte fixes this, and the result is the C# version runs about 15% faster.  Faster, yes, but still not fast enough to make a significant difference.  The most striking thing is that halving the memory usage only gives a 15% difference, in C++ the memory had direct linear impact on runtime.
2. Some people complained about using the fixed keyword to make the array part of the Data struct.  It’s trivial to replace it with a number of int or float to create a 128-byte struct and makes no difference in performance.  The point was to approximate C++ and to represent a larger class, fixed was a simple way to do this.
3. Some people complained this isn’t an apple to apple comparison – I disagree.  It’s about as direct a way as possible of comparing two very different languages and operating environments – perform the same task and see how long it takes.
4. I was running Visual Studio 2005 SP1 for both tests, newer/different versions may have varying results.

The purpose of the previous post isn’t to claim that C++ is better than C#, but many people seem to think that I am – I suppose this is what I get for posting such an inflammatory title. I use C# on a daily basis and like it.

There were a number of comments on my last post claiming to “debunk” my tests referring to what is  posted here.  This debunking itself admits it’s comparing apples to oranges – however it completely misses the point of creating a fixed size array in the first place.   Replacing the fixed array with a heap pointer shows nothing about C# performance – only that copying a smaller struct is faster than copying a larger one.  This is a fact I addressed previously.

The claim is made that this test isn’t representative of what a C# programmer would write.  True, a C# programmer would very rarely used a struct with a fixed array.  However, it’s fairly easy to create objects with 128-bytes or more of data.  That’s 32 4-bytes variables, which isn’t a huge amount of variables to have in a class, especially one with a few parent classes each with a few member variables.

Anyways, other that the char/byte mistake, I stand by the original results – C# takes about 10 times longer to sort an array on my computer, compared to a similar C++ verision.

Now given how straightforward the test is – sorting an array – I’d expected performance to not differ by much compared to the C++ version. Before I ran this code I was guessing that C# would be maybe 50% slower and at most half the speed than the C++ version.  Especially since the sorting should be mostly algorithmically and memory bandwidth bound.

You can get both versions of the code here: C++, C# to test for yourself. [EDIT: fixed link to C# code]

Now for the results.

[EDIT: a number of issues were raised about this test in the comments and elsewhere - here's my response]

The first issue was to figure out how to actually sort my array in C# – there are a variety of ways to store and sort objects in C# so I first tried the various combinations of Sort & ICompare,  Sort<T> & ICompare<T>, using a Comparison function pointer and even C#’s built-in indirect sort Sort(Keys, Items).  Let’s set the raw results:

There’s nothing too surprising really – the generic version of Sort<T> is faster than the non-generic one, probably because the CLR can remove dynamic casts or something.  Also no surprise that the indirect Sort<Key, Items> version is much, much faster than all the other version – this is what we looked into in my original Cache Pressure post.  The only surprise to me is that the IComparer version of the sort is faster than the Comparison function pointer version – I have no idea why, I’d expect both to be nearly identical.

Anyways, we’re here to compare C++ to C#, so lets put the results from our Cache Pressure test side by side with these C# tests:

So here’s the graph of the C++ direct sort vs the generic Sort<T>, IComparer<T> version.  In case you can’t read that properly, the C++ version is 10 times faster than the C# version.  You can also see that the lines are nice and parallel, so that’s no algorithmic difference in sorting routines, both implementations have the same runtime complexity.  We’re seeing 10 times overhead in execution.

Damn!  That’s pretty pathetic C#!

And you know what’s even sadder?  Here’s a chart of the slow direct C++ sort versus the fastest indirect C# sort:

So C++ still comes out  a head, over twice as fast in fact.  Considering this is the “fast” C# implementation and the “slow” C++ implementation beats is hands down, you can’t help but be disappointed by C# here.  I won’t even bother comparing C#’s sort to C++’s faster indirect sort, we already know the outcome.

Conclusion

So mostly I’ll let the number speak for themselves, the programs I ran are freely available for you to confirm for yourself.  I don’t pretend to be any type of C# expert, but my code is pretty much a textbook (or MSDN example) implementation.  I even use the exact timing method as the C++ version.

I was very surprised by how much faster C++ ran.  I never expected a 10 times difference in performance.  This doesn’t mean I’ll stop using C# – it’s still a very productive development environment, especially for GUI tools, which is still often a more important factor than exectuion time.  It does mean that I’ll probably think twice about implementing high-performance code in C#.

Let’s look at a common example – sorting.  Sorting is a computer science staple, and is very common in the real world.  A common usage in games is sorting draw calls.  There are a number of reasons to sort draws: front-to-back to optimize for Z rejection, back-to-front for alpha sorting and by shader or material to reduce state changes.

A naive implementation might look something like this:

class DrawCall
{
   int sortKey;
   bool operator< (const DrawCall &dc) { return sortKey < dc.sortKey; }

   int shaderIndex;
   Mesh *mesh;
   RenderState state;
   Color color;
   Matrix baseMatrix;
   // etc...
};

Basically we have a class that stores the information we need to draw a mesh and it’s sorting index or key.  We can simply sort our list of visible objects like so:

std::sort(drawCallList, drawCallList + numDrawCalls);

for (int i  = 0;  i < numDrawCalls; i++)
{
   drawCallList[i]->Draw();
}

Simple right? Yep.  Fast? Well, let’s look at some numbers.

Here’s the time in milliseconds it took to sort a similar 512 byte structure, for just up to just over a million items.

We don’t sort millions of items each frame in a normal game.  We do however often sort hundreds or a few thousand of draw calls in a single frame.  Our chart shows this will take 2-10ms to sort that many items.  That is a big chunk of time, and really is too slow to be useful.  We’re putting too much pressure on the cache. Or at least that’s what I claim.

Let’s take a more scientific approach and do an experiment or two.  My hypothesis is that our 512-byte object is too large and is putting too much pressure on the cache.  So by reducing the size of the object we’ll reduce the time required to sort.  Here are the results of sorting items of various size:

Ok, that’s a pretty clear picture.  The larger the object we’re sorting the longer it takes, as predicted. But what’s really important is the impact the size has on performance – 4 byte objects (well, 4 bytes + a 4 byte sorting key) sort almost two magnitudes faster than 512 byte objects.

The reason for this is pretty simple.  At the center of any comparison based sort (like the std::sort above) we’ll have a line like this:

if (item[i] < item[j])
   swap(item[i], item[j]);

That means for each comparison that swaps we’re copying two large objects structures, touching over hundreds of bytes of memory.  Those copies are putting a lot of pressure on the cache.  Especially since we only really need the sort key to order our DrawCall objects, and it’s only 4 bytes.  We have a lot of wasted memory access.

An improved method sorts a second indirect list of smaller structures than can then be used to access or sort the original list of larger objects.

struct IndirectObject
{
   int sortKey;
   int objectIndex;

   bool operator< (const DrawCall &dc) { return sortKey < dc.sortKey; }
}

We can initialize a list of these indirect objects from the original list, sort them, and then use the sorted indirect objects to access our larger objects in sorted order.

for (int i  = 0;  i < numDrawCalls; i++)
{
   indirectList[i].sortKey = drawCallList[i].sortKey;
   indirectList[i].sortKey = i;
}

std::sort(indirectList, indirectList + numDrawCalls);

for (int i  = 0;  i < numDrawCalls; i++)
{
   int index = indirectList[i].index;
   drawCallList[index]->Draw();
}

So, just how much faster is this?  Let’s take a look:

Again we’re looking a order of magnitude in speed difference.  The indirect line represents sorting the indirect array and then copying the draw calls into sorted order – so the end result is identical to the directly sorted array.  The no-copy line represents the time to sort the indirect array.  So if you only need to access your items in sorted order you’re probably best sorting an indirect array accessing your list indirectly.  This also has the advantage that you can keep sorted arrays around for different purposes – front to back, by material, etc., with out having to re-sort all the time or have multiple copies of the list.

Here’s the code if you want to run the tests yourself.

I think this is one of the clearest examples you’ll see on the impact of memory on performance.  This technique can be applied in a number of data structures to reduce the pressure on the cache. and as demonstrated can significantly affect performance.