Systematic Gaming

July 16, 2009

Asset Management – Content

Filed under: asset management, workflow — Tags: , , — systematicgaming @ 6:14 am

We’ve already looked at the runtime and processing layers of asset management, now we’re ready to look at the most involved and complex layer – the content layer.

This is the layer where the content producers – artists, level designers, sound artists,  anyone who creates game content truly interfaces with the game engine. At this level we’re not just dealing with runtime data – we need to manage digital content creation (DCC) tool data, such as PhotoShop PSD files, Maya files, etc.  In addition to custom data types produced from in-house tools or game engines.

This layer is all about how content creators use and interact with data and the game engine.  The key components of this layer are:

  • Asset tracking and versioning
  • Tool integration and workflow

(more…)

Advertisements

July 2, 2009

Asset Management – Processing

Filed under: asset management, game programming — Tags: , , — systematicgaming @ 4:21 am

Last time we looked into the lowest levels of asset management, which can be handled by referencing counting and intelligent loading.  With the next level, the processing layer, we look at how we prepare assets for our runtime.  This is an important step and where a lot of optimization occurs.  We can break the processing layer down into a few distinct stages:

  • Asset referencing
  • Building (or baking)
  • Packing

(more…)

June 25, 2009

Asset Management – Runtime

Filed under: asset management — Tags: , , — systematicgaming @ 1:00 am

The runtime layer is the lowest level of asset management encompassing:  file loading, reference counting, instancing and even procedural asset generation.  To meaningfully discuss handling assets at runtime we’ll need to define the different types of assets.  Here’s a few basic types of assets:

  1. Raw data
  2. Instanced data
  3. Procedural assets
  4. Composite data

(more…)

June 24, 2009

Asset Management

Filed under: asset management — Tags: , , — systematicgaming @ 2:25 am

Modern games have gigabytes of data, and thousands of individual assets.  Managing all this data well can be a very complex task, and impacts game development at all levels, from concept artist to low level bit-pushing coder.  The next few articles will look at the issues of asset management in games.  First we must clarify our goals with some simple definitions.

What exactly is an asset?

For the purpose of this series we’ll define an asset as a set of piece of data used by the game.  This is very inclusive, and should help illustrate the scope of the problem.  A texture, a model, an sound, an AI script, really just about any data used can be considered an asset.

What is asset management?

Its the process of tracking data used by the game: when it is used, how it is used and how individual pieces of data relate to each other.  For example, to load a character we need to need: a model, the material and shaders, the textures, the animation data, and possibly more (such as AI or motion graphs).  Asset management is an integral part of workflow, data processing and runtime optimizations.

Proper asset management is a major issue in game development.  Solid asset management is required to handle the amount of content modern games use.

In this series we’ll take a closer look at the major layers of asset management:

  • Runtime Layer – Where the game engine deals with assets
  • Processing Layer – Where assets are processed for the engine
  • Content Layer – Where assets are created and used by designers

Each layer has its own issues to handle, and we’ll investigate these issues and look into possible management solutions.

Part 1: The Runtime Layer

Part 2: The Processing Layer

Part 3: The Content Layer

Finally we have a checklist of important points to consider when building and maintaining your asset management system.

January 31, 2009

Game-Tool Communication

Filed under: game programming, workflow — Tags: , , — systematicgaming @ 12:20 am

Modern games are extremely content heavy – most of the people on a game team are part of the art or design departments.  The larger a the team get the greater the portion of content producers becomes, and the trend will only continue.

So what does this have to do with game systems programming?

The more content you game has the better your tools have to be.  The best tools are robust, efficient and as seamless as possible.  The key to a seamless workflow is close integration with your tools and game engine.

There are basically two ways to accomplish this:

  • Direct integration of the game engine with the tool
  • Remote integration via a communication layer

(more…)

October 23, 2008

Game Optimization 101

Filed under: game programming, optimization — Tags: , , — systematicgaming @ 6:23 am

In this article we’ll look at the process of optimization.  There’s no single way to optimize a game, but there are optimization methodologies to follow, which will help us prioritize our optimization strategy.

In previous articles we looked at how we can profile parts of our game.  At this point we only know how long sections of our game code are taking.  We haven’t looked into how to use this information to make our game faster.  We need to know how to target the slow parts of our game.

The process of optimization is two parts methodology, one part knowledge and one part intuition.  We’ll start with a simple process and show how it can help us optimize effectively.

(more…)

October 12, 2008

Game Profiling 102

Filed under: game programming, profiling — Tags: , — systematicgaming @ 1:41 am

In the last article we implemented a simple stopwatch, but didn’t really do much with our timing data other than print it out.  Simply printing out our timing isn’t very helpful either, first of all you can’t always see standard output on a console, and secondly it’s a pretty awkward way to collect our data, we can do much better.

Our goal this time is to design a system that allows us to profile out entire game and display the results in realtime.   We’re not going for detailed performance data, just a high level picture of our game’s performance.  The key goals are:

  • Collect data from each major subsystem
  • Present our profiling data to the user in an understandable format
  • Keep a multi-frame history of our data to detect performance spikes

(more…)

September 30, 2008

Game Profiling 101

Filed under: game programming, profiling — Tags: , — systematicgaming @ 8:21 am

On this blog I keep mentioning efficiency, resource usage and optimization as key requirement for any well designed game system.  When discussing memory I put a large emphasis on tracking and profiling memory usage.

This time we’ll look at the various ways we can profile our game. We’ll also look at how to instrument our game for custom profiling, focusing on CPU usage.

(more…)

September 24, 2008

Data Compression Evaluation

Filed under: file management — Tags: , — systematicgaming @ 1:38 pm

When discussing load times I mentioned that compression is an important way of reducing load times.  I also said that memory usage can be reduced by compressing data into smaller fixed sized blocks – instead of compressing a file as a single block.  There is also a variety of compression algorithms available, with different performance characteristics and strengths.

So how do you choose the proper compressor?  There is no single best algorithm, many have different strengths and weaknesses and are useful in different situations.  We’ll investigate a few different compression algorithms and more importantly evaluate what to look for when deciding on how to compress your data.

(more…)

September 18, 2008

Load Times: Layouts and More

Filed under: file management, game programming — Tags: , — systematicgaming @ 11:41 pm

We know that seek time is a major problem with optical media, so we’ll need to reduce it as much as possible to achieve our fast load times. There’s really only so much we can do to solve the problem

  • Seek less by reading less files
  • Seek less by organizing our files better

We’ve seen how to reduce the number of files read by using packfiles, which also helped by seek times a little bit.  Since seek times are related to the physical distance and direction a drive head has to move, we’ll look at how to reduce seek times by arranging files on disc to minimize their distance.

We’ll also wrap up this series on load times with some tips and tricks to get that last bit of optimization.

(more…)

September 16, 2008

Load Times: Packfiles

Filed under: file management, game programming — Tags: , — systematicgaming @ 5:27 am

A packfile is simply a single file containing one or more other files. They’re very useful for load times, and can be used in a number of ways. We can put files that get loaded together into a single packfile and load them at once, such as putting the data for a single level into a single packfile and loading in a single read. We can also put our entire filesystem into one or more packfiles, which lets us handle compression cleanly as well as making distribution easier.

So what do we need to do to create a packfile? No much:

  • File name and path, usually relative to a specific root directory
  • File size and compressed size

In this article now we’ll look at how to implement a packfile system.

(more…)

September 11, 2008

Load Times: Compression

Filed under: game programming, memory management — Tags: , — systematicgaming @ 11:49 am

We’ve gone over what happens when we read files, and came up with some ways to reduce stalls between our game and the OS and hardware. We designed a file manager that allows us to read files asynchronously and remove all the wait time. What next?

Well the simplest way to reduce load times is to load less data. If we simply compress our data we’ll reduce load times a lot. General compression algorithms can often get a 50% or more reduction in file size, which translates directly into reduced load time.

(more…)

September 8, 2008

Load Times: Asynchronous Loading

Filed under: file management, game programming — Tags: , — systematicgaming @ 5:26 am

We looked at how files are read earlier, and saw how much time is wasted by waiting for file operations to finish. So the only real solution is to not wait for operations to complete. This is called asynchronous reading, where the file operation happens in parallel with our game, controlled by the OS.

Asynchronous file loading requires more structure and organization than normal file reading, so we’ll need to create a file manager to provide portable asynchronous loading. We will need to change how we handle file data inside out game to support this functionality. But it’s worth it because it gives us a huge win in load times, and in the end gives us a more robust game engine as a bonus.

(more…)

September 1, 2008

Load Times: Reading Files

Filed under: file management, game programming — Tags: , — systematicgaming @ 1:23 am

Load times are a bane of console games, every game has load screens and nobody likes staring at them. Some games try to hide load times under cut-scenes or flashy effects. The elevator scenes in Mass Effect are a clever trick and are better than a “Loading” icon, but still get repetitive and players do feel like they’re standing still. Really the only solution is to make the load time as short as possible.

With console games content getting larger and larger and disc speeds not keeping pace, load times are aren’t getting any shorter. To get the shortest load times possible we need to handle file loading intelligently.
(more…)

August 29, 2008

Functional Gaming: Writing a game in OCaml

Filed under: game programming, ocaml — Tags: , , — systematicgaming @ 2:25 am

If you’re following the latest trends in multi-threading you may have seen some renewed interest in functional programming.

When you bring up functional programming you’ll often get one of two opinions:

  • It’s the silver bullet of parallelism, and will allow automatic parallelizing of code, and also is easier to read, write and creates less bugs.
  • I remember those from university, they made my head hurt and ears bleed. Nobody uses those type of languages in the real world.

I find you’ll get more of the second responses, because really, who codes in functional languages outside of university?

Of course I’m biased since I generally hang out with other game programmers. If you program for consoles you do it in C or C++, since there aren’t any other compliers provided. So nobody I know uses functional programming on a daily basis.

That said, I’m intrigued by the possibilities offered by functional programming. I can see how they can parallelize well, and they allow you to write some clean mathematical code. To form a more informed opinion I decided to start exploring the world of functional programming.

(more…)

August 26, 2008

Memory Management: Summary

Filed under: game programming, memory management — Tags: , — systematicgaming @ 2:59 am

This series hasn’t been much more than a quick over of the memory management issues facing console game development. Hopefully it’s been informative, but keep in mind it’s only an introduction and was fairly light on many of the nitty-gritty implementations details.

To wrap up this series here’s a few quick topics that deserve mentioning, as well as some links to some suggested reading.

(more…)

August 25, 2008

Memory Management: Budgets, Tracking and Profiling

Filed under: game programming, memory management — Tags: , — systematicgaming @ 2:16 am

The last few entries in this series mostly focused on memory allocation. However allocation is only one part of memory management. To create a complete memory management system we need to do a lot more than allocate and free blocks of memory. We need to look at memory as a system wide resource. This involves

  • budgeting memory for the various game sub-systems
  • tracking memory use, to ensure budgets are honoured and memory doesn’t leak
  • profiling memory use to look for worst case spots in or program and trend memory usage over time

We’ll look into how we can enhance the allocation interface we’ve designed to help accomplish these goals. As well, we’ll see how these features aid us in managing memory system-wide.

(more…)

August 19, 2008

Memory Management: Design

Filed under: game programming, memory management — Tags: , , — systematicgaming @ 12:29 pm

We’ve reviewed the different types of memory in part one and looked at issues specific to consoles in part two. Now we’re ready to start designing our own memory manager. Lets review our requirements:

  • Our memory manager must be fast and efficient. We don’t want to waste CPU cycles or extra bytes of RAM managing memory.
  • We need to allow memory to be allocated with arbitrary alignment.
  • We need to support managing multiple physical memory heaps as well as the ability to support both internal and external heap managers.

These are the main requirements we have above and beyond the basic functionality of allocating and releasing memory.

(more…)

August 15, 2008

Memory Management: Consoles

Filed under: game programming, memory management — Tags: , , — systematicgaming @ 9:23 am

In part one we described the main types of memory. This time we’ll take a look at the memory management issues specific to consoles, or issues that are at least more prevalent in console game programming. The key differences between your regular PC and consoles are:

  • No virtual memory, which means once you’re out of memory, you’re out of memory. No swap file is going to save you on a console. Also, this makes memory fragmentation a serious concern.
  • Multiple physical memory types. On the PC you have RAM and video RAM, but you have a video driver that manages it for you, not so on consoles.
  • Memory alignment is also a problem on consoles. The x86 CPU is very forgiving. However console CPUs are generally stricter. Alignment is also very important for performance.

Now let’s take a look at these issues and others in more detail.

(more…)

August 7, 2008

Memory Management: Overview of Memory

Filed under: game programming, memory management — Tags: , , — systematicgaming @ 2:19 pm

Before we can really delve into the design of a memory management system we need to have a solid understanding of memory. In this post we’ll go over different types of memory and look at how they are used in your average program. We’ll also take a look at the main issues faced with each memory type, such as lifetime, scope and problems such as fragmentation.

Disclaimer: You’ll see some code samples in the following article, they’re in pseudo-C, not compilable or even compliant. They’re included for illustration and discussion purposes.

(more…)

Older Posts »

Create a free website or blog at WordPress.com.