Valve Source Engine Multicore Port (presentation comment)

Game engine development company Valve has a nice presentation up on their website about how they attacked multithreading. It is a nice example of how to solve multicore programming for a particular domain by the classic layered approach.

The overall approach is to build a strong core that takes care of data sharing and work scheduling issues, and then expose a domain-specific API to game programmers (and modders and other higher-level programmers). They feel that native OS APIs are too low-level to be used efficiently, and compiler tools like OpenMP give too little control. In effect they build their own API level on top of the OS APIs (there is no way you can get around them unless you run on naked hardware).

Inside, there is a mix of data replication, locks, and lock-free algorithms depending on the nature of the particular type of algorithm being targeted by each part of the API. Note that different parts of a game engine have different properties regarding how best to parallelize and how to handle data.

I think this is a very nice example of how to build an infrastructure for multicore. It is lower-level than Erlang-style solutions where scheduling and data sharing is totally hidden, but I guess that is necessary to allow games to run fast. Performance is pretty much paramount for games, unlike many other applications.

One thought on “Valve Source Engine Multicore Port (presentation comment)”

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.