Warning! From time to time we are going to post some very technical material in this blog. This is one of those posts. Read on at your own risk.
Preface
Modern games tend to create new game objects by compositing them from separate reusable components. This is a very powerful concept as complex behavior can be built from relatively simple building blocks. Components can be things such as models, lights, animations, sound emitters and gameplay related components such as health and item components, just to give you some examples. In this blog post I’ll talk about how we use components to build the game objects in Druidstone.
The Dark Ages
Let’s start with some attempts that don’t work. Back in the 90s when C++ was hot and object-oriented programming was still considered a good idea, somebody had a great idea: hey, let’s build game objects using inheritance! Base-classes would be something like Weapon, Enemy, Player, Light or Vehicle. So if you needed an enemy that had a weapon and carried a light source you would create a new class and inherit from the necessary base-classes. Your typical textbook OOP solution. Needless to say this quickly led to a huge mess and to an explosion with the number of different classes.
Ok, what if you don’t use inheritance and just make the different components of the object direct data members of your class? Nope, this still doesn’t work because you end up with a new class for every combination of components in your game. Fast forward ten years. What you really need is some sort of entity-component-system where the components are decoupled from objects and game objects are no longer classes. What you have instead are “entities” which are usually just numeric IDs and components which are linked to these entity ID. This way components are maximally decoupled.
Something like this:
struct ModelComponent
{
Vertex* vertices;
int* triangles;
};
struct LightComponent
{
Vec3 color;
float brightness;
};
ModelComponent models[MAX_ENTITY_ID];
LightComponent lights[MAX_ENTITY_ID];
This kind of component system is how many game engines work. This is also basically how my previous codebase powering Legend of Grimrock 2 was structured (although the components were written in Lua, not C++). However, there are some problems. Inter-component communication and dependencies are kind of hard. With the dark age inheritance model you could at least call the methods in the same class without a fuzz. Not so with a component-entity system. If component A wants to talk to component B, or just do a simple thing such as look up a value in the other component, first it has to know the entity ID and look up the other component using the entity ID before it can do anything. This has to be done for every component look up. And believe me, there are many, many of those look-ups in a game! If this sounds too slow or cumbersome you can start adding coupling between the components by storing direct references to other components in your components… and you’re back to square 1. This direct coupling is what you wanted to avoid in the first place.
Enter Lua
In Lua you just have a single data structure, the table, which is basically a nifty, optimized hash map. So can we do better in Lua? Well, glad you asked! Sure we can.
First let’s throw that useless object-oriented garbage away ([1] and [2]). Once you realize that you no longer have to couple data and functions all sorts of fascinating possibilities start to emerge. Since you can dynamically add and remove table fields, why not just stash data for all of your components of a game object in a single table? Accessing another component’s data becomes trivial since it’s sitting right next to you in the same data structure. Once you have this then you can just use free functions to operate on your components and you can pass those object data tables around from function to function. No need for entity IDs.
A simple example:
local t = {} -- create a new empty object
-- create some components for t
init_model(t)
init_light(t)
-- call a component function
fade_light(t, 10.0)
-- and here is the implementation for the components
function init_model(obj)
-- init state for model component
obj.model_var1 = ...
obj.model_var2 = ...
end
function init_light(obj)
obj.light_var1 = ...
obj.light_var2 = ...
end
function fade_light(obj, target_brightness, time)
-- do whatever fancy stuff is required to fade light's brightness to target value
end
This 80s style no-nonsense programming is pretty much how objects in Druidstone are coded. Really, really simple and also reasonably efficient. There are just a few things to keep in mind. Since all components store their data in the same table, you want to avoid clashes with field names. A simple naming convention by prefixing the component name works like a charm. Secondly, you can have just one instance of a component per object. E.g. you can’t have two model components in the same object (unless you implement the multi model stuff directly into your component).
I could argue that in practice allowing multiple components of the same type per object is not so good idea. It’s quite expensive and unneeded special case for most objects. In those rare cases when you really need this, it’s usually better to just have multiple objects and perhaps link them somehow. The component system in Grimrock 2 allowed arbitrary number of components per object, but the price was that every component was stored in a separate table and every component had to be named so that when you referred to an component in an object, you always had to specify the component name. This system was very flexible but also more complex, less efficient and harder to use.
All in all, we have been building Druidstone with this new system for a year now and we’re happy to report that no major kinks have been found so far and working with the codebase has been a joy.
I hope this article was an interesting read. Thanks for reading!
Petri
I’m wondering why going back to “C OOP” style naming, when instead you could use compositing (which can also solve the “multiple components” problem):
local t = {} — create a new empty object
— create some components for t
t.model = model.load(…)
t.lights = { light.create(…), light.create(…) }
— call a component function
fade_lights(t, 10.0)
— and here is the implementation for the components
function model.load(…)
— init state for model component
return { var1 = …, var2 = … }
end
function light.create(…)
return { var1 = …, var2 = … }
end
function fade_lights(obj, target_brightness, time)
if obj.lights then
for _, l in ipairs(obj.lights) do
— do whatever fancy stuff is required to fade light’s brightness to target value
end
end
end
That’s pretty much how my previous codebase worked. The problem is that it requires having a separate table for each component and in many cases requires an extra indirection when accessing the fields, which I wanted to avoid. Just having a single table for storing all components of an object helps with garbage collection (less objects and references for the GC to track).
What a nice read. I am and old guy who never really took to OOP. Used an old 4GL named SuperNova ( a Dutch creation). Had very similar need for project-wide naming conventions and really felt it was not a burden that many proponents of OOP criticized it for. The freedom of grabbing a record from a table to populate the values I needed and knowing the field names would represent what they were.. well, I could do a lot very fast. And the naming conventions, done properly, would yield some fairly well self-documenting code. Looking forward to DruidStone and seeing what Lua is all about. I’m guessing this may end attracting a nice modding community.. if that is something desirable to you all.
Thanks for the spot on comments, Mark! Indeed, I’m seeing more and more OOP people abandoning the sinking ship.
We’d definitely love to see some community created mods for Druidstone. Actually mod support / editor has been a great success in our ealier projects (Legend of Grimrock games) and the new Druidstone editor is already in many ways more capable and easier to use than the Grimrock Dungeon Editor. It’s only a matter of spending our resources wisely — adding proper modding support and documenting everything is a lot of work, which at the moment is time off working on the actual game.
Simple and interesting!
I’ve done some entity-component systems in the past, both in Java and C++, and while a system like that is relatively simple and straightforward to implement, this table-based method sure beats the amount of boilerplate in a more verbose language…
Do you have any control of the memory layout this way, for stuff like SIMD operations etc?
I’m glad that you found the article interesting! Lua is a pretty high level language so there’s no access to SIMD, memory layouts or anything low level like that.
That said, I’m using C / C++ for speed critical things (mostly just rendering nowdays). When I’m working on that area of the codebase, I’m more concerned about data layout, caches, etc.
Wow, so you too! 🙂
First of all, thanks for a very good read, Petri!
It’s interesting how after 3 years of development on our game, we also came to a conclusion that this “80s no non-sense” style of coding is a really good fit for games. And I’m noticing this trend in more coding posts than before.
Just have functions that operate on components and possibly call other functions. So you can have what I call “atomic” functions that do the simplest stuff (like SetPosition), then middle functions that use these atomics and do something on their own (like TeleportActor, which certainly uses SetPosition), and then complex high-level functions that sort of compose their behavior out of many other functions.
When coding ECS in my Lua projects for Love framework, I was always skeptical to go this single-table approach with name-spacing the fields. But in reality, we’re making games, not public APIs, so as authors, we know what we’re doing, and so we can easily get away with this approach. The simpler the merrier, right?
A question pops up though. How do you separate the fields in the GameObject inspector if there is no logical separation?
One more question.
Where do the “component” functions live? Do you separate them into “modules” by category, or do they just live all together in some globally accessible table?
Sorry for the spam, but you really got me on for this! 🙂
How do you update the GOs? I suppose there is a function for every component that needs to update every frame and you just call these functions in the required order in the main loop.
function update_game()
for k,v = pairs(components) do
update_movement(v)
update_light_flicker(v)
update_whatever(v)
end
end
But then every such function would have to check if the component is present by checking for one of the fields. Which ain’t optimal. I’m intrigued how you solve this.
Thanks for the comments and interest in the article, Matej! Here are the answers to your questions:
> How do you separate the fields in the GameObject inspector if there is no logical separation?
I set up the gui for each component manually. E.g. gui_light() implements the inspector for the light component of an object. It’s just a single line of code per field using imgui.
> Where do the “component” functions live? Do you separate them into “modules” by category, or do they just live all together in some globally accessible table?
They are just global functions. I guess I could put them in modules, but ehh… globals work just fine for me.
> How do you update the GOs?
Each system updates their own components. E.g particle system components are stored in their own array and update_particle_systems() iterates the particle system array and does whatever is needed. This is actually a big win compared to classical inheritance based OOP solution, because there is no need to iterate objects without the specific component and there’s much fewer function calls.
Thank you for the reply!
> Each system updates their own components. E.g particle system components are stored in their own array and update_particle_systems() iterates the particle system array and does whatever is needed.
I see. So I guess when you call init_particle() (or any other init_component() function) on a GO, the function also adds that GO to the array of particles which the particle system then iterates on, using only the “particle_something…” fields.
About the OOP criticism – I share that opinion stated there too. Digging into a .NET framework based website, it’s a horror to see ‘at once’ what’s happening.
I’ve never seen/done anything more transparent and easily investigatable by using old school ASP/Javascript, with some XML and created the ‘objects’ myself (code snipplets) that get included in an agile manner and output different content depending on what parameters were given while calling them.
Please guys, do not leave us alone. Feed us with more druidstone.
Petri, as a pure OO business app developer, I’ve always been interested in (but ignorant about) game development. Short of buying a book and reading seriously about it, blog entries like this are perfect. Thanks!
Hi Petri,
I loved your programming posts about Grimrock, and am glad to find them here again.
That’s an interesting read, although it did surprise me a bit, since you are essentially the one who introduced (and converted) me to an entity-component system ;).
I understand the idea of having all properties in a single table, but I wonder if it’s really worth it to lose the logical organization in exchange for less indirection?
I don’t know the details of Lua performance/memory, but is
obj.lightProperty1
so much more efficient than
obj.light.property1 ?
And separating function from data is what ECS systems such as Artemis do, no? In their “pure” approach, components become simply data containers, and functions live in systems, which operate transversally on arrays of their respective components, exactly like you say yours do. I see of course how your approach is more free but I wonder if we’re not losing helpful code organization structures in the process.
As to supporting multiple components of the same type, I agree it’s generally not worth the hassle. You can get around that by identifying components that might need multiples (for example lights) and have one ligths component which handles an arbitrary number of lights, vs one light component which handles one light. That’s at least the approach I took in my (currently on hold) development of my own game. But you can even live without that like you say in your article.
As for modding support, well, I’ll gladly take it! 😀
Keep up the good work, looking forward to this game.
Hi Georges, good to see you here!
I haven’t measured it, but I’d guess “obj.component.property” is roughly twice as slow as “obj. component_property” because the former does two table accesses.
In practice I haven’t experienced any of the problems you mentioned. Actually I prefer this new codebase over Grimrock 2. For example, in Grimrock 2 the user-side scripting interface is more complex and more verbose than it needs to be because of the gameobject.component syntax. When all properties are stored in a single table, there are only objects to pass around between functions.