Architecture – Applications Architecture – fewer big systems vs more smaller systems

So the question is - how should I implement the interaction between the various systems?

Ideally they don't interact, not in any direct sense. The systems in an ECS all have access to the central ECS database where they can fetch entities and components attached to them. They don't talk to each other directly. They talk to the database and all run independently of each other.

Dependencies Flow Towards Raw Data, Not Abstractions

The dependencies in an ECS do not flow towards functions, not even abstract functionality. They all flow towards raw data which might sound like an epic violation of many accepted software engineering principles, and in my opinion it is, but yields something easier to maintain for some cases. Maybe some software engineering principles are wrong or at least not applicable for all scenarios. There are many situations where it's easier to achieve data stability than interface/design stability. As a basic example, it's much easier to reason about what data fields a raw matrix component should have once and for all and keep that stable (unchanging) for years to come. It's much harder to figure out all the functions an abstract IMatrix interface should provide once and for all and keep that perfectly stable (unchanging) for years to come without facing temptations, if not outright needs, to add and remove and change functions.

So in appropriate cases, when your dependencies flow towards data instead of abstract functionality, your codebase will find fewer and fewer and fewer reasons to have to face central design changes with cascading effects and potentially big parts having to be rewritten. To direct dependencies towards data in that case is directing them towards stability. It's worth asking yourself as a developer whether the tendency in your system is for developers to add, change, and remove functions or to add, change, and remove data from components. If it's the former case, you might benefit greatly from an ECS engine.

If systems start to depend on each other a lot, that's directing dependencies away from data and towards functionality, and many of the maintenance benefits and the ability to reason about the correctness of your engine and easily keep it stable at the design level will be lost. Of course a pragmatic solution might sometimes call for a system calling a function in another every once in a while, but you should generally seek to keep that to a bare minimum. Instead of talking directly to each other, you can have systems modify and attach components to entities in a way such that other systems can then pick up those changes and react accordingly.

System Interaction

[...] the rendering system must know the data from the positional component of an entity in order to draw it in a correct position. And so on.

That it can grab from the ECS database, looping through entities with renderable and position components, just as the physics system before it might loop through entities with position components and modify their position. Generally each system fits into a basic loop model:

for each entity with the components I'm interested in:
    do something with the components

... and you have to start thinking about doing things in passes, often multiple passes even if the intuitive solution is to do everything in one pass. For example, it might come more intuitively to loop through all your game entities and apply physics and respond to input and process AI and render them all in one go. That can minimize the amount of loops you have and also require less state. However, the ECS tackles this typically with multiple simpler passes and sometimes slightly more intermediary state to use from one pass to the next, but as a trade-off, it leads to a much easier system to maintain and one which is easier to change and potentially parallelize* and vectorize.

• As yuri mentioned, it could also make things harder to parallelize, at least across systems in an inter-system way, but could make things easier to parallelize in an intra-system way because it's easier to reason about the correctness of a parallel loop without locking if it's, say, making less state changes on the way and the code involved in the pass is much simpler. In my blunt opinion, it's often not worth multithreading the systems themselves so much as the loops they are performing inside for the most performance-critical systems.

Multiple, Simpler Passes

It's somewhat similar to GPU programming since GPUs aren't so good at doing complex things with each iteration, so they often excel instead at doing simple things per iteration that add up to a complex task after repeatedly running through the same data with multiple, simpler passes.

Unlike GPU programming, you can still potentially do much more complex things in a single pass, but each pass will represent like one logical thought: "for each of these components, apply physics", not both physics and rendering. The physics system performs its own pass just as the rendering system, living in its own isolated world, performs its own completely separate and detachable rendering pass. Each system lives in its own little world, seeing only the ECS database and being able to grab components and entities inside. They shouldn't have to bother with what other systems are doing.

In fact, in a well-designed ECS, you can remove any system from the engine and not have the codebase collapse horribly on itself because systems don't depend on each other to function. All they care about is the central database and the components (which are raw data) that they are interested in processing. They all live in their own isolated world. As a result you should be able to remove the physics system from your game, at which point motion components will cease to have physics applied, but everything else should keep on working just as before. It's extremely orthogonal in that respect.

Event-Driven Programming

Event-driven programming can be a bit awkward with ECS, but one straightforward way to solve that is to have event queue components. A system can push events to these queue components for another system to pop and process in a deferred fashion without the first system directly calling functions in the second. Again the bulk of your interactions should not be system->system, but system->ECS, and system->component.

However, is it reasonable to also create applications using the Component-Entity-System architecture common in game engines?

To me, absolutely. I work in visual FX and studied a wide variety of systems in this field, their architectures (including CAD/CAM), hungry for SDKs and any papers that would give me a sense of the pros and cons of the seemingly-infinite architectural decisions that could be made, with even the most subtle ones not always making a subtle impact.

VFX is rather similar to games in that there is one central concept of a "scene", with viewports that display the rendered results. There also tends to be a lot of central loopy processing going on revolving around this scene constantly in animation contexts, where there might be physics happening, particle emitters spawning particles, meshes being animated and rendered, motion animations, etc, and ultimately to render them all to the user at the end.

Another similar concept to at least very complex game engines was the need for a "designer" aspect where designers could flexibly design scenes, including the ability do some lightweight programming of their own (scripts and nodes).

I found, over the years, that ECS made the best fit. Of course that's never completely divorced from subjectivity, but I would say it strongly appeared to give the fewest problems. It solved a lot more major problems we were always struggling with, while only giving us some few new minor ones in return.

Traditional OOP

More traditional OOP approaches can be really strong when you have a firm grasp of the design requirements upfront but not the implementation requirements. Whether through a flatter multiple interface approach or a more nested hierarchical ABC approach, it tends to cement the design and make it more difficult to change while making the implementation easier and safer to change. There's always a need for instability in any product that goes past a single version, so OOP approaches tend to skew stability (difficulty of change and lack of reasons for change) towards the design level, and instability (ease of change and reasons for change) to the implementation level.

However, against evolving user-end requirements, both design and implementation may need to frequently change. You might find something weird like a strong user-end need for the analogical creature that needs to be both plant and animal at the same time, completely invalidating the entire conceptual model you built. Normal object-oriented approaches don't protect you here, and can sometimes make such unanticipated, concept-breaking changes even harder. When very performance-critical areas are involved, the reasons for design changes further multiply.

Combining multiple, granular interfaces to form the conforming interface of an object can help a lot in stabilizing client code, but it doesn't help in stabilizing the subtypes which could sometimes dwarf the number of client dependencies. You can have one interface being used by only part of your system, for example, but with a thousand different subtypes implementing that interface. In that case, maintaining the complex subtypes (complex because they have so many disparate interface responsibilities to fulfill) can become the nightmare rather than the code using them through an interface. OOP tends to transfer complexity to the object level, while ECS transfers it to the client ("systems") level, and that can be ideal when there are very few systems but a whole bunch of conforming "objects" ("entities").

A class also owns its data privately, and thus can maintain invariants all on its own. Nevertheless, there are "coarse" invariants that can actually still be hard to maintain when objects interact with each other. For a complex system as a whole to be in a valid state often needs to consider a complex graph of objects, even if their individual invariants are properly maintained. Traditional OOP-style approaches can help with maintaining granular invariants, but can actually make it difficult to maintain broad, coarse invariants if the objects focus on teeny facets of the system.

That's where these kinds of lego-block-building ECS approaches or variants can be so helpful. Also with systems being coarser in design than the usual object, it becomes easier to maintain those kinds of coarse invariants at the bird's-eye view of the system. A lot of teeny object interactions turn into one big system focusing on one broad task instead of teeny little objects focusing on teeny little tasks with a dependency graph that would cover a kilometer of paper.

Yet I had to look outside of my field, at the gaming industry, to learn about ECS, though I was always one of a data-oriented mindset. Also, funnily enough, I almost made my way towards ECS on my own just iterating through and trying to come up with better designs. I didn't make it all the way though and missed a very crucial detail, which is the formalization of the "systems" part, and squashing components all the way down to raw data.

I'll try to go through how I ended up settling on ECS, and how it ended up solving all the problems with previous design iterations. I think that'll help to highlight exactly why the answer here could be a very strong "yes", that ECS is potentially applicable far beyond the gaming industry.

1980s Brute Force Architecture

The first architecture I worked on in the VFX industry had a long legacy that was already going past a decade since I joined the company. It was brute force crude C coding all the way (not a slant on C, as I love C, but the way it was being used here was really crude). A miniature and oversimplistic slice resembled dependencies like this:

And this is an enormously simplified diagram of one tiny piece of the system. Each of these clients in the diagram ("Rendering", "Physics", "Motion") would get some "generic" object through which they would check a type field, like so:

void transform(struct Object* obj, const float mat[16])
{
    switch (obj->type)
    {
        case camera:
            // cast to camera and do something with camera fields
            break;
        case light:
            // cast to light and do something with light fields
            break;
        ...
    }
}

Of course with significantly uglier and more complex code than this. Often additional functions would be called from these switch cases which would recursively do the switch again and again and again. This diagram and code might almost look like ECS-lite, but there was no strong entity-component distinction ("is this object a camera?", not "does this object provide motion?"), and no formalization of "system" (just a bunch of nested functions going all over the place and mixing up responsibilities). In that case, just about everything was complicated, any function was a potential for a disaster waiting to happen.

Our testing procedure here often had to check things like meshes separate from other types of items, even if the identical thing was happening to both, since the brute force nature of the coding here (often accompanied by a lot of copy and paste) often made it very probable that what is otherwise the exact same logic could fail from one item type to the next. Trying to extend the system to handle new types of items was pretty hopeless, even though there was a strongly-expressed user-end need, as it was too difficult when we were struggling so much just to handle the existing types of items.

Some pros:

• Uhh... doesn't take any engineering experience, I guess? This system does not require any knowledge of even basic concepts like polymorphism, it's totally brute force, so I guess even a beginner might be able to understand some of the code even if a pro at debugging can barely maintain it.

Some cons:

• Maintenance nightmare. Our marketing team actually felt the need to boast the we fixed over 2000 unique bugs in one 3-year cycle. To me that's something to be embarrassed about that we had so many bugs in the first place, and that process probably still only fixed around 10% of the bugs total which were growing in number all the time.
• About the most inflexible solution possible.

1990s COM Architecture

Most of the VFX industry uses this style of architecture from what I've gathered, reading documents about their design decisions and glancing at their software development kits.

It may not exactly be COM at the ABI level (some of these architectures could only have plugins written using the same compiler), but shares a lot of similar characteristics with interface queries made on objects to see what interfaces their components support.

With this kind of approach, the analogical transform function above came to resemble this form:

void transform(Object obj, const Matrix& mat)
{
    // Wrapper that performs an interface query to see if the 
    // object implements the IMotion interface.
    MotionRef motion(obj);

    // If the object supported the IMotion interface:
    if (motion.valid())
    {
        // Transform the item through the IMotion interface.
        motion->transform(mat);
        ...
    }
}

This is the approach the new team of that old codebase settled on, to eventually refactor towards. And it was a dramatic improvement over the original in terms of flexibility and maintainability, but there were still some issues I'll cover in the next section.

Some pros:

• Dramatically more flexible/extensible/maintainable than the previous brute force solution.
• Promotes a strong conformance to many principles of SOLID by making every interface completely abstract (stateless, no implementation, only pure interfaces).

Some cons:

• Lots of boilerplate. Our components had to be published through a registry in order to instantiate objects, the interfaces they supported required both inheriting ("implementing" in Java) the interface and providing some code to indicate which interfaces were available in a query.
• Promoted duplicated logic all over the place as a result of the pure interfaces. For example, all components that implemented IMotion would always have the exact same state and exact same implementation for all of the functions. To mitigate this, we'd start centralizing base classes and helper functionality throughout the system for the things that would tend to be redundantly implemented the same way for the same interface, and possibly with multiple inheritance going on behind the hood, but it was pretty messy under the hood even though the client code had it easy.
• Inefficiency: vtune sessions often showed the basic QueryInterface function almost always showing up as a middle to upper hotspot, and occasionally even the #1 hotspot. To mitigate that, we'd do things like have rendering parts of the codebase cache a list of objects already known to support IRenderable, but that significantly escalated the complexity and maintenance costs. Likewise, this was more difficult to measure but we noticed some definite slowdowns compared to the C-style coding we were doing before when every single interface required a dynamic dispatch. Things like branch mispredictions and optimization barriers are difficult to measure outside a little facet of code, but the users were just generally noticing the responsiveness of the user interface and things like that getting worse by comparing previous and newer versions of the software side-by-side for areas where the algorithmic complexity didn't change, only the constants.
• Was still difficult to reason about correctness at a broader system level. Even though it was significantly easier than the previous approach, it was still hard to grasp the complex interactions between objects throughout this system, especially with some of the optimizations that started to become necessary against it.
• We had trouble getting our interfaces correct. Even though there might only be one broad place in the system that uses an interface, user-end requirements would change over versions, and we would end up having to do cascading changes to all classes that implement the interface to accommodate a new function added to the interface, e.g., unless there was some abstract base class that was already centralizing the logic under the hood (some of these would manifest in the middle of these cascading changes in hopes of not repeatedly doing this again and again).

Pragmatic Response: Composition

One of the the things we were noticing before (or at least I was) that was causing issues was that IMotion might be implemented by 100 different classes but with the exact same implementation and state associated. Furthermore, it would only be used by a handful of systems like rendering, keyframed motion, and physics.

So in such a case, we might have like a 3-to-1 relationship between the systems using the interface to the interface, and a 100-to-1 relationship between the subtypes implementing the interface to the interface.

The complexity and maintenance then would be drastically skewed to the implementation and maintenance of 100 subtypes, instead of 3 client systems who depend on IMotion. This shifted all of our maintenance difficulties to the maintenance of these 100 subtypes, not the 3 places using the interface. Updating 3 places in the code with few or no "indirect efferent couplings" (as in dependencies to it but indirectly through an interface, not a direct dependency), no big deal: updating 100 subtype places with a boatload of "indirect efferent couplings", pretty big deal *.

_{* I realize it's odd and wrong to screw with the definition of "efferent couplings" in this sense from an implementation perspective, I just haven't found a better way to describe the maintenance complexity associated when both interface and corresponding implementations of a hundred subtypes must change.}

So I had to push hard but I proposed that we try to become a little more pragmatic and relax the whole "pure interface" idea. It made no sense to me to make something like IMotion completely abstract and stateless unless we saw a benefit to it having a rich variety of implementations. In our case, for IMotion to have a rich variety of implementations would actually turn into quite a maintenance nightmare, as we didn't want variety. Instead we were iterating towards trying to make a single motion implementation that's really good against changing client requirements, and often were working around the pure interface idea a lot trying to force every implementor of IMotion to use the same implementation and state associated so that we don't duplicate goals.

Interfaces thus became more like broad Behaviors associated with an entity. IMotion would simply become a Motion "component" (I changed the way we defined "component" away from COM to one where is closer to the usual definition, of a piece making up a "complete" entity).

Instead of this:

class IMotion
{
public:
    virtual ~IMotion() {}
    virtual void transform(const Matrix& mat) = 0;
    ...
};

We evolved it to something more like this:

class Motion
{
public:
    void transform(const Matrix& mat)
    {
        ...
    }
    ...

private:
    Matrix transformation;
    ...
};

This is a blatant violation of the dependency inversion principle to start shifting away from the abstract back to the concrete, but to me such a level of abstraction is only useful if we can foresee a genuine need in some future, beyond a reasonable doubt and not exercising ridiculous "what if" scenarios completely detached from user experience (which would probably require a design change anyway), for such flexibility.

So we started evolving to this design. QueryInterface became more like QueryBehavior. Furthermore, it started to seem pointless to use inheritance here. We used composition instead. Objects turned into a collection of components whose availability could be queried and injected at runtime.

Some pros:

• Was a lot easier to maintain still in our case than the previous, pure-interface COM-style system. Unforeseen surprises like a change in requirements or workflow complaints could be accommodated more easily with one very central and obvious Motion implementation, e.g., and not dispersed across a hundred subtypes.
• Gave a whole new level of flexibility of the kind we actually needed. In our previous system, since inheritance models a static relationship, we could only effectively define new entities at compile-time in C++. We couldn't do it from the scripting language, e.g. With the composition approach, we could string together new entities on the fly at runtime by merely attaching components to them and adding them to a list. An "entity" turned into a blank canvas upon which we could just throw together a collage of whatever we needed on the fly, with relevant systems automatically recognizing and processing these entities as a result.

Some cons:

• We were still having a hard time in the efficiency department, and maintainability in the performance-critical areas. Each system would still end up wanting to cache components of entities that provided these behaviors to avoid looping through them all repeatedly and checking what was available. Each system demanding performance would do this ever so slightly-differently, and was prone to a different set of bugs in failing to update this cached list and possibly a data structure (if some form of search was involved like frustum culling or raytracing) on some obscure scene change event, e.g.
• There was still something awkward and complex that I couldn't put my finger on related to all these granular little behavioral, simple objects. We still spawned a lot of events to deal with interactions between these "behavior" objects that were sometimes necessary, and the result was very decentralized code. Each little object was easy to test for correctness and, taken individually, often were perfectly correct. Yet it still felt like we were trying to maintain a massive ecosystem composed of little villages and trying to reason about what they all individually do and add up to make as a whole. The C-style 80s codebase felt like one epic, overpopulated megalopolis which was definitely a maintenance nightmare, but dispersing that complexity across into too many teeny villages was also making it hard to think about the system at a bird's-eye view without getting overwhelmed by the complexity of the interactions between everything.
• Loss of flexibility with the lack of abstraction but in an area where we never actually encountered a genuine need for it, so hardly a practical con (though definitely at least a theoretical one).
• Preserving ABI compatibility was always hard, and this made it harder by requiring stable data and not just stable interface associated with a "behavior". However, we could easily add new behaviors and simply deprecate existing ones if a state change was needed, and that was arguably easier than doing backflips underneath the interfaces at the subtype-level to handle versioning concerns.

One phenomena that occurred was that, since we lost the abstraction on these behavioral components, we had more of them. For example, instead of an abstract IRenderable component, we'd attach an object with a concrete Mesh or PointSprites component. The rendering system would know how to render Mesh and PointSprites components and would find entities that provide such components and draw those. At other times, we had miscellaneous renderables like SceneLabel that we discovered we needed in hindsight, and so we'd attach a SceneLabel in those cases to relevant entities (possibly in addition to a Mesh). The rendering system implement would then be updated to know how to render entities that provided those, and that was a pretty easy change to make.

In this case, an entity composed of components could also then be used as a component to another entity. We'd build things up that way by hooking up lego blocks.

ECS: Systems and Raw Data Components

That last system was as far as I made it on my own, and we were still bastardizing it with COM. It felt like it was wanting to become an entity-component system but I was not familiar with it at the time. I was looking around at COM-style examples which saturated my field, when I should have been looking at AAA game engines for architectural inspiration. I finally started doing that.

What I was missing were several key ideas:

1. The formalization of "systems" to process "components".
2. "Components" being raw data rather than behavioral objects composed together into a bigger object.
3. Entities as nothing more than a strict ID associated to a collection of components.

I finally left that company and started working on an ECS as an indy (still working on it while draining my savings), and it has been the easiest system to manage by far.

What I noticed with the ECS approach was that it solved the problems I was still struggling with above. Most importantly to me, it felt like we were managing healthy-sized "cities" instead of teeny little villages with complex interactions. It wasn't as hard to maintain as a monolithic "megalopolis", too big in its population to effectively manage, but wasn't as chaotic as a world filled with tiny little villages interacting with each other where just thinking about the trade routes in between them formed a nightmarish graph. ECS distilled all the complexity towards bulky "systems", like a rendering system, a healthy-sized "city" but not an "overpopulated megalopolis".

Components becoming raw data felt really weird to me at first, as it breaks even the basic information hiding principle of OOP. It was kind of challenging one of the biggest values I held dear about OOP, which was its ability to maintain invariants which required encapsulation and information hiding. But it started to become a non-concern as it quickly became obvious what was happening with just a dozen or so broad systems transforming that data instead of such logic being dispersed across hundreds to thousands of subtypes implementing a combo of interfaces. I tend to think of it like still in an OOP-style fashion except spread out where the systems are providing the functionality and implementation that access the data, the components are providing the data, and the entities are providing components.

It became even easier, counter-intuitively, to reason about the side effects caused by the system when there were just a handful of bulky systems transforming the data in broad passes. The system became a lot "flatter", my call stacks became shallower than ever before for each thread. I could think about the system at that overseer level and not run into weird surprises.

Likewise, it made even the performance-critical areas simple with respect to eliminating those queries. Since the idea of "System" became very formalized, a system could subscribe to the components it was interested in, and just be handed a cached list of entities which satisfy that criteria. Each individual one didn't have to manage that caching optimization, it became centralized to a single place.

Some pros:

• Seems to just solve almost every major architectural problem I was encountering in my career without ever feeling trapped in a design corner when encountering unanticipated needs.

Some cons:

• I still have a hard time wrapping my head around it sometimes, and it's not the most mature or well-established paradigm even within the game industry, where people argue about exactly what it means and how to do things. It's definitely not something I could have done with the former team I worked with, which consisted of members deeply hooked to the COM-style mindset or the 1980s C-style mindset of the original codebase. Where I get confused sometimes is like how to model graph-style relationships between components, but I've always found a solution which didn't turn out to be horrible later where I can just make a component dependent upon another one ("this motion component depends on this other one as a parent, and the system will use memoization to avoid repeatedly doing the same recursive motion calculations", e.g.)
• ABI is still difficult, but so far I'd even venture to say that it's easier than pure interface approach. It's a shift in mindset: data stability becomes the sole focus for ABI, rather than interface stability, and in some ways it's easier to achieve data stability than interface stability (ex: no temptations to change a function just because it needs a new parameter. That kind of stuff happens inside coarse system implementations which don't break ABI).

However, is it reasonable to also create applications using the Component-Entity-System architecture common in game engines?

So anyway, I'd say absolutely "yes", with my personal VFX example being a strong candidate. But that's still fairly similar to the needs of gaming.

I haven't put it to practice in more remote areas completely detached from the concerns of game engines (VFX is quite similar), but it seems to me like far more areas are good candidates for an ECS approach. Maybe even a GUI system would be suitable for one, but I still use a more OOP approach there (but without deep inheritance unlike Qt, e.g.).

It's widely-unexplored territory, but it seems suitable to me whenever your entities can be composed of a rich combination of "traits" (and exactly what combo of traits they provide being ever subject to change), and where you have a handful of generalized systems that process entities that have the necessary traits.

It becomes a very practical alternative in those cases to any scenario where you might be tempted to use something like multiple inheritance or an emulation of the concept (mixins, e.g.) only to produce hundreds or more combos in a deep inheritance hierarchy or hundreds of combos of classes in a flat hierarchy implementing a specific combo of interfaces, but where your systems are few in number (dozens, e.g.).

In those cases, the complexity of the codebase starts to feels more proportional to the number of systems instead of the number of type combinations, since each type is now just an entity composing components which are nothing more than raw data. GUI systems naturally fit these kinds of specs where they might have hundreds of possible widget types combined from other base types or interfaces, but only a handful of systems to process them (layout system, rendering system, etc). If a GUI system used ECS, it would probably be so much easier to reason about the correctness of the system when all the functionality is provided by a handful of these systems instead of hundreds of different object types with inherited interfaces or base classes. If a GUI system used ECS, widgets would have no functionality, only data. Only the handful of systems that process widget entities would have functionality. How overridable events for a widget would be handled is beyond me, but just based on my limited experience so far, I haven't found a case where that type of logic could not be transferred centrally to a given system in a way that, in hindsight, yielded a much more elegant solution that I'd ever expect.

I would love to see it employed in more fields, as it was a lifesaver in mine. Of course it's ill-suited if your design doesn't break down this way, from entities aggregating components to coarse systems that process those components, but if they naturally fit this kind of model, it is the most wonderful thing I've encountered yet.