C++ – Is heap fragmentation a problem in bare-metal arm with g++

Any time you have an application which has a performance-intensive critical path, you should be concerned how you treat memory. Most end-user client-side applications don't fall into this category because they are primary event-driven and most events come from interactions with the user, and that doesn't have that many (if any at all) performance constraints.

However, a lot of back-end software should have some focus on how the memory is handled because a lot of that software can scale up to handle higher number of client, larger number of transactions, more data sources.... Once you start pushing the limits, you can start analyzing how your software users memory and write custom allocation schemes tailored to your software rather than rely on a completely generic memory allocator that was written to handle any imaginable use case.

To give you few examples... in my first company I worked on a Historian package, software responsible for collecting/storing/archiving of process control data (think of a factory, nuclear power plant or oil refinery with 10's of millions of sensors, we'd store that data). Any time we analyzed any performance bottleneck that prevented the Historian from processing more data, most of the time the problem was in how the memory was handled. We've gone through great lengths to make sure malloc/free were not called unless they were absolutely necessary.

In my current job, I work on surveillance video digital recorder and analysis package. At 30 fps, each channel receives a video frame every 33 milliseconds. On the hardware we sell, we can easily record a 100 channels of video. So that's another case to make sure that in the critical path (network call => capture components => recorder management software => storage components => disk) there isn't any dynamic memory allocations. We have a custom frame allocator, which contains fixed-size buckets of buffers and uses LIFO to reuse previously allocated buffers. If you need 600Kb of storage, you might end up with 1024Kb buffer, which waste space, but because it is tailored specifically for our use where each allocation is very short-lived, it works out very well because the buffer is used, free and reused for next channel without any calls to heap API.

In the type of applications I described (moving lots of data from A to B and handling large numbers of client requests) going to the heap and back is a major source of CPU performance bottlenecks. Keeping heap fragmentation to a minimum is a secondary benefit, however as far as I can tell most modern OSes already implement low-fragmentation heaps (at a minimum I know Windows does, and I would hope others do as well). Personally, in 12+ years working in these types of environments, I've seen CPU usage issues related to heap quite frequently, while never once have I seen a system that actually suffered from fragmented heap.

Some initial advantages include:

• Code safety because the operating system can enforce memory protection and run some tasks in supervisor mode and others in user mode. A few years ago when I had a class for a very popular RTOS I was shocked to see it used a flat, shared memory space. Not sure if this is common, but it seems like a big jump backward.
• Availability of wide ranging support for many peripherals through drivers.
• If a device is not available, a driver can implement just the functionality needed to fit with a corresponding stack, rather than everything that goes with having the device. Typically at least one similar device will have been written and can be used as an example.
• There may be compiler or library level support for thread safe support for Linux. If you have an executive, you may lack this and you may need to provide it yourself. If the executive has an active user base, there may be a combination of libraries available for purchase or via open source, but the quality may depend on how wide and deep the support that community provides. Pick your community with care.
• If you start without all the baggage that comes with a real operating system, how long until you can't get along without those extra features? Who writes, integrates, and maintains them?

Hope this helps.