my conclusion is that the answer to malloc performance problems is to malloc less

@ariadne
serverless
mallocless

like seriously, if you are chasing malloc pressure as a performance issue, you've royally fucked up your data structures, it is just that simple

yes, even on musl, or "MUSL" as the techbros call it for whatever reason

of course the folks who usually make this complaint are running some managed language like node.js or whatever, so when i mean data structures, what i really mean is "objects"

as in, you have too many of them and need to reevaluate your design because you're fragmenting your data all over the place

@ariadne yeah that is such a weird thing about managed languages

they tend to be object oriented in some way but at the same time every single object goes onto the heap and is passed by pointer value

which quite literally directly penalizes using objects

that is i think the main reason why minecraft has massive memory allocation pressure?

@ariadne I'll just be over here, allocating and then discarding 100,000 objects a second.

Ironically, even when I did that (on purpose mind), allocations weren't even close to a bottleneck.

Which makes me wonder wtf other people are doing.

@charlotte @ariadne garbage collection schemes can (often) change the factor of the performance impact from being the total number of objects allocated and freed to being, more or less, the number of objects that are live at any given time. Thats a calculus that's somewhat more in favor of being liberal with allocations for a lot of workloads.

and this fragmentation thing is real. modern processors depend heavily on L1D$, mispredictions are super expensive. you have 48KB of it per core on Zen 5.

if you have to dereference through a tree of pointers to get to the data, then you're blowing through L1D$ on every API call.

yes, malloc-ng is slower than ptmalloc, but not that much slower.

if you have malloc pressure, which is the case where malloc performance matters, because the time each malloc call takes to complete stacks up, then you almost always have fragmentation in how you're storing your data.

you're chasing the wrong symptom. yes faster malloc exists, but it's a crutch. the real solution is to refactor how data is structured.

@krutonium it isn't the malloc overhead itself that tanks performance, but the fragmentation. if you have to dereference a forest of pointers, that doesn't come for free.

but malloc gets blamed because it's a function call, while the dereferencing or other structural overhead is what actually tanks performance because you blow through your L1D$ lines.

@ariadne So, what you're saying is, I should have one global `uint8_t *memory = malloc(LARGE_NUMBER)` and then one subroutine uses memory[0x1000] and up, and another uses memory[0x2000] and up, etc, yes?

Because, I've seen projects that work that way. In Perl, but still ;)

@henryk no. I am saying that you should structure your data to avoid superfluous allocations. people blame system malloc for what is typically a data locality problem.

@ariadne @krutonium

Malloc often gets blamed because glibc continues to ship with an embarrassingly slow implementation. When we first released snmalloc, we got a report from someone whose end-to-end performance of a Rust codebase doubled when they switched to it from the glibc implementation.

This is especially true for multithreaded workloads. A lot of existing malloc implementations started as single threaded. Then they added a lock, which worked fine for low thread counts but rapidly became a problem. Then they added thread caching to avoid hitting the lock. But anything that has a producer-consumer model (allocate on one thread, free on another) hits the pathological case for thread-caching allocators: the produce thread fills its cache then allocates from the cache, then calls the global allocator to fill it again, the consumer thread is always filling its cache but never consuming from it, so the global allocator is still a contention point only now the thread caching is causing objects to sit for longer between being freed and reallocated, so your CPU caches churn more. Moving to a message-passing allocator (such as snmalloc or mimalloc) eliminates this bottleneck.

@david_chisnall @krutonium yes, malloc is slow (and in practice, malloc-ng is approximately ~half the raw performance of ptmalloc), but that's not my point.

my point is that in many cases, malloc performance issues are a symptom of a program design problem rather than the root cause.

(that doesn't mean that improving malloc performance isn't an interesting problem! snmalloc is great :))