Arenas have a lot of footguns like growing out of control if you don't have space in your logic to delete them (e.g. suppose every new HTTP connection required obtaining a new allocation from the primary arena -- now you have a memory leak). They also aren't necessarily faster than malloc: I remember hearing a case where mimalloc was faster than an arena for a one-shot program like a linker because reclaiming space improved cache locality. Plus it's difficult to implement things like growable vectors or hashmaps using an arena without wasting space.
I just don't see how "futz about with the allocator constantly" and "use an arena" is actually a good default.
[–] awesan 2 days ago
[–] latch 2 days ago
I don't think it's a huge problem, but it's more subtle than an explicit allocator is a better because it provides more flexibility.
[–] bunderbunder 1 day ago
Zig seems to be laser focused on systems programming domains where there is value in keeping unusually tight control over everything the program does. Aside from tinkering with Arduino, never been me. If it were, though, then I would probably not want to use a library that wants to be able to arbitrarily grow ArrayLists. I would probably want to structure my program so that it acquires pages of memory (actual memory, not just overcommit) in as few places as possible - ideally one - and from a location very close to the entry point. And a library that doesn’t play nice with that wouldn’t be much more palatable than a library that wants to bring its own memory management runtime along for the ride.
I don’t think that’s totally unreasonable? It’s more restrictive than what most of us are used to, sure. But it’s also quite a bit less restrictive than completely banning dynamic heap allocation as is common in aerospace applications.
[–] elcritch 2 days ago
I bought that as Zig's M.O., but then came the automatic "async/IO" colorless functions which is a huge chunk of implicit and hidden logic.
That's more implicit logic than what's in say JavaScript async AFAICT. Arbitrary int sizes with implicit casting rules, etc. More recently was the attempts of memory safety via an some sort of (optional) implicit GC mode. Personally I don't get it.
[–] Mond_ 2 days ago
The async design changed multiple times, and the current approach is that of requiring passing an I/O interface that encapsulates the logic, same approach as for passing around an allocator.
This effectively makes all I/O bound code "quasi-async", and it's all explicit and the caller is in control.
[–] simonask 1 day ago
This has a lot of implications, especially for function pointers and other forms of dynamic dispatch.
My personal take is that async code is fundamentally different from sync code, because the weights of various tradeoffs change dramatically. In my view, "colorless" functions are not really a worthwhile goal in the first place. But I've been wrong before, so it may turn out to be great, we'll see.
[–] lukaslalinsky 1 day ago
[–] elcritch 1 day ago
Yeah so that's a lot of hidden magic and implicit logic. ¯\_(ツ)_/¯
[–] lukaslalinsky 1 day ago
[–] imtringued 22 hours ago
const std = @import("std");
fn performTasks(io: std.Io) !void {
var group: std.Io.Group = .init;
errdefer group.cancel(io);
// Launch concurrent background jobs
try group.async(io, simulateWorker, .{ io, 1 });
try group.async(io, simulateWorker, .{ io, 2 });
// Wait for all group tasks to complete
try group.await(io);
}
fn simulateWorker(io: std.Io, id: usize) void {
std.log.info("Worker {d} started...", .{id});
// Any blocking operation (like sleeping) routes through 'io'
// Under the hood, this will either block an OS thread or yield an event loop stack.
io.sleep(.fromSeconds(1), .awake) catch {};
std.log.info("Worker {d} finished!", .{id});
}
pub fn main() !void {
var gpa_alloc = std.heap.GeneralPurposeAllocator(.{}){};
defer _ = gpa_alloc.deinit();
const allocator = gpa_alloc.allocator();
// Initialize the Threaded I/O backend
var threaded = std.Io.Threaded.init(allocator, .{
.async_limit = std.Io.Limit.limited(4), // Max concurrent tasks in the pool
});
defer threaded.deinit();
// Extract the generic I/O interface pointer
const io = threaded.io();
std.log.info("Running on Threaded backend:", .{});
try performTasks(io);
}
pub fn main() !void {
var gpa_alloc = std.heap.GeneralPurposeAllocator(.{}){};
defer _ = gpa_alloc.deinit();
const allocator = gpa_alloc.allocator();
// Initialize the Evented (non-blocking) backend
var evented: std.Io.Evented = undefined;
try evented.init(allocator, .{
.thread_limit = 4, // Worker threads running the event loop
});
defer evented.deinit();
// Extract the exact same generic interface
const io = evented.io();
std.log.info("Running on Evented backend:", .{});
try performTasks(io);
}
std.Io.Threaded and std.Io.Evented are two types of IO backends. std.Io.Threaded uses blocking IO primitives and Evented uses asynchronous IO primitives. Switching between backends is done via dynamic dispatch, the used backend is passed in via an explicit function parameter.The only real question you should have is, how does the "non-blocking" case work when the interface is still a synchronous call? Well, the answer is quite boring... You just call into the scheduler and it runs the other task inside the blocked operation.
I was promised hidden magic and implicit logic! Now I want a refund!
[–] maleldil 1 day ago
[–] lukaslalinsky 1 day ago
[–] the-tomster 2 days ago
I don't love malloc / free, managing frees one-to-one with allocations is a nightmare. An HTTP server is actually a great example of a good fit for arenas... you can have an arena created per connection, and then free the arena when the connection closes. Now all of your allocations during that connection are guaranteed to be freed instead of having to keep track of them individually. Need something to live across connections? Use the general purpose allocator for those particular things.
For one shot programs, why bother freeing anything at all? The gpa or malloc or whatever else is fine there.
Personally I mostly lean on fixed-size buffers, but if I'm writing Zig it's just little toy programs where blowing up when the buffer runs out of room is fine. But for big non-toy programs it might be good to use fixed-size buffers anyway and define your limits in terms of things actually relevant to your domain.
RAII and smart pointers are fine too, but C++ has you covered if you want those. Zig is specifically avoiding that kind of invisible code that destructors bring you.
[–] grg0 2 days ago
And how do you size the arena?
> keep track of them individually
I think this is an exaggeration of the problem. This is what RAII semantics in C++ are for.
> avoiding that kind of invisible code
The invisible code is traded for defer statements right after initialization that are obvious and therefore mostly just noise, and also a problem if you forget to call them. This isn't an issue with RAII. I get the point of not wanting invisible code, but defers don't seem much of an improvement either. A trade-off at best.
[–] the-tomster 2 days ago
Two options: you either say "eh 20MB per request ought to be big enough" and resize it later if you need to. Or you take time to figure out reasonable limits in terms of whatever domain you're dealing with, and return descriptive errors when those limits are exceeded. Maybe you're saying your service supports up to N HTTP headers and M query parameters in the URL and so on. For almost anything there's a limit you can place that's reasonable. Having no limits at all just means the limit is when you run out of memory / disk / whatever.
If you like RAII and C++ just use that though, Zig is an alternative not an upgrade, imo.
[–] grg0 1 day ago
Resize: realloc invalidates pointers. If you're saying allocate more chunks and link them, that is technically no longer an arena?
Requesting virtual memory and committing lazily seems a good trade-off except on Windows, which does not allow to over-commit. Somehow I don't find that very satisfying because it relies on the OS and on the hardware to provide virtual memory support. But maybe it's just what it is.
[–] MrBuddyCasino 1 day ago
[–] the-tomster 1 day ago
If you're doing something like per-request or per-frame arenas you can clear the arena without having to reallocate the whole mapping. And depending how fancy you want to get you can mark the pages unused when you clear the arena so that the OS can reclaim them if needed.
[–] rallyroger 1 day ago
Should you allow an http-server to allocate unbounded request sizes? Seems like it would lead to an easy denial of service.
[–] lenkite 2 days ago
All HTTP servers have a configurable maximum request size. Ex: client_max_body_size in Nginx.
[–] i2talics 2 days ago
I mean push in the sense of a pit-of-success, not making me do things because the alternative is unpalatable.
> managing frees one-to-one with allocations is a nightmare
It really isn't a nightmare if your language has tools to help you. I'm never particularly concerned about frees in Rust outside of thinking about room to optimize.
> Need something to live across connections? Use the general purpose allocator for those particular things.
Well, this is a determination you have to make very carefully and if you screw it up (like in the situation I was trying to outline) you have a subtle bug.
> Personally I mostly lean on fixed-size buffers, but if I'm writing Zig it's just little toy programs where blowing up when the buffer runs out of room is fine.
What's also fine is using malloc, which will have perfectly reasonable performance for your toy programs.
This is reinforcing my belief that Zig programmers actually just have a sort of aesthetic attachment to the idea of managing their allocations themselves and this has very little to do with the actual practical concerns of writing software that works. I'm a lot more sympathetic to the other reply that talked about library code improving flexibility for those relying on the library, than this comment that only thinks about the case where everything goes perfectly and using a fixed size buffer for no reason.
[–] nvme0n1p1 2 days ago
Zig's position, love it or hate it, is that you should be thinking about it. Dropping a Vec<String> in Rust is O(n). Have you ever clicked the X on an app and it takes 10 seconds to disappear? Stuff like this is part of the problem. You shouldn't have to traverse your process's whole memory space just to call exit(0).
If you have to write that for loop, the O(n) will jump out at you and push you towards an arena or FixedBufferAllocator or std.process.cleanExit, or any other solution to avoid the loop.
[–] kibwen 2 days ago
[–] lunar_mycroft 1 day ago
> Dropping vecs that contain strings does not make any Rust code in the real world slow. You would need to have a vec containing billions of strings, occupying several gigs of memory
I actually had a hobby project where that became very noticeable/too slow for my requirements at around 800k Strings, so several orders of magnitude less than your estimate. Switching to a solution that freed all the memory at once predictably made it run effectively instantly.
> And for extreme cases like that, you can simply use arenas
I think this is a fundamentally flawed way of thinking about things. Yes, each individual string allocation and drop isn't going to cost much, but in combinations with all the other similar decisions you're likely making if you think this way, plus the decisions of library authors, etc, it can add up even if strings specifically aren't a major problem. Avoiding pessimization by default instead of only when you can't ignore them any more is a good habit to be in.
[–] kibwen 1 day ago
[–] lunar_mycroft 1 day ago
(Also, you do not need to sell me on bumpalo, I use it heavily already)
[–] tialaramex 1 day ago
Measure, measure, measure.
So often when people insist that a thing they don't like is "part of the problem" they haven't even thought about how much that "part" would be, although I suppose it's worse when they have thought and decided to go ahead anyway because the weasel wording might convince the hard of thinking.
A guy who rounded up from $14.86 to $15 on some grant paperwork and got 14 cents he wasn't entitled to out of a government programme, and Ken Paxton inexplicably becoming very wealthy from a government job are both "part of the problem" but hopefully it's apparent which of these is worth fixing.
[–] simonask 2 days ago
(There are cases where it also takes time because the OS does have to flush all open files during process teardown, etc., but this shouldn't generally be user-visible.)
[–] the-tomster 2 days ago
Indeed if you don't want to manage everything yourself I don't think there's a need to use Zig. The whole pitch is making everything explicit and not hiding control flow. And I do think that limits its usefulness to smaller-scale codebases.
Writing quality software doesn't require much from the programming language until your codebase gets very large and you're collaborating with a ton of people. If I'm expecting a codebase to grow to a large scale I'm not reaching for Zig for that. I don't trust _myself_ to keep that much Zig code tidy and correct let alone a big mob of developers and agents :)
[–] alextingle 1 day ago
With a small program, it's rarely difficult to work out what's going on. If your small program develops a performance problem, it's usually fairly tractable to work out what's going on, and modify it. In extremis, you can just rewrite the whole thing.
Large programs are another matter. Hidden control flows in C++ (for example) can make pinning down performance problems extremely difficult. And then widespread refactoring can be extremely challenging.
Zig forces everything to be an explicit interface, which should make maintenance work on large programs much more tractable.
[–] scrubs 10 hours ago
I've spent quite a bit of time with zig after which I abandoned to rust despite starting with severe rust dislike. The switch was mainly due to:
- far better documented std lib. For example zig http stuff has bugs; and I could never find a public unzip inflate call despite the appearance of an impressive zig/unzip api.
- better library support esp for protocol. The zig stuff i found was few and had memory leaks.
- the fact struct fields are private by default (in zig all fields are public if you can get access to the struct). In complicated stuff the implementation has got to stay private
- the borrow checker was not a motivator, and rust compile times are nowhere annoying as I feared
- the zig i/o api i found too darn complicated and annoying
Things I still miss from zig:
- c/c++ ffi is solid. Gluing curl, zlib into zig was painless and worked day one. Rust takes more work.
- zig build and the ability to import a configuration as code is awesome.
- allocator story is good. It's darn simple to add custom allocators where performance matters
- Zig still has a couple of linker problems, or cross compliation bugs I immediately encountered when using it. One was fixed. One is still open.
[–] pron 1 day ago
2. Heap allocation is a very common cause of serious performance problems in low-level programs because malloc/free overhead is so high. Some languages have a moving GC to reduce the high overhead of heap allocation, but low-level languages typically can't use that optimisation, so you have to resort to arenas (which offer a similar optimisation to that of moving collectors). This isn't always faster than malloc/free, but it usually is, and if you want malloc/free you can pick that.
Anyway, I wouldn't say that the default in Zig is to use an arena, rather it's to give you the control that's the whole purpose of low-level languages.
[–] pebal 1 day ago
[–] pron 1 day ago
[–] pebal 1 day ago
[–] pron 1 day ago
[1]: More precisely, the work they need to do doesn't scale linearly with the allocation rate as is the case with non-moving approaches. Of course, arenas are sort of degenerate moving collectors, so they also display the same effect, but they can't always be used.
The way this works is that you allocate in the young generation with a pointer bump, and to free memory you compact (move) the surviving objects, which make up a small minority, a few times and then promote (move) them to the old generation (where you either manage memory with the same technique or a different one). And so the total amount of moving is not high (objects either never move or rarely move), and what you end up with is essentially two heaps, one where the allocation rate may be high but you can completely negate all the extra work by increasing the size of that heap, and one where the allocation rate is low. This ability to, optionally, not double the amount of work when the allocation rate doubles is what gives moving collectors their efficiency (and that you can bump-allocate in virtually all cases).
Of course, it's possible to come up with programs where objects do move a lot (or there's some other common work), e.g. programs that don't exhibit the behaviour of the generational hypothesis, but the point is that in a very large class of programs, this technique is hard to compete with.
[–] MathMonkeyMan 2 days ago
The trouble comes when you have data structures that manage memory, and you want to associate the allocation strategy with the particular instance of the data structure. Or, worse, you don't store the allocator pointer with the data and you require that all functions operating on the data structure happen to use compatible allocators.
At its simplest, though, passing allocators around is just removing an implicit global variable that is sometimes helpful to customize locally, but nearly impossible to customize locally if left as a global.
[–] hansvm 2 days ago
I distinctly remember one of my first "real" jobs, somebody had written a C# library for GraphQL. It had some sort of bug or lack of functionality I had to fix. The author had fallen hook, line, and sinker for sealed classes and all that nonsense, so the shortest path to using the rest of their code was to fork the library and re-write the thousand offending lines and all references thereto. Luckily it wasn't a deep dependency.
Contrast that with similar issues in typical Python. The library likely accepts my favourite duck-typed solution, if it doesn't then I can likely work around it with an appropriate choice of dunder methods in the code I have interacting with the library, and if it still doesn't yield then I can just rewrite (monkey-patch) the offending code at runtime.
Not that either of those scenarios are objectively optimal, but they illustrate one of the principles people have in mind when they use an interface like allocators (not the only one, I'm sure other commentors will talk your ear off), that when push comes to shove you'd prefer to be able to use your existing code to solve your new problems. And that does empirically mostly work here; you can use nearly the entire Zig stdlib (especially when ignoring things like the posix namespace) and most of the rest of the ecosystem as well on embedded devices, in WASM, in bounded-memory applications, in generic application code, in fast-running CLI apps not even bothering to free, etc.
I agree with much of your discussion about arenas by the way. They're a moderately poor fit for the current interface in Zig, largely because the free functionality is questionable at best. IME, most uses of arenas actually know they need such a thing. Functions taking allocator arguments and expecting the allocator to be an arena should take an arena argument. When using arenas for lifetime management, something should probably own that arena rather than implicitly assuming whatever gets passed through the stack is correct. Passing arenas to arbitrary functions expecting general-purpose allocators is prone to leaks, and you'd be better off using the arena (or a buffer created therefrom) as a backing allocator for a data structure capable of actually freeing if you can't trust the implementation you're calling into (and if you can trust it, they should expose a better API allowing you to detect that trustability at the callsite).
[–] sethops1 1 day ago
[–] hansvm 1 day ago
The bigger design point is who's responsible for knowing where the object lives. Zig has a balance of power more in favour of the caller than the callee, a point I tend to agree with since it's quite easy for a caller to know exactly what they want done and quite hard for a callee to know what all possible callers might want done. That principle is applied pretty liberally through the rest of the language.
[–] xedrac 1 day ago
[–] hansvm 1 day ago
Jokes aside, that's not the worst idea (you often see the stdlib do something in a similar vein with managed vs unmanaged data structures), but I bet it doesn't survive contact with real-world programmers well enough to actually provide the portability that normal Zig code has. For transitive "style" impositions, it only takes a low rate of non-compliance for the perceived benefit to disappear.
If that's a goal you have (and IME most code is lightning fast to write, with all the heavy lifting going to impedence mismatches like the one being described, so I tend to be happy to write a bit of extra boilerplate (I like Zig, but it's an absurdly verbose language) if it means the hairier integration points will go more smoothly, and I therefore do have that goal) then the allocator API empirically works, the other solutions I've seen work much less, and I'd want to have a pretty good idea of why a contender would sate my needs instead.
If that's not a goal you have, you do you man. I can shovel out sealed Scala classes or unmaintainable numpy with the best of them as the situation demands. Nobody's created "the best" language yet. We're just casually discussing tradeoffs.
[–] ozgrakkurt 1 day ago
You can limit the concurrent requests that can be handled at a time to 2000 and limit memory per request handler to 2MB.
So you allocate 2000 x 2MB at program start.
Then you crate an arena allocator with (2MB limit) when starting to handle a request.
It will just fail if a request uses more memory than 2MB. This means you will basically never have OOM issues or a request using unexpected amount of memory to crash the server.
You also have performance benefits like:
- Allocation operation is basically an integer increment.
- Memory for a request is laid out linearly in memory and likely is pre-populated so you won't have page fault into the kernel.
- It is very unlikely to have double free or other memory management mistakes.
You can also fuzz the system in several ways to make sure requests can really never exceed that amount of memory usage so you don't have surprises in runtime.
If a request by a user really exceeds that much memory usage, you can just fail that request and log it so it will be debugged and fixed later.
You also don't need data structures like SmallVec (rust version of the thing) that are hybrid stack/heap allocated.
There are many intricacies to this approach but it works much better on scale when you have a large program that handles a large amount of memory.
It is super difficult to debug and fix memory usage issues on global allocator only programs in my experience.
I tried to fix one such issue on a database-like program in rust and it was impossible.
Also this means you don't absolutely need ownership/lifetime safety.
I am using this pattern in a C-like C++ codebase I have and it works really well.
[–] slopinthebag 2 days ago
they make a lot less sense in languages where the compiler or runtime takes care of this for you.
[–] flohofwoe 1 day ago
[–] rk06 1 day ago
however, in languages that don't allow it, you can't pick and choose different allocators for different workflows. it may not be important to you, but it is important to many. and shoe horning support for multiple allocators is not easy for language
[–] lukaslalinsky 1 day ago
Zig is a language to use when you do need the control, mostly because of performance reasons. And once you are in, you start thinking how to manage your allocations and arenas are the logical outcome, once you notice that your application has scopes that can be destroyed at once.
If you need an argument why arenas are better than RAII, then you likely don't have the performance needs to even consider using Zig, because outside of performance, arenas are not better.
[–] elendilm 2 days ago
This is for a long running process where consuming compute and memory should only occur on processing a request and must revert to a baseline memory consumption once the request is completed.
malloc with default allocator always gradually increases the memory consumption. This continues with enough request bursts on our testing until the process runs out of memory. mimalloc doesn't.
On mobile, we are stuck with the default allocator. On mobiles, the phenomenon of the default allocator consuming memory is reproducible even with normal Qt apps. It is what it is.
[–] grg0 2 days ago
Curious, how much memory and how many requests does that take to exhaust?
[–] elendilm 1 day ago
But in recent years, the phenomenon was visible with Dip which was built on rust.
Dip does massive parallel fanout using green threads ('may' coroutines) with work being NUMA pinned (software hash) using a thread per core model.
A default request may span up to _chunk_count_ jobs (default 5k) configurable per request.
Every request (x 5k jobs) from the client on a stress test sees a jump of ~10-20 MB of memory. Sometimes the jumps use large swaths of memory - say 100s of MB and stabilizes for a while only to gradually climb up again later. Rinse and repeat - OOM kicks in and the process crashes.
Many times we thought we were leaking memory somewhere. But armed with the Qt experience a decade ago, we intuited maybe its the allocator - and switched to mimalloc on desktop executable.
Dip's memory, using mimalloc, stabilizes at ~60 MB post request completion no matter how many requests you throw at it.
Pity that on mobiles, more specifically Android, we have to live with the default allocator on even trivial Qt apps let alone embedded Dip. But people have lived with it for at least a decade, so it is what it is.
Thus on mobiles, we just have to be content like everybody else, unable to squeeze every last ounce of performance and efficiency out of the hardware on stress tests ensuring Dip remains "just better :(" :)
[–] giancarlostoro 2 days ago
[–] amw-zero 2 days ago
I don't think the footguns you're talking about are real by the way.
[–] GuB-42 1 day ago
Languages have opinions, that's why there are several of them and they are popular in some applications and not others. And Zig opinion is to be explicit so RAII is out, and the language does push you into avoiding allocation when unnecessary, as a side effect of not giving you the "easy to use interface" you want.
This makes it particularly suitable for fields like embedded software where you may not have malloc/free at all, but may be less so in a highly dynamic environment, where you may be better served with a GC anyways.
[–] silon42 1 day ago
[–] GuB-42 1 day ago
To me, the name "Resource Acquisition Is Initialization" implies that doing one means the other one is done implicitly. Now, in the end of the day, the result is machine language, with proper discipline you can follow the RAII principles in any languages, including assembly, but I have always understood RAII as being it done implicitly.
Also we can argue that Zig supports a weak form of RAII through the use of "defer", there are some nuances here.
In Zig app code you don't have to pass allocators all over if you don't want to, that's your code so do what you want. But for libraries it's the right thing to do because it puts the caller in control.
The reason you have to pass allocators instead of making this implicit like in e.g. Odin, is because Zig just takes a firm stance against implicit logic, firmer than any other language I've ever used. The end result is that people who like Zig tend to like passing allocators around even in their own code.