Handles are the better pointers (2018)

PaulHoule

A nice analysis of large-scale programming techniques that go beyond the novice version of "OO Sux" to the almost secret https://en.wikipedia.org/wiki/AoS_and_SoA and why game engines use entity-component systems, analytical databases are column-oriented, etc. That is, the classical organization of "objects" as structures has a lot of overhead when you're doing heavy work. For instance java reifies objects with a big header, a monitor lock, and other stuff that really adds up if you, say, are writing a chess engine and need to maintain transposition tables. And even if you are in C++ and there is a lot less overhead, AoS still has bad mechanical sympathy compared to SoA.

michaelcampbell

This article reminds me (on the surface, this is deeper than my experience) of how the recent Goose language orients things. I never worked with it but I seem to recall the old original Mac API's (Pascal based) used "handles" a lot to solve the issue of being able for the system to move allocations and memory out from under a program without it knowing, as a "pointer to pointer to <thing>". I was poring over those manuals in ~1988/1989 with visions of being a full-time Mac developer in my head.

throwitaway222

I remember back in the day in Windows C++ programming everything was a HANDLE memory is probably not serving me, but they were all really just void* renamed. And you use them with various win32 functions I guess to keep things straight. I don't miss those days.

Surac

my API always use handels, not because i design them that way, but in every iteration they get more and more handles

program_whiz

there are serious downsides to this, its a tradeoff and I wouldn't say they are "better pointers". There are definitely cases where it can work (e.g. uniform objects that either rapidly create/destroy or all die at a boundary). This is usually the case with a "simple memory model" domain like games, rendering, etc. All the work starts and ends at a request, frame, or scene. But for anything else, handles can be trouble. First, you're obviously paying indirection tax on EVERY lookup. Logic like `x->foo` becomes `memory_allocator.objects[x->index]->foo`. You're also paying the generation counter tax (assuming you want any kind of safety). So you also have to do something like this: if (x->index < 0 || x->index > memory_allocator.objects.size()) { // panic? } auto *ref = memory_allocator.objects[x->index]; if (!ref) { // panic? } if (ref->generation != x->generation) { /// panic? } return *ref; Next, you have type bloat. Hopefully you are making type-safe versions of all these indices so you can't just use an aribtrary index into any kind of array right? Ok well that means now you have double the types (one index type per pointer type). Ok maybe not the worst, just a lot of boilerplate everywhere. Prepare for a lot of this: struct Handle { uint64_t index; uint64_t generation } typedef PlayerHandle Handle; Handles are also a lot bigger. If you're memory constrained, adding an additional 16-32 bytes per pointer allocation is painful. For my programming lang I have a single test file that contains 300K AST nodes, that would be another 9.6MB to parse a single file for handles alone, for noticeably slower performance (300K+ indirections and cache misses). Next, you have the issue of resizing your memory array. You now are in "write your own garbage collector" land. You want to consolidate 50% of the memory that is suddenly free? Well you need to change every index to point the new compacted range (expensive and complex), or manage a free list which potentially increases the memory requirement, and adds complexity, cache coherence issues, etc. Do your memory lists grow indefinitely? Do you need to think about having separate lists for short lived and long lived (generational GC style)? Managing lifespan of objects all sharing one backing storage when the indexes are just integers strewn about is actually _harder_ than doing the same with pointers (just use compacting GC already). Then there's the issue of object pools. You want your data to live in arrays. Do you put all objects into one master array? You lose cache coherence / iteration speed, and type safety (unless you really bloat and start adding type info to every entry). If you use different arrays for each you might end up with dozens of arrays, one per type, and also their free lists. Now you need a way to manage the lifespans and scope of all these object pools. Suddenly, you've reinvented malloc / free and segfault signals. Or you could just wisely use pointers...

emtel

I’m not sure how applicable this is today. It is quite hard to beat jemalloc, especially for small allocations, which it handles essentially the same way a hand coded pool or arena would anyway.

mcphage

This sounds a lot like ECS—or at least, how ECS systems handle memory for the components.

FLeXMurphy

They always have been, and Win32 extensively uses HANDLE with some underlying restrictions. I remember reading up that certain numeric allocation buckets in HANDLE corresponded to the type of underlying resource (ie. Nth bits mean <this resource>), but it has been too long to remember the details.

Conscat

I made a real attempt to program this way in my C++ project (largely due to this article), and personally I found it extremely cumbersome because you have to carry the owner of this resource everywhere you want to use the handle. That itself can worsen the performance when it's in different functions, it made generic programming hard and such code has to be more template heavy, and it was difficult to refactor. But this was a very low level project, and I suspect Floooh was probably thinking more of end-user applications.

nickwanninger

My PhD thesis is on this topic, and I had an ASPLOS paper on handle based memory support in unmanaged languages (C/C++) through LLVM a few years ago: https://nickw.io/papers/asplos24-alaska.pdf The overhead we saw there is suprisingly just from instruction and branch overhead from using software handles - I have ongoing work to do it in hardware and the overheads are much lower.

boricj

That's basically what I've done at work on my embedded project. Object pools, with indices and generations. Way, way easier than meddling with raw pointers, cheaper than shared_ptr/weak_ptr and no clean-up required when deleting stuff because the stored generation won't match the new object's generation. You can even make that default-constructible if you increment the generation on instantiation too (so that only even generations are alive) and hide it all behind high-level abstractions with C++. Though my knowledge of these techniques come from 2000s video game architectures, as games like GTA: Vice City are known to use it.

peter_d_sherman

The most popular program of all time which used the concept of "Handles are the better pointers" -- is/was the Microsoft Windows series of operating systems (from a little company called Microsoft, in Seattle). You have have heard of it (or them!) You see, in that OS, individual windows (as GUI elements and "Operating System Objects") were not pointers, but Handles! Every other OS-controlled object (Bitmaps, Brushes, Fonts, Device Contexts, etc.) were also Handles! You see, Application Programmers (for normal applications, not Ring 0 device drivers) -- were never given direct pointer access to OS internal data structures! Instead the OS handed out Handles for different OS-managed objects to application code that asked for them, allocated and managed the memory they used during their lifetime, and disposed of the memory so allocated when the application using them had completed or terminated. Unix/Linux is similar -- except that the application programmer typically gets what's called a "File Handle" (because "everything's a file" under Unix/Linux...) But generally speaking (and yes, there are exceptions like mmap() on Unix/Linux and ring-0 drivers on Windows) the application programmer never manages the memory allocation or disposal themselves, they never use direct pointers for these objects, they go through an OS proxy to that OS structure, that proxy is called a "handle", and it moves the proper responsibility for memory allocation/deallocation of the resource to the OS, away from the programmer. Compare and contrast this with the use of handles (as proxies) and the concept of "Encapsulation" (from Object Oriented programming, but equally applicable here) in codebases where you have a central Senior developer who writes the central code in a language like C or C++, full of pointers, complex data structures, complex dynamic memory allocation/deallocation, etc., and you have regular developers who write code via the use of Handles around that central code, i.e., that central code does not expose pointers or direct memory -- instead it exposes an API or functions where regular programmers request handles, and subsequently pass the handles with additional API calls to the central codebase to gain functionality for whatever object or objects are represented by the handle. (You can also compare this to obtaining a token/cookie from a web API, then passing that token/cookie on subsequent requests with different additional parameters to different endpoints on a single website's API to gain different functions related to the use of the back end object or objects that that token/cookie is related to...) Also, I should point out that while replacing pointers with handles in code may make a programming a little bit slower, it also allows the handle allocation code to save/update a list of allocations. Hybrid code could replace pointer operations with handle allocation in a test/debug codebase, then replace it again with native pointer operations in a release codebase (faster, once all the bugs are worked out). A handle allocator is a powerful tool to check for memory allocation errors in code if it's programmed correctly... Allocate the same handle twice -- have your handle allocation code can check for that, and raise an immediate exception! Forget to free it at the end of use? Raise an exception for that! It's a great way to find bugs in pointer code! Anyway, great article! Related: https://en.wikipedia.org/wiki/Encapsulation_(computer_progra... https://en.wikipedia.org/wiki/Pointer_(computer_programming)

Panzerschrek

Nice approach! I did something similar in my recent project (even if it was not C++). But I didin't bother handling overflow of generation counter, since for my purposes 32 bits were more than enough.