Restartable Sequences

khuey

Maybe I'm just getting old but the "if you don't spend $20,000 on a workstation you're going to be left behind like a dinosaur" at the top of this article is a huge turn off to reading any further. And I say that as someone who owns a workstation with more cores than the author's.

GlenTheMachine

If you had no idea what a restorable sequence is the takeaway is about halfway down the OP: “This is why Linux now provides rseq() which is a much more enlightened solution. With restartable sequences, you actually can get rid of both the mutex and atomics, while the OS continues to fully abstract scheduling. The way it works is you advise the kernel whenever your program enters a critical section of code that you don't want interrupted. It's probably going to be maybe 10 assembly instructions tops. The first assembly opcode should be a move instruction that sets the rseq_cs field. The last instruction needs to be the thing that makes the modification to your global data structure. Think of it sort of like a really tiny database transaction. What makes it go fast, is that the bidirectional communication with the kernel happens via shared memory.”

Veserv

Restartable windows, or more generically introspection windows, are a really useful technique you can apply in any situation where you understand or control the sources of preemption. The earliest uses of this technique in operating systems that I am aware of are ~25 years old. The key insight is that the preempter can introspect the program counter of the code being preempted (which is now stable since it was preempted) and act accordingly. The simplest mechanism is to reset their program counter if in a critical section. The more generic mechanism is to jump them to a supplied address. This allows you to do things like hard abort and more. You can further remove the need for the preempter to understand the preempted code by having the preempted code create a self-introspection code snippet and supplying that with the program counter at preemption. So the preempter just vectors them to their own code which knows how to interpret its own state at any preemption point.

senderista

I'm surprised there was no reference to the librseq library, maintained by the rseq implementer: https://github.com/compudj/librseq This has helpers for common use cases like counters and linked lists. You shouldn't need to write assembly at all to use rseq in most applications.

yubblegum

> chances are the CPU's internal mutexes aren't as good as the ones you've implemented in userspace Anyone with an informed opinion on this statement? It's seems counter intuitive (npi).

keyle

... bidirectional communication with the kernel happens via shared memory. What could possibly go wrong?

squirrellous

IIUC rseq is similar to thread-local data with the additional benefit that it scales with number of CPU cores, not threads. However if you are an application developer and is able to control all the threads in an application, then rseq isn’t that superior. I fully agree that rseq should be more easily available to Linux developers, though.

matheusmoreira

This is amazing. I'll definitely use this in my projects.

smasher164

I was having a conversation with someone recently if RSEQ would be a good primitive to build a load-link/store-conditional implementation in user-space. It gives you a critical window, though you still have to deal with spurious restarts, and provide a way for one core to abort another.