Everyone should know SIMD

qurren

I just do gcc -O3 and get SIMD without having to learn it

wrl

i was having a conversation with a friend recently about simd in zig (which i have recently picked up and been having a pretty good time with). i find that simd writes decently well, though there's a few weird things: - some builtins purport to work on simd vectors but actually just unpack the vectors and do their work per-element (e.g. running `@sin()` on a `@Vector(4, f32)` will unpack the vector, run `@sin()` 4 times, and then pack it back into a vector). - a lot of `std.math` is scalar-only (some functions support vectors, though, and i've got a pr open for one of them and plan to do more). - i'm certainly missing some intrinsics that i get from xmmintrin.h (rcp, rsqrt, few others). in general though i'm finding it pretty capable. mitchell, i know you hang around some of these comments sometimes – i noticed that in ghostty you bring in some c++ libs to do the simd heavy lifting for you. any plans to port that to zig? anything missing from the language or libs that's preventing it?

eska

I don’t know zig syntax, but wouldn’t it be possible to put this common pattern into a macro and simplify it to mostly a lambda on V?

hnal943

Here's a helpful video about leveraging SIMD to solve a concrete performance problem for the dev team that made the game The Witness by Casey Muratori: https://www.youtube.com/watch?v=Ge3aKEmZcqY

pton_xd

"More importantly, when this loop matters enough for me to care about a 5x speedup, I want the vectorization to be explicit and predictable. I don't want an unrelated code change or compiler update to quietly turn it back into a scalar loop." Only tangentially related but this is by far the most painful part about optimizing code for JIT compilers like V8. Even changing a constant from 1 to 1.0 somewhere else can change the optimizations performed and lead to an unexpected performance decrease.

Rendello

I like SIMD, but before super-optimizing your code with SIMD and the like, really consider your data structures and access patterns. I've been singing Data-Oriented Design's praises, so I'll just collect all my comments here [1], but I think it's a good approach to optimization. I played around with SIMD in my old code (in Zig), but my approach to modelling datastructures was so antithetical to optimization, it was like putting high-performance racing tires on a lemon with a broken engine. It was the root-of-all-evil-type-premature-optimization , because I wasn't measuring performance, and I wasn't thinking about where the allocations were, etc. Now, I try to model my data as if it were SQL tables, see what my potential "primary keys" could be, and build my data structures around my access patterns. For example, I used to model trees as structs pointing to other structs on the heap: struct Tree { tag: TreeTag, children: Vec<&Tree> } Now my tree has all the bad characteristics of a linked list (* n nodes * m children), all the fragmentation of multiple heap vectors (* n nodes), and terrible set-up / tear-down time (in this case, Drop alone was taking up a good chunk of runtime). But a tree can be represented a million ways, and can always be linearized. So now I really consider my access/insert patterns of the tree, whether it's really a tree or some other sort of graph, whether I can store it in a Vec or a Struct of Vecs, etc. Since really looking at things through their access patterns and "primary keys", my code has been much faster and simpler. This has the added effect that a lot of your data ends up in homogeneous arrays / vecs, which means that the compiler can do its SIMD magic, the CPU can read it from your L1 cache a million times faster, etc. And then when you need to drop down into SIMD yourself, you can write some awesome branchless code. 1. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

derf_

To bolster the argument, even if you do not plan to write the SIMD yourself or will "just get AI to do it", it is important to know what can be fast in SIMD (and on what hardware). That allows you to design your algorithms and structure your code so that the SIMD is possible. Internalizing things like how data dependencies matter, how expensive it is to increase the width of your vector elements (and how to avoid the need), how to turn conditions and branches into masks, or simply things like "division does not exist" becomes a lot easier when you have spent at least some time trying to use SIMD yourself.

kristianp

Tangentially for Go programming, the last time I looked at optimising some Go code with SIMD there were a few different options available, but they were either not maintained any more or had incomplete support and required first writing your function in C++ with intrinsics and generating assembly, then converting it to go assembly with a tool [1]. I never got my function to work in go despite the C++ code working fine. In short, not really a production ready option for Go. This was a year or two ago, though. Edit, there's now an experimental official library at https://go.dev/pkg/simd/archsimd/ see https://go.dev/doc/go1.26#simd and at https://github.com/golang/go/issues/78902 so things have moved since I tried it last. [1] https://github.com/minio/c2goasm

waffletower

I was hand-rolling NEON SIMD 15 years ago, and in many cases the compiler (clang/llvm) simply out optimized me. I kept the attempts that were better than what the compiler could already do. That was ARM NEON, quite new at the time, not SSE, and again that was 15 years ago that the compiler could already beat me much of the time. I hate the naive cult coder adage concerning premature optimization, but this might be a situation where you peruse your compiler output before you start writing code in a manner the compiler can for you. In Clang, auto-vectorization is enabled by default at optimization levels -O2 and -O3.

arijun

> Every developer should… most importantly, not be scared of SIMD Seems like he should be recommending fearless_simd [1], the Rust crate by Raph Levian and the folks at Linebender :) More seriously, if you’re looking to add SIMD to your Rust code, that’s the package to start with. [1] https://crates.io/crates/fearless_simd

andix

99% of developers should just ignore SIMD. Most projects have a lot of low hanging fruit to increase performance, and still nobody finds the time to solve them.

pjmlp

Not really, not as SIMD remains an esoteric art from packing matrix and vector operations in endless opcodes. I rather let the compiler auto vectorise itself, or with AI help.

mamcx

Is interesting that this is how array langs work. I bet will be easy to turn into a lib.

gblargg

My compiler knows SIMD. However, knowing the limitations of SIMD might help avoid a calculation that can't be optimized to use it.

ktimespi

The biggest barrier I faced learning SIMD is the weird naming convention for intrinsics. Once I got past that, it was fairly simple. mcyoung's articles were also super helpful

losvedir

This is an interesting article. I don't really work with low level enough languages for this to matter (unless - does this ever show up in Javascript somehow?). I guess I don't understand the "reduce" step. It seems like you have to be careful not to "undo" all the benefit from SIMD. Sure, it can compare 8 values in parallel, but then if you have to look at each of the 8 answers in turn you're back to where you began. Is the `@reduce()` function in the example a special Vector one that tells you if all the values are true or not in one "step"?

boricj

Everyone doesn't need to know SIMD. Mechanical sympathy is an important passive perk for software architects to cut down the number of reworks down the line, but I would rate benchmarking and being able to identify bottlenecks as more important everyday skills. I'm working on a voxel space renderer homebrew for the PlayStation. I only have so many cycles to spend on rendering before it becomes a slideshow, so I count them in my hot rendering loop and parallelize work as much stuff as I can, even across memory load stalls from main RAM. I've worked on a basic network accessory card with a STM32 MCU that is extremely overkill for what it needs to do. We haven't bothered making any performance or memory optimizations whatsoever, writing plain C++ almost as if we were on server-class hardware because we had such egregious margins. The first question to ask is not whether something can leverage SIMD, it's whether the performance requirements are met or not (although it's far too easy to not care when it's not your hardware that's struggling...).

magarnicle

Everyone should know about SIMD, so you can know when to ask your good friend Al, who knows how to do it, to use it for you in the right places.

ivanjermakov

I respect (and fear a little) those who intentionally utilize SIMD in their implementations, but I believe it's a bit too much of a semantic shift for 2-5x performance gain. Good news is that modern compilers are more than capable of emitting SIMD code even if original source is nothing but. Most software's poor performance would be fixed long before SIMD comes into play. Reduce obvious DB/server round trips, bad data structure/cache locality, poor algorithm complexity, doing redundant computations, etc.

guess_who_is

SIMD does not pay, speaking from 20yr exp in the field in various semis.