Alternative(s) to run CUDA on non-Nvidia hardware

pjmlp

Most of these "alternatives" focus on CUDA C++, and overlook what actually makes CUDA interesting. Already in 2020, https://developer.nvidia.com/blog/cuda-refresher-the-gpu-com...

maxloh

There is also ZLUDA, which is open source and works on pre-compiled binaries. https://github.com/vosen/ZLUDA

DiabloD3

Its easier to just get rid of your legacy code entirely and use Vulkan for compute, or have your compiler emit SPIR-V directly. No reason to tie yourself to Nvidia's moat.

lulzx

I have been trying for cuda -> metal, to run it on mac, https://github.com/lulzx/cuda-metal

luciana1u

every CUDA alternative follows the same arc: bold launch, works for 3 operations, then a Discord server where the last message is 'any updates?' from 2024

woctordho

There's nothing wrong to run CUDA on non-Nvidia hardware. CUDA has an interface that is reasonably well-designed, well-documented/reverse-engineered, and battle-tested for decades. What we need is not to invent another interface just under the name of 'open standard', but to implement the same interface. ROCm is exactly doing this, and so are other hardware SDKs such as MooreThread and Alibaba T-Head.

puschkinfr

In this context AdaptiveCpp should also be mentioned. Started as a SYCL implementation, but recently-ish added a compiler for compiling a CUDA dialect to GPUs and CPUs from basically all vendors

dachworker

Why should I not just port my kernel to Triton? What's the appeal of Scale?

asdaqopqkq

aren't llms smart enough to directly write custom kernels for custom hardware from cuda code?

u1hcw9nx

Alternatives exist, but little demand outside hyperscalers and special uses. Neocloud customers just want plug-and-play CUDA. It works, it's tested, it adapts faster, and has known performance. Alternatives give no significant benefits. Things can change, but they are not changing now.

cactusplant7374

Isn't the future of the industry specialized chips like those that Broadcom and Cerebras are making? I don't know how much longer I can tolerate 50 tokens per second. It feels like the dial-up era.

woodrowbarlow

i'm also interested in tenstorrent. they're building GPUs with cheap GDDR6 using a fast SRAM cache, and writing their own compiler stack (used instead of CUDA) that pipelines data to the SRAM ahead-of-time so you (in theory) never need to suffer the slow speed of GDDR6 for AI workloads. also they've got built-in SFP cages where the video ports would normally be.

villgax

@claude add this to the graveyard of wannabees

HarHarVeryFunny

These efforts to support CUDA on non-Nvidia hardware seem to me misguided. If all you want is to be able to easily use non-NVidia hardware then high level tools like PyTorch already let you do that (and torch.compile uses Triton for target-specific optimization). OTOH if you want to be programming close to the metal to achieve top performance then you are probably not using CUDA in the first place, and using some CUDA translation layer on non-NVidia hardware would be an even worse idea.

alightsoul

I fail to see how scale is not just another form of vendor lock in, given that their compiler is not open source. Every compiler used today except cuda's is open source. And Nvidia can get away with it because no one else cares about development experience