GPU Offload in Rust: Portable, Safe, and Fast

Thomashuet

That's promising but did they publish any code? I can't find anything in the abstract.

jasonjmcghee

> the rust-gpu project has to emulate pointers[8], which we consider a blocking issue for most HPC benchmarks. Why is it a blocking issue? I feel like this is very aligned with the goals of rust-gpu.

rfgplk

Fascinating how many people still overcomplicate offloading to GPUs.

maxchisto

does anyone know Mojo well enough to comment how Rust + gpu-offload compares to it?

Alephinitesimal

The NVIDIA+AMD support is the part I find really interesting. I know OpenMP and SYCL can already target multiple GPU vendors, but doing this while keeping Rust's safety model seems pretty compelling. I'm curious how portable the performance is in practice.

boywitharupee

is this mainly about making host binaries self-contained for heterogenous workloads? also, seems like this is mostly targeted towards HPC audience?

whateverboat

> This module is under active development. Once upstream, it should allow Rust developers to run Rust code on GPUs. We aim to develop a rusty GPU programming interface, which is safe, convenient and sufficiently fast by default. This includes automatic data movement to and from the GPU, in a efficient way. We will (later) also offer more advanced, possibly unsafe, interfaces which allow a higher degree of control. I really appreciate the work and the effort that went into this. However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?

YuechenLi

So... why go through LLVM at all instead of having the MIR target PTX/HIP C directly then? If they really wanted a vendor neutral solution for Rust GPU, that already exists: you write the CPU side code, including buffering, allocation, concurrency, etc through Vulkan binding and consume the compute kernel in SPIR-V from HLSL/GLSL/WGSL etc. As it stands, the way they use Rust here feels more like using it like TypeScript types/interfaces than anything else. Again, the size of most operations that should be done on the GPU is known ahead of time before compilation, so it's very much possible to statically allocate memory at compile time instead of going through all this trouble to write what's essentially a Rust shaped DSL for GPU compute.

bicepjai

I write all my code in Rust because I am a Rustacean. In many of my custom LLM inference engine projects, the biggest fight has always been bindings. I don’t want to maintain and write bindings; also, if I use an existing project that provides bindings, then I have to wait for the owner to update or fork it and then maintain it on top. It has been a big headache. Running Rust core on GPU sounds like something I will try from day one. Kudos to the team and will watch it closely.