The work by Valve's Timur Kristóf on improving old AMD GPUs on Linux

segmondy

I wonder if some of these could carry over for LLM inference. It will be nice to turn more ewaste GPUs into capable processing units for LLM.

LaurensBER

I just bought a used Ayaneo 2 handheld, it has an old(er) mobile RDNA 2 GPU and I was blown away by how well this thing performed under Linux. Almost everything (that's not a recent AAA game) runs beautiful and a lot faster/smoother than it does under Windows. The experience has been so good that I'm considering switching my main pc (with a 9070XT) to Linux as well. No doubt Timur contributed heavily to this given Valves Steamdeck (which uses a very similar but slower GPU). Given the current hardware prices it's pretty awesome to see someone squeezing maximum performance out of old hardware!

wewewedxfgdf

If only AMD did this.

gary_0

Direct link to the talk, with timestamp: https://youtu.be/j5W5ErEMnvM?t=21385

vkaku

Honestly, the folks writing inference drivers for Llama.cpp / GGML would sort of benefit from better compiler work like this. In general, Valve's work has been exceptional and supplementing AMD's own ROCM/OpenCL and Vulkan team, they've gotten a lot of defaults right and people should work together with Valve to improve support for their chips.

bugake

Some other benefits of older GPUs: Use as a dedicated GPU for encoding and decoding video. Post processing like frame interpolation or superresolution. Use for GPGPU workloads. Run additional monitors independently. Use for GPU passthrough to virtual machines. Use as a backup GPU for troubleshooting. Use for test code without breaking the main GPU