What if we stopped using GPUs? [video]

tolugenius

I wonder what would "compute" mean if cpus were more efficient at matrix multiplication say 15 years ago. And on the flipside, what it would take to say train a frontier model entirely on cpus in the future.

librasteve

As the video points out, there is a hardware/software feedback loop at play here. Since the hardware is deeply pipelined SIMD FPU datapath, the software is hand tuned machine coded transformations. In addition to limiting flexibility (every variation has to be ground out in CUDA), this prevents sparse matrix type optimisations. I predict that a set of general purpose CPUs - non shared memory at this scale - would be a much better use of transistor/power. And you can code that at high level give a CSP style approach such as https://bil-lang.org

vivzkestrel

- stupid question - how exactly does apple silicon s unified gpu + ram thingy work? - how come intel and nvidia cannot do the same? - is there an actual difference in terms of hardware architecture or something or is it pure apple marketing hype?