CUDA Books

phoronixrly

In an age when your company mandates you to raise your productivity right now with hundreds of percentage points using LLMs, how do you find an excuse to sit down and read a book?

chrsw

"AI Systems Performance Engineering" might deserve a mention, even though it's not strictly CUDA.

zparky

I liked going through https://www.olcf.ornl.gov/cuda-training-series/ for an intro and some fundamentals.

juvoly

Increasingly (for instance ADSP podcast [1]) those in nvidia's inner circle are advocating against writing your own CUDA kernels. (Unless that's your full time job at nvidia, that is). [1] https://adspthepodcast.com/2024/08/30/Episode-197.html

pwython

First one I clicked on is 404: Programming Massively Parallel Processors: A Hands-on Approach (3rd Edition) https://www.cambridge.org/core/books/programming-in-parallel...

dahart

Regarding the section on Python and high-level CUDA, anyone interested should maybe first take a peek at Warp, which I’m guessing is too new to have a book yet. Warp lets you write CUDA kernels directly in Python, and it’s a breeze to get started. https://github.com/nvidia/warp

somethingsome

Having read or at least skimmed most of those books, I think the best intro is 'CUDA Programming: A Developer's Guide to Parallel Computing with GPUs' Massively Parallel Processors: A Hands-on Approach is not really good in my opinion, many small mistakes and confusing sentences (even when you know cuda). CUDA by Example: An Introduction to General-Purpose GPU Programming is too simple and abstract too much the architecture. Next year I'm planning to start writing a cuda book that starts by engineering the hardware, and goes up to the optimization part on that harware (which is basically a nvidia card) including all the main algorithms (except for graphs). I'm already teaching the course in this way at uni, and it is quite successful among students.

brcmthrowaway

Any good MOOCs on Parallel programming/NVIDIA?

fwx

Does anyone know of any good resources for the newer paradigms like cuTile?

SkiFreeWin3

I wish the README had a solid “what cool things you can do with this” right at the top. In this day and age when programming is so accessible, why not have a more tempting pitch than just book titles categorized by difficulty.

saagarjha

Probably worth noting that writing performant kernels for modern Nvidia hardware looks almost nothing like what the books from 2012 are going to teach you. You can read them for fun if you'd like but they're basically irrelevant.

wces

This is highly condensed video of all important concepts in CUDA from Stephen Jones, one of the CUDA architects: https://www.youtube.com/watch?v=QQceTDjA4f4 Understand everything he talks about and you understand CUDA.