Grace Blackwell needs a new scheduler: 27% more usable GPUs from the same racks

jhgg

This blog post spends a lot of time telling why gang scheduling is needed here (a fact obvious to anyone using the nvl72s), but doesn't actually go into detail about how their scheduler works. At work, we have built something very similar, a custom topology aware scheduler for workloads, and the most interesting things about such a project are entirely absent from this article, making it more of a fluff/marketing piece than an actually interesting technical blog in this area. Our next iteration of this scheduler aims to scale to O(millions) of GPUs, with a scheduling cycle time measured in the milliseconds.

novalite

This reads a bit disingenuous and click-baity. There's no "new" scheduler. Gang scheduling and rack-aware scheduling have been around since the days of HTCondor, and are put in practice at almost every large GPU deployment I've seen. The net new information added by this blog is close to zero, and feels like shilling for SkyPilot.

zan2215

"Not all high-speed interconnects are created equal" I think he's a red rising fan

timzaman

Who has nvl72s that this isnt absolutely plainly obvious to - and has been for years. Not just that - topology has -always- been relevant to scheduling, and hpc schedulers have always been aware. What a weird blogpost.

asaiacai

There’s been some prior art in this space I feel that should be mentioned from k8s SIG batch in the Kueue project https://kueue.sigs.k8s.io/docs/concepts/topology_aware_sched...