The same H100 costs $2.69 or $12.29 an hour depending where you rent it

jdc0589

runpod is incredible, im a big fan, but good luck with availability on any specific GPU above 20gb of vram when you actually want to use it. last time I was trying to run some workloads there the pricing of an h100 was largely irrelevant.

dpflan

This is premise for https://skypilot.ai/ , right? Cloud provider arbitrage.

plantain

You are completely mad if you are running any kind of valuable workload on Runpod community or Vast. Hosts are absolutely peeking inside the VM's (you can see it openly discussed in their Discords).

zizzigy

But a ps5.48xlarge has 8x H100s. Are the other options here comparable?

qoez

I wish there was some kind of GPU marketplace so one could just arb this opportunity.

uneekname

> 4.57× between cheapest and dearest, for the same chip > What the 4.57× is not > - InfiniBand > - A datacentre SLA > - Enterprise compliance, committed capacity... ok

fg137

I guess the author hasn't discovered vast.ai which is a minefield.

nico

Yes. And the prices a lot of times don’t even make sense I’ve run into many hosted GPUs with processes stuck, with outdated drivers/libraries, or poor Internet connections

buredoranna

There are a couple of new indices out there. [1] "Silicon Index". Built against pricing observations, including listed prices and transaction data. [2] "Ornn Compute Price Index (OCPI)". Claiming to use only executed transactions. Your mileage may vary, but I find these new indices to be quite compelling. [1] https://www.silicondata.com/products/silicon-index [2] https://data.ornn.com/ And yup, its not just how much these resources cost, but how much people are willing to pay for them. (edit: formatting)

williamgators

Seems once the government controls it the prices go haywire

tokepi

There’s much more data here, with more providers and splits by the different H100 types: https://cloud-gpus.com/nvidia-h100/

happyPersonR

lol watch the $12 people just subcontract out to the $2 and hope no one notices lolll

matt3210

These prices don't seem real. For me, runpod is 8.99 an hour for b300 right now.

pwython

I've only used Modal.com -- They give you $30 month of usage free. Otherwise, $3.95/hr for an H100 SXM5.

adeptima

Speaking of Runpod.io GPU availability, it is definitely increased in the past 6 months, getting 80Gb+ vram on demand takes few mins (not 5-6 hours like it used to be) at 1-3 usd an hour rate ... which made me thought a week ago - finally GPU bubble peak arrived Another thought - i want to audit the actual usage of GPU like actual efficient inference time not billed one without relying on "trust me bro" billing. Especially on AWS. Had no trust issue with Runpod.io Any effective inference audit ideas are appreciated

dyauspitr

I mean after the initial capex you’re just paying for electricity.

covi

The spread is indeed very big. Lowest $1.90/hr Average $7.08/hr Median $7.91/hr https://gpus.skypilot.co/?gpu=H100&count=8

gigatexal

if they're all just an API away is anyone buying for 2.69 somewhere and then selling at 12.28 elsewhere arbitraging things?

Havoc

Nice that they clearly explained why it is different

cvhc

I tried the TensorDock, which now offers H100 "from $2.25/hr", two years ago. The SSD IO was terrible, maybe oversold. My training workers was stuck at loading the dataset and cannot fully utilize the GPU.