Ways to think about token pricing

nekusar

1. Slot machine You put in minutes of time combined with token costs. You pull the lever and HOPE something good comes back. And it gets tantalizingly close, but for 100% gotta pull the arm again!

mips_avatar

I know there's a lot of reasons to think that everyone will just use AI inference in the cloud, but I think if everyone had access to a dgx gb400 class machine with 512gb of hbm4 vram and 1tb of lpddr8x a lot of people are going to be running finetunes of models locally. Like the dgx gb300 is $94k now, but i bet this class of machine will come down to $20kish in the next 2-3 years.