GPU Memory Math for LLMs: Formula That Tells You What Fits on Your GPU
XMasterrrr
12 points
2 comments
May 20, 2026
Related Discussions
Found 5 related stories in 758.3ms across 14,015 title embeddings via pgvector HNSW
- Right-sizes LLM models to your system's RAM, CPU, and GPU bilsbie · 76 pts · March 01, 2026 · 64% similar
- Mapping with In-Memory Layers to Reduce LLM Overload Buckwheat469 · 14 pts · July 04, 2026 · 54% similar
- Real-time LLM Inference on Standard GPUs: 3k tokens/s per request NicoConstant · 202 pts · May 29, 2026 · 52% similar
- Unified Memory, Explained: Why Mini PCs Can Run 70B Models a Big GPU Can't ermantrout · 71 pts · July 10, 2026 · 52% similar
- 768GB Intel Optane DIMMs to run 1T-parameter LLM with single GPU at 4tps walterbell · 26 pts · May 30, 2026 · 52% similar
Discussion Highlights (2 comments)
metadat
This is super useful. Most of the time I go to run a model off Hugging Face on my 64GB MBP I run into issues where I drastically overestimated what it could do. :>
DiabloD3
This isn't very useful. V of context is not equal across models. Also, huggingface tells you how big the model is for the exact one you have in your hand, why the weird guesswork? Dynamic quants are not going to magically fit some formula.