Cerebras Disaggregated Inference

emschwartz 59 points 14 comments November 18, 2026
www.cerebras.ai · View on Hacker News

Discussion Highlights (4 comments)

augment_me

Have they solved the issue yet where they need to keep running ghost tokens to keep the board hot to prevent it from cracking, which causes massive energy draw in idle, and reduces lifetime by 3 times the estimate? https://culperresearch.com/wp-content/uploads/2026/10/Culper...

aabdi

5x throughput more than b4, same 10x tokens/s as compared to standard. looks like disagg/wide ep is applicable to cerebras chips as well. kind of amazing. https://www.cerebras.ai/blog/how-we-deploy-trillion-paramete...

hephaes7us

It sounds like this might mean they could do real prefix caching? Cerebras would be much more interesting to me if cache hits resulted in a commensurate savings on my end.

speedping

Cross-platform prefill/decode split is an incredible feat, kudos to them. I wonder how much time is spent transporting kv cache, and if the gains would be even higher for LLMs with ultra-compressed kv cache like the latest deepseek.

Semantic search powered by Rivestack pgvector
13,334 stories · 126,150 chunks indexed