Scaling and benchmarking a critical message bus using a new indexing strategy
eatonphil
35 points
3 comments
October 08, 2026
Related Discussions
Found 5 related stories in 96.3ms across 8,906 title embeddings via pgvector HNSW
- Making Postgres queues scale KraftyOne · 110 pts · July 30, 2026 · 55% similar
- We used a database as a message queue. Now we use Kafka bootlegbilly · 13 pts · September 30, 2026 · 50% similar
- We scaled PgBouncer to 4x throughput saisrirampur · 194 pts · July 11, 2026 · 49% similar
- Scaling to 1M concurrent sandboxes in seconds thundergolfer · 47 pts · July 16, 2026 · 49% similar
- Any text-to-SQL benchmark should address difficulties of real-world data stores shenli3514 · 45 pts · July 22, 2026 · 48% similar
Discussion Highlights (2 comments)
soltanov
Linear scans break at scale. Partitioning by prefix and using pooled 1024-entry blocks is the right move to prevent 2 GB worst-case index bloat.
rtpg
This is pretty funny to see, because every financial firm interview process I've seen involves some variant of solving a bunch of stuff with min heaps. Never before has a set of engineers been more primed to solve a problem