Show HN: LavinMQ, an open-source message broker written in Crystal
Hi HN. I'm Carl, creator of LavinMQ. LavinMQ is an open-source message broker. AMQP 0-9-1, MQTT, HTTP and streaming. Single binary, minimal resource use. If you know RabbitMQ, it's that (but running on a fraction of hardware). We're the team behind CloudAMQP where we've hosted RabbitMQ for 14 years. Sometimes customers hit issues we couldn't fully explain / work around. So we built our own broker, in order to provide solutions where we previously couldn't (which is much easier when you control the full stack). It all started by me building an open-source AMQP proxy to handle short-lived connections for a customer, to avoid heavy connection churn on cpu. Once the protocol was implemented, adding a persistence layer seemed like a fun challenge. How hard could it be? Turns out the protocol was the easy part. Reliable disk storage, ack handling, and replication was harder. It took years. Our first release was pushed in 2020 and today it runs on 5,000+ production instances on CloudAMQP. LavinMQ is written in Crystal (LLVM-compiled, Go-like concurrency, Ruby-like syntax). Messages go straight to disk via memory-mapped files, no in-memory cache. Append-only writes for both messages and acks. The whole design is built around minimizing memory copies, allocations and syscalls. Numbers: ~580k msgs/sec on a t4g.micro (2 vCPU, 1 GB RAM). Over 1M msgs/sec on a c8g.large (2 vCPU, 4 GB). Try it: docker run -p 15672:15672 -p 5672:5672 cloudamqp/lavinmq We are curious to hear: how are you handling messaging today? What would make you consider switching broker?
Discussion Highlights (2 comments)
jhk482001
When a leader fails, does a publisher confirm guarantee the message survived on at least one follower?
sdogruyol
Impressive benchmarks. Reaching 1M msgs/sec on a 2 vCPU instance is a great showcase for Crystal. As a Crystal core member, I've always seen LavinMQ as a prime example of performance-oriented engineering, especially with how it handles I/O and minimizes syscalls to get the most out of the hardware. If you want to see Crystal's concurrency and type system in a serious production environment, this is the project to check out. Kudos to the LavinMQ team for their work since 2020.