Show HN: We Beat MLPerf: Modern Storage for KV Offload and LLM Training

arnav__1

Hey HN, Today we are launching OpenLake, a KV offloading solution for modern LLM workloads. OpenLake achieved 1M+ iops in 1ms on a 96 node cluster. We achieved this through rust compio (thread per core design) with on GPU compression. In MLPerf Storage v3.0, OpenLake ranked #1 for read and write bandwidth among the five comparable S3 submissions 8B checkpoint. It achieved 6.72 GiB/s writes and 11.55 GiB/s reads. Would love to get your feedback and hear what you think. Thanks a lot! GitHub: https://github.com/openlake-project/openlake