OpenData Vector: MIT-Licensed Vector Search on Object Storage

oliverio

Very interesting, thanks for sharing. This has a lot of nods to Turbopuffer's architecture [0]. My impression is they've spent a lot of time optimizing at the hardware/firmware layer to achieve extremely fast query results. Inarticulately - how ~close is OpenData Vector to Turbopuffer in terms of performance today and where are the major gaps + mountains to scale? Really excited to keep an eye on the repos, great read! [0] https://turbopuffer.com/blog/turbopuffer

Reubend

Stupid question: I was under the impression that object storage was super expensive compared to "normal" SSDs if the QPS numbers got high. Is that not the case for DBs based on object storage because they cache data before sending it to the object storage? Or because they do some other processing on the DB server before it hits storage?