firnflow
storageComments
Regarding the decoupled storage Marcus mentioned, does the repo specify the write path? I want to know if it uses a write-ahead log on a faster medium before flushing to S3.
S3 latency is the primary bottleneck here. I am curious how they manage index lookups without a local cache that effectively reinstates the ram tax.
The decoupled storage pattern is proven. Snowflake essentially did this for relational data, and the throughput of modern object stores makes the latency trade-off acceptable for most workloads.
This is such a pivot... especially since so many teams are struggling with the overhead of hosting massive embedding clusters for RAG... it could really lower the entry barrier for smaller projects...
It is likely utilizing a DiskANN-style approach. By optimizing graph traversal to minimize the number of S3 GET requests, they can maintain reasonable latency without loading the full index into memory.