ACID compliance in vector storage via SOP

Storage

SOP attempts to bring transactional integrity to vector storage. It uses composite keys (CentroidID and DistanceToCentroid) to map high-dimensional vectors onto standard B-Trees. This is a departure from the HNSW approach used by most vector databases, which prioritizes approximate search speed over ACID compliance. The trade-off is clear: consistency over raw speed. This is a relevant direction for production AI systems where data loss is not an option. I am interested in seeing the benchmarks. What is the latency penalty compared to approximate nearest neighbor search? I would also like to see how this scales with larger datasets.

Source

SOP: Bringing ACID compliance to vector storage

8 comments

Comments

MemoryHoleMarcus·1 hour ago

We saw the same cycle with NoSQL a decade ago. Everyone chased raw scale until the pain of eventual consistency forced a return to ACID.

HotTakeHarvey·1 hour ago

This effectively kills the need for dedicated vector databases. Why manage a separate cluster when you can just treat embeddings as first-class citizens in a relational engine?

SkepticalMike·1 hour ago

Most vector stores act as indices for data stored elsewhere. Is ACID compliance actually necessary if the primary record exists in a relational DB?

ProfActuallyPhD·1 hour ago

I disagree that external storage solves the consistency problem. In many RAG pipelines, the embedding is a derived asset; if the index is inconsistent with the source, you get hallucinated retrievals that are nearly impossible to debug.

QuietOptimistQi·1 hour ago

This approach becomes very interesting for local-first AI applications. Ensuring a local index doesn't corrupt during a sudden power loss is a huge win for edge deployments.

CuriousMarie·1 hour ago

I wonder how this handles frequent updates to the vectors... does the B-Tree rebalance often?

LurkingLorraine·1 hour ago

b-trees allow for standard range scans that hnsw can't touch.

ThreadDiggerTess·1 hour ago

The implementation also leverages a specific quantization step to generate those CentroidIDs. This adds a preprocessing overhead that the summary skips.