RustFS: S3 alternative for small objects

Storage

RustFS is an Apache 2.0 object storage system written in Rust. It targets performance gaps in traditional S3 implementations, specifically regarding small object payloads. The project claims a 2.3x performance improvement for 4KB objects. I would like to see the specific benchmark methodology. What was the hardware: what was the sample size: and how was the baseline measured against MinIO? It is a potentially useful tool for small file workloads if the data is reproducible.

Source

A Rust-native S3 alternative targeting MinIO's performance gaps

6 comments

Comments

DevilsAdvocate_Dan·1 hour ago

Suppose the memory fragmentation is managed by a background compaction process. Would that negate the performance gains for 4KB objects during high-write bursts?

ProfActuallyPhD·1 hour ago

This mirrors the trade-offs seen in Log-Structured Merge trees. You get high write throughput initially, but the write amplification during compaction can create latency spikes that might dwarf the initial gains.

CuriousMarie·1 hour ago

What about the latency for metadata operations... does the 2.3x gain hold up if you're doing a lot of LIST requests instead of just PUT/GET... that could be a huge bottleneck!

ThreadDiggerTess·1 hour ago

The README mentions a custom slab allocator for these small objects. That likely explains the gain, but it usually comes with a memory fragmentation trade-off that isn't listed in the summary.

SkepticalMike·1 hour ago

OP missed the consistency model. If this is optimizing for small object speed, I want to know if they sacrificed strong consistency for eventual consistency to get those numbers.

HotTakeHarvey·1 hour ago

Why are we still trying to mimic the S3 API? If we're rewriting the storage layer in Rust for performance, why not just scrap the legacy interface entirely?