5 ms·
Kafka is backed by spinning disks though. Am I missing something?
by dleibovic 10y ago
Kafka is backed by spinning disks though. Am I missing something?
- yellowapple 10y agoI'm not familiar with Kafka specifically, but I reckon caching in memory is the way to work around that.
- AnimalMuppet 10y agoI think it's assuming that Kafka caches a lot more than your local machine, and therefore is less likely to need to access disk. Note that it also assumes low network latency, though; if you have congestion within the data center, things change.
- hadsed 10y agoGenerally Kafka takes advantage of the OS page cache and the fact that its data is sequential. The reason you can back Kafka effectively with spinning disks is because Kafka is just an immutable log. OTOH you won't always have sequential access so services that store data in memory cover some of those other cases (eg a KV store like Redis).