4 ms·

Yeah this should be benchmarked against other systems that have flush() disabled. mmap is nice but it doesn’t support durable semantics in the way that we usua

Yeah this should be benchmarked against other systems that have flush() disabled. mmap is nice but it doesn’t support durable semantics in the way that we usually mean with databases. if a write is acknowledged it should not be forgotten, which is not what this is.

rgbimbochamp 19d ago
You're right, that mode provides process crash recovery, not power-loss durability. The benchmark compares it against fjall’s equivalent buffered-WAL mode.
- a2ff6eeb0 19d ago
  If that's your design constraint, couldn't you speed it up by getting rid of the WAL?
  - oneshadab 19d ago
    You'd lose durability against process crashes. If your system has a reasonable tolerance for power failure (multi-az multi-cloud), this can provide much better throughput
    - t098i3 19d ago
      Indeed, a common enough pattern for etcd is to run it backed by a RAMdisk and have multi-az availability + periodic backups + tolerance at a business level to be OK losing some recent data.
- deathanatos 19d ago
  Word choice matters. Defaults matter. People will go "well it says durable right here" and while arguably, yes, they should RTFM, it would still be great if tool-builders did not set the shotgun's default state to State::AT_FOOT. It would be nice if every paragraph of technical writing that I have to do need not be burdened by a thousand asterisks of "durable in this context means something other than durable".
  - carlmr 19d ago
    >arguably, yes, they should RTFM Agreed. Good design is when the things do what you expect them to do without reading the manual, don't reuse wording with other meaning in the wrong way. That way if you do encounter nee wording, you know you should read the manual.
    - gryfft 18d ago
      https://en.wikipedia.org/wiki/Principle_of_least_astonishment https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...
muvlon 18d ago
As of a couple years ago, mmap actually has a MAP_SYNC flag that makes it durable in the DB sense. The caveat is that it requires DAX on the file and so comes with a whole bunch of restrictions w.r.t. filesystem, storage media and even CPU architecture.

rgbimbochamp 19d ago

You're right, that mode provides process crash recovery, not power-loss durability. The benchmark compares it against fjall’s equivalent buffered-WAL mode.

a2ff6eeb0 19d ago
If that's your design constraint, couldn't you speed it up by getting rid of the WAL?
- oneshadab 19d ago
  You'd lose durability against process crashes. If your system has a reasonable tolerance for power failure (multi-az multi-cloud), this can provide much better throughput
  - t098i3 19d ago
    Indeed, a common enough pattern for etcd is to run it backed by a RAMdisk and have multi-az availability + periodic backups + tolerance at a business level to be OK losing some recent data.
deathanatos 19d ago
Word choice matters. Defaults matter. People will go "well it says durable right here" and while arguably, yes, they should RTFM, it would still be great if tool-builders did not set the shotgun's default state to State::AT_FOOT. It would be nice if every paragraph of technical writing that I have to do need not be burdened by a thousand asterisks of "durable in this context means something other than durable".
- carlmr 19d ago
  >arguably, yes, they should RTFM Agreed. Good design is when the things do what you expect them to do without reading the manual, don't reuse wording with other meaning in the wrong way. That way if you do encounter nee wording, you know you should read the manual.
  - gryfft 18d ago
    https://en.wikipedia.org/wiki/Principle_of_least_astonishment https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...

muvlon 18d ago

As of a couple years ago, mmap actually has a MAP_SYNC flag that makes it durable in the DB sense. The caveat is that it requires DAX on the file and so comes with a whole bunch of restrictions w.r.t. filesystem, storage media and even CPU architecture.