4 ms·

Without specifying a particular representation[0] on the struct you want to put in "permanent memory", this seems liable to break any time you recompile the cod

Without specifying a particular representation[0] on the struct you want to put in "permanent memory", this seems liable to break any time you recompile the code with different optimization flags, or with a newer or older compiler that may optimize differently, or even if you just change seemingly unrelated code that then triggers or de-triggers some optimization. [0] https://doc.rust-lang.org/reference/type-layout.html#representations https://doc.rust-lang.org/reference/type-layout.html#represe...

WirelessGigabit 3y ago
Yup, at a minimum they should apply #[repr(C)] to avoid this optimization.
o11c 3y ago
Worse, it breaks all pointers since you can't guarantee `mmap` can give you the same address.
- astrange 3y ago
  I think you could fixup the pointers if you tried. It's more or less the same problem as compacting GC.
  - o11c 3y ago
    Certainly there are solutions, but they aren't transparent, and at some point you have to ask "would serialization be easier?"
    - GTP 3y ago
      This is exactly what I thought: why not just save state to a file?
- junon 3y ago
  You can detect when it doesn't, though, I think. If memory serves you can request a specific memory address to mount it at and it'll do it if it's not already mapped. It should (in theory?) return an error in such a case, no?
  - deciduously 3y ago
    I believe on conflict it will just fall back to placing it where it can by default. It will return the address it actually used, so you can catch the mismatch.
- kouteiheika 3y ago
  You absolutely can, with `MAP_FIXED`. But it's not trivial to do because you'd have to handle the case where the rtld and/or the kernel has already put something in memory where you want to put down your map. So it'd be tricky to implement, but definitely possible.
  - mike_hearn 3y ago
    That can be fixed with a linker script, to reserve the range using ELF headers. But then you're going to hit other problems. This isn't a new idea. Presumably one of the reasons you're restarting your server is to upgrade it ...
conradev 3y ago
The rkyv crate has stable in-memory representations for many types in the Rust standard library: https://rkyv.org/ https://rkyv.org/ I feel like it is the missing piece here because you could do hash table lookups in shared memory, too!
graham_king_3 3y ago
Good point, thank you! I added `repr(C)`.

WirelessGigabit 3y ago

Yup, at a minimum they should apply #[repr(C)] to avoid this optimization.

o11c 3y ago

Worse, it breaks all pointers since you can't guarantee `mmap` can give you the same address.

astrange 3y ago
I think you could fixup the pointers if you tried. It's more or less the same problem as compacting GC.
- o11c 3y ago
  Certainly there are solutions, but they aren't transparent, and at some point you have to ask "would serialization be easier?"
  - GTP 3y ago
    This is exactly what I thought: why not just save state to a file?
junon 3y ago
You can detect when it doesn't, though, I think. If memory serves you can request a specific memory address to mount it at and it'll do it if it's not already mapped. It should (in theory?) return an error in such a case, no?
- deciduously 3y ago
  I believe on conflict it will just fall back to placing it where it can by default. It will return the address it actually used, so you can catch the mismatch.
kouteiheika 3y ago
You absolutely can, with `MAP_FIXED`. But it's not trivial to do because you'd have to handle the case where the rtld and/or the kernel has already put something in memory where you want to put down your map. So it'd be tricky to implement, but definitely possible.
- mike_hearn 3y ago
  That can be fixed with a linker script, to reserve the range using ELF headers. But then you're going to hit other problems. This isn't a new idea. Presumably one of the reasons you're restarting your server is to upgrade it ...

conradev 3y ago

The rkyv crate has stable in-memory representations for many types in the Rust standard library: https://rkyv.org/ https://rkyv.org/ I feel like it is the missing piece here because you could do hash table lookups in shared memory, too!

graham_king_3 3y ago

Good point, thank you! I added `repr(C)`.