10 ms·

reinterpret_cast comes up a lot when handling binary protocols - e.g. reading a message struct off a buffer / packet. How would this be handled in Rust or other

reinterpret_cast comes up a lot when handling binary protocols - e.g. reading a message struct off a buffer / packet. How would this be handled in Rust or otherwise without copying the data?

EdSchouten 5y ago
You can always just create a helper type that wraps the byte range and provides getter/setter functions for the individual fields. There is no need to cast the thing to an actual struct.
- Denvercoder9 5y ago
  That's true, but a lot less convenient and I bet you'll be missing out on a bunch of optimizations if you do that.
  - tialaramex 5y ago
    In the medium term I'd say it's more likely you miss out on optimizations by not clearly expressing what you actually wanted here. The more clarity is available as to what you meant, the more aggressive the optimiser can be in delivering precisely that. If you don't express exactly what is intended, the optimisers end up getting muzzled because the effect is so confusing.
    - gpderetta 5y ago
      What specific optimizations would a wrapper type allow that a accessing directly a pointer to a reinterpret casted, property aligned StandardLayout type would not allow?
      - tialaramex 5y ago
        The concern I have isn't about optimisations that are impossible but optimisations which are judged unwise and so either disabled by default or not implemented at all. Maybe reinterpret_cast on StandardLayout with the correct alignment really does express exactly what we want here, I'm not sure. If so you're correct there wouldn't be a difference because a valid optimisation will always be exactly what everybody wanted. But if there are any other uses of these techniques the optimiser has to be pessimistic. That's why I prefer to be as explicit as possible. This is how so many extra types of cast came into existence in C++ in the first place right, the recognition that casting could have different intents.
        gpderetta 5y ago
        > StandardLayout with the correct alignment really does express exactly what we want here, I'm not sure Surely if you are unsure the default position should be to go to the obvious and simple solution instead of overengineering it? But I might be completely misunderstanding your point. [If the struct is not correctly aligned, I would either memcpy or apply any required compiler pragmas, while cursing the protocol designer]
  - deleted 5y ago
    [deleted]
roblabla 5y ago
In current Rust, you'd do it with an unsafe std::mem::transmute - possibly using a crate that wraps this call behind a safe interface that asserts that alignment and size are correct, and that your type doesn't have any padding, etc... See for instance the bytemuck crate. In the future, rust might gain "Safe Transmutes", which would allow you to safely transmute from one type to another under some limited conditions. See https://github.com/rust-lang/lang-team/issues/21 https://github.com/rust-lang/lang-team/issues/21
pornel 5y ago
The exact same approach as C is doing is possible in Rust. There are pointer casts and mem::transmute(). It does require unsafe, since you must be aware of gotchas with alignment, padding, and uninitialized memory. If you want to be fancy, there's https://lib.rs/rkyv https://lib.rs/rkyv deserialization framework that will auto-generate appropriate zero-copy code for you. There's also serde (de)serialization framework and bincode format, which aren't exactly zero-copy, but are fast 1-copy.
webkike 5y ago
I’ve fixed undefined behavior in large C++ code based caused by misaligned reinterpret casts - the solution is to use memcpy
- sgerenser 5y ago
  And in most cases (at least in optimized builds) the memcpy will be optimized out.
- allo37 5y ago
  It's such a good idea that they made a special cast that does just that in C++20: https://en.cppreference.com/w/cpp/numeric/bit_cast https://en.cppreference.com/w/cpp/numeric/bit_cast

reinterpret_cast comes up a lot when handling binary protocols - e.g. reading a message struct off a buffer / packet. How would this be handled in Rust or otherwise without copying the data?

EdSchouten 5y ago

You can always just create a helper type that wraps the byte range and provides getter/setter functions for the individual fields. There is no need to cast the thing to an actual struct.

Denvercoder9 5y ago
That's true, but a lot less convenient and I bet you'll be missing out on a bunch of optimizations if you do that.
- tialaramex 5y ago
  In the medium term I'd say it's more likely you miss out on optimizations by not clearly expressing what you actually wanted here. The more clarity is available as to what you meant, the more aggressive the optimiser can be in delivering precisely that. If you don't express exactly what is intended, the optimisers end up getting muzzled because the effect is so confusing.
  - gpderetta 5y ago
    What specific optimizations would a wrapper type allow that a accessing directly a pointer to a reinterpret casted, property aligned StandardLayout type would not allow?
    - tialaramex 5y ago
      The concern I have isn't about optimisations that are impossible but optimisations which are judged unwise and so either disabled by default or not implemented at all. Maybe reinterpret_cast on StandardLayout with the correct alignment really does express exactly what we want here, I'm not sure. If so you're correct there wouldn't be a difference because a valid optimisation will always be exactly what everybody wanted. But if there are any other uses of these techniques the optimiser has to be pessimistic. That's why I prefer to be as explicit as possible. This is how so many extra types of cast came into existence in C++ in the first place right, the recognition that casting could have different intents.
      - gpderetta 5y ago
        > StandardLayout with the correct alignment really does express exactly what we want here, I'm not sure Surely if you are unsure the default position should be to go to the obvious and simple solution instead of overengineering it? But I might be completely misunderstanding your point. [If the struct is not correctly aligned, I would either memcpy or apply any required compiler pragmas, while cursing the protocol designer]
- deleted 5y ago
  [deleted]

roblabla 5y ago

In current Rust, you'd do it with an unsafe std::mem::transmute - possibly using a crate that wraps this call behind a safe interface that asserts that alignment and size are correct, and that your type doesn't have any padding, etc... See for instance the bytemuck crate. In the future, rust might gain "Safe Transmutes", which would allow you to safely transmute from one type to another under some limited conditions. See https://github.com/rust-lang/lang-team/issues/21 https://github.com/rust-lang/lang-team/issues/21

pornel 5y ago

The exact same approach as C is doing is possible in Rust. There are pointer casts and mem::transmute(). It does require unsafe, since you must be aware of gotchas with alignment, padding, and uninitialized memory. If you want to be fancy, there's https://lib.rs/rkyv https://lib.rs/rkyv deserialization framework that will auto-generate appropriate zero-copy code for you. There's also serde (de)serialization framework and bincode format, which aren't exactly zero-copy, but are fast 1-copy.

webkike 5y ago

I’ve fixed undefined behavior in large C++ code based caused by misaligned reinterpret casts - the solution is to use memcpy

sgerenser 5y ago
And in most cases (at least in optimized builds) the memcpy will be optimized out.
allo37 5y ago
It's such a good idea that they made a special cast that does just that in C++20: https://en.cppreference.com/w/cpp/numeric/bit_cast https://en.cppreference.com/w/cpp/numeric/bit_cast