6 ms·

The more pertinent question to me is can we implement some new static analysis that understands buffer re-use and can hoist buffer initialization outside the lo

The more pertinent question to me is can we implement some new static analysis that understands buffer re-use and can hoist buffer initialization outside the loop? Rather than make the programmer write obfuscated code for efficiency, it is usually better to have the compiler do the heavy lifting. P.S. Also, folks, don't re-use buffers without zeroing unless you absolutely need the performance and know what you're doing.

vlovich123 2y ago
I like that direction better but it requires the ability to declare data-flow based contracts whereas Rust’s tools are only lifetime and type contracts. Is there a language that has data-flow based contracts?
- queuebert 2y ago
  That would be easier but is not required. There are no compiler hints these days to unroll loops or hoist invariants, even though if done incorrectly it could change the result. It would take some complicated analysis, but I think it could be done safely in some cases.
  - gpm 2y ago
    I was going to make this argument, but I actually don't think it's true in almost any case. Most functions could be inferred, but the ultimate source of basically all of these write only APIs is FFI functions, which in turn call systemcalls. You're at least going to need a way to annotate the FFI calls and systemcalls to describe to the compiler how they access data.
    - queuebert 2y ago
      If you're calling FFIs in an inner loop, you have bigger issues than the time it takes to clear the buffer, right?
      - gpm 2y ago
        No? It depends on your definition of inner loop I guess. If you're doing some sort of zero-copy IO, the time to clear the buffer might be non-trivial (not huge, but non-trivial). It's true that you need a large enough buffer that syscall/ffi overhead doesn't dominate, but that's not unrealistic. It's rare that we care about this, that's true, that's why generally rust has been fine with "just zero buffers". There are definitely domains that care though.
      - thayne 2y ago
        In some languages, like Java, go and probably Javascript, this is probably true, depending on how much memory needs to to be initialized. But in rust FFI isn't any more expensive than any other non-inlined function call.
  - vlovich123 2y ago
    The loop unrolling & invariant hoisting is a static transformation. What the “read” function does semantically isn’t captured today within that and the compiler wouldn’t be able to automatically infer it. It would have to be told that information and there would need to be unsafe annotations for things like syscalls and FFI boundaries. The other approach is to change the API which is what BorrowedBuf is. If you can think of a different approach of how the compiler can figure out automatically what memory has become initialized by a random function call I’m all ears.
    - queuebert 2y ago
      That's what I glossed over as "complicated analysis". In my mind, if a compiler can understand register and stack use (required for static transformations), it can (theoretically, and with some effort) understand heap use. Am I wrong?
      - vlovich123 2y ago
        Yes, you are wrong. This isn’t basic constant hoisting. The compiler doesn’t reasonably have any of that information to understand what read is filling in at runtime because that information is encoded purely at runtime and the compiler has no reasoning mechanism even close to answering runtime data flow questions. There’s also all sorts of complexity that has to do with the kinds of transformations that are possible as the legal information that exists at the language level is often erased before it gets to the stack/register piece and vice versa the language layer knows nothing about registers and minimal stuff about stack. This is the same reason that the compiler fails to compile something like: for _ in 1..10 { let x: String = create_new_string(); eprintln(“{x}”); } Fails to hoist x out of the loop even if the returned string is String::new(“ABC”) unless maybe LTO is on (and even then maybe not). Basically the compiler’s “magic” is very limited to static transformations that follow as-if - the compiler must know the static transformation is blindly identical and the amount of reasoning about the structure is often very limited. Said another way, if the compiler could do the optimizations you’re hypothesizing, it would be equivalent to applying a mid level performance engineer to every code base it encounters.
      - 2y ago
- ijustlovemath 2y ago
  what do you mean by this?
  - vlovich123 2y ago
    There would need to be contractual declarations on the read method that the compiler is able to enforce that tells it that the input &mut slice has N elements clobbered based on the returned length. That’s basically what BorrowedBuf is accomplishing via the type system and runtime enforcement of the contract. Using a non-existent syntax: fn read<T, N: size_t>(&mut self, buf: &mut [MaybeUninit<T>] becomes &[T; N] after call) -> N { … enforces the body initializes N elements out of buf } and then rules that &mut [T] can also be supplied to such functions that today could only accept a &mut [MaybeUninit<T>] transparently. A more likely interface you could write today would look like: fn read_uninit<T>(&mut self, buf: &mut [MaybeUninit<T>]) -> (&[T], &[MaybeUninit<T>]) { … enforces the body initializes N elements out of buf } You still have to cast &[T] into &[MaybeUninit<T>] somehow.
    - Someone 2y ago
      I think an ergonomic way to do that would to have read return not an integer, but a slice of that integer’s length. Problem would be: how do you express “you can only access the buffer you sent me through the read-only slice I returned, but you have to free that same buffer when you’re done calling me? I think that can be done using a function creating a read buffer for a given input stream that - during calls to read is ‘owned for writing’ by that stream (so, it has to borrow a capability that the creator of the buffer doesn’t have. I don’t think Rust currently supports that) - where stream.read returns a read only slice whose lifetime is bound to that of the buffer So, the creator of the buffer can only pass it to read to get a slice back that contains precisely the data read. The stream can write to the entire buffer.
    - gpm 2y ago
      > You still have to cast &[T] into &[MaybeUninit<T>] somehow. unsafe{ std::mem::transmute(slice) } This is probably the only way that will ever exist, because let slice: &mut [NonZeroU8] = ...; let slice_uninit: &mut [MaybeUninit<NonZeroU8>] = ...; let nonzero_uninit: &mut MaybeUninit<NonZeroU8> = &mut slice_uninit[0]; *nonzero_uninit = MaybeUninit::zeroed(); slice[0]; // Undefined behavior for sure by now. Is all safe except for the cast. I.e. MaybeUninit<T> allows you to write invalid bit-patterns to T, so you can't safely cast a reference to T to it (and if you do unsafely cast a reference to T to it you can't soundly write an invalid bit pattern). All current forms of safely making a MaybeUninit take ownership of the value they are declaring to be MaybeUninit for this reason. I guess at some point we might get methods for this on types that can take on all bit patterns - if/when that's encoded as a trait.
mrpf1ster 2y ago
Why would re-using a buffer be bad? Assuming you write to it with the contents of the file/stream before it is read.
- kohbo 2y ago
  You just answered your own question
  - rendaw 2y ago
    Why is it particularly more dangerous or likely than other logic errors?
    - benschulz 2y ago
      Because the compiler optimizes based on the assumption that consecutive reads yield the same value. Reading from uninitialized memory may violate that assumption and lead to undefined behavior. (This isn't the theoretical ivory tower kind of UB. Operating systems regularly remap a page that hasn't yet been written to.)
      - kazinator 2y ago
        If you read something where you have not written, who cares whether the compiler optimizes things such that if you read from there again, you get the same value, even though that is not true?
        lmm 2y ago
        Anyone who wants to be able to sanely debug. Code is imperfect, mistakes happen. If the compiler can optimise so that any mistake anywhere in your program could mean insane behaviour anywhere else in your program, then you get, well, C. (E.g. imagine doing a write to an array at offset x - this is safe in Rust, so the compiler turns that into code that checks that x is within the bounds of that array, then writes at that offset. If the value of x can change, then now this code can overwrite some other variable anywhere in your program, giving you a bug that's very hard to track down)
jvanderbot 2y ago
Fair, but note there is a significant subset of Rust-targeted programmers who dislike the compiler doing things like that. They also dislike the compiler doing things like auto-initializing every loop iteration, but two wrongs wouldn't make it right, just less wrong.

vlovich123 2y ago

I like that direction better but it requires the ability to declare data-flow based contracts whereas Rust’s tools are only lifetime and type contracts. Is there a language that has data-flow based contracts?

queuebert 2y ago
That would be easier but is not required. There are no compiler hints these days to unroll loops or hoist invariants, even though if done incorrectly it could change the result. It would take some complicated analysis, but I think it could be done safely in some cases.
- gpm 2y ago
  I was going to make this argument, but I actually don't think it's true in almost any case. Most functions could be inferred, but the ultimate source of basically all of these write only APIs is FFI functions, which in turn call systemcalls. You're at least going to need a way to annotate the FFI calls and systemcalls to describe to the compiler how they access data.
  - queuebert 2y ago
    If you're calling FFIs in an inner loop, you have bigger issues than the time it takes to clear the buffer, right?
    - gpm 2y ago
      No? It depends on your definition of inner loop I guess. If you're doing some sort of zero-copy IO, the time to clear the buffer might be non-trivial (not huge, but non-trivial). It's true that you need a large enough buffer that syscall/ffi overhead doesn't dominate, but that's not unrealistic. It's rare that we care about this, that's true, that's why generally rust has been fine with "just zero buffers". There are definitely domains that care though.
    - thayne 2y ago
      In some languages, like Java, go and probably Javascript, this is probably true, depending on how much memory needs to to be initialized. But in rust FFI isn't any more expensive than any other non-inlined function call.
- vlovich123 2y ago
  The loop unrolling & invariant hoisting is a static transformation. What the “read” function does semantically isn’t captured today within that and the compiler wouldn’t be able to automatically infer it. It would have to be told that information and there would need to be unsafe annotations for things like syscalls and FFI boundaries. The other approach is to change the API which is what BorrowedBuf is. If you can think of a different approach of how the compiler can figure out automatically what memory has become initialized by a random function call I’m all ears.
  - queuebert 2y ago
    That's what I glossed over as "complicated analysis". In my mind, if a compiler can understand register and stack use (required for static transformations), it can (theoretically, and with some effort) understand heap use. Am I wrong?
    - vlovich123 2y ago
      Yes, you are wrong. This isn’t basic constant hoisting. The compiler doesn’t reasonably have any of that information to understand what read is filling in at runtime because that information is encoded purely at runtime and the compiler has no reasoning mechanism even close to answering runtime data flow questions. There’s also all sorts of complexity that has to do with the kinds of transformations that are possible as the legal information that exists at the language level is often erased before it gets to the stack/register piece and vice versa the language layer knows nothing about registers and minimal stuff about stack. This is the same reason that the compiler fails to compile something like: for _ in 1..10 { let x: String = create_new_string(); eprintln(“{x}”); } Fails to hoist x out of the loop even if the returned string is String::new(“ABC”) unless maybe LTO is on (and even then maybe not). Basically the compiler’s “magic” is very limited to static transformations that follow as-if - the compiler must know the static transformation is blindly identical and the amount of reasoning about the structure is often very limited. Said another way, if the compiler could do the optimizations you’re hypothesizing, it would be equivalent to applying a mid level performance engineer to every code base it encounters.
    - 2y ago
ijustlovemath 2y ago
what do you mean by this?
- vlovich123 2y ago
  There would need to be contractual declarations on the read method that the compiler is able to enforce that tells it that the input &mut slice has N elements clobbered based on the returned length. That’s basically what BorrowedBuf is accomplishing via the type system and runtime enforcement of the contract. Using a non-existent syntax: fn read<T, N: size_t>(&mut self, buf: &mut [MaybeUninit<T>] becomes &[T; N] after call) -> N { … enforces the body initializes N elements out of buf } and then rules that &mut [T] can also be supplied to such functions that today could only accept a &mut [MaybeUninit<T>] transparently. A more likely interface you could write today would look like: fn read_uninit<T>(&mut self, buf: &mut [MaybeUninit<T>]) -> (&[T], &[MaybeUninit<T>]) { … enforces the body initializes N elements out of buf } You still have to cast &[T] into &[MaybeUninit<T>] somehow.
  - Someone 2y ago
    I think an ergonomic way to do that would to have read return not an integer, but a slice of that integer’s length. Problem would be: how do you express “you can only access the buffer you sent me through the read-only slice I returned, but you have to free that same buffer when you’re done calling me? I think that can be done using a function creating a read buffer for a given input stream that - during calls to read is ‘owned for writing’ by that stream (so, it has to borrow a capability that the creator of the buffer doesn’t have. I don’t think Rust currently supports that) - where stream.read returns a read only slice whose lifetime is bound to that of the buffer So, the creator of the buffer can only pass it to read to get a slice back that contains precisely the data read. The stream can write to the entire buffer.
  - gpm 2y ago
    > You still have to cast &[T] into &[MaybeUninit<T>] somehow. unsafe{ std::mem::transmute(slice) } This is probably the only way that will ever exist, because let slice: &mut [NonZeroU8] = ...; let slice_uninit: &mut [MaybeUninit<NonZeroU8>] = ...; let nonzero_uninit: &mut MaybeUninit<NonZeroU8> = &mut slice_uninit[0]; *nonzero_uninit = MaybeUninit::zeroed(); slice[0]; // Undefined behavior for sure by now. Is all safe except for the cast. I.e. MaybeUninit<T> allows you to write invalid bit-patterns to T, so you can't safely cast a reference to T to it (and if you do unsafely cast a reference to T to it you can't soundly write an invalid bit pattern). All current forms of safely making a MaybeUninit take ownership of the value they are declaring to be MaybeUninit for this reason. I guess at some point we might get methods for this on types that can take on all bit patterns - if/when that's encoded as a trait.

mrpf1ster 2y ago

Why would re-using a buffer be bad? Assuming you write to it with the contents of the file/stream before it is read.

kohbo 2y ago
You just answered your own question
- rendaw 2y ago
  Why is it particularly more dangerous or likely than other logic errors?
  - benschulz 2y ago
    Because the compiler optimizes based on the assumption that consecutive reads yield the same value. Reading from uninitialized memory may violate that assumption and lead to undefined behavior. (This isn't the theoretical ivory tower kind of UB. Operating systems regularly remap a page that hasn't yet been written to.)
    - kazinator 2y ago
      If you read something where you have not written, who cares whether the compiler optimizes things such that if you read from there again, you get the same value, even though that is not true?
      - lmm 2y ago
        Anyone who wants to be able to sanely debug. Code is imperfect, mistakes happen. If the compiler can optimise so that any mistake anywhere in your program could mean insane behaviour anywhere else in your program, then you get, well, C. (E.g. imagine doing a write to an array at offset x - this is safe in Rust, so the compiler turns that into code that checks that x is within the bounds of that array, then writes at that offset. If the value of x can change, then now this code can overwrite some other variable anywhere in your program, giving you a bug that's very hard to track down)

jvanderbot 2y ago

Fair, but note there is a significant subset of Rust-targeted programmers who dislike the compiler doing things like that. They also dislike the compiler doing things like auto-initializing every loop iteration, but two wrongs wouldn't make it right, just less wrong.