6 ms·

> The compiler is allowed to transform your code if it can prove that the result would interact with the outside world in exactly the same way Well, in C/C++,

by titzer 4d ago

> The compiler is allowed to transform your code if it can prove that the result would interact with the outside world in exactly the same way Well, in C/C++, as soon as your program has one UB bug, the compiler has absolutely no obligation whatsoever.

uecker 4d ago
This is not quite correct in C. ISO C at least requires that observable behavior until this point is preserved.
- titzer 4d ago
  I don't think this is true. You're going to have to point to the exact place in the spec that says this, because optimizations in the presence of UB in most compilers make absolutely no assumptions.
  - uecker 4d ago
    It follows from the definition of UB: undefined behavior: "behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this document imposes no requirements" The C++ spec at some point changed this to explicitly allow changing anything in the program not just the specific behavior implied by the "for which". The C spec never did this. Because people were confused about this, in C23 we added the following note. "Note 3 to entry: Any other behavior during execution of a program is only affected as a direct consequence of the concrete behavior that occurs when encountering the erroneous or non-portable program construct or data. In particular, all observable behavior (5.1.2.4) appears as specified in this document when it happens before an operation with undefined behavior in the execution of the program." Compilers mostly follow this. GCC has bugs related to volatile. Clang often follows the C++ standard, where it is different from C, so probably does not conform to the standard here (as for some other things). The new C++ standard will have UB "barrier", i.e. std::observable that will limit the effect of UB to things before this barrier.
    - titzer 4d ago
      Thanks for pointing to this. I'm surprised the specification verbiage differs so much between the two standards. However I think the language for C is aspirational and there are cases where compiler optimizations will fail this.
      - uecker 4d ago
        There are a few rare cases where compiler optimization will fail this in GCC. There are also cases where the C spec has defined behavior that optimizers break. Where I know about them, I file bugs. It would be up for users to insist that those get fixed. This includes optimization issues that affect Rust as well.
      - Georgelemental 4d ago
        Some of those cases have been fixed. For example, Clang used to reorder UB around volatile loads, but the latest version no longer does so.
        uecker 4d ago
        This is good news! Clang representatives were very basically the only ones against this clarification. Do you have a pointer?
        Georgelemental 4d ago
        https://llvm.org/docs/LangRef.html#volatile-memory-accesses https://llvm.org/docs/LangRef.html#volatile-memory-accesses: > Volatile operations are permitted to trap. The compiler may not assume that execution will continue after a volatile operation.
- tialaramex 4d ago
  This is important. Undefined Behaviour is a behaviour, and so if we can ensure the behaviour doesn't happen, our problem is averted. For example if there's UB when a zero size file is loaded by our software, we can instruct operators to check the file has a non-zero size and we're preventing whatever horrible UB would arise. We can even rope off whole parts of the software. If the Postscript printing code has UB, simply instruction operators only to use the HP inkjet printers for which we know Postscript is not used can prevent this UB from happening.
mpyne 4d ago
Well yes, that's what UB means. It's a singularity, you run into it and there's no longer a specified requirement for the behavior that will follow. Rust also has UB, btw, https://doc.rust-lang.org/reference/behavior-considered-undefined.html https://doc.rust-lang.org/reference/behavior-considered-unde..., so I don't know where it is that people have imagined this is something the C and C++ language designers went out of their way to foist upon you. If you want to write code for a VAX, then you can use the K&R C compiler where it had defined outcomes for everything. If you want to write portable C code for modern CPUs then it's fair to ask what the C language standard is supposed to define for each of those CPUs and OSes and ABIs. And a bunch of people were nice enough to do that for you and I, but because they are not deities, there are things that they had to leave out to make the language useful, so they did.
- titzer 4d ago
  > I don't know where it is that people have imagined this is something the C and C++ language designers went out of their way to foist upon you. They did. Most other languages, the vast majority of which are also memory safe languages, go out of their way to do the opposite, and give meaning even to erroneous programs. Some things slip through the cracks, and generally language designers and implementers work hard to get rid of UB. C/C++ is the only ecosystem that has fully embraced UB as a way of life. They are the only compilers that make full use of "UB is bad and cannot ever happen" as a core tenet in how optimizations are designed. Rust UB is at least a little different. Rust UB can only be the result of unsafe code and is meant to be limited in blast radius, and is absolutely not meant as a loophole for compilers to just do whatever to make the code faster. C/C++ have a surprisingly large set of UB, too. Thankfully, the rest of the software world is rising up and the committees are starting to make things like gasp signed arithmetic overflow into defined behavior. But don't hold your breath.
  - uecker 4d ago
    For, C we have already removed at lot of UB from the working draft for C2y. My hope is that only the UB is left that is difficult to remove without requiring extensive changes to code or compilers, and that this can then be addressed by a technical specification that defines a safe subset of C. But note that UB also does not necessarily mean your program has no meaning. A good compiler can do something reasonable by defining the UB. There is no mandate in the specification that a compiler has to break things. It is also the user's responsibility to put pressure on compiler developers to do something reasonable.
tialaramex 4d ago
If you don't want UB to mean "Absolutely anything might happen" which necessarily has to include "... forever" then you need Fil-C or similar runtime handling so that any time its behaviour would become undefined the program exits instead. To some extent in C and even more in C++ there's a much worse problem, IFNDR [Ill-formed No Diagnostic Required]. Programs which the language specification insists mean nothing at all, but your tools won't (in many cases can't) notice so the result might do anything. It's not Undefined Behaviour, your program never had any defined behaviour at all.
- mirashii 4d ago
  > Fil-C or similar runtime handling so that any time its behaviour would become undefined the program exits instead It’s worth being clear here that this is not what Fil-C does, it still has UB, and can still explode in many of the same ways as C and all (after all, it’s a clang fork). Fil-C takes one particular class of allocation related bugs and UB off the table, but leaves many of them behind.
  - uecker 4d ago
    Whatever happens for the other UB remains bounded memory safely in Fil-C. (according to a definition of memory safety that excludes protection of subobject bounds, but Rust also redefines memory safety to what the Rust compiler can do, e.g. excludes memory leaks).
    - mirashii 4d ago
      I'm not so sure you can make such a strong statement about what happens when UB is invoked. The presence of UB allows the compiler to make all kinds of weird assumptions, and it seems very unlikely that there exists no series of allowable transforms that results in a pointer capabilities check being elided or similar.
      - tialaramex 4d ago
        Presumably Martin intends the usual caveat that this is subject to bugs. So, "there might be a bug in the compiler" isn't interesting. There are bugs in LLVM, bugs in Rust's trait resolution, bugs in Javascript implementations, obviously code has bugs and that's generally not very interesting - we can fix bugs. Are you claiming that there must be such transforms or only that it is likely that bugs exist which isn't interesting.

by titzer 4d ago

uecker 4d ago

This is not quite correct in C. ISO C at least requires that observable behavior until this point is preserved.

titzer 4d ago
I don't think this is true. You're going to have to point to the exact place in the spec that says this, because optimizations in the presence of UB in most compilers make absolutely no assumptions.
- uecker 4d ago
  It follows from the definition of UB: undefined behavior: "behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this document imposes no requirements" The C++ spec at some point changed this to explicitly allow changing anything in the program not just the specific behavior implied by the "for which". The C spec never did this. Because people were confused about this, in C23 we added the following note. "Note 3 to entry: Any other behavior during execution of a program is only affected as a direct consequence of the concrete behavior that occurs when encountering the erroneous or non-portable program construct or data. In particular, all observable behavior (5.1.2.4) appears as specified in this document when it happens before an operation with undefined behavior in the execution of the program." Compilers mostly follow this. GCC has bugs related to volatile. Clang often follows the C++ standard, where it is different from C, so probably does not conform to the standard here (as for some other things). The new C++ standard will have UB "barrier", i.e. std::observable that will limit the effect of UB to things before this barrier.
  - titzer 4d ago
    Thanks for pointing to this. I'm surprised the specification verbiage differs so much between the two standards. However I think the language for C is aspirational and there are cases where compiler optimizations will fail this.
    - uecker 4d ago
      There are a few rare cases where compiler optimization will fail this in GCC. There are also cases where the C spec has defined behavior that optimizers break. Where I know about them, I file bugs. It would be up for users to insist that those get fixed. This includes optimization issues that affect Rust as well.
    - Georgelemental 4d ago
      Some of those cases have been fixed. For example, Clang used to reorder UB around volatile loads, but the latest version no longer does so.
      - uecker 4d ago
        This is good news! Clang representatives were very basically the only ones against this clarification. Do you have a pointer?
        Georgelemental 4d ago
        https://llvm.org/docs/LangRef.html#volatile-memory-accesses https://llvm.org/docs/LangRef.html#volatile-memory-accesses: > Volatile operations are permitted to trap. The compiler may not assume that execution will continue after a volatile operation.
tialaramex 4d ago
This is important. Undefined Behaviour is a behaviour, and so if we can ensure the behaviour doesn't happen, our problem is averted. For example if there's UB when a zero size file is loaded by our software, we can instruct operators to check the file has a non-zero size and we're preventing whatever horrible UB would arise. We can even rope off whole parts of the software. If the Postscript printing code has UB, simply instruction operators only to use the HP inkjet printers for which we know Postscript is not used can prevent this UB from happening.

mpyne 4d ago

Well yes, that's what UB means. It's a singularity, you run into it and there's no longer a specified requirement for the behavior that will follow. Rust also has UB, btw, https://doc.rust-lang.org/reference/behavior-considered-undefined.html https://doc.rust-lang.org/reference/behavior-considered-unde..., so I don't know where it is that people have imagined this is something the C and C++ language designers went out of their way to foist upon you. If you want to write code for a VAX, then you can use the K&R C compiler where it had defined outcomes for everything. If you want to write portable C code for modern CPUs then it's fair to ask what the C language standard is supposed to define for each of those CPUs and OSes and ABIs. And a bunch of people were nice enough to do that for you and I, but because they are not deities, there are things that they had to leave out to make the language useful, so they did.

titzer 4d ago
> I don't know where it is that people have imagined this is something the C and C++ language designers went out of their way to foist upon you. They did. Most other languages, the vast majority of which are also memory safe languages, go out of their way to do the opposite, and give meaning even to erroneous programs. Some things slip through the cracks, and generally language designers and implementers work hard to get rid of UB. C/C++ is the only ecosystem that has fully embraced UB as a way of life. They are the only compilers that make full use of "UB is bad and cannot ever happen" as a core tenet in how optimizations are designed. Rust UB is at least a little different. Rust UB can only be the result of unsafe code and is meant to be limited in blast radius, and is absolutely not meant as a loophole for compilers to just do whatever to make the code faster. C/C++ have a surprisingly large set of UB, too. Thankfully, the rest of the software world is rising up and the committees are starting to make things like gasp signed arithmetic overflow into defined behavior. But don't hold your breath.
- uecker 4d ago
  For, C we have already removed at lot of UB from the working draft for C2y. My hope is that only the UB is left that is difficult to remove without requiring extensive changes to code or compilers, and that this can then be addressed by a technical specification that defines a safe subset of C. But note that UB also does not necessarily mean your program has no meaning. A good compiler can do something reasonable by defining the UB. There is no mandate in the specification that a compiler has to break things. It is also the user's responsibility to put pressure on compiler developers to do something reasonable.

tialaramex 4d ago

If you don't want UB to mean "Absolutely anything might happen" which necessarily has to include "... forever" then you need Fil-C or similar runtime handling so that any time its behaviour would become undefined the program exits instead. To some extent in C and even more in C++ there's a much worse problem, IFNDR [Ill-formed No Diagnostic Required]. Programs which the language specification insists mean nothing at all, but your tools won't (in many cases can't) notice so the result might do anything. It's not Undefined Behaviour, your program never had any defined behaviour at all.

mirashii 4d ago
> Fil-C or similar runtime handling so that any time its behaviour would become undefined the program exits instead It’s worth being clear here that this is not what Fil-C does, it still has UB, and can still explode in many of the same ways as C and all (after all, it’s a clang fork). Fil-C takes one particular class of allocation related bugs and UB off the table, but leaves many of them behind.
- uecker 4d ago
  Whatever happens for the other UB remains bounded memory safely in Fil-C. (according to a definition of memory safety that excludes protection of subobject bounds, but Rust also redefines memory safety to what the Rust compiler can do, e.g. excludes memory leaks).
  - mirashii 4d ago
    I'm not so sure you can make such a strong statement about what happens when UB is invoked. The presence of UB allows the compiler to make all kinds of weird assumptions, and it seems very unlikely that there exists no series of allowable transforms that results in a pointer capabilities check being elided or similar.
    - tialaramex 4d ago
      Presumably Martin intends the usual caveat that this is subject to bugs. So, "there might be a bug in the compiler" isn't interesting. There are bugs in LLVM, bugs in Rust's trait resolution, bugs in Javascript implementations, obviously code has bugs and that's generally not very interesting - we can fix bugs. Are you claiming that there must be such transforms or only that it is likely that bugs exist which isn't interesting.