11 ms·

C++'s `noexcept` can sometimes help or hurt performance

100 points by def-pri-pub 2y ago

TillE 2y ago
> I didn't know std::uniform_int_distribution doesn't actually produce the same results on different compilers I think this is genuinely my biggest complaint about the C++ standard library. There are countless scenarios where you want deterministic random numbers (for testing if nothing else), so std's distributions are unusable. Fortunately you can just plug in Boost's implementation.
- chipdart 2y ago
  > There are countless scenarios where you want deterministic random numbers (for testing if nothing else), so std's distributions are unusable. Fortunately you can just plug in Boost's implementation. I don't understand what's your complain. If you're already plugging in alternative implementations,what stops you from actually stubbing these random number generators with any realization at all?
  - akira2501 2y ago
    It's a compromised and goofy implementation with lots of warts. What's the point it in having a /standard/ library then?
    - chipdart 2y ago
      > It's a compromised and goofy implementation with lots of warts. I don't think this case qualifies as an example. I think the only goofy detail in the story is expecting a random number generator to be non-random and deterministic with the only conceivable usecase being poorly designed and implemented test fxtures. > What's the point it in having a /standard/ library then? The point of standardized components is to provide reusable elements that can be used across all platforms and implementations, thus saving on the development effort of upgrading and porting the code across implementations and even platforms. If you cannot design working software, that's not a problem you can pin on the tools you don't know how to use.
      - kevin_thibedeau 2y ago
        > with the only conceivable usecase being poorly designed and implemented test fxtures. Reproducible pseudo-randomness is a necessity with fuzz testing. It is not a poor design approach when it is actually useful.
        leni536 2y ago
        it is reproducible within a single standard library implementation, so usable for fuzz testing
      - forrestthewoods 2y ago
        > The point of standardized components is to provide reusable elements that can be used across all platforms and implementations, thus saving on the development effort of upgrading and porting the code across implementations and even platforms. It's a shame that C++'s "standardized" components ARE COMPLETELY DIFFERENT on different platforms. Some of the C++ standard requires per-platform implementation work. For example std::thread on Linux and Windows obviously must have a different implementation. However a super majority of the standard API is just vanilla C++ code. For example std::vector or std::unordered_map. The fact that the standard defines a spec which is then implemented numerous times is absurd, stupid, and bad. The specs are simultaneously over-constrained and under-constrained. It's a disaster.
        gumby 2y ago
        I consider the current tradeoff to be a feature. It permits implementations to take advantage of target-specific affordances (your thread case is an example) as well as taking different implementation strategies (e.g. the small string optimization is different in libc++ and libstdc++). Also you may use another, independent standard library because you prefer its implementation decisions. Meanwhile they remain compatible at the source level.
        Maxatar 2y ago
        Unlike in C, in C++ it is not possible to use an independent implementation of the standard library. Clang is compatible with GCC's standard library/libstdc++ and MSVC's standard library because the clang compiler explicitly supports them, but it's not possible to use clang's standard library with GCC in a standard conforming way or interchange GCC's with MSVC's standard library. There are some hacks that let you use some parts of libc++ with GCC by using the nostdlib flag, but this disables a lot of C++ functionality such as exception handling, RTTI, type traits. These features are in turn used by things like std::vector, std::map, etc... so you won't be able to use those classes either, and so on so forth...
- quotemstr 2y ago
  > I think this is genuinely my biggest complaint about the C++ standard library What do you think of Abseil hash tables randomizing themselves (piggybacking on ASLR) on each start of your program?
  - slaymaker1907 2y ago
    Their justification is here https://github.com/abseil/abseil-cpp/issues/720 https://github.com/abseil/abseil-cpp/issues/720 However, I personally disagree with them since I think it's really important to have _some_ basic reproducibility for things like reproducing the results of a randomized test. In that case, I'm going to avoid changing as much as possible anyways.
- nwallin 2y ago
  It's actually really important that uniform_int_distribution is implementation defined. The 'right' way to do it on one architecture is probably not the right way to do it on a different architecture. For instance, Apple's new CPUs has very fast division. A convenient and useful tool to implement uniform_int_distribution relies on using modulo. So the implementation that runs on Apple's new CPUs ought to use the modulo instructions of the CPU. On other architectures, the ISA might not even have a modulo instruction. In this case, it's very important that you don't try to emulate modulo in software; it's much better to rely other more complicated constructs to give a uniform distribution. C++ is also expected to run on GPUs. NVIDIA's CUDA and AMD's HIP are both implementations of C++. (these implementations are non-compliant given the nature of GPUs, but both they and the C++ standard's committee have a shared goal of narrowing that gap) In general, std::uniform_int_distribution uses loops to eliminate redundancies; the 'happy path' has relatively easily predicted branches, but they can and do have instances where the branch is not easily predicted and will as often as not have to loop in order to complete. Doing this on a GPU might be multiple orders of magnitude slower than another method that's better suited for a GPU. Overzealously dictating an implementation is why C++ ended up with a relatively bad hash table and very bad regex in the standard. It's a mistake that shouldn't be made again.
  - aw1621107 2y ago
    > Overzealously dictating an implementation is why C++ ended up with a relatively bad hash table and very bad regex in the standard. What parts of the standard dictate a particular regex implementation? IIRC the performance issues are usually blamed on ABI compatibility constraints rather than the standard making a fast(er) implementation impossible.
  - lifthrasiir 2y ago
    But reproducibility is as important as performance for the vast majority of use cases, if these implementation-defined bits start to affect the observable outcomes. (That's why we define the required time complexity for many container-related functions but do not actually specify the exact algorithm; difference in Big-O time complexity is just large enough to be "observed".) A common solution is to provide two versions of such features, one for the less reproducible but maximally performant version and another for common middle grounds that can be reproduced reasonably efficiently across many common platforms. In fact I believe `std::chrono` was designed in that way to sidestep many uncertainties in platform clock implementations.
compiler-guy 2y ago
Even a speedup of around 1% (if it is consistent and in a carefully controlled experiment) is significant for many workloads, if the workload is big enough. The OP has this as in the fuzz, which it may be for that particular workload. But across a giant distributed system like youtube or Google search, it is a real gain.
rwmj 2y ago
Shouldn't the compiler deduce noexcept for you?
- terrymah 2y ago
  It absolutely does, and even better, the compiler deduced "this function doesn't throw" doesn't come with the overhead of implementing noexcept proper
- Eyas 2y ago
  It probably can in a .cc file but if you're importing another library and just have access to the header, it wouldn't know how to.
- compiler-guy 2y ago
  The compiler can tell about the immediate function, but not any functions it calls. If a function marked noexcept calls a function that throws an exception, then the program is terminated with an uncaught exception. A called function can throw through a non-noexcept function to a higher-level exception handler no problem. So in order to avoid changing the semantics of the function, the compiler would have to be able to determine that that transitive closure of called functions dynamically don't throw, and that problem is undecidable, even assuming the requirement that "the compiler can see the source of all those functions" is somehow met, which it won't be.
  - terrymah 2y ago
    No, we compile in bottom up order, starting with leaf functions, and collecting information about functions as we go. So "not throwing" sort of trickles up when possible to a certain degree. In LTCG (MSVC)/O3 (GCC/Clang) there are prepasses over the entire callgraph to collect this order
    - deleted 2y ago
      [deleted]
    - deleted 2y ago
      [deleted]
hoten 2y ago
I don't feel like this article illuminates anything about how noexcept works. The asm diff at the end suggests _there is no difference_ in the emitted code. I plugged it into godbolt myself and see absolutely no difference. https://godbolt.org/z/jdro5jdnG https://godbolt.org/z/jdro5jdnG It seems the selected example function may not be exercising noexcept. I suppose the assumption is that operator[] is something that can throw, but ... perhaps the machinery lives outside the function (so should really examine function calls), or is never emitted without a try/catch, or operator[] (though not marked noexcept...) doesn't throw b/c OOB is undefined behavior, or ... ?
- quuxplusone 2y ago
  > I don't feel like this article illuminates anything about how noexcept works. The asm diff at the end suggests _there is no difference_ in the emitted code. You are absolutely correct. The OP is basically testing the hypothesis "Wrapping a function in `noexcept` will magically make it faster," which is (1) nonsense to anyone who knows how C++ works, and also (2) trivially easy to falsify, because all you have to do is look at the compiled code. Same codegen? Then it's not going to be faster (or slower). You needn't spend all those CPU cycles to find out what you already know by looking. There has been a fair bit of literature written on the performance of exceptions and noexcept, but OP isn't contributing anything with this particular post. Here are two of my own blog posts on the subject. The first one is just an explanation of the "vector pessimization" which was also mentioned (obliquely) in OP's post — but with an actual benchmark where you can see why it matters. https://quuxplusone.github.io/blog/2022/08/26/vector-pessimization/#conclusion-the-vector-pessimization https://quuxplusone.github.io/blog/2022/08/26/vector-pessimi... https://godbolt.org/z/e4jEcdfT9 https://godbolt.org/z/e4jEcdfT9 The second one is much more interesting, because it shows where `noexcept` can actually have an effect on codegen in the core language. TLDR, it can matter on functions that the compiler can't inline, such as when crossing ABI boundaries or when (as in this case) it's an indirect call through a function pointer. https://quuxplusone.github.io/blog/2022/07/30/type-erased-inplace-printable/#benefits-from-noexcept https://quuxplusone.github.io/blog/2022/07/30/type-erased-in...
  - hoten 2y ago
Arech 2y ago
That's quite interesting and a huge work has been done here, respect for that. Here's what has jumped out at me: `noexcept` qualifier is not free in some cases, particularly, when a qualified function could actually throw, but is marked `noexcept`. In that case, a compiler still must set something up to fulfil the main `noexcept` promise - call `std::terminate()` if an exception is thrown. That means, that putting `noexcept` on each and every function blindly without any regard to whether the function could really throw or not (for example, `std::vector::push_back()` could throw on reallocation failure, hence if a `noexcept` qualified function call it, a compiler must take into account) doesn't actually test/benchmark/prove anything, since as the author correctly said, - you won't ever do this in a real production project. It would be really interesting to take a look into a full code of cases that showed very bad performance, however, here we're approaching the second issue: if that's the core benchmark code: https://github.com/define-private-public/PSRayTracing/blob/acb04979c49ea8adef8c4afc349a96015834835e/experiments/noexcept_keyword/noexcept_list_iteration_test.cpp https://github.com/define-private-public/PSRayTracing/blob/a... then unfortunately it's totally invalid since it measures time with the `std::chrono::system_clock` which isn't monotonic. Given how long the code required to run, it's almost certain that the clock has been adjusted several times...
- zokier 2y ago
  > then unfortunately it's totally invalid since it measures time with the `std::chrono::system_clock` which isn't monotonic. Given how long the code required to run, it's almost certain that the clock has been adjusted several times monotonic clocks are mostly useful for short measurement periods. for long-term timing wall-time clocks (with their adjustments) are more accurate because they will drift less.
  - Arech 2y ago
    Ah, that's a great correction, thank you! Yes, indeed, due to a drift, in order to discern second+ (?) differences on different machines (or same machines, but different OSes?), one definitely needs to use a wall-clock time, otherwise it's comparing apples to oranges. There's a lot of interesting questions related to that, but they out of the scope of the thread. If I'm not mistaken the author has also timed some individual small functions, which, if correct, still poses a problem to me, but for measuring huge long running tasks like a full suite running 10+ hours, they are probably right in choosing wall-clock timer indeed. However, before researching into results any further (for example, -10% difference for `noexcept` case is extremely interesting to debug up to the root cause), I'd still like to understand how the code was run and measured exactly. I didn't find a plausible looking benchmark runner in their code base.
Night_Thastus 2y ago
I thought I saw this post, or a very similar one, a couple years ago. Does anyone else remember that? Yet I don't see it in the post history.
plorkyeran 2y ago
The most common place where noexcept improves performance is on move constructors and move assignments when moving is cheaper than copying. If your type is not nothrow moveable std::vector will copy it instead of moving when resizing, as the move constructor throwing would leave the vector in an invalid state (while the copy constructor throwing leaves the vector unchanged). Platforms with setjmp-longjmp based exceptions benefit greatly from noexcept as there’s setup code required before calling functions which may throw. Those platforms are now mostly gone, though. Modern “zero cost” exceptions don’t execute a single instruction related to exception handling if no exceptions are thrown (hence the name), so there just isn’t much room for noexcept to be useful to the optimizer. Outside of those two scenarios there isn’t any reason to expect noexcept to improve performance.
- 10tacobytes 2y ago
  This is the correct analysis. The article's author could have saved themselves (and the reader) a good amount of blind data diving by learning more about exception processing beforehand.
- jzwinck 2y ago
  There is another standard library related scenario: hash tables. The std unordered containers will store the hash of each key unless your hash function is noexcept. Analogous to how vector needs noexcept move for fast reserve and resize, unordered containers need noexcept hash to avoid extra memory usage. See https://gcc.gnu.org/onlinedocs/libstdc++/manual/unordered_associative.html https://gcc.gnu.org/onlinedocs/libstdc++/manual/unordered_as...
  - anonymoushn 2y ago
    For many key types and access patterns, storing the hash is faster anyway. I assume people who care about performance are already not using std::unordered_map though.
olliej 2y ago
I would like to have seen a comparison that actually includes -fno-exceptions, rather than just noexcept. My assumption is that to get a consistent gain from noexcept, you would need every function called to be explicitly noexcept, because a bunch of the cost of exceptions is code size and state required to support unwinding. So if the performance cost exception handling is causing is due to that, then if _anything_ can cause an exception (or I guess more accurately unless every opaque call is explicitly indicated to not cause an exception) then that overhead remains. That said, I'm still confused by the perf results of the article, especially the perlin noise vs MSVC one. It's sufficiently weird outlier that it makes me wonder if something in the compiler has a noexcept path that adds checks that aren't usually on (i.e imagine the code has a "debug" mode that did bounds checks or something, but the function resolution you hit in the noexcept path always does the bounds check - I'm really not sure exactly how you'd get that to happen, but "non-default path was not benchmarked" is not exactly an uncommon occurrence)
quotemstr 2y ago
There's a lot of mysticism and superstition surrounding C++ exceptions. It's instructive to sit down with godbolt and examine specific scenarios in which noexcept (or exceptions generally) can affect performance. Read the machine code. Understand why the compiler does what it does. Don't want to invest at that level? You probably want to use a higher level language.
- r2vcap 2y ago
  Or set the compiler flag -fno-exceptions and ban the use of exceptions. While it isn’t standard-compliant, a surprisingly large number of companies and projects follow these practices.
- Maxatar 2y ago
  You won't get a sense of how bad exceptions can be by using Godbolt. A lot of the magic of exceptions is handled behind the scenes by the compiler and/or the Itanium ABI. For example one disasterous consequence of using exceptions in GCC is that there is a global application wide lock used to manage stack unwinding. This means that only one single thread can unwind a stack at a time and the lock is held from the start of the exception being thrown until the very last destructor is called. If you have a multicore server with 100 threads, and one of those threads throws an exception, you better hope that no other thread throws an exception because even if those two threads are entirely independent of one another, one of them will block. You won't see this by looking at Godbolt.
  - aw1621107 2y ago
    > For example one disasterous consequence of using exceptions in GCC is that there is a global application wide lock used to manage stack unwinding. This might have been (partially?) fixed? GCC Bug 71744 "Concurrently throwing exceptions is not scalable" is marked "RESOLVED FIXED" [0], and commit 6e80a1d164d1 in particular looks interesting: > eliminate mutex in fast path of __register_frame > > <snip> > > This commit eliminates both the mutex and the sorted list from the atomic fast path, and replaces it with a btree that uses optimistic lock coupling during lookup. This allows for fully parallel unwinding and is essential to scale exception handling to large core counts. I'm not particularly familiar with the unwinding machinery though so I don't know if the issue is fully resolved. [0]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744
Squeeeez 2y ago
Is this some kind of new clickbait title? Something "can" "sometimes" do something (already 0 information) - ooor sometimes it also does the opposite. The only possibility not allowed is that it would not make any difference, but this one is actually also possible. Sigh.
terrymah 2y ago
Oh man, don't get me started. This was a point in a talk I gave years ago called "Please Please Help the Compiler" (what I thought was a clever cut at the conventional wisdom at the time of "Don't Try to Help the Compiler") I work on MSVC backend. I argued pretty strenuously at the time that noexcept was costly and being marketed incorrectly. Perhaps the costs are worth it, but none the less there is a cost The reason is simple: there is a guarantee here that noexcept functions don't throw. std::terminate has to be called. That has to be implemented. There is some cost to that - conceptually every noexcept function (or worse, every call to a noexcept function) is surrounded by a giant try/catch(...) block. Yes there are optimizations here. But it's still not free Less obvious; how does inlining work? What happens if you inline a noexcept function into a function that allows exceptions? Do we now have "regions" of noexceptness inside that function (answer: yes). How do you implement that? Again, this is implementable, but this is even harder than the whole function case, and a naive/early implementation might prohibit inlining across degrees of noexcept-ness to be correct/as-if. And guess what, this is what early versions of MSVC did, and this was our biggest problem: a problem which grew release after release as noexcept permeated the standard library. Anyway. My point is, we need more backend compiler engineers on WG21 and not just front end, library, and language lawyer guys. I argued then that if instead noexcept violations were undefined, we could ignore all this, and instead just treat it as the pure optimization it was being marketed as (ie, help prove a region can't throw, so we can elide entire try/catch blocks etc). The reaction to my suggestion was not positive.
- rockwotj 2y ago
  Added to my list why I compile with -fno-exceptions
  - jcelerier 2y ago
    -fno-exceptions only prevents you from calling throw. If you don't want overhead likely you want -fno-asynchronous-unwind-tables + that clang flag that specifies that extern "C" functions don't throw
    - 2y ago
tolmasky 2y ago
Is there any compiler option to have it yell at you if you mark something that can throw as `noexcept`, which seems to be the cause of (at least some of) the slowdowns where the compiler is forced to accommodate with `std::terminate`? I feel like these situations are more commonly mistakes, and not the user wanting to "collapse" exceptions into terminations. So the current approach to dealing with these cases seems to be suboptimal not only from a performance perspective, but a behavior perspective as well.
- terrymah 2y ago
  No, calling throw in a noexcept function is a defined behavior (call std::terminate), and that behavior is not a diagnostic I think maybe WG21 was concerned a compiler engineer would be clever if throwing in noexcept were UB, for example and assume any block that throws is unreachable and could just be removed along with all blocks it postdominates. Compiler guys love optimizations that just remove code. The fastest and smallest code is code that can’t run and doesn’t exist
  - account42 2y ago
    Compilers are allowed to and do diagnose defined but undesirable behavior though, even more so if that's only enabled with an option.
    - terrymah 2y ago
      Until yesterday I thought I was the only person in the world who thought the designed behavior was undesirable, though
hoseja 2y ago
Apologize less for using completely benign standard macros. They are an okay tool if not abused.
shultays 2y ago
I can't find the explanation on why noexcept could hurt performance. One reason I can see itt is some containers like unordered_map can inline the hash along with the key with noexcept, which may not worth additional memory overhead if the hashing is relatively cheap. He talks a bit about it in "Intel+Windows+MSVC" but not much info. I wish there was noexcept helps in some cases that author doesn't seem to be using and any performance gain or loss is basically due to some (unrelated?) optimization decisions the compiler takes differently in noexcept builds if I am understanding correctly?
maccard 2y ago
This is super unrelated to the optimisation, and is just related to the cmake setup - instead of common.hpp having #ifdef USE_NOEXCEPT #define NOEXCEPT noexcept #else #define NOEXCEPT #endif and cmake being: if (WITH_NOEXCEPT) message(STATUS "Using `noexcept` annotations (faster?)") target_compile_definitions(PSRayTracing_StaticLibrary PUBLIC USE_NOEXCEPT) else() message(STATUS "Turned off use of `noexcept` (slower?)") endif() , the cmake could just be: if (WITH_NOEXCEPT) message(STATUS "Using `noexcept` annotations (faster?)") target_compile_definitions(PSRayTracing_StaticLibrary PUBLIC USE_NOEXCEPT=noexcept) else() message(STATUS "Turned off use of `noexcept` (slower?)") target_compile_definitions(PSRayTracing_StaticLibrary PUBLIC USE_NOEXCEPT=) endif() No need for these shared "common" config headers. Back on topic, this doesn't surprise me. There's this idea that C++ is fast, and that people who work with C++ are focused on optimisation and in my experience there's as many of these theoretical ideas about performance which aren't backed up by numbers, but are now ingrained in people. See https://news.ycombinator.com/item?id=41095814 https://news.ycombinator.com/item?id=41095814 from last week for another example of dogmatic guidelines having the wrong impact.
muth02446 2y ago
RE: unexpected performance degradation programs can be quite sensitive to how code is laid out because of cache line alignment, cache conflicts etc. So random changes can have a surprising impact. There was a paper a couple of years ago explaining this and how to measure compiler optimizations more reliably. Sadly, I do not recall the title/author.
- Arech 2y ago
  It would be super interesting to read the paper. Please post a link or some more details if you will remember them.

100 points by def-pri-pub 2y ago

> I didn't know std::uniform_int_distribution doesn't actually produce the same results on different compilers I think this is genuinely my biggest complaint about the C++ standard library. There are countless scenarios where you want deterministic random numbers (for testing if nothing else), so std's distributions are unusable. Fortunately you can just plug in Boost's implementation.

chipdart 2y ago
> There are countless scenarios where you want deterministic random numbers (for testing if nothing else), so std's distributions are unusable. Fortunately you can just plug in Boost's implementation. I don't understand what's your complain. If you're already plugging in alternative implementations,what stops you from actually stubbing these random number generators with any realization at all?
- akira2501 2y ago
  It's a compromised and goofy implementation with lots of warts. What's the point it in having a /standard/ library then?
  - chipdart 2y ago
    > It's a compromised and goofy implementation with lots of warts. I don't think this case qualifies as an example. I think the only goofy detail in the story is expecting a random number generator to be non-random and deterministic with the only conceivable usecase being poorly designed and implemented test fxtures. > What's the point it in having a /standard/ library then? The point of standardized components is to provide reusable elements that can be used across all platforms and implementations, thus saving on the development effort of upgrading and porting the code across implementations and even platforms. If you cannot design working software, that's not a problem you can pin on the tools you don't know how to use.
    - kevin_thibedeau 2y ago
      > with the only conceivable usecase being poorly designed and implemented test fxtures. Reproducible pseudo-randomness is a necessity with fuzz testing. It is not a poor design approach when it is actually useful.
      - leni536 2y ago
        it is reproducible within a single standard library implementation, so usable for fuzz testing
    - forrestthewoods 2y ago
      > The point of standardized components is to provide reusable elements that can be used across all platforms and implementations, thus saving on the development effort of upgrading and porting the code across implementations and even platforms. It's a shame that C++'s "standardized" components ARE COMPLETELY DIFFERENT on different platforms. Some of the C++ standard requires per-platform implementation work. For example std::thread on Linux and Windows obviously must have a different implementation. However a super majority of the standard API is just vanilla C++ code. For example std::vector or std::unordered_map. The fact that the standard defines a spec which is then implemented numerous times is absurd, stupid, and bad. The specs are simultaneously over-constrained and under-constrained. It's a disaster.
      - gumby 2y ago
        I consider the current tradeoff to be a feature. It permits implementations to take advantage of target-specific affordances (your thread case is an example) as well as taking different implementation strategies (e.g. the small string optimization is different in libc++ and libstdc++). Also you may use another, independent standard library because you prefer its implementation decisions. Meanwhile they remain compatible at the source level.
        Maxatar 2y ago
        Unlike in C, in C++ it is not possible to use an independent implementation of the standard library. Clang is compatible with GCC's standard library/libstdc++ and MSVC's standard library because the clang compiler explicitly supports them, but it's not possible to use clang's standard library with GCC in a standard conforming way or interchange GCC's with MSVC's standard library. There are some hacks that let you use some parts of libc++ with GCC by using the nostdlib flag, but this disables a lot of C++ functionality such as exception handling, RTTI, type traits. These features are in turn used by things like std::vector, std::map, etc... so you won't be able to use those classes either, and so on so forth...
quotemstr 2y ago
> I think this is genuinely my biggest complaint about the C++ standard library What do you think of Abseil hash tables randomizing themselves (piggybacking on ASLR) on each start of your program?
- slaymaker1907 2y ago
  Their justification is here https://github.com/abseil/abseil-cpp/issues/720 https://github.com/abseil/abseil-cpp/issues/720 However, I personally disagree with them since I think it's really important to have _some_ basic reproducibility for things like reproducing the results of a randomized test. In that case, I'm going to avoid changing as much as possible anyways.
nwallin 2y ago
It's actually really important that uniform_int_distribution is implementation defined. The 'right' way to do it on one architecture is probably not the right way to do it on a different architecture. For instance, Apple's new CPUs has very fast division. A convenient and useful tool to implement uniform_int_distribution relies on using modulo. So the implementation that runs on Apple's new CPUs ought to use the modulo instructions of the CPU. On other architectures, the ISA might not even have a modulo instruction. In this case, it's very important that you don't try to emulate modulo in software; it's much better to rely other more complicated constructs to give a uniform distribution. C++ is also expected to run on GPUs. NVIDIA's CUDA and AMD's HIP are both implementations of C++. (these implementations are non-compliant given the nature of GPUs, but both they and the C++ standard's committee have a shared goal of narrowing that gap) In general, std::uniform_int_distribution uses loops to eliminate redundancies; the 'happy path' has relatively easily predicted branches, but they can and do have instances where the branch is not easily predicted and will as often as not have to loop in order to complete. Doing this on a GPU might be multiple orders of magnitude slower than another method that's better suited for a GPU. Overzealously dictating an implementation is why C++ ended up with a relatively bad hash table and very bad regex in the standard. It's a mistake that shouldn't be made again.
- aw1621107 2y ago
  > Overzealously dictating an implementation is why C++ ended up with a relatively bad hash table and very bad regex in the standard. What parts of the standard dictate a particular regex implementation? IIRC the performance issues are usually blamed on ABI compatibility constraints rather than the standard making a fast(er) implementation impossible.
- lifthrasiir 2y ago
  But reproducibility is as important as performance for the vast majority of use cases, if these implementation-defined bits start to affect the observable outcomes. (That's why we define the required time complexity for many container-related functions but do not actually specify the exact algorithm; difference in Big-O time complexity is just large enough to be "observed".) A common solution is to provide two versions of such features, one for the less reproducible but maximally performant version and another for common middle grounds that can be reproduced reasonably efficiently across many common platforms. In fact I believe `std::chrono` was designed in that way to sidestep many uncertainties in platform clock implementations.

compiler-guy 2y ago

Even a speedup of around 1% (if it is consistent and in a carefully controlled experiment) is significant for many workloads, if the workload is big enough. The OP has this as in the fuzz, which it may be for that particular workload. But across a giant distributed system like youtube or Google search, it is a real gain.

rwmj 2y ago

Shouldn't the compiler deduce noexcept for you?

terrymah 2y ago
It absolutely does, and even better, the compiler deduced "this function doesn't throw" doesn't come with the overhead of implementing noexcept proper
Eyas 2y ago
It probably can in a .cc file but if you're importing another library and just have access to the header, it wouldn't know how to.
compiler-guy 2y ago
The compiler can tell about the immediate function, but not any functions it calls. If a function marked noexcept calls a function that throws an exception, then the program is terminated with an uncaught exception. A called function can throw through a non-noexcept function to a higher-level exception handler no problem. So in order to avoid changing the semantics of the function, the compiler would have to be able to determine that that transitive closure of called functions dynamically don't throw, and that problem is undecidable, even assuming the requirement that "the compiler can see the source of all those functions" is somehow met, which it won't be.
- terrymah 2y ago
  No, we compile in bottom up order, starting with leaf functions, and collecting information about functions as we go. So "not throwing" sort of trickles up when possible to a certain degree. In LTCG (MSVC)/O3 (GCC/Clang) there are prepasses over the entire callgraph to collect this order
  - deleted 2y ago
    [deleted]
  - deleted 2y ago
    [deleted]

hoten 2y ago

I don't feel like this article illuminates anything about how noexcept works. The asm diff at the end suggests _there is no difference_ in the emitted code. I plugged it into godbolt myself and see absolutely no difference. https://godbolt.org/z/jdro5jdnG https://godbolt.org/z/jdro5jdnG It seems the selected example function may not be exercising noexcept. I suppose the assumption is that operator[] is something that can throw, but ... perhaps the machinery lives outside the function (so should really examine function calls), or is never emitted without a try/catch, or operator[] (though not marked noexcept...) doesn't throw b/c OOB is undefined behavior, or ... ?

quuxplusone 2y ago
> I don't feel like this article illuminates anything about how noexcept works. The asm diff at the end suggests _there is no difference_ in the emitted code. You are absolutely correct. The OP is basically testing the hypothesis "Wrapping a function in `noexcept` will magically make it faster," which is (1) nonsense to anyone who knows how C++ works, and also (2) trivially easy to falsify, because all you have to do is look at the compiled code. Same codegen? Then it's not going to be faster (or slower). You needn't spend all those CPU cycles to find out what you already know by looking. There has been a fair bit of literature written on the performance of exceptions and noexcept, but OP isn't contributing anything with this particular post. Here are two of my own blog posts on the subject. The first one is just an explanation of the "vector pessimization" which was also mentioned (obliquely) in OP's post — but with an actual benchmark where you can see why it matters. https://quuxplusone.github.io/blog/2022/08/26/vector-pessimization/#conclusion-the-vector-pessimization https://quuxplusone.github.io/blog/2022/08/26/vector-pessimi... https://godbolt.org/z/e4jEcdfT9 https://godbolt.org/z/e4jEcdfT9 The second one is much more interesting, because it shows where `noexcept` can actually have an effect on codegen in the core language. TLDR, it can matter on functions that the compiler can't inline, such as when crossing ABI boundaries or when (as in this case) it's an indirect call through a function pointer. https://quuxplusone.github.io/blog/2022/07/30/type-erased-inplace-printable/#benefits-from-noexcept https://quuxplusone.github.io/blog/2022/07/30/type-erased-in...
- hoten 2y ago

Arech 2y ago

That's quite interesting and a huge work has been done here, respect for that. Here's what has jumped out at me: `noexcept` qualifier is not free in some cases, particularly, when a qualified function could actually throw, but is marked `noexcept`. In that case, a compiler still must set something up to fulfil the main `noexcept` promise - call `std::terminate()` if an exception is thrown. That means, that putting `noexcept` on each and every function blindly without any regard to whether the function could really throw or not (for example, `std::vector::push_back()` could throw on reallocation failure, hence if a `noexcept` qualified function call it, a compiler must take into account) doesn't actually test/benchmark/prove anything, since as the author correctly said, - you won't ever do this in a real production project. It would be really interesting to take a look into a full code of cases that showed very bad performance, however, here we're approaching the second issue: if that's the core benchmark code: https://github.com/define-private-public/PSRayTracing/blob/acb04979c49ea8adef8c4afc349a96015834835e/experiments/noexcept_keyword/noexcept_list_iteration_test.cpp https://github.com/define-private-public/PSRayTracing/blob/a... then unfortunately it's totally invalid since it measures time with the `std::chrono::system_clock` which isn't monotonic. Given how long the code required to run, it's almost certain that the clock has been adjusted several times...

zokier 2y ago
> then unfortunately it's totally invalid since it measures time with the `std::chrono::system_clock` which isn't monotonic. Given how long the code required to run, it's almost certain that the clock has been adjusted several times monotonic clocks are mostly useful for short measurement periods. for long-term timing wall-time clocks (with their adjustments) are more accurate because they will drift less.
- Arech 2y ago
  Ah, that's a great correction, thank you! Yes, indeed, due to a drift, in order to discern second+ (?) differences on different machines (or same machines, but different OSes?), one definitely needs to use a wall-clock time, otherwise it's comparing apples to oranges. There's a lot of interesting questions related to that, but they out of the scope of the thread. If I'm not mistaken the author has also timed some individual small functions, which, if correct, still poses a problem to me, but for measuring huge long running tasks like a full suite running 10+ hours, they are probably right in choosing wall-clock timer indeed. However, before researching into results any further (for example, -10% difference for `noexcept` case is extremely interesting to debug up to the root cause), I'd still like to understand how the code was run and measured exactly. I didn't find a plausible looking benchmark runner in their code base.

Night_Thastus 2y ago

I thought I saw this post, or a very similar one, a couple years ago. Does anyone else remember that? Yet I don't see it in the post history.

plorkyeran 2y ago

The most common place where noexcept improves performance is on move constructors and move assignments when moving is cheaper than copying. If your type is not nothrow moveable std::vector will copy it instead of moving when resizing, as the move constructor throwing would leave the vector in an invalid state (while the copy constructor throwing leaves the vector unchanged). Platforms with setjmp-longjmp based exceptions benefit greatly from noexcept as there’s setup code required before calling functions which may throw. Those platforms are now mostly gone, though. Modern “zero cost” exceptions don’t execute a single instruction related to exception handling if no exceptions are thrown (hence the name), so there just isn’t much room for noexcept to be useful to the optimizer. Outside of those two scenarios there isn’t any reason to expect noexcept to improve performance.

10tacobytes 2y ago
This is the correct analysis. The article's author could have saved themselves (and the reader) a good amount of blind data diving by learning more about exception processing beforehand.
jzwinck 2y ago
There is another standard library related scenario: hash tables. The std unordered containers will store the hash of each key unless your hash function is noexcept. Analogous to how vector needs noexcept move for fast reserve and resize, unordered containers need noexcept hash to avoid extra memory usage. See https://gcc.gnu.org/onlinedocs/libstdc++/manual/unordered_associative.html https://gcc.gnu.org/onlinedocs/libstdc++/manual/unordered_as...
- anonymoushn 2y ago
  For many key types and access patterns, storing the hash is faster anyway. I assume people who care about performance are already not using std::unordered_map though.

olliej 2y ago

I would like to have seen a comparison that actually includes -fno-exceptions, rather than just noexcept. My assumption is that to get a consistent gain from noexcept, you would need every function called to be explicitly noexcept, because a bunch of the cost of exceptions is code size and state required to support unwinding. So if the performance cost exception handling is causing is due to that, then if _anything_ can cause an exception (or I guess more accurately unless every opaque call is explicitly indicated to not cause an exception) then that overhead remains. That said, I'm still confused by the perf results of the article, especially the perlin noise vs MSVC one. It's sufficiently weird outlier that it makes me wonder if something in the compiler has a noexcept path that adds checks that aren't usually on (i.e imagine the code has a "debug" mode that did bounds checks or something, but the function resolution you hit in the noexcept path always does the bounds check - I'm really not sure exactly how you'd get that to happen, but "non-default path was not benchmarked" is not exactly an uncommon occurrence)

quotemstr 2y ago

There's a lot of mysticism and superstition surrounding C++ exceptions. It's instructive to sit down with godbolt and examine specific scenarios in which noexcept (or exceptions generally) can affect performance. Read the machine code. Understand why the compiler does what it does. Don't want to invest at that level? You probably want to use a higher level language.

r2vcap 2y ago
Or set the compiler flag -fno-exceptions and ban the use of exceptions. While it isn’t standard-compliant, a surprisingly large number of companies and projects follow these practices.
Maxatar 2y ago
You won't get a sense of how bad exceptions can be by using Godbolt. A lot of the magic of exceptions is handled behind the scenes by the compiler and/or the Itanium ABI. For example one disasterous consequence of using exceptions in GCC is that there is a global application wide lock used to manage stack unwinding. This means that only one single thread can unwind a stack at a time and the lock is held from the start of the exception being thrown until the very last destructor is called. If you have a multicore server with 100 threads, and one of those threads throws an exception, you better hope that no other thread throws an exception because even if those two threads are entirely independent of one another, one of them will block. You won't see this by looking at Godbolt.
- aw1621107 2y ago
  > For example one disasterous consequence of using exceptions in GCC is that there is a global application wide lock used to manage stack unwinding. This might have been (partially?) fixed? GCC Bug 71744 "Concurrently throwing exceptions is not scalable" is marked "RESOLVED FIXED" [0], and commit 6e80a1d164d1 in particular looks interesting: > eliminate mutex in fast path of __register_frame > > <snip> > > This commit eliminates both the mutex and the sorted list from the atomic fast path, and replaces it with a btree that uses optimistic lock coupling during lookup. This allows for fully parallel unwinding and is essential to scale exception handling to large core counts. I'm not particularly familiar with the unwinding machinery though so I don't know if the issue is fully resolved. [0]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71744

Squeeeez 2y ago

Is this some kind of new clickbait title? Something "can" "sometimes" do something (already 0 information) - ooor sometimes it also does the opposite. The only possibility not allowed is that it would not make any difference, but this one is actually also possible. Sigh.

terrymah 2y ago

Oh man, don't get me started. This was a point in a talk I gave years ago called "Please Please Help the Compiler" (what I thought was a clever cut at the conventional wisdom at the time of "Don't Try to Help the Compiler") I work on MSVC backend. I argued pretty strenuously at the time that noexcept was costly and being marketed incorrectly. Perhaps the costs are worth it, but none the less there is a cost The reason is simple: there is a guarantee here that noexcept functions don't throw. std::terminate has to be called. That has to be implemented. There is some cost to that - conceptually every noexcept function (or worse, every call to a noexcept function) is surrounded by a giant try/catch(...) block. Yes there are optimizations here. But it's still not free Less obvious; how does inlining work? What happens if you inline a noexcept function into a function that allows exceptions? Do we now have "regions" of noexceptness inside that function (answer: yes). How do you implement that? Again, this is implementable, but this is even harder than the whole function case, and a naive/early implementation might prohibit inlining across degrees of noexcept-ness to be correct/as-if. And guess what, this is what early versions of MSVC did, and this was our biggest problem: a problem which grew release after release as noexcept permeated the standard library. Anyway. My point is, we need more backend compiler engineers on WG21 and not just front end, library, and language lawyer guys. I argued then that if instead noexcept violations were undefined, we could ignore all this, and instead just treat it as the pure optimization it was being marketed as (ie, help prove a region can't throw, so we can elide entire try/catch blocks etc). The reaction to my suggestion was not positive.

rockwotj 2y ago
Added to my list why I compile with -fno-exceptions
- jcelerier 2y ago
  -fno-exceptions only prevents you from calling throw. If you don't want overhead likely you want -fno-asynchronous-unwind-tables + that clang flag that specifies that extern "C" functions don't throw
  - 2y ago

tolmasky 2y ago

Is there any compiler option to have it yell at you if you mark something that can throw as `noexcept`, which seems to be the cause of (at least some of) the slowdowns where the compiler is forced to accommodate with `std::terminate`? I feel like these situations are more commonly mistakes, and not the user wanting to "collapse" exceptions into terminations. So the current approach to dealing with these cases seems to be suboptimal not only from a performance perspective, but a behavior perspective as well.

terrymah 2y ago
No, calling throw in a noexcept function is a defined behavior (call std::terminate), and that behavior is not a diagnostic I think maybe WG21 was concerned a compiler engineer would be clever if throwing in noexcept were UB, for example and assume any block that throws is unreachable and could just be removed along with all blocks it postdominates. Compiler guys love optimizations that just remove code. The fastest and smallest code is code that can’t run and doesn’t exist
- account42 2y ago
  Compilers are allowed to and do diagnose defined but undesirable behavior though, even more so if that's only enabled with an option.
  - terrymah 2y ago
    Until yesterday I thought I was the only person in the world who thought the designed behavior was undesirable, though

hoseja 2y ago

Apologize less for using completely benign standard macros. They are an okay tool if not abused.

shultays 2y ago

I can't find the explanation on why noexcept could hurt performance. One reason I can see itt is some containers like unordered_map can inline the hash along with the key with noexcept, which may not worth additional memory overhead if the hashing is relatively cheap. He talks a bit about it in "Intel+Windows+MSVC" but not much info. I wish there was noexcept helps in some cases that author doesn't seem to be using and any performance gain or loss is basically due to some (unrelated?) optimization decisions the compiler takes differently in noexcept builds if I am understanding correctly?

maccard 2y ago

This is super unrelated to the optimisation, and is just related to the cmake setup - instead of common.hpp having #ifdef USE_NOEXCEPT #define NOEXCEPT noexcept #else #define NOEXCEPT #endif and cmake being: if (WITH_NOEXCEPT) message(STATUS "Using `noexcept` annotations (faster?)") target_compile_definitions(PSRayTracing_StaticLibrary PUBLIC USE_NOEXCEPT) else() message(STATUS "Turned off use of `noexcept` (slower?)") endif() , the cmake could just be: if (WITH_NOEXCEPT) message(STATUS "Using `noexcept` annotations (faster?)") target_compile_definitions(PSRayTracing_StaticLibrary PUBLIC USE_NOEXCEPT=noexcept) else() message(STATUS "Turned off use of `noexcept` (slower?)") target_compile_definitions(PSRayTracing_StaticLibrary PUBLIC USE_NOEXCEPT=) endif() No need for these shared "common" config headers. Back on topic, this doesn't surprise me. There's this idea that C++ is fast, and that people who work with C++ are focused on optimisation and in my experience there's as many of these theoretical ideas about performance which aren't backed up by numbers, but are now ingrained in people. See https://news.ycombinator.com/item?id=41095814 https://news.ycombinator.com/item?id=41095814 from last week for another example of dogmatic guidelines having the wrong impact.

muth02446 2y ago

RE: unexpected performance degradation programs can be quite sensitive to how code is laid out because of cache line alignment, cache conflicts etc. So random changes can have a surprising impact. There was a paper a couple of years ago explaining this and how to measure compiler optimizations more reliably. Sadly, I do not recall the title/author.

Arech 2y ago
It would be super interesting to read the paper. Please post a link or some more details if you will remember them.