7 ms·

Yes, indeed it is the case with fused multiply-add operations, as I wrote below in another thread. By default at -O3, unfortunately, gcc and icc both take respo

by thxg 5y ago

Yes, indeed it is the case with fused multiply-add operations, as I wrote below in another thread. By default at -O3, unfortunately, gcc and icc both take responsibility for deciding whether or not to use FMA instructions when they see (a x b + c) expressions. As a result we can even see slightly different result across compiles!! However, as far as I understand, specifying a proper language standard (for example -std=c99, -std=c++14, etc.) restores sane behavior (see -ffast-math, and more specifically -fexcess-precision). Are there other cases where -O3 breaks IEEE754 on modern setups?

jcranmer 5y ago
Fast math flags are actually far more complicated in C/C++ compilers than just a simple -ffast-math flag, although I suppose most users may not be acquainted with the degree of control possible. In LLVM, for example, fast math flags are actually a set of 7 independently-toggled flags (no-signed-zero, no-infinity, no-NaN, allow FMAs, allow reassociation, allow approximate reciprocal, and allow approximate functions), that at the C/C++ level can be toggled either by command-line options or on a per-scope basis. ICC actually has two different levels of fast math [1]. The dark underside of IEEE 754 is that full support for it also requires features that most languages don't provide access for: rounding mode and exception handling (aka sticky bits). In C/C++, using these features correctly requires STDC FENV_ACCESS support which is only in ICC and recent clang (neither GCC nor MSVC support it). And there's usually two hardware bits for handling denormals (flush-to-zero and denormals-are-zero) that aren't in IEEE 754, which tend to be on by default when you're compiling for some targets (most notably GPUs). [1] https://software.intel.com/content/www/us/en/develop/documentation/cpp-compiler-developer-guide-and-reference/top/compiler-reference/compiler-options/compiler-option-details/floating-point-options/fp-model-fp.html https://software.intel.com/content/www/us/en/develop/documen... -- note fp-model=fast=[1|2]
- BeeOnRope 5y ago
  What is a "per scope basis"?
  - jcranmer 5y ago
    You can change it between any pair of curly braces. E.g.: void foo(int N) { for (int i = 0; i < N; i++) { #pragma STDC FENV_ACCESS ON // This applies only to the for loop... } // But not here, for example! }
wumpus 5y ago
> Yes, indeed it is the case with fused multiply-add operations, as I wrote below in another thread This is not always true. Some FMA operations round after the multiply, and some do not.

by thxg 5y ago

jcranmer 5y ago

Fast math flags are actually far more complicated in C/C++ compilers than just a simple -ffast-math flag, although I suppose most users may not be acquainted with the degree of control possible. In LLVM, for example, fast math flags are actually a set of 7 independently-toggled flags (no-signed-zero, no-infinity, no-NaN, allow FMAs, allow reassociation, allow approximate reciprocal, and allow approximate functions), that at the C/C++ level can be toggled either by command-line options or on a per-scope basis. ICC actually has two different levels of fast math [1]. The dark underside of IEEE 754 is that full support for it also requires features that most languages don't provide access for: rounding mode and exception handling (aka sticky bits). In C/C++, using these features correctly requires STDC FENV_ACCESS support which is only in ICC and recent clang (neither GCC nor MSVC support it). And there's usually two hardware bits for handling denormals (flush-to-zero and denormals-are-zero) that aren't in IEEE 754, which tend to be on by default when you're compiling for some targets (most notably GPUs). [1] https://software.intel.com/content/www/us/en/develop/documentation/cpp-compiler-developer-guide-and-reference/top/compiler-reference/compiler-options/compiler-option-details/floating-point-options/fp-model-fp.html https://software.intel.com/content/www/us/en/develop/documen... -- note fp-model=fast=[1|2]

BeeOnRope 5y ago
What is a "per scope basis"?
- jcranmer 5y ago
  You can change it between any pair of curly braces. E.g.: void foo(int N) { for (int i = 0; i < N; i++) { #pragma STDC FENV_ACCESS ON // This applies only to the for loop... } // But not here, for example! }

wumpus 5y ago

> Yes, indeed it is the case with fused multiply-add operations, as I wrote below in another thread This is not always true. Some FMA operations round after the multiply, and some do not.