13 ms·

Please do not require AVX support for your software

KingLancelot 3y ago
[dead]
PaulHoule 3y ago
Apple switched to Apple Silicon precisely because Intel was dragging it's feet on acceleration and by doing so could fire customers like you. Seriously though, Intel has not been good at keeping all of its chips up to date with AVX and one consequence of that is that developers don't use it at all or they use a version that is 10 years out of date. If you look at those graphs in the article, single thread performance really is dying out there, the one thing Intel could offer to make their newer offering compelling is the latest version of AVX but they have been so hypnotized by smartphones that they just had to make a "big-little" clone that disable the most advanced AVX on the good cores so the weak cores not only waste your money, time and power, but actually sap performance from your strong cores. Customers buy AMD or Apple Silicon instead. Intel and their paid shills in the tech media will make as many excuses for this as AMZN will for why you can only get 5 day shipping with Prime, but all it means is 20 years from now some people will remember Intel the way we remember Sears or Atari or AMC.
- LoganDark 3y ago
  > they have been so hypnotized by smartphones that they just had to make a "big-little" clone that disable the most advanced AVX on the good cores so the weak cores not only waste your money, time and power, but actually sap performance from your strong cores. Absolutely, absolutely, amen. I'm going to hunt for a good 12600 with AVX-512 because my BIOS has an option to force-enable it—but I have to find a CPU that doesn't have the AVX-512 fused off on a hardware level (despite it being on the silicon).
rany_ 3y ago
This is, at its core a software distribution problem. Windows, Linux distros, etc. must have a way wherein the OS requests to download a more optimized version of the package. I think this is most easily fixed for Linux distros, all it takes is for them to create a new architecture, say amd64_avx, which only contains packages with avx optimizations enabled where applicable.
- basementcat 3y ago
  One complication is that AVX is not one but more than a dozen ISA extensions each of which may or may not be implemented on a particular processor. This means software delivered to a customer should ideally check CPUID at runtime to dispatch the appropriate processing kernel. https://en.m.wikipedia.org/wiki/AVX-512 https://en.m.wikipedia.org/wiki/AVX-512
  - dilippkumar 3y ago
    I also want to point out that there’s also the extreme-crazy option that is Gentoo linux. All packages are distributed as source and compiled on the destination machine before being installed there. And yes, you can modify build flags per package to enable/disable compile flags. I recommend doing it for fun. It’s a crazy world.
  - boulos 3y ago
    If someone just says "AVX" they usually mean AVX(1). And that's what the article is discussing. AVX-512 forking into tons of different beasts is a separate, but related, problem. But it's more like how SSE2, 3, 4, 4.1 and so on existed. Sometimes people said "SSE" and they might have meant one the later versions, but I don't hear the same statement with AVX, since people very explicitly seem to say AVX2 and AVX512.
  - rany_ 3y ago
    The article was referring to the original AVX1 not the subsequent variants. Likewise I was referring to the first set of extensions.
- vlmutolo 3y ago
  It also seems like Clickhouse could offer runtime detection of AVX and dispatch to the optimized functions in that case.
  - zX41ZdbW 3y ago
    Yes, here is the article about the techniques: https://maksimkita.com/blog/cpu-dispatch-in-clickhouse.html https://maksimkita.com/blog/cpu-dispatch-in-clickhouse.html
rany_ 3y ago
If anyone is interested, Intel has its own Linux distribution called Clear Linux which has all optimizations enabled. Of course this has the issue of not running on systems which cannot support all these optimizations, in particular, very few chips have AVX512 unless you have the latest 11th and 12th gen Intel CPUs.
- LoganDark 3y ago
  > very few chips have AVX512 unless you have the latest 11th and 12th gen Intel CPUs 12th gen had AVX-512 physically fused off because the E-cores didn't support it.
  - Cold_Miserable 3y ago
    No. AVX512 was disabled because the money-men want to force you to buy the far more expensive Sapphire-Rapids which are the same chip. Same old Intel. Learned nothing. They didn't just fuse it off, they also forced microcode updates that permanently disable it for working alderlakes.
    - LoganDark 3y ago
      It's almost as if people make these throwaway accounts just to post hot takes without affecting their real karma value.
- 7speter 3y ago
  Wasn’t avx-512 disabled rather shortly after alderlakes release?
HenrikB 3y ago
FWIW, the 'AVX' CPU feature is part of the x86-64-v3 level of the x86-64 microarchitecture standard [0]. There are chips produced in 2015 that are only supporting x86-64-v2 [0]. Also, according to [1], "The new server-class CPUs released in 2020 [2] do not implement the AVX instruction set." FWIW2, RedHat Enterprise Linux 9 (RHEL9) requires x86-64-v2 or newer [1]. So, as a reference, they decided to not yet require support for AVX. [0] https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels https://en.wikipedia.org/wiki/X86-64#Microarchitecture_level... [1] https://developers.redhat.com/blog/2021/01/05/building-red-hat-enterprise-linux-9-for-the-x86-64-v2-microarchitecture-level# https://developers.redhat.com/blog/2021/01/05/building-red-h... [2] https://www.intel.com/content/www/us/en/products/details/processors/atom/p.html https://www.intel.com/content/www/us/en/products/details/pro...
- pavel_odintsov 3y ago
  Thank you for sharing such great insights.
xet7 3y ago
Ok, I asked from MongoDB: https://jira.mongodb.org/browse/SERVER-74457 https://jira.mongodb.org/browse/SERVER-74457
zX41ZdbW 3y ago
Here is a discussion about introducing AVX by default in ClickHouse: https://github.com/ClickHouse/ClickHouse/issues/40459 https://github.com/ClickHouse/ClickHouse/issues/40459 TLDR: runtime CPU dispatching and JIT - ok, AVX by default - not ok.
- justsomehnguy 3y ago
  Amusingly this discussion led to SSE2-only build and discovery of some issues with their tests. Nothing serious, but still! https://github.com/ClickHouse/ClickHouse/pull/41498 https://github.com/ClickHouse/ClickHouse/pull/41498
jiggawatts 3y ago
"Please do not require racing slicks for your formula one cars, some of us are still racing with wooden wagon wheels and can't afford to upgrade to Bridgestone rubber tyres." The listed excuses are all in the category: "We've been doing things wrong, sometimes for a decade or more, please don't make us change our erroneous ways!" First of all, if anyone is using any kind of cloud hosting, then there is no excuse. Zero. None. All public clouds allow the choice of CPUs with instruction sets up to and including AVX-512. It's a dropdown menu. Go look at the options it has. You didn't, did you? You deployed your high performance data warehouse cluster (Clickhouse) with the default VM SKU, didn't you? Admit it. That's a pile of money you'd rather burning than admit that you are too lazy to even glance at the menu options when creating a $50K/month VM cluster.[1] The next one is the VMware Enhanced vMotion Compatibility (EVC), which allows specific CPU features to be masked out, enabling old servers to coexist with new servers in a single cluster. This too, is a setting that was almost certainly set once when the cluster was created, and nobody bothered to revisit the setting a decade later. I bet 95% VMware clusters with AVX masked are running on hardware 100% capable of using AVX. Again, this is leaving a ton of performance on the floor. Heck, I've seen brand new, uniform-hardware clusters with AVX-512 capable CPUs lobotomised down to SSE4.2 because admins have no clue what EVC actually does. I remember reading the same articles by admins just as lazy as the guy from that rant when 64-bit-only server software first appeared. "It's too hard to ask the SOE team to make a 64-bit SOE!", or "Our backup software from 1999 doesn't support 64-bit!", etc... [1] I watched several customers make this mistake, not just occasionally, but literally every time. We eventually had to block the 11-year-old VM SKUs using cloud policies to stop the unfathomable laziness.
- CommanderData 3y ago
  Quite an edge case but Graylog 5 requires MongoDb 5 which requires AVX. I was unable to get this setup on my 3 year old nas (920+) and had to resort to running an older version which will most likely stop getting updates very soon. AVX is old but apparently Intel decided to keep it out of certain line ups.
  - pavel_odintsov 3y ago
    Yep that's serious issue and it's similar to our case. Our main product can work just fine even without SSE 4.2 but MongoDB requires it and then indirectly leads to AVX1 support as we use MongoDB as storage. We did PoC with FerretDB last month and I think it may be good option for Gerylog: https://www.ferretdb.io https://www.ferretdb.io
thorncorona 3y ago
This entire article is just laziness. Including the data. I mean the graph provided is just incredibly outdated.. it gives hard data until 2010 and projects outwards.. charitably laziness, uncharitably cherry picked data. https://mlech26l.github.io/pages/2020/12/17/cpus.html https://mlech26l.github.io/pages/2020/12/17/cpus.html You can see there's about a 10-15% uplift in perf every year in single thread performance. Multi-thread increases performance significantly as 16+ core cpus become more common.
olavgg 3y ago
I have still a Intel Westmere cpu running, which is the CPU iteration before Sandy Bridge. Honestly this platform is outdated, its inefficient and slow. It's not even cheap, considering you can get Broadwell/Epyc Rome cheap from Ebay today. I'm just running it, because I have not had time to migrate yet and it works. I could probably run it for another 10 years, as long I don't update anything. However if Linux should require a new CPU generation, I would quickly migrate it to my Epyc Rome VM server. No questions asked. A 10 years old CPU is outdated, its single core performance may be ok. But this generation had 4-6 cores max and sucked 100watts at full load, like 25 watt per cor. My Epyc uses 2 watt per core and is also faster per core.
rpep 3y ago
I dont much like this article since SIMD is not “niche” as the author says, even if people aren’t aware of it. It’s hard to pick up a single piece of software and say that “yes my overall workload overall will improve” by turning it on, but if you are aware of it, it will speed up most non-branching loop operations via auto-vectorisation. Having had to work on software that did runtime dispatch for SIMD - there is a small performance cost to this. For most software it’s just not worth it. Compiling multiple versions is preferable but more confusing for users, and more costly for the developer in CI and build time. There are therefore good reasons not to support multiple microarchitectures and picking one released 12 years ago is a good compromise. I wouldn’t at all advocate picking AVX512 but this is not that - almost all consumer and professional grade hardware sold in the last 10 years supports AVX, and it’s unreasonable to expect vendors to continue supporting an ever shrinking niche…
- LeoNatan25 3y ago
  “Runtime dispatch” does not require that the CPU feature be queried on each call. There are designs that can do a single query on launch and set up the environment so that future calls do not incur any additional penalty.
SleepyMyroslav 3y ago
Games used to shy away from AVX support for various reasons. Tbh even SSE4.2 was in bad shape due to AMD chips that were before Ryzen. But not anymore. With hardware requirements of "new generation only" games are going to require AVX and many even AVX2. This is the only practical way to have consistency between platforms. Especially when crossplay is the norm. Idea that software should be 'portable everywhere' is impractical for games.
exabrial 3y ago
Seems like JIT to the rescue here?
PeterZaitsev 3y ago
I think this article gives poor advice. You should provide binaries which provide best experience for magiority of your users. Old CPUs, Old Operating systems become edge cases at certain point which has cost to maintain which should be considered wisely. It might be good option to provide alternative "legacy" binaries or enable users to compile their own binaries if users like based on project resources Too many folks in particular in Open Source are very forceful about maintainers meeting their needs but not understanding resources can be limited and if you're sayting yes to something you're sayting no to something else - new features, better quality for majority of users etc.
johnklos 3y ago
Sometimes it's interesting to read the comments on articles like these. Many if not most of the comments which lean towards ignoring support for older or lesser CPUs don't seem to care about much more than sheer popularity, as though something that 90% of the world has should become the de facto standard. That's the same logic that used to be used to rationalize not supporting free Unix / Linux. Why do people who have money and fancy, sometimes expensive things so often insist on getting even more, when getting more means that those without the means to afford better get less? I suppose we could fix the planet if we could figure out how to deal with this. So, in a nutshell, I dismiss those who have fancy CPUs and who insist that everyone use AVX, even when it affects those who can't. Really, though, the answer is quite simple: run-time code paths for various CPU feature sets. This problem would be moot if programmers just programmed and let compilers and assemblers output AVX optimized code for CPUs that have it and regular x86 assembly for those that don't, in the same binary.

[dead]

Apple switched to Apple Silicon precisely because Intel was dragging it's feet on acceleration and by doing so could fire customers like you. Seriously though, Intel has not been good at keeping all of its chips up to date with AVX and one consequence of that is that developers don't use it at all or they use a version that is 10 years out of date. If you look at those graphs in the article, single thread performance really is dying out there, the one thing Intel could offer to make their newer offering compelling is the latest version of AVX but they have been so hypnotized by smartphones that they just had to make a "big-little" clone that disable the most advanced AVX on the good cores so the weak cores not only waste your money, time and power, but actually sap performance from your strong cores. Customers buy AMD or Apple Silicon instead. Intel and their paid shills in the tech media will make as many excuses for this as AMZN will for why you can only get 5 day shipping with Prime, but all it means is 20 years from now some people will remember Intel the way we remember Sears or Atari or AMC.

LoganDark 3y ago
> they have been so hypnotized by smartphones that they just had to make a "big-little" clone that disable the most advanced AVX on the good cores so the weak cores not only waste your money, time and power, but actually sap performance from your strong cores. Absolutely, absolutely, amen. I'm going to hunt for a good 12600 with AVX-512 because my BIOS has an option to force-enable it—but I have to find a CPU that doesn't have the AVX-512 fused off on a hardware level (despite it being on the silicon).

rany_ 3y ago

This is, at its core a software distribution problem. Windows, Linux distros, etc. must have a way wherein the OS requests to download a more optimized version of the package. I think this is most easily fixed for Linux distros, all it takes is for them to create a new architecture, say amd64_avx, which only contains packages with avx optimizations enabled where applicable.

basementcat 3y ago
One complication is that AVX is not one but more than a dozen ISA extensions each of which may or may not be implemented on a particular processor. This means software delivered to a customer should ideally check CPUID at runtime to dispatch the appropriate processing kernel. https://en.m.wikipedia.org/wiki/AVX-512 https://en.m.wikipedia.org/wiki/AVX-512
- dilippkumar 3y ago
  I also want to point out that there’s also the extreme-crazy option that is Gentoo linux. All packages are distributed as source and compiled on the destination machine before being installed there. And yes, you can modify build flags per package to enable/disable compile flags. I recommend doing it for fun. It’s a crazy world.
- boulos 3y ago
  If someone just says "AVX" they usually mean AVX(1). And that's what the article is discussing. AVX-512 forking into tons of different beasts is a separate, but related, problem. But it's more like how SSE2, 3, 4, 4.1 and so on existed. Sometimes people said "SSE" and they might have meant one the later versions, but I don't hear the same statement with AVX, since people very explicitly seem to say AVX2 and AVX512.
- rany_ 3y ago
  The article was referring to the original AVX1 not the subsequent variants. Likewise I was referring to the first set of extensions.
vlmutolo 3y ago
It also seems like Clickhouse could offer runtime detection of AVX and dispatch to the optimized functions in that case.
- zX41ZdbW 3y ago
  Yes, here is the article about the techniques: https://maksimkita.com/blog/cpu-dispatch-in-clickhouse.html https://maksimkita.com/blog/cpu-dispatch-in-clickhouse.html

rany_ 3y ago

If anyone is interested, Intel has its own Linux distribution called Clear Linux which has all optimizations enabled. Of course this has the issue of not running on systems which cannot support all these optimizations, in particular, very few chips have AVX512 unless you have the latest 11th and 12th gen Intel CPUs.

LoganDark 3y ago
> very few chips have AVX512 unless you have the latest 11th and 12th gen Intel CPUs 12th gen had AVX-512 physically fused off because the E-cores didn't support it.
- Cold_Miserable 3y ago
  No. AVX512 was disabled because the money-men want to force you to buy the far more expensive Sapphire-Rapids which are the same chip. Same old Intel. Learned nothing. They didn't just fuse it off, they also forced microcode updates that permanently disable it for working alderlakes.
  - LoganDark 3y ago
    It's almost as if people make these throwaway accounts just to post hot takes without affecting their real karma value.
7speter 3y ago
Wasn’t avx-512 disabled rather shortly after alderlakes release?

HenrikB 3y ago

FWIW, the 'AVX' CPU feature is part of the x86-64-v3 level of the x86-64 microarchitecture standard [0]. There are chips produced in 2015 that are only supporting x86-64-v2 [0]. Also, according to [1], "The new server-class CPUs released in 2020 [2] do not implement the AVX instruction set." FWIW2, RedHat Enterprise Linux 9 (RHEL9) requires x86-64-v2 or newer [1]. So, as a reference, they decided to not yet require support for AVX. [0] https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels https://en.wikipedia.org/wiki/X86-64#Microarchitecture_level... [1] https://developers.redhat.com/blog/2021/01/05/building-red-hat-enterprise-linux-9-for-the-x86-64-v2-microarchitecture-level# https://developers.redhat.com/blog/2021/01/05/building-red-h... [2] https://www.intel.com/content/www/us/en/products/details/processors/atom/p.html https://www.intel.com/content/www/us/en/products/details/pro...

pavel_odintsov 3y ago
Thank you for sharing such great insights.

xet7 3y ago

Ok, I asked from MongoDB: https://jira.mongodb.org/browse/SERVER-74457 https://jira.mongodb.org/browse/SERVER-74457

zX41ZdbW 3y ago

Here is a discussion about introducing AVX by default in ClickHouse: https://github.com/ClickHouse/ClickHouse/issues/40459 https://github.com/ClickHouse/ClickHouse/issues/40459 TLDR: runtime CPU dispatching and JIT - ok, AVX by default - not ok.

justsomehnguy 3y ago
Amusingly this discussion led to SSE2-only build and discovery of some issues with their tests. Nothing serious, but still! https://github.com/ClickHouse/ClickHouse/pull/41498 https://github.com/ClickHouse/ClickHouse/pull/41498

jiggawatts 3y ago

"Please do not require racing slicks for your formula one cars, some of us are still racing with wooden wagon wheels and can't afford to upgrade to Bridgestone rubber tyres." The listed excuses are all in the category: "We've been doing things wrong, sometimes for a decade or more, please don't make us change our erroneous ways!" First of all, if anyone is using any kind of cloud hosting, then there is no excuse. Zero. None. All public clouds allow the choice of CPUs with instruction sets up to and including AVX-512. It's a dropdown menu. Go look at the options it has. You didn't, did you? You deployed your high performance data warehouse cluster (Clickhouse) with the default VM SKU, didn't you? Admit it. That's a pile of money you'd rather burning than admit that you are too lazy to even glance at the menu options when creating a $50K/month VM cluster.[1] The next one is the VMware Enhanced vMotion Compatibility (EVC), which allows specific CPU features to be masked out, enabling old servers to coexist with new servers in a single cluster. This too, is a setting that was almost certainly set once when the cluster was created, and nobody bothered to revisit the setting a decade later. I bet 95% VMware clusters with AVX masked are running on hardware 100% capable of using AVX. Again, this is leaving a ton of performance on the floor. Heck, I've seen brand new, uniform-hardware clusters with AVX-512 capable CPUs lobotomised down to SSE4.2 because admins have no clue what EVC actually does. I remember reading the same articles by admins just as lazy as the guy from that rant when 64-bit-only server software first appeared. "It's too hard to ask the SOE team to make a 64-bit SOE!", or "Our backup software from 1999 doesn't support 64-bit!", etc... [1] I watched several customers make this mistake, not just occasionally, but literally every time. We eventually had to block the 11-year-old VM SKUs using cloud policies to stop the unfathomable laziness.

CommanderData 3y ago
Quite an edge case but Graylog 5 requires MongoDb 5 which requires AVX. I was unable to get this setup on my 3 year old nas (920+) and had to resort to running an older version which will most likely stop getting updates very soon. AVX is old but apparently Intel decided to keep it out of certain line ups.
- pavel_odintsov 3y ago
  Yep that's serious issue and it's similar to our case. Our main product can work just fine even without SSE 4.2 but MongoDB requires it and then indirectly leads to AVX1 support as we use MongoDB as storage. We did PoC with FerretDB last month and I think it may be good option for Gerylog: https://www.ferretdb.io https://www.ferretdb.io

thorncorona 3y ago

This entire article is just laziness. Including the data. I mean the graph provided is just incredibly outdated.. it gives hard data until 2010 and projects outwards.. charitably laziness, uncharitably cherry picked data. https://mlech26l.github.io/pages/2020/12/17/cpus.html https://mlech26l.github.io/pages/2020/12/17/cpus.html You can see there's about a 10-15% uplift in perf every year in single thread performance. Multi-thread increases performance significantly as 16+ core cpus become more common.

olavgg 3y ago

I have still a Intel Westmere cpu running, which is the CPU iteration before Sandy Bridge. Honestly this platform is outdated, its inefficient and slow. It's not even cheap, considering you can get Broadwell/Epyc Rome cheap from Ebay today. I'm just running it, because I have not had time to migrate yet and it works. I could probably run it for another 10 years, as long I don't update anything. However if Linux should require a new CPU generation, I would quickly migrate it to my Epyc Rome VM server. No questions asked. A 10 years old CPU is outdated, its single core performance may be ok. But this generation had 4-6 cores max and sucked 100watts at full load, like 25 watt per cor. My Epyc uses 2 watt per core and is also faster per core.

rpep 3y ago

I dont much like this article since SIMD is not “niche” as the author says, even if people aren’t aware of it. It’s hard to pick up a single piece of software and say that “yes my overall workload overall will improve” by turning it on, but if you are aware of it, it will speed up most non-branching loop operations via auto-vectorisation. Having had to work on software that did runtime dispatch for SIMD - there is a small performance cost to this. For most software it’s just not worth it. Compiling multiple versions is preferable but more confusing for users, and more costly for the developer in CI and build time. There are therefore good reasons not to support multiple microarchitectures and picking one released 12 years ago is a good compromise. I wouldn’t at all advocate picking AVX512 but this is not that - almost all consumer and professional grade hardware sold in the last 10 years supports AVX, and it’s unreasonable to expect vendors to continue supporting an ever shrinking niche…

LeoNatan25 3y ago
“Runtime dispatch” does not require that the CPU feature be queried on each call. There are designs that can do a single query on launch and set up the environment so that future calls do not incur any additional penalty.

SleepyMyroslav 3y ago

Games used to shy away from AVX support for various reasons. Tbh even SSE4.2 was in bad shape due to AMD chips that were before Ryzen. But not anymore. With hardware requirements of "new generation only" games are going to require AVX and many even AVX2. This is the only practical way to have consistency between platforms. Especially when crossplay is the norm. Idea that software should be 'portable everywhere' is impractical for games.

exabrial 3y ago

Seems like JIT to the rescue here?

PeterZaitsev 3y ago

I think this article gives poor advice. You should provide binaries which provide best experience for magiority of your users. Old CPUs, Old Operating systems become edge cases at certain point which has cost to maintain which should be considered wisely. It might be good option to provide alternative "legacy" binaries or enable users to compile their own binaries if users like based on project resources Too many folks in particular in Open Source are very forceful about maintainers meeting their needs but not understanding resources can be limited and if you're sayting yes to something you're sayting no to something else - new features, better quality for majority of users etc.

johnklos 3y ago

Sometimes it's interesting to read the comments on articles like these. Many if not most of the comments which lean towards ignoring support for older or lesser CPUs don't seem to care about much more than sheer popularity, as though something that 90% of the world has should become the de facto standard. That's the same logic that used to be used to rationalize not supporting free Unix / Linux. Why do people who have money and fancy, sometimes expensive things so often insist on getting even more, when getting more means that those without the means to afford better get less? I suppose we could fix the planet if we could figure out how to deal with this. So, in a nutshell, I dismiss those who have fancy CPUs and who insist that everyone use AVX, even when it affects those who can't. Really, though, the answer is quite simple: run-time code paths for various CPU feature sets. This problem would be moot if programmers just programmed and let compilers and assemblers output AVX optimized code for CPUs that have it and regular x86 assembly for those that don't, in the same binary.