6 ms·
auto-vectorization is not nearly as good as you would hope it to be. The best SIMD optimizations likely require changing your data format from AoS to SoA.
by forrestthewoods 2mo ago
auto-vectorization is not nearly as good as you would hope it to be.
The best SIMD optimizations likely require changing your data format from AoS to SoA.
- raegis 2mo agoWhat are AoS and SoA?
- nylonstrung 2mo agoArray of Structs and Struct of Arrays
- Georgelemental 2mo agoArray of Structs and Struct of Arrays https://en.wikipedia.org/wiki/AoS_and_SoA https://en.wikipedia.org/wiki/AoS_and_SoA
- Rendello 2mo agoA good introduction to SoA (for anyone curious) are the two most famous Data-Oriented Design talks by Mike Acton (game engine dev) [1] and Andrew Kelley (Zig lead dev) [2] respectively. I read a book book about DoD [3] really which confused me at first with all its talk about database table design (in a book about a high-performance C++ game engine?), but when it finally clicked it was amazing. The point is that you want to think hard about your access patterns and what could constitute good "primary keys", then model it accordingly. SoA ends up being useful a lot of the time, because having your data in homogeneous arrays/vectors is great for cache locality and branch elimination. Even without SIMD you can get huge speedups from that, but that's also where your compiler (or you as a programmer) can get incredible SIMD gains. SoA is not a silver bullet as it may not align well with your access patterns, but it can great to add to your toolkit. --- Mike Acton: Data-Oriented Design and C++: https://www.youtube.com/watch?v=rX0ItVEVjHc https://www.youtube.com/watch?v=rX0ItVEVjHc Andrew Kelley: A Practical Guide to Applying Data Oriented Design: https://www.youtube.com/watch?v=IroPQ150F6c https://www.youtube.com/watch?v=IroPQ150F6c Richard Fabian: Data-Oriented Design: https://www.dataorienteddesign.com/dodbook/ https://www.dataorienteddesign.com/dodbook/
- formerly_proven 2mo agoAnd -march=native or at least -march=x86-64-v3 or similar, alternatively identifying relevant functions and manually invoking FMV and uarch specialization via target_clones. Plus non-integer code can generally not be autovectorized in normal-math mode since FP is non-commutative.
- nylonstrung 2mo agoThe one feature in Jonathan Blow's Jai language I really envy is a a single keyword to switch AoS to SoA and visa-versa at comptime
- mbStavola 2mo agoDidn't he drop this feature years ago?
- Joker_vD 2mo agoWell, then I just prompt Claude and get SIMD without having to learn it /s
- ethin 2mo agoEither this or you have to do special tricks like pairwise tree reductions and hand-unroll certain portions of loops.
- pjmlp 2mo agoWhile C++ may be reaching levels of Algol 68, PL/I complexity, with C++26 reflection you can do automatically. See https://github.com/cern-nextgen/reflmempp https://github.com/cern-nextgen/reflmempp
- forrestthewoods 2mo agoThat’s a research project. So no.
- exDM69 2mo ago> The best SIMD optimizations likely require changing your data format from AoS to SoA. We do have gather load instructions in SIMD instruction sets these days (AVX2 and newer), so AoS vs SoA is not nearly as important as it was once. Scatter stores are also available but only in newer CPUs.
- inigyou 2mo agoIt remains extremely important. Gather loads are much more expensive than sequential loads, for obvious reasons.