5 ms·

No (sorry, I post this every time the Mythical Compiler Myth reappears), the problem with VLIW is that it fundamentally doesn’t work for anything with unpredict

by bri3d 2mo ago

No (sorry, I post this every time the Mythical Compiler Myth reappears), the problem with VLIW is that it fundamentally doesn’t work for anything with unpredictable memory access patterns, and modern general purpose computing has moved almost exclusively in this direction. For VLIW to work, you need to either guess correctly what is in cache or not have a cache at all; as soon as you mispredict what has been loaded, you stall while an OoO processor keeps going and a speculative OoO processor even keeps guessing. With multiple workloads on the same hardware (virtualization, multitasking, multitenancy) this becomes an intractable problem even in the presence of the magic compiler which can solve for software based unpredictability (branch likelihood and pointer chasing), because every context switch clobbers an unknown set of cache lines and blows the entire thing up. VLIW works for single workloads. It works exceptionally well for single workloads with no or explicit cache like DSP. You can trade the footprint and complexity from OoO for a wider execution unit and more SRAM. It works well for HPC, too, for the same reason. But for anything where more than one process exists, it just really doesn’t work, and that’s most modern workload. Itanium also has a unique set of self inflicted issues due in large part to Intel trying to make a wide variety of cross compatible parts, but IMO even if they’d got it right, it still would have died.

Joel_Mckay 2mo ago
We agree it probably would have still failed, but mostly it was the legacy code-motion compatibility/performance issues that were practically inescapable without refactoring millions of lines of code. gcc maintained the ia64 target a long time for unclear reasons, but it was also still inefficient on other platforms. The FOSS compiler worked, but that was its only performance metric that counted for many users. =3 Not sure why you think the Intel compilers were a myth, as they are still around working far better than gcc in many use-cases: "An Overview of the Intel® IA-64 Compiler" https://webdocs.cs.ualberta.ca/~amaral/courses/605/papers/IntelIA64Compiler.pdf https://webdocs.cs.ualberta.ca/~amaral/courses/605/papers/In...
- bri3d 2mo ago
  The myth I was referring to was the overarching theme that “VLIW would have been practical if only a better compiler existed,” which I don’t believe to be true for modern or Itanium-contemporary general purpose computing patterns.
  - Joel_Mckay 2mo ago
    Indeed, people are still bad a parallelism today, and most compilers still suck at reliably unrolling abstracted concurrent source intended functionality. Very few modern languages handle parallel scaling gracefully, and bodged on CUDA still isn't great either. As Moore's law ends, people have to reevaluate how they approach traditionally monolithic architectural design. =3
monocasa 2mo ago
Itanium had all sorts of specialized machinery so that you could issue a load that's required in the future, and then keep chugging away at the instruction stream, similar to what an OoO processor does with it's reordering logic. One example was "advanced loads" which allowed you to issue a load as soon as you knew the address, even in the face of potential pointer aliasing in the future, and then later complete the load when you actually hit a data dependency that requires it. https://devblogs.microsoft.com/oldnewthing/20150805-00/?p=91171 https://devblogs.microsoft.com/oldnewthing/20150805-00/?p=91... Another example is "speculative loads" which lets you issue a load before you even know if it's valid, such as unrolling a loop for an array that you don't know if is a multiple of the unrolled loop chunk length (and therefore might trap with a page fault if fully resolved). https://devblogs.microsoft.com/oldnewthing/20150804-00/?p=91181 https://devblogs.microsoft.com/oldnewthing/20150804-00/?p=91...
- bri3d 2mo ago
  These worked well for “known” intra-task pointer aliasing situations, but if you don’t know what will be in cache due to preemption of any kind, you still don’t know how many cycles the speculative loads will take, so you get the same stall risks across a dependency hazard.
  - monocasa 2mo ago
    OoO cores have load stalls too after preemptions. Even something like a Apple M core basically has to stall if the load has to go out to the memory controller. The goal is to move far enough ahead in the stream that you can at least issue the next load, and advaned loads lets you do that. Some Intel cores will occasionally speculatively predict zero for a load, but AFAIK that got turned off as part of the Spectre mitigations.
    - eigenform 2mo ago
      The difference is that this is not the programmer's responsibility in modern machines: instead we bake-in some hardware that watches the online state of the machine and then actively decides on what to do. If you're trying to create a compiler that approaches the effectiveness of this statically, you're condemned to do a ton of extra work (you're essentially writing an emulator for your CPU core, and then a compiler that uses that model to produce optimal code - even then, you might not be accounting for nondeterminism on the actual target machine, and some information is simply not accessible to you when you are not on the target machine)
seanmcdirmid 2mo ago
VLIW is being heavily used in some AI inference tasks (by Apple, AMD, Google, mainly in optimizing latency). It also is a nice way to do inference on edge devices that aren’t using constantly advancing models. It’s pretty much dead in general purpose computing though, and lacks the versatility of GPUs to go beyond inference.
imtringued 2mo ago
Honestly, you're conflating two things. 1. VLIW exposes microarchitectural details, locking them in like an ABI. Updates to the microrachitecture will require changes to the ISA, thereby breaking backwards compatibility with every generation. 2. The dominant programming paradigm is sequential code, often just old C code with heavy pointer aliasing and little compile time extractable parallelism. This reduces static parallelism in the code base and shifts it into a runtime problem. The CPU discovers the dependencies at runtime instead. If you built a programming language that exposes more parallelism (think something like ParaSail), this problem wouldn't be as big as it is for C style programming languages. Nobody will build a language for architectures that suffer from backwards compatibility problems, so why bother? VLIW is primarily suited for ASIPs and not much else.
- bri3d 2mo ago
  I don't think I'm really conflating anything, and I don't agree with your thesis even though I agree with individual points. 1) Yes, unless you add abstraction at the ABI/ISA level in hardware or microcode, which then defeats the point of VLIW to some extent. I mention this in my comment; Itanium in particular was hamstrung by trying to patch over this. This is one of the "sliders" you're dragging with VLIW; trading floor plan for cross compatibility. 2) Well, kind of, but this is just the Mythical Compiler I discussed in my own parent comment, regardless of language, and I find this argument tangental to the core issue. Yes, C makes things bad by encouraging both aliasing and pointer-chasing, but one can also argue that a sufficiently advanced compiler can discover the intra-program hazards and prefetch accordingly (see monocasa's discussion); this is moving where the parallelism is expressed, but isn't patching the fundamental issue. Overall, I don't think the issue is that nobody will build a language or compiler for an architecture with compatibility problems. We see good compilers and lots of language research in this space for DSP and AI workloads. I still believe that the fundamental issue is that VLIW is not suited for general purpose computing workloads due to cache residency issues caused by context switching.

by bri3d 2mo ago

Joel_Mckay 2mo ago

We agree it probably would have still failed, but mostly it was the legacy code-motion compatibility/performance issues that were practically inescapable without refactoring millions of lines of code. gcc maintained the ia64 target a long time for unclear reasons, but it was also still inefficient on other platforms. The FOSS compiler worked, but that was its only performance metric that counted for many users. =3 Not sure why you think the Intel compilers were a myth, as they are still around working far better than gcc in many use-cases: "An Overview of the Intel® IA-64 Compiler" https://webdocs.cs.ualberta.ca/~amaral/courses/605/papers/IntelIA64Compiler.pdf https://webdocs.cs.ualberta.ca/~amaral/courses/605/papers/In...

bri3d 2mo ago
The myth I was referring to was the overarching theme that “VLIW would have been practical if only a better compiler existed,” which I don’t believe to be true for modern or Itanium-contemporary general purpose computing patterns.
- Joel_Mckay 2mo ago
  Indeed, people are still bad a parallelism today, and most compilers still suck at reliably unrolling abstracted concurrent source intended functionality. Very few modern languages handle parallel scaling gracefully, and bodged on CUDA still isn't great either. As Moore's law ends, people have to reevaluate how they approach traditionally monolithic architectural design. =3

monocasa 2mo ago

Itanium had all sorts of specialized machinery so that you could issue a load that's required in the future, and then keep chugging away at the instruction stream, similar to what an OoO processor does with it's reordering logic. One example was "advanced loads" which allowed you to issue a load as soon as you knew the address, even in the face of potential pointer aliasing in the future, and then later complete the load when you actually hit a data dependency that requires it. https://devblogs.microsoft.com/oldnewthing/20150805-00/?p=91171 https://devblogs.microsoft.com/oldnewthing/20150805-00/?p=91... Another example is "speculative loads" which lets you issue a load before you even know if it's valid, such as unrolling a loop for an array that you don't know if is a multiple of the unrolled loop chunk length (and therefore might trap with a page fault if fully resolved). https://devblogs.microsoft.com/oldnewthing/20150804-00/?p=91181 https://devblogs.microsoft.com/oldnewthing/20150804-00/?p=91...

bri3d 2mo ago
These worked well for “known” intra-task pointer aliasing situations, but if you don’t know what will be in cache due to preemption of any kind, you still don’t know how many cycles the speculative loads will take, so you get the same stall risks across a dependency hazard.
- monocasa 2mo ago
  OoO cores have load stalls too after preemptions. Even something like a Apple M core basically has to stall if the load has to go out to the memory controller. The goal is to move far enough ahead in the stream that you can at least issue the next load, and advaned loads lets you do that. Some Intel cores will occasionally speculatively predict zero for a load, but AFAIK that got turned off as part of the Spectre mitigations.
  - eigenform 2mo ago
    The difference is that this is not the programmer's responsibility in modern machines: instead we bake-in some hardware that watches the online state of the machine and then actively decides on what to do. If you're trying to create a compiler that approaches the effectiveness of this statically, you're condemned to do a ton of extra work (you're essentially writing an emulator for your CPU core, and then a compiler that uses that model to produce optimal code - even then, you might not be accounting for nondeterminism on the actual target machine, and some information is simply not accessible to you when you are not on the target machine)

seanmcdirmid 2mo ago

VLIW is being heavily used in some AI inference tasks (by Apple, AMD, Google, mainly in optimizing latency). It also is a nice way to do inference on edge devices that aren’t using constantly advancing models. It’s pretty much dead in general purpose computing though, and lacks the versatility of GPUs to go beyond inference.

imtringued 2mo ago

Honestly, you're conflating two things. 1. VLIW exposes microarchitectural details, locking them in like an ABI. Updates to the microrachitecture will require changes to the ISA, thereby breaking backwards compatibility with every generation. 2. The dominant programming paradigm is sequential code, often just old C code with heavy pointer aliasing and little compile time extractable parallelism. This reduces static parallelism in the code base and shifts it into a runtime problem. The CPU discovers the dependencies at runtime instead. If you built a programming language that exposes more parallelism (think something like ParaSail), this problem wouldn't be as big as it is for C style programming languages. Nobody will build a language for architectures that suffer from backwards compatibility problems, so why bother? VLIW is primarily suited for ASIPs and not much else.

bri3d 2mo ago
I don't think I'm really conflating anything, and I don't agree with your thesis even though I agree with individual points. 1) Yes, unless you add abstraction at the ABI/ISA level in hardware or microcode, which then defeats the point of VLIW to some extent. I mention this in my comment; Itanium in particular was hamstrung by trying to patch over this. This is one of the "sliders" you're dragging with VLIW; trading floor plan for cross compatibility. 2) Well, kind of, but this is just the Mythical Compiler I discussed in my own parent comment, regardless of language, and I find this argument tangental to the core issue. Yes, C makes things bad by encouraging both aliasing and pointer-chasing, but one can also argue that a sufficiently advanced compiler can discover the intra-program hazards and prefetch accordingly (see monocasa's discussion); this is moving where the parallelism is expressed, but isn't patching the fundamental issue. Overall, I don't think the issue is that nobody will build a language or compiler for an architecture with compatibility problems. We see good compilers and lots of language research in this space for DSP and AI workloads. I still believe that the fundamental issue is that VLIW is not suited for general purpose computing workloads due to cache residency issues caused by context switching.