27 ms·

The natural question is why doesn't Jank use MLIR?

pjmlp 4mo ago
No language using MLIR uses it directly out of the box, in that sense the right question is why did Jank not create their MLIR dialect.
- debugnik 4mo ago
  MLIR dialects have to be lowered into the basic LLVM one eventually, don't they? Does MLIR add anything over a custom IR for host languages that aren't deficient at manipulating data structures?
  - eigenspace 4mo ago
    'MLIR dialects' is just a term for teaching MLIR how to manipulate and understand your own custom IR. MLIR is just very good at producing good vectorized code in the presence of stuff like nested loops compared to LLVM or even some of the most carefully crafted custom compilers. It's not about whether your custom compiler is 'deficient' at handling data structures, MLIR is just genuinely very good at some of this stuff compared to basically anyone else. For most projects it's just more trouble than it's worth though, because maintaining and using an MLIR dialect definition is hard.
    - debugnik 4mo ago
      But AFAIK those aren't features of MLIR, but of lowering to existing MLIR dialects and running their passes. My genuine question is whether these passes provide any benefit before lowering, because otherwise a custom dialect doesn't add anything over lowering from a custom IR for anyone not using C++; and the only example I've seen is forced inlining.
erichocean 4mo ago
I suspect the author created his own IR after being offered that suggestion earlier, he's definitely aware it exists.
Jeaye 4mo ago
I spoke with a couple Clang and LLVM devs about MLIR when I was doing the original design for jank's IR. The general consensus was that MLIR added a great deal of complexity on top of designing/implementing an IR and nobody was confident it was actually worth the effort. Since I knew exactly what I wanted, I just built that.
- CalChris 4mo ago
  Your custom IR is above LLVM’s IR, correct? Is it like SwiftIR then? Maybe you could add a paragraph or two going through that design decision.
  - Jeaye 4mo ago
    jank's custom IR is completely separate and unrelated to LLVM IR, aside from both of them being SSA-based IRs. We go from jank's AST into jank's IR into C++, which we then give to Clang compile into the LLVM JIT runtime. So LLVM IR is used in the pipeline, but we don't touch it directly. More info on that, and a diagram, is here: https://book.jank-lang.org/dev/ir.html https://book.jank-lang.org/dev/ir.html (which I linked in the post)

pjmlp 4mo ago

No language using MLIR uses it directly out of the box, in that sense the right question is why did Jank not create their MLIR dialect.

debugnik 4mo ago
MLIR dialects have to be lowered into the basic LLVM one eventually, don't they? Does MLIR add anything over a custom IR for host languages that aren't deficient at manipulating data structures?
- eigenspace 4mo ago
  'MLIR dialects' is just a term for teaching MLIR how to manipulate and understand your own custom IR. MLIR is just very good at producing good vectorized code in the presence of stuff like nested loops compared to LLVM or even some of the most carefully crafted custom compilers. It's not about whether your custom compiler is 'deficient' at handling data structures, MLIR is just genuinely very good at some of this stuff compared to basically anyone else. For most projects it's just more trouble than it's worth though, because maintaining and using an MLIR dialect definition is hard.
  - debugnik 4mo ago
    But AFAIK those aren't features of MLIR, but of lowering to existing MLIR dialects and running their passes. My genuine question is whether these passes provide any benefit before lowering, because otherwise a custom dialect doesn't add anything over lowering from a custom IR for anyone not using C++; and the only example I've seen is forced inlining.

erichocean 4mo ago

I suspect the author created his own IR after being offered that suggestion earlier, he's definitely aware it exists.

Jeaye 4mo ago

I spoke with a couple Clang and LLVM devs about MLIR when I was doing the original design for jank's IR. The general consensus was that MLIR added a great deal of complexity on top of designing/implementing an IR and nobody was confident it was actually worth the effort. Since I knew exactly what I wanted, I just built that.

CalChris 4mo ago
Your custom IR is above LLVM’s IR, correct? Is it like SwiftIR then? Maybe you could add a paragraph or two going through that design decision.
- Jeaye 4mo ago
  jank's custom IR is completely separate and unrelated to LLVM IR, aside from both of them being SSA-based IRs. We go from jank's AST into jank's IR into C++, which we then give to Clang compile into the LLVM JIT runtime. So LLVM IR is used in the pipeline, but we don't touch it directly. More info on that, and a diagram, is here: https://book.jank-lang.org/dev/ir.html https://book.jank-lang.org/dev/ir.html (which I linked in the post)