> I know that. I also know that JITs cannot optimize to the same amount as LLVM due to the time limit Yeah, this is not true, and there's no time limit. I mean

> I know that. I also know that JITs cannot optimize to the same amount as LLVM due to the time limit Yeah, this is not true, and there's no time limit. I mean, maybe some JIT compilers, like JavaScript's have a time limit, but their goal is to run JS at an acceptable speed. Java's JIT is intended to reduce the runtime overheads of AOT compilers, and the only way to do that is by optimising significantly more than AOT compilers, obviously not less (otherwise, we'd just always use an AOT compiler). You can easily see why there's no time limit if you understood how Java's optimising JIT works. First, code is run in the interpreter and some profiles are collected, then a non-optimising JIT runs and continues to collect profile, and finally the optimising JIT runs. The vast majority of the time is spent waiting for profiles to collect, and so if compilation itself runs, say, even 3x slower, it won't even be perceptible. Also, because we have profiles, we don't have to compile much of the program at all, because we know what the hot spots are. Initialisation code that runs once is never compiled (remember, the focus is long-running programs, exactly those that low-level languages have trouble with). Finally, the reason sophisticated JIT compilers can optimise more - which is why they're used in the first place - is thanks to speculative optimisation. AOT compilers need to spend a lot of time on optimisation, and even then they are limited, because they need to prove that the program transformation is valid (i.e. that there's no miscompilation). The power of JIT compilers is that they don't. They only need to speculate that a certain profile will continue to be in effect. So if so far some virtual call always hits a certain target, they can go ahead and inline it (not only to the cost of a regular call, but to no call at all, and then they optimise the whole inlined code). If they're wrong, a fault triggers and they decompile the relevant subroutine going back to the interpreter and non-optimising compiler. > and that C++ and Rust are allocating much, much less than Java and even C# or Go, so a faster allocation scheme is much less needed there This is true, but the causaility here is that the reason we avoid allocation in C++ is precisely because it's so slow. > but even in those cases it's usually possible to alleviate the costs with wise organization of allocations The problem is that this is true in principle. In practice this is certainly true in smaller programs. In larger programs, this work is not easy at all, and you find yourself doing harder and harder work just to keep up. > including using arenas etc. in some places One of the reasons I'm excited about Zig (I'm a low-level programmer) is that it makes arenas much more viable. Arenas in C++ and especially Rust are not really a pleasure to work with, and they're viral and a constant maintenance burden. BTW, the reason moving GCs are so fast is that they work quite similarly to arenas. > and they're also offloaded from the better-optimizing compiler. It's a worse-optimising compiler. In C++, I use templates to achieve similar optimisation to what Java does, and in Zig I can use comptime, and again, it's certainly possible but it's hard work. You can't let the templates explode all over the codebase, and, as it evolves, you have to go back and profile and take out the ones that no longer help, replacing them with new ones. Just to tell you a bit about me, I was a C++ programmer for many years, and when Java showed up, like many, I was sceptical. When I saw that the JIT + moving collector hypothesis actually accomplishes its goal in reducing the overheads we were seeing in C++ in many situations, I went to work on the JVM. Back then there were still latency tradeoffs due to GC pauses, but GC pauses no longer exist as of three years ago. Now, a lot of people, including some of the world's top compiler and memory management experts, believe that the vision of using JITs and moving collectors to address the performance problem of low-level languages is working exceedingly well. We can certainly argue about under which conditions Java wins and under which C++ (or Zig it Rust etc.) win and how common they are, but people who think low-level languages win across the board or almost across the board clearly don't know what's going on. Early on it was people who were sceptical about how effectively JITs and moving collectors could do their job in practice (even though the theory was clear), but these days I think it's mostly people who haven't struggled with performance issues in low-level languages long enough, and just see that for small or young programs they work fine. They always were. Writing a new program in C++ was never harder than writing a new program in Java, and the performance was great (and people weren't concerned about memory safety in particular). The problems came later - in the 5th year, the 10th year, etc.., when the cost of evolution and trying to keep performance good were piling up.