ribit

36 ms·

1.
by ribit 1y ago
Yep. My wife just started as a professor (humanities) and she entered on H1B visa last week, as green card takes years to obtain. I have been offered a teaching job at the same institution as a partner hire and they have filed an H1B petiti
2.
by ribit 2y ago
Execution with masking is pretty much how broaching works on GPUs. What’s more relevant however is that conditional statements add overhead on terms of additional instructions and execution state management. Eliminating small branches using
3.
by ribit 2y ago
Quick note: I looked at the bindless proposal linked from the blog post and their description of Metal is quite outdated. MTLArgumentEncoder has been deprecated for a while now, the layout is a transparent C struct that you populate at will
4.
by ribit 2y ago
No.
5.
by ribit 2y ago
M3 GPU uses a new instruction encoding, among other things. Also, it has a new memory partitioning scheme (aka. Dynamic Caching), which probably requires a bunch of changes to both the driver interface and the shader compiler. I hope the As
6.
by ribit 2y ago
Are you talking about Vulkan or about geometry shaders? The later is simple: because geometry shaders are a badly designed feature that sucks on modern GPUs. Apple has designed Metal to only support things that are actually fast. Their solu
7.
by ribit 2y ago
Apple not supporting Vulkan is a business decision. They wanted a lean and easy to learn API that they can quickly iterate upon, and they want you to optimize for their hardware. Vulkan does not cater to either of these goals. Interestingly
8.
by ribit 2y ago
Have you looked at the Swift error model? I really like their design. They use a dedicated try statement to mark call sites that can fail — note that try is not the same as try...catch — Swift has an additional block construct for catching
9.
by ribit 2y ago
Is this really a new approach? On a cursory look this seems like implicit error propagation with checked exceptions. I am Also curious about authors presentation of the topic. To me, an important feature of error handling design is whether
10.
by ribit 2y ago
While I understand the argument, it would also be good to see some empirical evidence. So far all x86 built need more power to reach the same performance level as ARM. Of course, Apple is still the outlier.
11.
by ribit 2y ago
> Yes, that my understanding, and that's why I claim it's different from "classical" SIMD I understand, yes, it makes sense. Of course, other architectures can make other optimizations, like selecting warps that are m
12.
by ribit 2y ago
Most NPUs are not directly end-user programmable. The vendor usually provides a custom SDK that allows you to run models created with popular frameworks on their NPUs. Apple is a good example since they have been doing it for a while. They
13.
by ribit 2y ago
> Not sure what you mean by lockstep here. When an operand-collector entry is ready it dispatch it to execute as soon as possible (write arbitration aside) even if other operand-collector entries from the same warp are not ready yet (so
14.
by ribit 2y ago
Modern GPUs are exposing the SIMD behind the SIMT model and heavily investing into SIMD features such as shuffles, votes, and reduces. This leads to an interesting programming model. One interesting challenge is that flow control is done ve
15.
by ribit 2y ago
In an operand-collector architecture the threads are still executed in lockstep. I don't think this makes the basic architecture less "SIMD-y". Operand collectors are a smart way to avoid multi-ported register files, which en
16.
by ribit 2y ago
How would you envision that working at the hardware level? GPUs are massively parallel devises, they need to keep the scheduler and ALU logic as simple and compact as possible. SIMD is a natural way to implement this. In real world, SIMT is
17.
by ribit 2y ago
You need to consider this in the context of the relevant task. Nvidia GPUs have extremely high peak performance for GEMM, but when working with LLMs, bandwidth (and RAM capacity) becomes the limiting factor. There is a reason why real ML-fo
18.
by ribit 2y ago
I remember last year (?) Quancomm proposing an ISA extension that brings ARM-like addressing modes and paired stores to RISC-V, and the community reaction being very negative. Happy to hear that there are now initiatives to streamline these
19.
by ribit 2y ago
I fully support the idea of open instruction sets. I am not as much sold on the idea of cookie-cutter one-size-fits-all instruction sets. RISC-V is very nice for teaching CPU basics, and it is a great fit for tiny cores or specialized mic
20.
by ribit 2y ago
Bugs notwithstanding (which I agree are a significant concern for Metal), I'd frankly much prefer to work with a well-designed, streamlined API like Metal instead of a needlesly verbose and complex Vulkan.
21.
by ribit 2y ago
> Note that compact/compress are not actually the key enablers: also with AVX-512 we use table lookups for u64 keys, because this allows us to actually partition a vector and write it both to the left and write sides, as opposed to
22.
by ribit 2y ago
> Perhaps, though on VQSort it was more like 50% the performance. I looked at the paper and my interpretation is that the performance delta between M1 (Neon) and the Xeon (AVX2) can be fully explained by the difference in clock (3.7 vs 3
23.
by ribit 2y ago
> I agree with you we do not only want "very wide SIMD", and it seems to me that 2x512-bit (Intel) or 4x256 (AMD) are actually a good middle ground. I'd already classify this as "very wide". And the story is far
24.
by ribit 2y ago
If you do streaming-type operations on long arrays, yes. If your data sizes are small, however, four smaller units might be more flexible. As a naive example, let's take the popular SIMD acceleration of hash tables. Since the key is li
25.
by ribit 2y ago
Yes, OoO is expensive — after all, that is the cost of performance. Very wide SIMD is great for energy efficiency if that is what your compute patterns require (there is a good reason why GPUs are in-order very wide SMT SIMD processors). Is
26.
by ribit 2y ago
I don't see how this would work out beneficially. Let's say your hardware can join 4x128b units as a virtual 512-bit SVE SIMD unit. This means you have to advertise VL as 512bit for reasons of consistency. Yes, you will save some
27.
by ribit 2y ago
I am curious, which SVE instructions imply variable execution time? I’d guess that first fault load could be tricky to implement…
28.
by ribit 2y ago
What do you mean? Apple is the only one who has an SME/SSVE implementation.
29.
by ribit 2y ago
My guess is that Apple is simply not interested in some of the ARMv9 features. They are not eager to implement SVE and the se Ure virtualization features are probably not that relevant to them.
30.
by ribit 2y ago
No, per-clock performance improvements between M3 and M4 range from 0% to 20%, this is ignoring the two subtests that benefit from SME. That Twitter post is moot. GB results show high variation, it is easy enough to cherry pick pairs of re

More ›