n7g

6 ms·

1.
Asura: Looped Language Models done better(neel04.github.io)
2 points by n7g 7mo ago|1 comments
2.
by n7g 7mo ago
Recursive models like TRM/CTM/UT have create a lot of buzz lately. But they're rarely used outside of static, toy domains - especially language. In 2018, we saw "Universal Transformers" try this. However, follow-up
3.
by n7g 2y ago
> There are plenty of compilers that place restrictions that you barely notice. gcc, clang, nvcc -- they're fairly flexible, and "dynamic" In the context of scientific computing - this is completely, blatantly false. We&#x
4.
by n7g 2y ago
I haven't used TinyGrad but I'm not really sure what its goal is. To be the best autograd framework? to be a minimal one? I'm glad they've removed the (rather arbitrary, and admittedly stupid) loc cap. And from the littl
5.
by n7g 2y ago
Hey patrick, love your work! I think the biggest, well "con" I've seen is non-technical - the fear of JAX being killed by Google. I mention in the blog as well [here]( https://neel04.github.io/my-website/b
6.
by n7g 2y ago
> with torch.compile the main advantage of jax is disappearing. Interesting take - I agree here somewhat. But also, wouldn't you think a framework that has been from the ground-up designed around a specific, mature compiler stack be
7.
by n7g 2y ago
> In another timeline AI would have made Lua popular. I wonder if it'd have been hated more than Python is - especially with the 1-based indexing...
8.
by n7g 2y ago
Hey, thanks for actually engaging with the blog's points instead of "Google kills everything it touches" :) 1. I'm well aware of the PyTorch stack, but this point: > PyTorch is building towards a multi-backend future
9.
by n7g 2y ago
Curious to know how the OP is biased when they have no conflict of interest, and they explicitly mention the `torch.compile` stack like... a few dozen times in the blog?