terafo

8 ms·

1.
by terafo 8mo ago
Dynamic libraries are a dumpster fire with how they are implemented right now, and I'd really prefer everything to be statically linked. But ideally, I'd like to see exploration of a hybrid solution, where library code is tagged i
2.
by terafo 9mo ago
This article specifically talks about PC laptops and discusses changes in them.
3.
by terafo 9mo ago
Having access to the text and being trained on the text are two different things.
4.
by terafo 9mo ago
There are LLM's that can process 1 million token context window. Amazon Nova 2 for one, even though it's definitely not the highest quality model. You just put whole book in context and make LLM answer questions about it. And give
5.
by terafo 1y ago
Yes
6.
by terafo 1y ago
MLA uses way more flops in order to conserve memory bandwidth, H20 has plenty of memory bandwidth and almost no flops. MLA makes sense on H100/H800, but on H20 GQA-based models are a way better option.
7.
by terafo 2y ago
https://www.youtube.com/watch?v=b2F-DItXtZs
8.
by terafo 2y ago
Because you have to do inference distributed between multiple nodes at this point. For prefill because prefill is actually quadratic, but also for memory reasons. KV Cache for 405B at 10M context length would take more than 5 terabytes (at
9.
by terafo 2y ago
Why mention Microsoft twice?
10.
by terafo 2y ago
There was. Now second gen of that goes for $15.
11.
by terafo 2y ago
Overwhelming majority of flops is indeed spent on matmuls, but softmax disproportionately uses memory bandwidth, so it generally takes much longer than you'd expect from just looking at flops.
12.
by terafo 2y ago
t5 is an architecture, t5x is a framework for training models that was created with that architecture in mind, but can be used to train other architectures, including decoder-only ones(there is one in examples).
13.
by terafo 3y ago
To quote their official response "If the WSE weren't rectangular, the complexity of power delivery, I/O, mechanical integrity and cooling become much more difficult, to the point of impracticality.".
14.
by terafo 3y ago
Not right now.
15.
by terafo 3y ago
Because SRAM stopped getting smaller with recent nodes.
16.
by terafo 3y ago
This thing targets training, which isn't affected by tiny accelerators inside CPUs.
17.
by terafo 3y ago
No, it's comparable to 230Mb of SRAM on Groq chip, since both of them are SRAM-only chips that can't really use external memory.
18.
by terafo 3y ago
I would say that Bradley is actually more valuable, since it can serve wider range of missions, while having higher crew survival rate and being more maneuverable.
19.
by terafo 3y ago
> they have 24,700,000 left of fighting age Without equipment, logistics and ammo to support it it's a dead weight. Also, it's very interesting that you omitted Gulf War, which would be the most similar conflict in terms of p
20.
by terafo 3y ago
Wrong. Shells, artillery, drone components, engineering vehicles, tanks, APCs, jets, long-range missiles, anti-air defenses. 10x that and Ukraine starts winning again. 10x manpower won't do that.
21.
by terafo 3y ago
"Escalation management" through "slowly boiling the frog" combined with lack of coherent strategy for the war, partially because some western politicians are afraid of Russia's tantrum in case of Ukrainian victory(u
22.
by terafo 3y ago
Ukrainan Air Force has HARMs, but they are VERY limited in their capabilities due to them being employed from soviet-era jets. Basically area where target resides have to be pre-programmed on the ground, rocket then flies to that area and l
23.
by terafo 3y ago
For video to be a major contributor to a game size is a very rare occurrence. Games are 120GB+ in size because texture sizes continue to grow.
24.
by terafo 3y ago
Jetson Nano is a lot more expensive. But at the same price point as Coral you can buy Orange Pi 5 with 4 gigs of ram, 6 tops NPU(not supported as good as Coral though, but is generally more open and has support of much more frameworks) and
25.
by terafo 3y ago
Vision Pro isn't really supposed to be used outside.
26.
by terafo 3y ago
Attention is shared. It's ~30% of params here. So ~2B params are shared between experts and ~5B params are unique to each expert.
27.
by terafo 3y ago
Yes, but you wouldn't want to do that. You will be able to run that on a single 24gb GPU by the end of this weekend.
28.
by terafo 3y ago
v4 chips are basically the same in bf16 performance as A100(slower in int8).
29.
by terafo 3y ago
They are mostly incompatible.
30.
by terafo 3y ago
I don't have to imagine.

More ›