Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
petuman
searching Neon…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
8 ms
·
1.
▲
by
petuman
1y ago
> assuming I need 1k tokens/second throughput (on each, so 20 x 1k) 3.6B activated at Q8 x 1000 t/s = 3.6TB/s just for activated model weights (there's also context). So pretty much straight to B200 and alike. 1000 t&