chessgecko

5 ms·

1.
by chessgecko 1y ago
The real issue is that housing is heavily underweighted in the cpi basket. How many people do you know that are only spending 12.9% of their after tax take home on housing, water and fuel? Only people with paid off mortgages.
2.
by chessgecko 1y ago
Wouldn’t say never harder, but it’s been pretty flat. https://fred.stlouisfed.org/series/FYFRGDA188S
3.
by chessgecko 1y ago
My girlfriend loves it. She’s an immigrant from Kazakhstan and apparently the Kazakh community on Threads is very active. Reminds her of home.
4.
by chessgecko 1y ago
if you bought a nvidia h100 at wholesale prices (around $25k) and ran it 24/7 at commercial electric rates (lets say $0.1 per kwh), then it would take you over 40 years to spend the purchase price of the gpu in electricity. Maybe bump
5.
by chessgecko 1y ago
Actually since the qe to fix 2008 foreign investors/governments cut down on bond buying. Treasury has foreign bond holdings up about 2T since 2014 and current account deficits sum to around 7T. Nowadays we're mostly selling compan
6.
by chessgecko 1y ago
Or maybe a lot of it can't deliver returns in line with the current price. The world realizes that and stops sending us stuff for those investments. Then we're stuck with limited manufacturing, high inflation and relatively low ow
7.
by chessgecko 1y ago
It is tracked! there are a few different numbers. by the BEA: https://fred.stlouisfed.org/graph/?g=gX0f by the treasury: https://ticdata.treasury.gov/resource-center/data-chart-cent... It's a
8.
by chessgecko 2y ago
1800 on the h100s is with 2/4 sparsity, it’s half of that without. Not sure if the tpu number is doing that too, but I don’t think 2/4 is used that heavily so I probably would compare without it.
9.
by chessgecko 2y ago
I remember reading that it’s too hard to get good memory bandwidth/l2 utilization in the fancy algorithms, you need to read contiguous blocks and be able to use them repeatedly. But I also haven’t looked at the gpu blas implementations
10.
by chessgecko 2y ago
I guess it's hard to know how well this would compete with integrated gpus, especially at a reasonable pricepoint. If you wanted to spend $4000+ on it, it could be very competitive and might look something like nvidias grace-hopper sup
11.
by chessgecko 2y ago
I think we don't really disagree, I just think that this shape isn't really a gpu its just a cpu because it isn't very good for graphics at that point.
12.
by chessgecko 2y ago
For ai inference you definitely have other options, but for low end graphics? the lpddr that apple (and nvidia in grace) use would be super expensive to get a comparable bandwidth (think $3+/gb and to get 500GB/sec you need at lea
13.
by chessgecko 2y ago
GDDR isnt like the ram that connects to cpu, it's much more difficult and expensive to add more. You can get up to 48GB with some expensive stacked gddr, but if you wanted to add more stacks you'd need to solve some serious signal
14.
by chessgecko 2y ago
One day is doing some heavy heavy lifting here, we’re currently off by ~3-4 orders of magnitude…
15.
by chessgecko 2y ago
The real reason is that it's basically impossible to produce a cheap new car that is a better deal than a Toyota with 80k miles on it.
16.
by chessgecko 2y ago
For decode steps it depends on the number of inputs you run at a time. If your batch size is 1 then it runs in line with active params, then as you get to like batch size 8 it runs in line with all params, then as you increase to 128ish it
17.
by chessgecko 2y ago
Not just gpus, the k20 was at 3.9 Tflops (fp32) and the new iPhone is at 4.3 (fp16). If you don’t need the precision it got passed by the phones
18.
by chessgecko 2y ago
You’d get better perf training on a current gen phone than that gpu, but it probably functions
19.
by chessgecko 2y ago
Are you one of the authors?
20.
by chessgecko 2y ago
I feel like that blogpost was almost just ragebait for ai researchers. It goes between calling not including the +1 an error (which to me implies it would improve training losses, which it doesn't really https://news.ycombin
21.
by chessgecko 2y ago
My hypothesis for why this works that it mitigates the downsides of rope to eli5: rope is the modern strategy used to give information to the model about how far a query and a key are apart when doing attention. It's the best strategy
22.
by chessgecko 2y ago
I think they mitigated the extra memory/compute from this by using half the number of overall heads and doubling V and O. Without actually checking the math I think it should be equivalent in flops, not counting the extra (cheap) multi
23.
by chessgecko 2y ago
I wonder how much of the value here is from canceling out the positional noise rope produces. I would love to see a table comparing an alibi version of this to an alibi baseline in addition to the rope models here. Crazy gains though congra
24.
by chessgecko 2y ago
Their numbers are for a single input, I assume the throughput is much higher given the prices they are quoting and the cost of a single cs3.
25.
by chessgecko 2y ago
On die communication isn’t free, a lot of things here are sequential and within matrix multiplies the cores have to transfer output and mem loads have to be distributed. It’s really fast but not like one cycle
26.
by chessgecko 2y ago
*edit neverming below this is a character level model that probably has a small vocab so it wouldn’t make a massive difference Is this taking into account the parameters in the embedding and the output ffn? Because normally when models are
27.
by chessgecko 2y ago
Having played with this stuff its definitely spots in the expert buffers (the other comment in the thread has the link to explanation) and not the extremely small differences in floating point arithmetic. The effect from this is much much l
28.
by chessgecko 2y ago
jorbs manages to get 15 win streaks on a20 beating the heart. I wouldn’t call it a slot machine
29.
by chessgecko 2y ago
This is the sparsest model thats been put out in a while (maybe ever, kinda forget the shapes of googles old sparse models). This probably wont be a great tradeoff for chat servers, but could be good for local stuff if you have 512GB of ram
30.
by chessgecko 2y ago
I think its almost certainly using at least two experts per token. It helps a lot during training to have two experts to contrast when putting losses on the expert router.

More ›