mrfox321

8 ms·

1.
by mrfox321 2mo ago
No. Distillation trains on a teach model's logits or output tokens.
2.
by mrfox321 10mo ago
... the toppings contains potassium benzoate
3.
by mrfox321 1y ago
I know right? They have always been behind. Why would this time be any different?
4.
by mrfox321 1y ago
Read the paper. They control for the data being in-distribution Their dataset also has examples of the problem being solved.
5.
by mrfox321 1y ago
Sounds like excuses
6.
by mrfox321 2y ago
They didn't show this, they just increased the length where accuracy breaks down.
7.
by mrfox321 2y ago
What's your leg circumference at, now? Mine is also smaller, due to patella tendinopathy.
8.
by mrfox321 2y ago
Who do you think the shareholders are? What demographic are they?
9.
by mrfox321 2y ago
It's because they originally built their recommendation system to recommend friends and their content. Here, the social graph makes complete sense as the foundation for their simple search algorithm. But as they expanded their recommen
10.
by mrfox321 2y ago
I mean, it's their serving infra, but not their features. See my reply in another parent comment.
11.
by mrfox321 2y ago
It's simpler intuition but more complex from a data / ml perspective. Their algorithm is really built around their features. Specifically, temporal representations of user interest: https://ieeexplore.ieee.org/doc
12.
by mrfox321 2y ago
IG users and Tiktok users were / are quite similar. Especially when Tiktok wasn't yet eating IGs lunch.
13.
by mrfox321 2y ago
IG has plenty of data (I did ML at IG). Don't be naive.
14.
by mrfox321 2y ago
Not true. Why do IG reels and YouTube shorts suck, then? They clearly built something superior. And it can't seem to be matched by the biggest tech companies.
15.
by mrfox321 2y ago
At a big co I worked at, the lack of consistency between trading systems caused money to (dis)appear (into)out of thin air. Prior to one of these hiccups, I hypothesized, given how shitty the codebase was, that they must be tracking this
16.
by mrfox321 2y ago
Second order methods, and their approximations, can be used in weight updating, too.
17.
by mrfox321 2y ago
20 to 15 is insane. Are these low 15s or high 15s? Regardless, that's fantastic progress. Do you respond better to intervals or longer aerobic efforts? I ask because Ive always wondered if training should be specialized for fast/
18.
by mrfox321 2y ago
simultaneity is relative. That's all.
19.
by mrfox321 2y ago
More importantly, he invented diffusion models: http://proceedings.mlr.press/v37/sohl-dickstein15.pdf
20.
by mrfox321 2y ago
China does this, for better or for worse.
21.
by mrfox321 2y ago
Similar, but specialized to softmax(QK)V computation
22.
by mrfox321 2y ago
These people want to win. It's not just society wanting to define winners.
23.
by mrfox321 2y ago
Michael Phelps is an alien..
24.
by mrfox321 3y ago
When you need the best possible model, full stop. E.g. finance In a sufficiently competitive space, good enough doesn't cut it.
25.
by mrfox321 3y ago
I guess op may be envisioning an end-to-end solution that can train a model in the context of an external document store. I.e. One day we want to be able to backprop through the database. Search systems face equivalent problems. The hiera
26.
by mrfox321 3y ago
the police in America
27.
by mrfox321 3y ago
For the same reason why he brought up his iq tests in the first place. Because he self-identifies with some definition of iq.
28.
by mrfox321 3y ago
Nit to your nit, which is incorrect w.r.t. parent reply: A Fourier transform is not a projection, it's a change of basis represented by a unitary transformation.
29.
by mrfox321 3y ago
A square orbit would require infinite acceleration. Specifically, a dirac delta.
30.
by mrfox321 3y ago
This is my belief as well. In California, the haves will have houses. The have nots will have ADUs. Our world is structuring itself to (more) obviously reveal the wealth disparity.

More ›