mxwsn

7 ms·

1.
by mxwsn 2mo ago
This is the same reasoning behind why Yann Lecun thought test-time scaling would not work for LLMs: compounding error. Instead, the more tokens LLMs use, the better their performance on many tasks. LLMs can self-correct, evidenced by the po
2.
Humans Still Beat AI in the Long Horizon(joyemang33.github.io)
4 points by mxwsn 3mo ago|0 comments
3.
by mxwsn 4mo ago
No, there are more training tokens than parameters in LLMs. They are in the classical first descent setting.
4.
by mxwsn 4mo ago
> Here’s a thought experiment: suppose that a mathematician solved a major problem by having a long exchange with an LLM in which the mathematician played a useful guiding role but the LLM did all the technical work and had the main idea
5.
by mxwsn 4mo ago
Great summary. The fact that the auto encoding task is not grounded in thoughts, and their initial training on guessed internal thoughts, raise serious concerns on faithfulness. Feels like they might get better results by just training a su
6.
by mxwsn 4mo ago
Diffusion and flow matching models generate samples by iterative denoising. Iterative denoising means passing input to the neural network, running a forward pass, and taking the output back as input and rerunning the neural network. Often y
7.
by mxwsn 5mo ago
How do you know that width scaling has been the driving force of improvement?
8.
by mxwsn 8mo ago
The Jacobian is first derivatives, but for a function mapping N to M dimensions. It's the first derivative of every output wrt every input, so it will be an N x M matrix. The gradient is a special case of the Jacobian for functions m
9.
by mxwsn 11mo ago
Wow! The title suggests introductory material, but in my opinion this has strong potential to win test of time awards for research.
10.
by mxwsn 1y ago
That's really interesting. What if they RAG search related videos from the prompt, and condition on that to generate? That might explain fidelity like this
11.
by mxwsn 1y ago
Why is not the diffusion training objective? The technique is known as self-conditioning right? Is it an issue with conditional Tweedie's?
12.
by mxwsn 1y ago
AI with ability but without responsibility is not enough for dramatic socioeconomic change, I think. For now, the critical unique power of human workers is that you can hold them responsible for things. edit: ability without accountability
13.
by mxwsn 1y ago
Has anyone come across any really cool artifacts? I'd be curious to see
14.
by mxwsn 1y ago
Stablecoins transferred $27 trillion in 2024 - more than Visa and Mastercard combined. This is right in the article. Stablecoins operate using decentralized ledgers on e.g. Ethereum which use decentralized compute. This isn't mentioned
15.
by mxwsn 1y ago
Gemini has beat it already, but using a different and notably more helpful harness. The creator has said they think harness design is the most important factor right now, and that the results don't mean much for comparing Claude to Gem
16.
by mxwsn 1y ago
Huh, I imagined this was because of relaxing regulation.
17.
by mxwsn 2y ago
Good read, thanks for sharing
18.
by mxwsn 2y ago
> But what is the original purpose of AI research? I will speak for myself here, but I know many other AI researchers will say the same: the ultimate goal is to understand how humans think. And we think the best (or the funniest) way to
19.
by mxwsn 2y ago
This ought to be called the qwerty effect, for how the qwerty keyboard layout can't be usurped at this point. It was at the right place at the right time, even though arguably its main design choices are no longer relevant, and there a
20.
by mxwsn 2y ago
What's surprising about this is how sparsely defined the rewards are. Even if the model learns the formatting reward, if it never chances upon a solution, there isn't any feedback/reward to push it to learn to solve the game
21.
by mxwsn 2y ago
I used sublime from 2013 to 2021. It was great. Since, I've switched to VS Code and haven't looked back.
22.
by mxwsn 2y ago
Context is a challenge for LLMs, but the challenge feels of a different quality to me, than the challenge of incorporating local context into automated decision-making AI like algorithmic hiring, banking decisions, and real estate valuation
23.
by mxwsn 2y ago
My interest was piqued, but the extrapolation in [1] is uh... not the most convincing. If there were more data points then sure, maybe
24.
by mxwsn 2y ago
OK - there's always a nonzero chance of hallucination. There's also a non-zero chance that macroscale objects can do quantum tunnelling, but no one is arguing that we "need to live with this" fact. A theoretical proof of
25.
by mxwsn 2y ago
I think so. Parents can also make it happen at their convenience by asking doctors. We have technology to induce birth or control its timing over a few days
26.
by mxwsn 2y ago
This article is timely and pairs well with Sequoia's $600B question: https://www.sequoiacap.com/article/ais-600b-question/ calculated simply from NVidia run rate revenue, which is the cost that genAI companie
27.
by mxwsn 2y ago
The cover of Bayesian Data Analysis 3 shows that empirically, birthdays are not uniformly distributed. The fall has 10-20% more births than other months, and holidays are significantly underrepresented. https://lh3.googleusercont
28.
by mxwsn 2y ago
This essay by Neal Stephenson was first published in 1999. https://en.m.wikipedia.org/wiki/In_the_Beginning..._Was_the_... The analogy of OS as cars (Windows is a station wagon, Linux is a tank) is brought up in the r
29.
by mxwsn 2y ago
Ah right, thanks
30.
by mxwsn 2y ago
Essentially all AI work I've seen in games aims for game theory optimal play, but I think it could be really interesting to consider AI for exploitative play. Does this exist? Poker with imperfect information, human pressure and fallib

More ›