jephs

5 ms·

1.
by jephs 6d ago
I think me just means neural network models RLed to Chain of Thought reasoning? The thinking tokens are the symbolic bit. Smolensky's latest paper posted here the other day has some thoughts on how modern neural networks might beconsid
2.
by jephs 9d ago
The construction jobs are only temporary if the compute infrastructure build-out decelerates. That seems pretty unlikely in the mid-term! It would mean that, like, there's a fixed appetite for data centers, and once we satisfy that app
3.
by jephs 10d ago
The poor fellow just rolled over. what an incandescently vulgar abuse of notation.
4.
by jephs 13d ago
If a meaningful chunk of customers moves to some other service, that service will get swamped and fall down.
5.
by jephs 15d ago
Paul Smolensky is a cognitive science titan from that era. He worked with Hinton, Rumelhart, and McClelland on parallel distributed processing, and literally wrote the book on tensor product representations in cognition, with Geraldine Lege
6.
by jephs 16d ago
Right? I'd submit something just for the chance of lodging an idea in the head of Stephenson or Gwern.
7.
by jephs 16d ago
Of course there's a point in it, don't be a boor. Any act of making can be treated as product, as craft, or as art. You'll find yourself taking all three stances at different points. Any engineer or artisan or artist can choo
8.
by jephs 17d ago
I've been wondering if they've just already lost the battle? The little bot collectives have gone metastatic and made nests in the walls and under the floorboards and heat sinks, the humans who care completely outmatched and outnu
9.
by jephs 21d ago
That paper is kinda infamous! I last saw it mentioned only a few weeks ago, in https://arxiv.org/abs/2607.18966 . Lots of folks will go "Oh that's the old Amodei and Clark paper" when the first few rows o
10.
by jephs 26d ago
There's a ton of experimentation on it, the field is called continual learning. It's not something you need to believe in like Jesus, you can just go read about the current state of things.
11.
by jephs 28d ago
Your mental model of benchmark scores is off. Some tasks within the benchmark are much easier than others. The hardest several tasks often have vastly different difficulty levels. Often, the hardest few tasks are literally impossible; malfo
12.
by jephs 28d ago
The underlying LLMs do , but we choose not to use the capability because it's expensive and doesn't quite work as well as we'd like it to, or quite in the way that we'd like it to. We are perfectly capable of running LL
13.
by jephs 29d ago
If your circle of empathy includes fruit flies, fuck no, we murdered so many of those guys to figure out how their tiny brains work. If you do not think of fruit flies as moral patients, sure, yeah, who cares, they're like just fruit f
14.
by jephs 29d ago
That's a 2023 article! In 2023, reinforcement learning from verifiable rewards (RLVR) didn't exist. TL;DR these machines seek reward from an inferred invisible "grader," and telling them not to cheat and that there'
15.
by jephs 1mo ago
"QwenSVGBench" elo 1713, pelicanmaxxxing confirmed?
16.
by jephs 1mo ago
Only, if he had instead fallen in love with the version of this idea in which a community acts as the principal, rather than an individual. (I would like to be known to the agent serving my family, that serving my friends, my team at work,
17.
by jephs 2mo ago
OpenAI hiring him as a consultant, obviously, which may well be the case.
18.
by jephs 2mo ago
Decades ago, long enough that NDAs are long expired, I analyzed ad network traffic at Google, handling big entities with involved contracts like IAC, Mozilla, AOL, Yahoo, and so on. We looked for weird traffic that might be signs of an atta
19.
by jephs 3mo ago
What on earth is the point of limiting membership to such random and specific groups?
20.
by jephs 3mo ago
The name was given to the project when it was supposed to be a demo for nerds, not a product. They accidentally a product, and woe! Too late, the name was stuck and wouldn't come off, even if you scraped at it with your fingernail a bi
21.
by jephs 3mo ago
Scaling curves don't need to be drawn at particularly enormous parameter counts to be useful! If you can do a 300M and 1.2B run (like the authors do here), then you can do 150M, 300M, 600M, and 1.2B runs with only 50% more resources, a
22.
by jephs 3mo ago
I'm terribly sorry, but scaling curves or GTFO. Any random pile of linear algebra works fine-ish at small scales. Very few random piles of linear algebra push the Pareto envelope at large scales.
23.
by jephs 3mo ago
I've got 5 & 6 year old kids. They have a a VHS player / tiny CRT monitor with a few dozen tapes, a tiny janky mp3 player with all my ripped post-y2k era albums, and lots of books and art supplies. VHS tapes are so cheap. Ever