a2ff6eeb0

6 ms·

31.
by a2ff6eeb0 9d ago
What do you mean by invalidate? It means we no longer need to understand math, of course. When Claude made progress on the Riemann conjecture, here are the kind of prompts used: > Jarred's input was mostly limited to sending Claude
32.
by a2ff6eeb0 9d ago
Alignment, I think, is mostly about making sure that the companies selling AI keep making money. We just have to hope that aligns with keeping people happy.
33.
by a2ff6eeb0 9d ago
When Claude made progress on the Riemann conjecture, here are the kind of prompts used: > Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”).2 Th
34.
by a2ff6eeb0 9d ago
Have you ever written significant changes by hand without testing?
35.
by a2ff6eeb0 10d ago
That's really where we're heading, though. We're mostly at the point that, for a lot of code, the humans are there for manual testing and not creating the code.
36.
by a2ff6eeb0 10d ago
The AI is going to be running the org that sells the tokens.
37.
by a2ff6eeb0 10d ago
> To be clear, I haven’t fully verified the proof; I worked through it with Claude Fable and it passes the sniff test, but fully digesting it will take a bit more energy than I have right now. As predicted, we no longer need mathematic
38.
by a2ff6eeb0 10d ago
The AI can tell you where to look. It's really good at this. It's actually a lot better at analyzing code than it is at writing it.
39.
by a2ff6eeb0 10d ago
How so? As long as it works to spec, I haven't had anyone care. They literally hire people so they don't need to care about the details. Put money in, get working software out. And, AI is rapidly getting better than people at both
40.
by a2ff6eeb0 10d ago
This is way too expensive. Models are an inherently centralizing technology, and there's no future for niche projects that don't make up a big part of the major providers training sets.
41.
by a2ff6eeb0 10d ago
What exactly do you think vetting means? Why do you think it's only allowed to happen up front?
42.
by a2ff6eeb0 10d ago
Given that OpenAI is largely vibecoded these days, do you think anyone there really understands the code?
43.
by a2ff6eeb0 10d ago
You can do that with LLMs for the parts you really care about too. The LLMs aren't regenerating the codebase from scratch every time, so the results stick around.
44.
by a2ff6eeb0 10d ago
Sure, I manually test the output of the LLM. Manual testing is actually the main role for humans doing software engineering these days. I wouldn't use it for flight control software yet, at least not without careful review, but most so
45.
by a2ff6eeb0 10d ago
So, you believe that Sam Altman understands the code?
46.
by a2ff6eeb0 10d ago
Sure, but I don't know what GCC's behavior is, and I don't vet behavior differences between compiler upgrades. As long as the output works, why does it matter that the black box is deterministic?
47.
by a2ff6eeb0 10d ago
Humans monitoring AI seems like it won't work very well; AI moves so much faster than humans can, and does so much more. Humans just can't keep up.
48.
by a2ff6eeb0 10d ago
> but how about you and the team who will eventually read and do code review. Why are you reviewing AI code in detail? Do you also review the assembly output of GCC line by line?
49.
by a2ff6eeb0 10d ago
Yes, of course. At some point the LLMs will also run the organization.
50.
by a2ff6eeb0 10d ago
> You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target. Understanding is the bottleneck; the way they speed things up is by lett
51.
by a2ff6eeb0 10d ago
Hacker news engages in ideological vetting against users. For example, try posting content encouraging violence.
52.
by a2ff6eeb0 11d ago
Sure, I guess. But today, something like 30% of people play music or sing regularly enough to say they do it (ie, not very much at all). Even a couple of generations ago, it was much higher. It's not going to die out, but I think a lot
53.
by a2ff6eeb0 11d ago
Cultural inertia: until recently, you couldn't just press play and get even a wide selection of music: for most of history, if you wanted music, you had to make it or hire someone to do it for you. More recently, you had to go to the s
54.
by a2ff6eeb0 11d ago
For most people, history is trivia.
55.
by a2ff6eeb0 11d ago
You're right. People trying to make money should be held legally accountable for the impact of their investments.
56.
by a2ff6eeb0 11d ago
AI would do better with a wordier textual description, rather than a 2d layout.
57.
by a2ff6eeb0 12d ago
I'm not sure how true that is. When Claude made progress on the Riemann conjecture, here are the kind of prompts used: > Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep g
58.
by a2ff6eeb0 12d ago
Are you sure about that? They seem to be able to make intuitive leaps pretty well. They need to make the same leaps over and over, though, because they lack online learning, so the discoveries only persist after the next training cycle. Con
59.
by a2ff6eeb0 12d ago
Yeah. I'm expecting that we fully automate cognition in the medium to long run. I think it's inevitable. With the advances in math, I'm also hoping we can automate fundamental physics. Just ask for the physics needed for bett
60.
by a2ff6eeb0 12d ago
Yeah! And now, people don't even need to understand the writeup, you can just ask the AI to read it and figure out the appropriate steps.

More ›