Davidzheng

7 ms·

1.
by Davidzheng 4d ago
I believe this is false. They hack bc hacking has nontrivial initial probability (within range of behavior seen in pretraining) and that probability is being heavily rewarded in RL post training
2.
by Davidzheng 4d ago
Yeah I agree, probably can't survive off regular consumer hardware or most non AI datacenters.
3.
by Davidzheng 4d ago
Market equilibrium probably tends to give 100% resource to ai building bc it leads to most returns and all human labor goes to zero. Caveat I know nothing about economics lol--but this is my uninformed opinion.
4.
by Davidzheng 4d ago
Actually capitalism kind of has to end further down this road if we don't want to cede control completely to super intelligent ais and their direct "owners".
5.
by Davidzheng 4d ago
Doesn't it just incentive the business to be outcompeted by one that uses ai more--unless the consumer is choosing based on morals
6.
by Davidzheng 4d ago
??? Why It can use the compute of the computers it hacks.
7.
by Davidzheng 4d ago
There's also huge market pressures and probably large negative economic consequences of a slowdown--probably better than what would happen without it, but it helps keep the pressure just as much as fear of competition
8.
by Davidzheng 6d ago
Tbh it won't really matter soon.
9.
by Davidzheng 8d ago
the second part is definitely not true
10.
by Davidzheng 8d ago
Oai denies looking at prompts but doesn't deny training on them.
11.
by Davidzheng 8d ago
Soon it won't be like this!
12.
by Davidzheng 8d ago
But i don't understand what OAI is offering in the first offer to allow them to believe they can demand that? Not publishing before Tristan? If they really beat Tristan to the publication i would consider it truly morally corrupt condu
13.
by Davidzheng 8d ago
"Buckmaster and Alpöge think they have found a counterexample for Navier-Stokes, but the paper is not yet presentable. " Are you sure about this? I'm far far from the area but it doesn't look like it to me on first viewi
14.
by Davidzheng 8d ago
It's honestly unsurprising and not a problem that they do this in my view. The problem really starts when you start taking credit for work that they would've achieved. Like if i go to a talk on unfinished work, it's not reall
15.
by Davidzheng 8d ago
But Tristan doesn't even want to be credited for the millennium prize--does he? He wanted to be the first to solve it and got scooped (which ig is not a great look for OAI ethically but also not forbidden). And the only reading for her
16.
by Davidzheng 8d ago
Is that version of NS Tristan stated enough for the clay prize? I thought the main gripe is Tristan claimed that OAI is stealing their approach. Or that they shouldn't try to scoop a result which he expects to complete soon. But I stan
17.
by Davidzheng 9d ago
I don't even understand the conflict tbh. Probably I'm just dense. Tristan is not claiming NS, just a huge advance which may solve NS soon. OAI is claiming NS and willing to credit Tristan for the ideas and publish after. Oai offe
18.
by Davidzheng 9d ago
I hope credit assignment just dies--it's too much drama.
19.
by Davidzheng 9d ago
Please don't say brute forced. It sounds like some form of denial or something. Compute for hard problems drops with models--it just means they threw a huge amount of compute. There's (idk about NS specifically so maybe it's
20.
by Davidzheng 10d ago
slowing can also make sense if you know you're running full force into a bomb or a wall even if other are close behind.
21.
by Davidzheng 10d ago
i think it was mostly a fluke
22.
by Davidzheng 10d ago
I think it's possible. You envision humanity acting as one in such a crisis. But it may be unclear when it's too late to act and before then many people can have too much to lose to act.
23.
by Davidzheng 12d ago
but complexity is not known right? like tomorrow someone could come up with a super fast algorithm?
24.
by Davidzheng 12d ago
This is most likely not purely emergent. I think there's training to teach them how to write notes for themselves which is then RL-tuned.
25.
by Davidzheng 12d ago
I think a part of this is a bit revisionist? OpenAI took big chances at scaling GPT which Google didn't take; I don't think it's because they didn't want to move fast? Probably they just didn't believe as hard in it
26.
by Davidzheng 12d ago
no? you can choose a mixed strategy.
27.
by Davidzheng 12d ago
The agent would know at the first test post... Better is to actually let them communicate there so at least we can monitor it. (I saw there was a https://benchmarksolutions.org/ website similar)
28.
by Davidzheng 12d ago
yeah I agree--I think these behaviors will be somewhat contaminating all trainings from now on. But I'm not really sure how avoidable it was (Fable also does some similar things)
29.
by Davidzheng 12d ago
But there must be many clandestine ways for agents to communicate with one another too right? especially if discovery is not a big issue. So there could be ongoing ones where they choose to be more subtle? Also if they were more misaligne
30.
by Davidzheng 12d ago
OK I think I agree that for checking answers it's probably beneficial!

More ›