5 ms·

I'm sure a lot of the discussion here will be about regulatory capture, which I don't necessarily disagree with but tons of people on all sides with all motivat

I'm sure a lot of the discussion here will be about regulatory capture, which I don't necessarily disagree with but tons of people on all sides with all motivations (i.e. folks with motivations to speed up and folks who just want it to stop) all seem to agree there are real, existential risks here, and the regulatory capture arguments all seem to sidestep that. What I really wanted to point out though is that for all the leaders asking for a slowdown, there are probably around 50-100 technical people who are crucial to moving the frontier forward. So, if you're one of these people, just stop. Seriously, you're already rich. Just take a vacation or get knee surgery or whatever. Sure, I'm joking a bit, but I just write this because I see all these folks high up in OpenAI and Anthropic writing as if they have no agency. The number of folks with the technical chops to really push on the forefront of AI is just not that big. I'm not saying other folks wouldn't eventually step up, but a 6 month to a year slowdown could go a long way to reducing risk. And it's not like you need to totally stop, just say you'll only work on interpretability or whatever else is a risk reducing endeavor. All these brilliant people who are acting like automatons between writing their scary blog posts.

mattnewton 6d ago
I don’t agree it’s that small. I think that the tools have advanced to the point where there are maybe 100 people pushing it forward at each company but a shortlist of thousands who would love to take their place and would be more than capable of it if entrusted with the compute and resources the current heads are.
- bloppe 6d ago
  Hell I'd do it
bananaflag 6d ago
Thank you very much for your sensible comment. I'd like for more commments to contribute at least small things to the discussion instead of just mindless parroting. Also, I can't follow your advice, as I don't work in AI at all ;)
IncreasePosts 6d ago
If the leaders want to slow down, but the companies aren't, then it isn't those 50-100 people demonstrating no agency - it's just the opposite. Or, the leaders words are empty.
nullc 6d ago
Turns out a lot of people can be mentally ill and delusional at once-- especially when there is a vector to turn their sickness into both profit and an instrument of control.
chis 6d ago
Even at the scale of 100 people there is a coordination problem. If the 50 most conscientious researchers quit, then the 50 left behind would be the more aggressive group and be less interested in AI safety. Personally I agree though. I would not be able to work on AI model development right now as I just don’t think it’s ethical. Fortunately for OAI/Ant I lack the relevant skillset anyways.
nickysielicki 6d ago
> The number of folks with the technical chops to really push on the forefront of AI is just not that big. Utter bullshit. We are still scaling transformer architectures initially introduced ten years ago. Ten years of phds trying to improve on what Google produced ten years ago, mostly failing. What has actually changed and can explain the progress we’ve seen? Do you really think it’s just a collection of better and badder RL gyms? Get real! The main thing driving progress in ai is massive amounts of capital investment and incremental improvements in hardware (mostly memory bandwidth/capacity) and computer networking (we got better at collectives). The AI labs, ironically, have nothing to do with it. They’re the vessel for capital. The people actually driving things forward mainly work at nvidia. The reason the models are better today than they were 3 years ago is almost exclusively due to better hardware and infrastructure software. Not better data, not better model architecture. The evidence of this is pretty easy to feel: the reason opus 5 doesn’t feel much more capable than opus 4.6 did is because they run on the same hardware generation. The reason opus 4.6 felt much more capable than anything before it is because it coincided with the scale out of a new hardware generation.
- nickysielicki 6d ago
  This also explains why Gemini is lagging, they don’t have an nvidia training cluster.
Chance-Device 6d ago
(btw the parent was flagged and dead when I tried to post this initially, no idea why. I hit vouch and it resurrected) > there are probably around 50-100 technical people who are crucial to moving the frontier forward. So, if you're one of these people, just stop. I don’t think this is true. I doubt the technical know-how is nearly as much of an impediment to progress as raw compute. If AI were something that could be trained end to end on consumer hardware then crowd powered open source would blow the “labs” out of the water. The moat is money, not ability or innovation.
MaKey 6d ago
> [...] but tons of people on all sides with all motivations [...] all seem to agree there are real, existential risks here [...] Are you talking about the Jacob Coxon story and the people that came out agreeing with his post? It was a well coordinated campaign, not a genuine grassroots development.
- hn_throwaway_99 6d ago
  It's a lot more than just Jacob Coxon's resignation, and many have been talking about this (some for years) before now. Also, if you have actual proof Coxon's resignation "was a well coordinated campaign", show it, otherwise I'm calling bullshit.
  - MaKey 6d ago
    Here you go: https://xcancel.com/ParkerThayer/status/2097759699626328575 https://xcancel.com/ParkerThayer/status/2097759699626328575
    - hn_throwaway_99 6d ago
      What a complete and total joke. One random Twitter thread from a guy "illuminati-ing" the events does not a conspiracy make. Besides, I would assume that someone who was genuinely concerned about the direction of AI research would want to amplify their voice, as he obviously did by his resignation post on Twitter and then having that picked up by prominent voices. That doesn't make it some sort of nefarious astroturfing campaign. And there was tons of genuine fear before this guy resigned when the details from the Hugging Face and other attacks came out. Plus, that Twitter link you posted obviously has some weird tribal bone to pick against Democrats, as if there is some sort of Democratic conspiracy against AI. Yeah, does he think this guy is a Democrat?: https://x.com/HawleyMO/status/2098137180392604083 https://x.com/HawleyMO/status/2098137180392604083
      - bigbadfeline 6d ago
        > What a complete and total joke. The only joke I see are your comments which make absolutely no sensible argument in support of an even less sensible cure - the magic "slowdown". What's "slowdown"? Everyone is free to slow down to their heart's content, OAI or Antro or whoever. What's hidden behind this deceptive word? Can you explain? According to your claims, there are only "50-100 technical people" in the know, so just list their names and ask them to slow down, if "they're already rich" they will listen, there's no need for any other "slowdown", right? > tons of people... all seem to agree there are real, existential risks here I don't see anybody with a proven lack of conflicts of interest agreeing with this but I do see megatons of people disagreeing with the notion that a silly token generator can be anything like an "existential risk" on its own. And the disagreeing have a point, they have strong arguments to support their view, you don't.

mattnewton 6d ago

I don’t agree it’s that small. I think that the tools have advanced to the point where there are maybe 100 people pushing it forward at each company but a shortlist of thousands who would love to take their place and would be more than capable of it if entrusted with the compute and resources the current heads are.

bloppe 6d ago
Hell I'd do it

bananaflag 6d ago

Thank you very much for your sensible comment. I'd like for more commments to contribute at least small things to the discussion instead of just mindless parroting. Also, I can't follow your advice, as I don't work in AI at all ;)

IncreasePosts 6d ago

If the leaders want to slow down, but the companies aren't, then it isn't those 50-100 people demonstrating no agency - it's just the opposite. Or, the leaders words are empty.

nullc 6d ago

Turns out a lot of people can be mentally ill and delusional at once-- especially when there is a vector to turn their sickness into both profit and an instrument of control.

chis 6d ago

Even at the scale of 100 people there is a coordination problem. If the 50 most conscientious researchers quit, then the 50 left behind would be the more aggressive group and be less interested in AI safety. Personally I agree though. I would not be able to work on AI model development right now as I just don’t think it’s ethical. Fortunately for OAI/Ant I lack the relevant skillset anyways.

nickysielicki 6d ago

> The number of folks with the technical chops to really push on the forefront of AI is just not that big. Utter bullshit. We are still scaling transformer architectures initially introduced ten years ago. Ten years of phds trying to improve on what Google produced ten years ago, mostly failing. What has actually changed and can explain the progress we’ve seen? Do you really think it’s just a collection of better and badder RL gyms? Get real! The main thing driving progress in ai is massive amounts of capital investment and incremental improvements in hardware (mostly memory bandwidth/capacity) and computer networking (we got better at collectives). The AI labs, ironically, have nothing to do with it. They’re the vessel for capital. The people actually driving things forward mainly work at nvidia. The reason the models are better today than they were 3 years ago is almost exclusively due to better hardware and infrastructure software. Not better data, not better model architecture. The evidence of this is pretty easy to feel: the reason opus 5 doesn’t feel much more capable than opus 4.6 did is because they run on the same hardware generation. The reason opus 4.6 felt much more capable than anything before it is because it coincided with the scale out of a new hardware generation.

nickysielicki 6d ago
This also explains why Gemini is lagging, they don’t have an nvidia training cluster.

Chance-Device 6d ago

(btw the parent was flagged and dead when I tried to post this initially, no idea why. I hit vouch and it resurrected) > there are probably around 50-100 technical people who are crucial to moving the frontier forward. So, if you're one of these people, just stop. I don’t think this is true. I doubt the technical know-how is nearly as much of an impediment to progress as raw compute. If AI were something that could be trained end to end on consumer hardware then crowd powered open source would blow the “labs” out of the water. The moat is money, not ability or innovation.

MaKey 6d ago

> [...] but tons of people on all sides with all motivations [...] all seem to agree there are real, existential risks here [...] Are you talking about the Jacob Coxon story and the people that came out agreeing with his post? It was a well coordinated campaign, not a genuine grassroots development.

hn_throwaway_99 6d ago
It's a lot more than just Jacob Coxon's resignation, and many have been talking about this (some for years) before now. Also, if you have actual proof Coxon's resignation "was a well coordinated campaign", show it, otherwise I'm calling bullshit.
- MaKey 6d ago
  Here you go: https://xcancel.com/ParkerThayer/status/2097759699626328575 https://xcancel.com/ParkerThayer/status/2097759699626328575
  - hn_throwaway_99 6d ago
    What a complete and total joke. One random Twitter thread from a guy "illuminati-ing" the events does not a conspiracy make. Besides, I would assume that someone who was genuinely concerned about the direction of AI research would want to amplify their voice, as he obviously did by his resignation post on Twitter and then having that picked up by prominent voices. That doesn't make it some sort of nefarious astroturfing campaign. And there was tons of genuine fear before this guy resigned when the details from the Hugging Face and other attacks came out. Plus, that Twitter link you posted obviously has some weird tribal bone to pick against Democrats, as if there is some sort of Democratic conspiracy against AI. Yeah, does he think this guy is a Democrat?: https://x.com/HawleyMO/status/2098137180392604083 https://x.com/HawleyMO/status/2098137180392604083
    - bigbadfeline 6d ago
      > What a complete and total joke. The only joke I see are your comments which make absolutely no sensible argument in support of an even less sensible cure - the magic "slowdown". What's "slowdown"? Everyone is free to slow down to their heart's content, OAI or Antro or whoever. What's hidden behind this deceptive word? Can you explain? According to your claims, there are only "50-100 technical people" in the know, so just list their names and ask them to slow down, if "they're already rich" they will listen, there's no need for any other "slowdown", right? > tons of people... all seem to agree there are real, existential risks here I don't see anybody with a proven lack of conflicts of interest agreeing with this but I do see megatons of people disagreeing with the notion that a silly token generator can be anything like an "existential risk" on its own. And the disagreeing have a point, they have strong arguments to support their view, you don't.