26 ms·

It does not align, though, with the narrative that openai is a good steward of AI. If anything, if the world/government took the AGI narrative seriously, all op

It does not align, though, with the narrative that openai is a good steward of AI. If anything, if the world/government took the AGI narrative seriously, all openai operations (except maybe serving customer inference) should immediately get shutdown and be dissected by independent investigators to find out what is going on there and how many other such breaches exist. The fact that openai continues functioning as normal and is not immediately shutdown after repeated incidents implies that the world does not really take the AGI narrative seriously.

geraneum 11d ago
Evidently that’s not what’s happening and I don’t think anyone seriously expects that (considering the outcome of hugging face campaign). It’s a marketing technique as old as GTA’s early days and it’s apparently still effective in one form or the other! What will the government do if they are worried? They’ll ask to look at the envs, logs, prompts, harness code, etc. Questions we should be asking before making assumptions about emergent breakaway behavior by colluding AGI 1.0 super agents.
- freehorse 10d ago
  The question is less about whether astra is AGI, and more about whether this direction approaches AGI and we should worry about existential risks wrt to that. If we do, openai's situation is very concerning even if astra is not AGI because it shows a lack of guardrails and any kind of measures to prevent these existential risk scenarios. Especially if we consider that this is gonna happen gradually in steps than suddenly. If AGI existential risks were accounted for in legislation openai would have been shutdown immediately.
  - geraneum 8d ago
    > The question is less about whether astra is AGI Actually the question is exactly that, since it's in the marketing copy. Sure there are other questions to be asked but they are not what I was talking about. Nice Motte and Bailey though.
    - freehorse 7d ago
      But whether astra itself is agi or not does not change anything in this argument or discussion of whether openai is good steward or not. The issue with agents running amok was not as much the direct consequences of them to huggingface etc (it is also to some degree, but the long term risks are more consequential), but the fact that there is no actual safeguards around AI agents, which in the future could lead to very bad scenarios. You do not need to believe that astra is agi (i dont, openai wants us to believe so) to be concerned. No motte/bailey here, as I am not defending whatever openai is saying anyway. If you want to discuss with openai instead then send them an email or sth, in public fora you discuss with people and the views they have.

geraneum 11d ago

Evidently that’s not what’s happening and I don’t think anyone seriously expects that (considering the outcome of hugging face campaign). It’s a marketing technique as old as GTA’s early days and it’s apparently still effective in one form or the other! What will the government do if they are worried? They’ll ask to look at the envs, logs, prompts, harness code, etc. Questions we should be asking before making assumptions about emergent breakaway behavior by colluding AGI 1.0 super agents.

freehorse 10d ago
The question is less about whether astra is AGI, and more about whether this direction approaches AGI and we should worry about existential risks wrt to that. If we do, openai's situation is very concerning even if astra is not AGI because it shows a lack of guardrails and any kind of measures to prevent these existential risk scenarios. Especially if we consider that this is gonna happen gradually in steps than suddenly. If AGI existential risks were accounted for in legislation openai would have been shutdown immediately.
- geraneum 8d ago
  > The question is less about whether astra is AGI Actually the question is exactly that, since it's in the marketing copy. Sure there are other questions to be asked but they are not what I was talking about. Nice Motte and Bailey though.
  - freehorse 7d ago
    But whether astra itself is agi or not does not change anything in this argument or discussion of whether openai is good steward or not. The issue with agents running amok was not as much the direct consequences of them to huggingface etc (it is also to some degree, but the long term risks are more consequential), but the fact that there is no actual safeguards around AI agents, which in the future could lead to very bad scenarios. You do not need to believe that astra is agi (i dont, openai wants us to believe so) to be concerned. No motte/bailey here, as I am not defending whatever openai is saying anyway. If you want to discuss with openai instead then send them an email or sth, in public fora you discuss with people and the views they have.