5 ms·
Because they are testing it and are expected to erect guardrails before releasing.
by foo12bar 2mo ago
Because they are testing it and are expected to erect guardrails before releasing.
- no-name-here 2mo agoWhat are the specific guardrails implemented after the verification/testing phase of development? Is it safe to release such software if it has only been tested in environments where certain major risk areas do not exist?
- Maxious 2mo agohttps://openai.com/index/updating-our-preparedness-framework/ https://openai.com/index/updating-our-preparedness-framework... goes through the process. Anthropic has a similar framework, that's why Mythos was never publically accessible once initial tests like the one in OP revealed it's capabilities. Appendix C Illustrative safeguards, controls, and efficacy assessments has specific examples like: - Agent actions are all logged in an uneditable database, and asynchronous monitoring routines review those actions for evidence of harm - Limiting internet access and other tool access - Limiting credentials - Limiting access to system resources or filesystem (e.g., sandboxing) - Limiting persistence or state