5 ms·
It's important that humans are still able to understand the actions of these systems for safety reasons; the area focused on the problem you identify is "scalab
by nmca 3y ago
It's important that humans are still able to understand the actions of these systems for safety reasons; the area focused on the problem you identify is "scalable oversight", see for example:
https://arxiv.org/abs/2206.05802 https://arxiv.org/abs/2206.05802
https://arxiv.org/abs/2211.03540 https://arxiv.org/abs/2211.03540
- KyeRussell 3y agoUnderstanding them probably isn’t as important (or as likely) as someone- the right person-simply being held responsible for their actions, or for relying on them.
- kypro 3y agoImportant for who? And what do you mean by "safety" in this context? Systems which can produce code unaided are obviously dangerous to humanity, but this isn't the concern of any single company. Businesses will do whatever makes sense for their business. If you're talking about safety in the context of cyber security, then where the stakes are high (say in the development of banking software) you will probably see slower adoption of automation, but even there it will quickly become absurd to believe that humans are more capable at producing reliable and safe software. No human can be have a real-time database of every known cyber security venerability in their head for example, but an LLM could. I'd argue GPT-4 already has super human coding knowledge, it's really just on the intelligence side of things (reasoning, planning, logical consistency) that they're lacking a bit.