5 ms·

Yes, we do have that already, and it's kind of odd that people who see no point in thinking hard about risks to humans from this particular type of inhuman opti

Yes, we do have that already, and it's kind of odd that people who see no point in thinking hard about risks to humans from this particular type of inhuman optimization process would readily agree that other types of inhuman optimization processes (with much better understood risk profiles and limitations) can be very harmful. :)

mercer 9y ago
Perhaps the 'singularity' type stuff makes a lot of people disengage? So far I'm inclined to think that the whole 'sentient AI mucking things up' is jumping the gun (to put it mildly). I might very well be wrong though. However, it's only because of HN that I even bother to make the distinction between that particular fear (realistic or otherwise) and the much more directly useful discussion of how AI-type stuff (even just facebook feed algorithms) can be dangerous or harmful.
circlefavshape 9y ago
I don't think it's odd. I agree that optimising for the wrong thing (or, probably, for any particular thing) can be harmful, especially when it's done on a very large scale. AI-as-we-know-it doesn't introduce a new class of risk, it just makes some of the optimisations faster. I don't see any point in thinking about AGI risk until we get some experimental evidence that AGI is possible
- convexfunction 9y ago
  Maybe. I think the idea is that it's preferable to have wasted a lot of effort on coming up with irrelevant abstractions addressing a nonexistent problem, than to be surprised by something weird and poorly-understood presenting novel and not-clearly-bounded risk and not have effective tools at hand to address it. I won't claim to know what the right amount of effort directed toward this is though, nor do I really think AI safety research has a >epsilon chance of accomplishing its goal if it turns out to be relevant at all. ¯\_(ツ)_/¯

5 ms·

Yes, we do have that already, and it's kind of odd that people who see no point in thinking hard about risks to humans from this particular type of inhuman opti

by convexfunction 9y ago

mercer 9y ago
Perhaps the 'singularity' type stuff makes a lot of people disengage? So far I'm inclined to think that the whole 'sentient AI mucking things up' is jumping the gun (to put it mildly). I might very well be wrong though. However, it's only because of HN that I even bother to make the distinction between that particular fear (realistic or otherwise) and the much more directly useful discussion of how AI-type stuff (even just facebook feed algorithms) can be dangerous or harmful.
circlefavshape 9y ago
I don't think it's odd. I agree that optimising for the wrong thing (or, probably, for any particular thing) can be harmful, especially when it's done on a very large scale. AI-as-we-know-it doesn't introduce a new class of risk, it just makes some of the optimisations faster. I don't see any point in thinking about AGI risk until we get some experimental evidence that AGI is possible
- convexfunction 9y ago
  Maybe. I think the idea is that it's preferable to have wasted a lot of effort on coming up with irrelevant abstractions addressing a nonexistent problem, than to be surprised by something weird and poorly-understood presenting novel and not-clearly-bounded risk and not have effective tools at hand to address it. I won't claim to know what the right amount of effort directed toward this is though, nor do I really think AI safety research has a >epsilon chance of accomplishing its goal if it turns out to be relevant at all. ¯\_(ツ)_/¯