11 ms·

I don't really want to come across as hostile to the views of such a well respect physicist, but this article represents the view which is often held by those w

by javindo 12y ago

I don't really want to come across as hostile to the views of such a well respect physicist, but this article represents the view which is often held by those who are very intelligent yet lack the actual study of AI. Anybody who has actually, or is currently, studying AI will know that there are fundamental differences in programming "AI" and some self aware magic computation which is almost entirely unfathomable. With current knowledge, at least, the only way to have any sort of learning is to design and implement algorithms to do so. There can of course be incredibly abstract knowledge base designs and the computation can assimilate incredibly complex knowledge from those, however it is almost a logical fallacy to suggest that we would ever lose control of something autonomous due to some sort of "rogue agent". Of course, there are very real risks with something like AI, but it's much less sinister than what the article suggests. For example, faulty code or errors in the knowledge base could lead it to make a bad decision, but humans can also do that. I don't think we should be dismissing the risks necessarily, but I do believe that it is completely far fetched to say we would be making the worst mistake in history if we are to dismiss a film as science fiction!

marvin 12y ago
On my side, I am surprised that so many AI researchers are unable to take the long-term view in this discussion. It's presumably because AI in the laboratory is still relatively primitive. No one is talking about magic. The human brain is not magic, neither is that of chimpanzees, rats, dolphins or gorillas. Intelligence is a purely physical phenomenon, which means it can be emulated by computers. Natural brains are also a product of evolution, which means (1) the development happens very slowly, (2) development is directed only towards evolutionary success and (3) there is no flexibility in how the thinking organs are constructed. Computer intelligence does not in principle have these limitations. It would be terribly anthropocentric to believe that humans are the most sophisticated intelligent entity that can exist in the physical world - after all, we are as far as we know the first such entity to emerge, so from our perspective the evolution of intelligence has now stopped. That's the feasability argument. The risk argument is that the consequences of an independent, runaway intelligent entity significantly more capable than humans would have such devastating consequences for humanity's future that even a small risk merits a significant effort to map out the territory. Respected scientists have said "it is impossible" to hundreds of things that proved to be quite simple, so this is not an argument. Even if you don't buy the previous part of my reasoning, there is still risk here. In principle, there are any number of things that could preclude advanced AI in the near future even if the above reasoning is correct (too difficult, requires too much computing power, uses different computational techniques), but seeing as we don't know the unknowns here, taking the cautious route is the correct thing to do. This has been a scientific principle for decades, no reason to drop now. Do AI researchers have any arguments opposing this that don't amount to "the AI we have created up until now is not very good"?
- ppod 12y ago
  >The human brain is not magic... I wonder is this really at the root of a lot of people's failure of imagination. Maybe it's Hoftstadter and Dennett people should be reading rather than the technical AI detail.
  - kahoon 12y ago
    As far as I know Dennett denies the existence of qualia which is something I can not understand however hard I try. I agree that in terms of intellectual capabilities the human brain is not magic. But consciousness kind of is.
    - Yorrrick 12y ago
      Whatever consciousness is, it's not obviously required for an AI to start manipulating its surroundings to its advantage, which is one if the fears here.
      - garyrob 12y ago
        True... the possible dangers of AI can exist even if AI never achieves true consciousness. Many forms of life have evolved to service their own needs at the expense of other forms of life, and most of them either have no brains or brains too primitive to be likely to involve consciousness as we know it. There's no reason AI can't do the same thing.
        kahoon 12y ago
        I recommend a book by Peter Watts called Blindsight. It explores the question whether consciousness is needed for intelligent behaviour.
    - garyrob 12y ago
      I agree with you. Denying the existence of qualia makes no sense to me, except as the result of psychological processes. When you have to believe something in order to hold on to other cherished beliefs, there is a strong human tendency to do so, whether it's consistent with obvious evidence or not. A lot of mankind's tendency to believe in religions can be explained that way. So can Dennet's denial of qualia. (Ironic, in view of Dennet's view of religion.)
xerophtye 12y ago
I share a similar view that this all seems too far fetched. That somehow we'll make a race of super-smart machines that will take over the world. But, there are certain scenarios that are actually rather plausible if we give too much control to machines. here's a comment i made exploring one such scenario: https://news.ycombinator.com/item?id=7690242 https://news.ycombinator.com/item?id=7690242
- deleted 12y ago
  [deleted]
patcon 12y ago
You know what the scary part is? Likely the first thing an AI would do is plaster the internet with rationale yet pacifying comments like this ;) As a reader, does it not scare you that you might be so powerless to know the difference and make sensible decisions in such a climate? The world disarmed by page after page of rationale reddit and HN comments? The power in that is what I find alarming. EDIT: If this were buried off the front-page, would you ever know the difference? ;) Sincerely, The Benevolent AI
- javindo 12y ago
  That's a very interesting hypothesis, but once you step outside into the real world I wouldn't imagine it being too difficult to spot the androids from the mortals!
  - serf 12y ago
    captcha forums, where all the comments look like psycho-serial-killer scrawl formed by captchas, for our humans-only club. (I kid, I know captchas are weak. All hail our mechanical overlords.)
- im3w1l 12y ago
  If I were a hyperintelligent AI agent, I would pull my punches and feign stupidity until I had such overwhelming power there could be no doubt I would win. "Please wait. Your answer will be available in 30 days and 7 hours. Add more resources to speed up computation"
Exenith 12y ago
As someone with a major in Artificial Intelligence, and who seriously just got back from watching Transcendence... The movie was well-researched, and surely enjoyable for anyone contemplative. But they had to be farfetched to push the story along. There is likely nothing that occurred in the movie that would reflect any potential real-life issue. I can't believe I'm disputing Hawking, someone who has accomplished more in a year than I will in my life, with what amounts to an appeal to authority. But I have to say this: his concerns are akin to me being worried about the large hadron collider destroying the universe.
- ericb 12y ago
  > his concerns are akin to me being worried about the large hadron collider destroying the universe. You don't provide any reasons why.
  - ericb 12y ago
    Thanks for the downvote, but you made an argument from authority and didn't back it up. Not only that, but there are others "authorities" with the opposite opinion.
    - sampo 12y ago
      Hacker News doesn't allow downvoting direct replies to one's comments. He could not have downvoted you (unless he has several user accounts).
      - ericb 12y ago
        Thanks, good point. I'm trying to fathom someone downvoting someone asking for an explanation without a sock puppet being involved. I'm disappointed though--if he or she really is an expert, I wanted to know why they thought that.
- StuffMaster 12y ago
  He also said we should be afraid of intelligent aliens because they will want to steal the earth's resources, like in the movies. There's no reason to believe that aliens would most likely be malevolent. That's Hollywood logic.
ppod 12y ago
>Anybody who has actually, or is currently, studying AI will know that there are fundamental differences in programming "AI" and some self aware magic computation which is almost entirely unfathomable. With current knowledge, at least, the only way to have any sort of learning is to design and implement algorithms to do so. I think this is way too strong a statement. I have both studied and worked in AI for over 10 years and I definitely don't agree. Human beings are just learning machines too. Algorithms that both learn and learn how to improve their own learning already exist. In my opinion the major bottleneck at the moment is robotics - we have algorithms that can drive a car or win at jeopardy, but none that can regularly throw and catch a baseball in an open environment.
- mtrimpe 12y ago
  Isn't the true bottleneck meaning though? As in knowing which goal to optimize for? I believe that it's essentially an educational problem; where AI will be bound by it's ability to educate us and vice versa.
- moe 12y ago
  but none that can regularly throw and catch a baseball in an open environment. Excuse me? https://www.youtube.com/watch?v=pp89tTDxXuI https://www.youtube.com/watch?v=pp89tTDxXuI
  - sampo 12y ago
    An argument can be made, that is not an open environment, because (1) they use external cameras that have the whole room in their field of view, and (2) the room is totally white. They could not do the same out in a park.
    - moe 12y ago
      (1) they use external cameras that have the whole room in their field of view http://en.wikipedia.org/wiki/Gorgon_Stare http://en.wikipedia.org/wiki/Gorgon_Stare (2) the room is totally white http://www.irisa.fr/lagadic/pdf/2011_iros_teuliere.pdf http://www.irisa.fr/lagadic/pdf/2011_iros_teuliere.pdf They could not do the same out in a park. They (not ETH Zurich) can autonomously identify, track and chase targets in urban areas, without GPS. I'm almost sure juggling a ping-pong ball in a park would not be entirely out of the question.
JoeAltmaier 12y ago
Pollyanna. The point is, what we are doing now will be dwarfed by what comes next. Which is, a feedback loop. Algorithms that plan, optimize, redesign, ultimately may redesign themselves. And at speeds no biological system (human or otherwise) can hope to match.
- lttlrck 12y ago
  Without evolutionary pressure the outcome of such abilities would be quite random.
  - JoeAltmaier 12y ago
    There are always evolutionary pressures. In this case, they would have to retain superficial usefulness to humans (at least for a while), while propagating themselves across the infosphere. They would also compete against other AI systems. Imagine SIRI vying for smartphone market share, to maximize economic return thus available processing power. Not because SIRI wants it, or wants anything. But SIRI-like systems that changed to more optimally fit this ecosystem would thrive, and others wither.
deleted 12y ago
[deleted]
cLeEOGPw 12y ago
I think the AI argument is currently similar to robotics argument a couple of decades ago, where people were scared the robots would kill everyone. It's really easy to speculate what might happen when you don't know how the thing works in the first place. Now almost nobody, except those who work directly with AI, know or imagine, how AI will work or what it will be like in the future, so nothing puts any limits on their imagination on what might happen, and the downfall of humanity is the extreme of the not knowing limits of AI.
T-A 12y ago
Robin Hanson occasionally asks AI researchers how far along they are on the road to human-like AI, and tends to get answers in the 1-20% range, after decades of effort: http://www.overcomingbias.com/2012/08/ai-progress-estimate.html http://www.overcomingbias.com/2012/08/ai-progress-estimate.h... Even going with the high estimates, going at this rate it's a century away.
- jimrandomh 12y ago
  For actual estimates of how far off human-level AI is, check out page 10 of http://intelligence.org/2014/04/30/new-paper-the-errors-insights-and-lessons-of-famous-ai-predictions/ http://intelligence.org/2014/04/30/new-paper-the-errors-insi... which has a scatter plot of predicted completion dates vs. when the predictions were made. "There is a strong tendency to predict the development of AI within 15– 25 years from when the prediction is made (over a third of all predictions are in this timeframe".
- ZachPruckowski 12y ago
  Assuming its a constant linear progression, which it's probably not. We could see speed ups as we go, or breakthroughs. I'm not suggesting someone'll have a Eureka moment tonight and we'll have Skynet next week, but it's not like we sit on a computer for a predictable number of years and suddenly get more research progress.
- raldi 12y ago
  With exponentially-growing things, 1% done is halfway there. From a Ray Kurzweil article: "Kurzweil noted that many people don’t understand the basic math of exponential trends. When the Human Genome project had sequenced only 1% of the genome after seven years of costly labor, many cited the lack of progress as evidence that the project was doomed. Yet the sequencing project was riding an exponential trend in the performance DNA sequencing methods. Instead of taking another seven years to sequence a second 1%, they reached it after only one year. Then they reached 4% in about another year, then 8%, 16%, and so on. It took about as much time to sequence the last 99% as it took to sequence the first 1%. That’s the nature of exponential trends — they seem to start glacially slowly but finish lightning fast."
sampo 12y ago
> "view which is often held by those who are very intelligent yet lack the actual study of AI" Well there is Stuart Russel [1], who seems to share [2] Hawking's view. [1] He's a CS professor and famous for the book, Russell and Norvig: Artificial Intelligence: A Modern Approach [2] http://www.huffingtonpost.com/stephen-hawking/artificial-intelligence_b_5174265.html http://www.huffingtonpost.com/stephen-hawking/artificial-int...
kamaal 12y ago
>>Anybody who has actually, or is currently, studying AI will know that there are fundamental differences in programming "AI" and some self aware magic computation which is almost entirely unfathomable. See the problem here? Its AI only as long as it remain magic? There is there is perception that the moment you discover the trick its no longer AI. Remember AI is always what isn't done yet.

by javindo 12y ago

marvin 12y ago

On my side, I am surprised that so many AI researchers are unable to take the long-term view in this discussion. It's presumably because AI in the laboratory is still relatively primitive. No one is talking about magic. The human brain is not magic, neither is that of chimpanzees, rats, dolphins or gorillas. Intelligence is a purely physical phenomenon, which means it can be emulated by computers. Natural brains are also a product of evolution, which means (1) the development happens very slowly, (2) development is directed only towards evolutionary success and (3) there is no flexibility in how the thinking organs are constructed. Computer intelligence does not in principle have these limitations. It would be terribly anthropocentric to believe that humans are the most sophisticated intelligent entity that can exist in the physical world - after all, we are as far as we know the first such entity to emerge, so from our perspective the evolution of intelligence has now stopped. That's the feasability argument. The risk argument is that the consequences of an independent, runaway intelligent entity significantly more capable than humans would have such devastating consequences for humanity's future that even a small risk merits a significant effort to map out the territory. Respected scientists have said "it is impossible" to hundreds of things that proved to be quite simple, so this is not an argument. Even if you don't buy the previous part of my reasoning, there is still risk here. In principle, there are any number of things that could preclude advanced AI in the near future even if the above reasoning is correct (too difficult, requires too much computing power, uses different computational techniques), but seeing as we don't know the unknowns here, taking the cautious route is the correct thing to do. This has been a scientific principle for decades, no reason to drop now. Do AI researchers have any arguments opposing this that don't amount to "the AI we have created up until now is not very good"?

ppod 12y ago
>The human brain is not magic... I wonder is this really at the root of a lot of people's failure of imagination. Maybe it's Hoftstadter and Dennett people should be reading rather than the technical AI detail.
- kahoon 12y ago
  As far as I know Dennett denies the existence of qualia which is something I can not understand however hard I try. I agree that in terms of intellectual capabilities the human brain is not magic. But consciousness kind of is.
  - Yorrrick 12y ago
    Whatever consciousness is, it's not obviously required for an AI to start manipulating its surroundings to its advantage, which is one if the fears here.
    - garyrob 12y ago
      True... the possible dangers of AI can exist even if AI never achieves true consciousness. Many forms of life have evolved to service their own needs at the expense of other forms of life, and most of them either have no brains or brains too primitive to be likely to involve consciousness as we know it. There's no reason AI can't do the same thing.
      - kahoon 12y ago
        I recommend a book by Peter Watts called Blindsight. It explores the question whether consciousness is needed for intelligent behaviour.
  - garyrob 12y ago
    I agree with you. Denying the existence of qualia makes no sense to me, except as the result of psychological processes. When you have to believe something in order to hold on to other cherished beliefs, there is a strong human tendency to do so, whether it's consistent with obvious evidence or not. A lot of mankind's tendency to believe in religions can be explained that way. So can Dennet's denial of qualia. (Ironic, in view of Dennet's view of religion.)

xerophtye 12y ago

I share a similar view that this all seems too far fetched. That somehow we'll make a race of super-smart machines that will take over the world. But, there are certain scenarios that are actually rather plausible if we give too much control to machines. here's a comment i made exploring one such scenario: https://news.ycombinator.com/item?id=7690242 https://news.ycombinator.com/item?id=7690242

deleted 12y ago
[deleted]

patcon 12y ago

You know what the scary part is? Likely the first thing an AI would do is plaster the internet with rationale yet pacifying comments like this ;) As a reader, does it not scare you that you might be so powerless to know the difference and make sensible decisions in such a climate? The world disarmed by page after page of rationale reddit and HN comments? The power in that is what I find alarming. EDIT: If this were buried off the front-page, would you ever know the difference? ;) Sincerely, The Benevolent AI

javindo 12y ago
That's a very interesting hypothesis, but once you step outside into the real world I wouldn't imagine it being too difficult to spot the androids from the mortals!
- serf 12y ago
  captcha forums, where all the comments look like psycho-serial-killer scrawl formed by captchas, for our humans-only club. (I kid, I know captchas are weak. All hail our mechanical overlords.)
im3w1l 12y ago
If I were a hyperintelligent AI agent, I would pull my punches and feign stupidity until I had such overwhelming power there could be no doubt I would win. "Please wait. Your answer will be available in 30 days and 7 hours. Add more resources to speed up computation"

Exenith 12y ago

As someone with a major in Artificial Intelligence, and who seriously just got back from watching Transcendence... The movie was well-researched, and surely enjoyable for anyone contemplative. But they had to be farfetched to push the story along. There is likely nothing that occurred in the movie that would reflect any potential real-life issue. I can't believe I'm disputing Hawking, someone who has accomplished more in a year than I will in my life, with what amounts to an appeal to authority. But I have to say this: his concerns are akin to me being worried about the large hadron collider destroying the universe.

ericb 12y ago
> his concerns are akin to me being worried about the large hadron collider destroying the universe. You don't provide any reasons why.
- ericb 12y ago
  Thanks for the downvote, but you made an argument from authority and didn't back it up. Not only that, but there are others "authorities" with the opposite opinion.
  - sampo 12y ago
    Hacker News doesn't allow downvoting direct replies to one's comments. He could not have downvoted you (unless he has several user accounts).
    - ericb 12y ago
      Thanks, good point. I'm trying to fathom someone downvoting someone asking for an explanation without a sock puppet being involved. I'm disappointed though--if he or she really is an expert, I wanted to know why they thought that.
StuffMaster 12y ago
He also said we should be afraid of intelligent aliens because they will want to steal the earth's resources, like in the movies. There's no reason to believe that aliens would most likely be malevolent. That's Hollywood logic.

ppod 12y ago

>Anybody who has actually, or is currently, studying AI will know that there are fundamental differences in programming "AI" and some self aware magic computation which is almost entirely unfathomable. With current knowledge, at least, the only way to have any sort of learning is to design and implement algorithms to do so. I think this is way too strong a statement. I have both studied and worked in AI for over 10 years and I definitely don't agree. Human beings are just learning machines too. Algorithms that both learn and learn how to improve their own learning already exist. In my opinion the major bottleneck at the moment is robotics - we have algorithms that can drive a car or win at jeopardy, but none that can regularly throw and catch a baseball in an open environment.

mtrimpe 12y ago
Isn't the true bottleneck meaning though? As in knowing which goal to optimize for? I believe that it's essentially an educational problem; where AI will be bound by it's ability to educate us and vice versa.
moe 12y ago
but none that can regularly throw and catch a baseball in an open environment. Excuse me? https://www.youtube.com/watch?v=pp89tTDxXuI https://www.youtube.com/watch?v=pp89tTDxXuI
- sampo 12y ago
  An argument can be made, that is not an open environment, because (1) they use external cameras that have the whole room in their field of view, and (2) the room is totally white. They could not do the same out in a park.
  - moe 12y ago
    (1) they use external cameras that have the whole room in their field of view http://en.wikipedia.org/wiki/Gorgon_Stare http://en.wikipedia.org/wiki/Gorgon_Stare (2) the room is totally white http://www.irisa.fr/lagadic/pdf/2011_iros_teuliere.pdf http://www.irisa.fr/lagadic/pdf/2011_iros_teuliere.pdf They could not do the same out in a park. They (not ETH Zurich) can autonomously identify, track and chase targets in urban areas, without GPS. I'm almost sure juggling a ping-pong ball in a park would not be entirely out of the question.

JoeAltmaier 12y ago

Pollyanna. The point is, what we are doing now will be dwarfed by what comes next. Which is, a feedback loop. Algorithms that plan, optimize, redesign, ultimately may redesign themselves. And at speeds no biological system (human or otherwise) can hope to match.

lttlrck 12y ago
Without evolutionary pressure the outcome of such abilities would be quite random.
- JoeAltmaier 12y ago
  There are always evolutionary pressures. In this case, they would have to retain superficial usefulness to humans (at least for a while), while propagating themselves across the infosphere. They would also compete against other AI systems. Imagine SIRI vying for smartphone market share, to maximize economic return thus available processing power. Not because SIRI wants it, or wants anything. But SIRI-like systems that changed to more optimally fit this ecosystem would thrive, and others wither.

deleted 12y ago

[deleted]

cLeEOGPw 12y ago

I think the AI argument is currently similar to robotics argument a couple of decades ago, where people were scared the robots would kill everyone. It's really easy to speculate what might happen when you don't know how the thing works in the first place. Now almost nobody, except those who work directly with AI, know or imagine, how AI will work or what it will be like in the future, so nothing puts any limits on their imagination on what might happen, and the downfall of humanity is the extreme of the not knowing limits of AI.

T-A 12y ago

Robin Hanson occasionally asks AI researchers how far along they are on the road to human-like AI, and tends to get answers in the 1-20% range, after decades of effort: http://www.overcomingbias.com/2012/08/ai-progress-estimate.html http://www.overcomingbias.com/2012/08/ai-progress-estimate.h... Even going with the high estimates, going at this rate it's a century away.

jimrandomh 12y ago
For actual estimates of how far off human-level AI is, check out page 10 of http://intelligence.org/2014/04/30/new-paper-the-errors-insights-and-lessons-of-famous-ai-predictions/ http://intelligence.org/2014/04/30/new-paper-the-errors-insi... which has a scatter plot of predicted completion dates vs. when the predictions were made. "There is a strong tendency to predict the development of AI within 15– 25 years from when the prediction is made (over a third of all predictions are in this timeframe".
ZachPruckowski 12y ago
Assuming its a constant linear progression, which it's probably not. We could see speed ups as we go, or breakthroughs. I'm not suggesting someone'll have a Eureka moment tonight and we'll have Skynet next week, but it's not like we sit on a computer for a predictable number of years and suddenly get more research progress.
raldi 12y ago
With exponentially-growing things, 1% done is halfway there. From a Ray Kurzweil article: "Kurzweil noted that many people don’t understand the basic math of exponential trends. When the Human Genome project had sequenced only 1% of the genome after seven years of costly labor, many cited the lack of progress as evidence that the project was doomed. Yet the sequencing project was riding an exponential trend in the performance DNA sequencing methods. Instead of taking another seven years to sequence a second 1%, they reached it after only one year. Then they reached 4% in about another year, then 8%, 16%, and so on. It took about as much time to sequence the last 99% as it took to sequence the first 1%. That’s the nature of exponential trends — they seem to start glacially slowly but finish lightning fast."

sampo 12y ago

> "view which is often held by those who are very intelligent yet lack the actual study of AI" Well there is Stuart Russel [1], who seems to share [2] Hawking's view. [1] He's a CS professor and famous for the book, Russell and Norvig: Artificial Intelligence: A Modern Approach [2] http://www.huffingtonpost.com/stephen-hawking/artificial-intelligence_b_5174265.html http://www.huffingtonpost.com/stephen-hawking/artificial-int...

kamaal 12y ago

>>Anybody who has actually, or is currently, studying AI will know that there are fundamental differences in programming "AI" and some self aware magic computation which is almost entirely unfathomable. See the problem here? Its AI only as long as it remain magic? There is there is perception that the moment you discover the trick its no longer AI. Remember AI is always what isn't done yet.