4 ms·

If their learning material is based on expert human games, how can it ever get better than that?

by someotheridiot 11y ago

space_fountain 11y ago
How can a human ever get better than their teacher? In this case though they play and optimize against themselves
- kazinator 11y ago
  > How can a human ever get better than their teacher? By learning from other teachers, and by applying original thought. Also, due to innately superior intelligence. If your IQ is 140, and that of the teacher is 105, you will eventually outstrip the teacher.
  - jibalt 11y ago
    The question was rhetorical. And what is needed is aptitude for the specific task, not "IQ" ... the two are often very different.
sawwit 11y ago
It's because they have a much larger stack size than a human brain (which does not have a stack at all, but just various kinds of short term memories). An expert Go player can realistically maybe consider 2-3 moves into the future and can have a rough idea about what will happen in the coming 10 moves, while this method does tree search all the way to the end of the game on multiple alternative paths for each move.
- donmaq 11y ago
  Not true. Profession go players read out 20+ moves consistently. Go Seigan's nemesis Kitani Minoru regularly read-out 30-40 moves. As an AGAAmateur 4 dan I read 10 moves pretty regularly, that's including variations. And if the sequence includes joseki (known optimal sequences of 15-20+ moves), then pros will read even deeper...
  - sawwit 11y ago
    Yes, the latter number was perhaps too conservative; no doubt about deeper predictions being easily possible, but I doubt even expert players consider many alternative paths in the search tree. They might recognize overall strategies which reach many moves into the future, but extensive consideration of what will happen in the upcoming moves is probably constrained to a only few steps; at least relative to the number and depths of paths that AlphaGo considers.
- jibalt 11y ago
  "while this method does tree search all the way to the end of the game" No it doesn't. You seem quite happy to just make stuff up that you know nothing about, like "2-3 moves into the future".
zodiac 11y ago
the expert human games are used just to predict future moves
reddytowns 11y ago
If you took one expert and faced him against a room full of experts who all together decided on the next move, who would win?
- blackskad 11y ago
  The one expert, because the others would not be able to reach a decision on which move to play.
  - reacweb 11y ago
    In fact, no. A big group of average experts appears to be better than a single super expert. This is the principal justification for the success of AI in oil prospective (https://books.google.fr/books?id=6DNgIzFNSZsC&pg=SA30-PA5&lpg=SA30-PA5&dq=AI+oil+system+experts&source=bl&ots=A53ahNSAXV&sig=C49TeZpEwpXuI2uGPyvLDFjrK2U&hl=fr&sa=X&ved=0ahUKEwiWg93crMzKAhVEjnIKHWV_DzUQ6AEIMzAE#v=onepage&q=AI%20oil%20system%20experts&f=false https://books.google.fr/books?id=6DNgIzFNSZsC&pg=SA30-PA5&lp...)
    - Jach 11y ago
      Counterpoint: https://en.wikipedia.org/wiki/Kasparov_versus_the_World https://en.wikipedia.org/wiki/Kasparov_versus_the_World I think a key missing component to crowd success on real expert knowledge (as opposed to trivia) is captured by the concept of prediction markets. (https://en.wikipedia.org/wiki/Prediction_market https://en.wikipedia.org/wiki/Prediction_market) The experts who are correct will make more money than the incorrect ones and eventually drive them out of the market for some particular area.
      - jibalt 11y ago
        That's no counterpoint because the World team (of which I was a member) was made up of boobs on the internet, not players of Kasparov's strength, which was the premise of the question you responded to.
    - blackskad 11y ago
yvsong 11y ago
I concluded that the all time no. 1 master Go Seigen's secret is 1. learn from all masters; 2. keep inventing/innovating. Most experts do 1 well, and are pretty much stuck there. Few are good at 2. I doubt if computers can invent/innovate.
- kitd 11y ago
  I would have thought (he says casually) that some kind of genetic algorithm of introducing random moves and evaluating outcomes for success would be entirely possible, no?
  - DanBC 11y ago
    There's a large space of random moves. How many are likely to be useful?
    - jibalt 11y ago
      Do you ask that of natural evolution, too?
- jibalt 11y ago
  "I doubt if computers can invent/innovate." Sheer ignorance.
brian_cloutier 11y ago
This was the question which originally led me to lose faith in deep learning for solving go. Existing research throws a bunch of professional games at a DCNN and trains it to predict the next move. It generally does quite well but fails hilariously when you give it a situation which never comes up in pro games. Go involves lots of implicit threats which are rarely carried out. These networks learn to make the threats but, lacking training data, are incapable of following up. The first step of creating AlphaGo worked the same way (and actually was worse at predicting the next move than current state of the art), but Deep Mind then took that base network and retrained it. Instead of playing the move a pro would play it now plays the move most likely to result in a win. For pros, this is the same move. But for AlphaGo, in this completely different MCTS environment, they are quite different. Deep Mind then played the engine against older versions of itself and used reinforcement learning to make the network as accurate as possible. They effectively used the human data to bootstrap a better player. The paper used a lot of other cool techniques and optimizations, but I think this one might be the coolest.
- mourner 11y ago
  Fantastic explanation, thank you!

If their learning material is based on expert human games, how can it ever get better than that?

space_fountain 11y ago

How can a human ever get better than their teacher? In this case though they play and optimize against themselves

kazinator 11y ago
> How can a human ever get better than their teacher? By learning from other teachers, and by applying original thought. Also, due to innately superior intelligence. If your IQ is 140, and that of the teacher is 105, you will eventually outstrip the teacher.
- jibalt 11y ago
  The question was rhetorical. And what is needed is aptitude for the specific task, not "IQ" ... the two are often very different.

sawwit 11y ago

It's because they have a much larger stack size than a human brain (which does not have a stack at all, but just various kinds of short term memories). An expert Go player can realistically maybe consider 2-3 moves into the future and can have a rough idea about what will happen in the coming 10 moves, while this method does tree search all the way to the end of the game on multiple alternative paths for each move.

donmaq 11y ago
Not true. Profession go players read out 20+ moves consistently. Go Seigan's nemesis Kitani Minoru regularly read-out 30-40 moves. As an AGAAmateur 4 dan I read 10 moves pretty regularly, that's including variations. And if the sequence includes joseki (known optimal sequences of 15-20+ moves), then pros will read even deeper...
- sawwit 11y ago
  Yes, the latter number was perhaps too conservative; no doubt about deeper predictions being easily possible, but I doubt even expert players consider many alternative paths in the search tree. They might recognize overall strategies which reach many moves into the future, but extensive consideration of what will happen in the upcoming moves is probably constrained to a only few steps; at least relative to the number and depths of paths that AlphaGo considers.
jibalt 11y ago
"while this method does tree search all the way to the end of the game" No it doesn't. You seem quite happy to just make stuff up that you know nothing about, like "2-3 moves into the future".

zodiac 11y ago

the expert human games are used just to predict future moves

reddytowns 11y ago

If you took one expert and faced him against a room full of experts who all together decided on the next move, who would win?

blackskad 11y ago
The one expert, because the others would not be able to reach a decision on which move to play.
- reacweb 11y ago
  In fact, no. A big group of average experts appears to be better than a single super expert. This is the principal justification for the success of AI in oil prospective (https://books.google.fr/books?id=6DNgIzFNSZsC&pg=SA30-PA5&lpg=SA30-PA5&dq=AI+oil+system+experts&source=bl&ots=A53ahNSAXV&sig=C49TeZpEwpXuI2uGPyvLDFjrK2U&hl=fr&sa=X&ved=0ahUKEwiWg93crMzKAhVEjnIKHWV_DzUQ6AEIMzAE#v=onepage&q=AI%20oil%20system%20experts&f=false https://books.google.fr/books?id=6DNgIzFNSZsC&pg=SA30-PA5&lp...)
  - Jach 11y ago
    Counterpoint: https://en.wikipedia.org/wiki/Kasparov_versus_the_World https://en.wikipedia.org/wiki/Kasparov_versus_the_World I think a key missing component to crowd success on real expert knowledge (as opposed to trivia) is captured by the concept of prediction markets. (https://en.wikipedia.org/wiki/Prediction_market https://en.wikipedia.org/wiki/Prediction_market) The experts who are correct will make more money than the incorrect ones and eventually drive them out of the market for some particular area.
    - jibalt 11y ago
      That's no counterpoint because the World team (of which I was a member) was made up of boobs on the internet, not players of Kasparov's strength, which was the premise of the question you responded to.
  - blackskad 11y ago

yvsong 11y ago

I concluded that the all time no. 1 master Go Seigen's secret is 1. learn from all masters; 2. keep inventing/innovating. Most experts do 1 well, and are pretty much stuck there. Few are good at 2. I doubt if computers can invent/innovate.

kitd 11y ago
I would have thought (he says casually) that some kind of genetic algorithm of introducing random moves and evaluating outcomes for success would be entirely possible, no?
- DanBC 11y ago
  There's a large space of random moves. How many are likely to be useful?
  - jibalt 11y ago
    Do you ask that of natural evolution, too?
jibalt 11y ago
"I doubt if computers can invent/innovate." Sheer ignorance.

brian_cloutier 11y ago

This was the question which originally led me to lose faith in deep learning for solving go. Existing research throws a bunch of professional games at a DCNN and trains it to predict the next move. It generally does quite well but fails hilariously when you give it a situation which never comes up in pro games. Go involves lots of implicit threats which are rarely carried out. These networks learn to make the threats but, lacking training data, are incapable of following up. The first step of creating AlphaGo worked the same way (and actually was worse at predicting the next move than current state of the art), but Deep Mind then took that base network and retrained it. Instead of playing the move a pro would play it now plays the move most likely to result in a win. For pros, this is the same move. But for AlphaGo, in this completely different MCTS environment, they are quite different. Deep Mind then played the engine against older versions of itself and used reinforcement learning to make the network as accurate as possible. They effectively used the human data to bootstrap a better player. The paper used a lot of other cool techniques and optimizations, but I think this one might be the coolest.

mourner 11y ago
Fantastic explanation, thank you!