6 ms·

Apparently 2 stones is a huge advantage. An estimate is that the computer is roughly 4-600 ELO stronger on an even match. Also, the human played a strategy tai

by dlevine 16d ago

Apparently 2 stones is a huge advantage. An estimate is that the computer is roughly 4-600 ELO stronger on an even match. Also, the human played a strategy tailored to that huge initial advantage. He said that the AI did not handle this particularly well, and played high probability moves instead of trying to lure him into a mistake. Also, even though this was the best Go engine, it was not running on a supercomputer, and had a relatively limited amount of time per move. So, this was an important victory for a human, but not a sign that humans are now stronger than AIs at Go.

kadoban 15d ago
> Also, the human played a strategy tailored to that huge initial advantage. He said that the AI did not handle this particularly well, and played high probability moves instead of trying to lure him into a mistake. Yeah, katago's training is not really focused at all on handicap games, because it's by nature learning from even games against similar-strength opponents. It doesn't have specific training from playing in a way to exploit a weaker player. In a handicap game you have to give your opponent opportunities to fuck up if you want to play optimally. If a move loses 0.0005 points if the opponent plays optimally, katago won't play it even if there's ~zero chance a weaker player would play it right. There have been go AIs that tried to train more directly on uneven opponents, one called "sai" comes to mind, but katago has huge advantages otherwise and won out over the others (for very good reason, it's a great project).
- rudi-c 15d ago
  While AlphaGo originally only had win rate as a metric, modern Go AIs have more knobs, including an evaluation of "complexity". Just stating this off the top of my head so I could be misremembering, but I heard that the KataGo settings used were tweaked to favor complexity. This was most apparent in Game 1 which Shin Jinseo lost, where the AI had an unusual opening. However, the last game was quite plain leading me to wonder whether that setting was present in the last game (or at all).
  - foota 15d ago
    Ah this is interesting. Essentially the idea is that the compute can try and move into positions that it can evaluate but humans might have trouble evaluating because of the board state's complexity?
    - pmontra 14d ago
      By the way that is exactly what humans do when playing with white in high stone handicap games. They place their stones all around the board, start little fights everywhere and wait for the weaker player to misread or misevaluate something. Suddenly two or three fights merge in a one sided larger one and part of the handicap is gone.
    - kadoban 13d ago
      That's the basic idea, yeah. When playing white in handicap games, you want to make your opponent uncomfortable. Play moves where the simple/safe/obvious move is just a little bit bad. Force them to choose between complex fights or a slow death of 100 slightly suboptimal moves. It feels really wrong to defend like 10 times in a row, so if you make them do that they'll lash out at the wrong time and you can take advantage. You also want to look for moves where...even if their best response means it's even or a little bit worse for you, there's ~reasonable responses where you win out or it goes complex. A lot of the time it's not even crazy complex fights, it's more just situations where the judgement of what is more points is difficult. (Note: most of this stops applying as strongly if it's a teaching game, which most handicap games are, there you have other considerations besides winning)
  - kadoban 15d ago
    There are tweaks to move choice like that, but it's all evaluation time. None of that happens in training. Zero times in the RL loop does katago see a two stone game against a weaker player. You can kind of tweak towards play this metric or that, but it's not the same.
    - Alpha3031 14d ago
      Maybe not to the extent you mentioned earlier (I definitely agree that it's not "focused" on this) but there is definitely asymmetric playout training in the training data (see comments on lightvector/KataGo issues #39 and #162 mentioning it for example), that is presumably how they got the tweak for playoutDoublingAdvantage (i.e. actually having a few thousand of the millions upon millions of training games be games where playouts have been doubled for advantage).
wslh 15d ago
> So, this was an important victory for a human, but not a sign that humans are now stronger than AIs at Go. Another way to look at this: Go's handicap system gives us a genuinely interesting metric for the distance between a human and a machine at this specific game. Instead of just "computers beat humans" we get a quantified gap.
prmph 15d ago
The question of whether machines or humans are stronger is moot, isn't it? In any intellectual contest between human and machine, all the machine winning implies is that the endeavor is algorithmic. The machine can be given practically unlimited memory and compute; we consider it cheating if the human would use memory aids. The machine could be implemented as many agents cooperating; we'd think it's not right if thousands of humans collaborated to face the machine, etc. So statements like "not a sign that humans are now stronger than AIs at Go" are pretty meaningless, IMO
- fn-mote 15d ago
  > not a sign that humans are now stronger than AIs at Go Totally wrongheaded, actually, since the computer gave the human a 2 stone advantage from the start.
tucnak 15d ago
I'm sorry, but Elo is not an acronym for Electronic Light Orchestra. You don't write ELO, but simply Elo.

by dlevine 16d ago

kadoban 15d ago

> Also, the human played a strategy tailored to that huge initial advantage. He said that the AI did not handle this particularly well, and played high probability moves instead of trying to lure him into a mistake. Yeah, katago's training is not really focused at all on handicap games, because it's by nature learning from even games against similar-strength opponents. It doesn't have specific training from playing in a way to exploit a weaker player. In a handicap game you have to give your opponent opportunities to fuck up if you want to play optimally. If a move loses 0.0005 points if the opponent plays optimally, katago won't play it even if there's ~zero chance a weaker player would play it right. There have been go AIs that tried to train more directly on uneven opponents, one called "sai" comes to mind, but katago has huge advantages otherwise and won out over the others (for very good reason, it's a great project).

rudi-c 15d ago
While AlphaGo originally only had win rate as a metric, modern Go AIs have more knobs, including an evaluation of "complexity". Just stating this off the top of my head so I could be misremembering, but I heard that the KataGo settings used were tweaked to favor complexity. This was most apparent in Game 1 which Shin Jinseo lost, where the AI had an unusual opening. However, the last game was quite plain leading me to wonder whether that setting was present in the last game (or at all).
- foota 15d ago
  Ah this is interesting. Essentially the idea is that the compute can try and move into positions that it can evaluate but humans might have trouble evaluating because of the board state's complexity?
  - pmontra 14d ago
    By the way that is exactly what humans do when playing with white in high stone handicap games. They place their stones all around the board, start little fights everywhere and wait for the weaker player to misread or misevaluate something. Suddenly two or three fights merge in a one sided larger one and part of the handicap is gone.
  - kadoban 13d ago
    That's the basic idea, yeah. When playing white in handicap games, you want to make your opponent uncomfortable. Play moves where the simple/safe/obvious move is just a little bit bad. Force them to choose between complex fights or a slow death of 100 slightly suboptimal moves. It feels really wrong to defend like 10 times in a row, so if you make them do that they'll lash out at the wrong time and you can take advantage. You also want to look for moves where...even if their best response means it's even or a little bit worse for you, there's ~reasonable responses where you win out or it goes complex. A lot of the time it's not even crazy complex fights, it's more just situations where the judgement of what is more points is difficult. (Note: most of this stops applying as strongly if it's a teaching game, which most handicap games are, there you have other considerations besides winning)
- kadoban 15d ago
  There are tweaks to move choice like that, but it's all evaluation time. None of that happens in training. Zero times in the RL loop does katago see a two stone game against a weaker player. You can kind of tweak towards play this metric or that, but it's not the same.
  - Alpha3031 14d ago
    Maybe not to the extent you mentioned earlier (I definitely agree that it's not "focused" on this) but there is definitely asymmetric playout training in the training data (see comments on lightvector/KataGo issues #39 and #162 mentioning it for example), that is presumably how they got the tweak for playoutDoublingAdvantage (i.e. actually having a few thousand of the millions upon millions of training games be games where playouts have been doubled for advantage).

wslh 15d ago

> So, this was an important victory for a human, but not a sign that humans are now stronger than AIs at Go. Another way to look at this: Go's handicap system gives us a genuinely interesting metric for the distance between a human and a machine at this specific game. Instead of just "computers beat humans" we get a quantified gap.

prmph 15d ago

The question of whether machines or humans are stronger is moot, isn't it? In any intellectual contest between human and machine, all the machine winning implies is that the endeavor is algorithmic. The machine can be given practically unlimited memory and compute; we consider it cheating if the human would use memory aids. The machine could be implemented as many agents cooperating; we'd think it's not right if thousands of humans collaborated to face the machine, etc. So statements like "not a sign that humans are now stronger than AIs at Go" are pretty meaningless, IMO

fn-mote 15d ago
> not a sign that humans are now stronger than AIs at Go Totally wrongheaded, actually, since the computer gave the human a 2 stone advantage from the start.

tucnak 15d ago

I'm sorry, but Elo is not an acronym for Electronic Light Orchestra. You don't write ELO, but simply Elo.