11 ms·

Strongest chess player, ever

170 points by gren 12y ago

EGreg 12y ago
I bet it would be unnerving for someone like Kasparov or Magnus Carlsen to play this program, where it would have 1 minute on the clock and they could have the whole day. It would make many of the moves in under a second and they'd be better than the grandmasters' moves!
- radicalbyte 12y ago
  It wouldn't surprise me if the top players regularly played against the best algorithms as part of their training.
  - _flag 12y ago
    It's actually not that common. Players will use chess engines to analyze positions, but they will rarely play against them because it doesn't provide realistic practice for an actual chess match. For example, it may be reasonable to play an aggressive variation against an opponent because you think they might have difficulty finding a response in time pressure, but a computer can make precise calculations in any situation, and so such a strategy almost always backfires. What's more, chess is often abstracted at higher levels in terms of things like long term plans ("I want to put pressure on the c7 pawn and control c6") instead of concrete material gains, which allows players to make progress even in positions where there are no direct threats and no sensible exchanges of pieces. Computers when faced with such situations will know that the position is objectively drawn and so will just shuffle their pieces around aimlessly, and playing against this kind of thing is not very good practice for actual human opponents who will try to find ways to beat you anyway.
  - Isofarro 12y ago
    Only realistic if they were preparing for some sort of Man vs Computer tournament. Computers don't play like humans, and they are consistent in their weaknesses as well as their strengths. As players are more exposed to playing against various chess programs, they learn what positions computers are not so good at (long-range strategical themes, beyond the typical move horizon). A local optimisation, of sorts. So players start adopting an anti-computer style of play, which requires a different mind set to pull off. That in turn affects their natural style of play. So I argue the opposite, for preparing for top-flight tournaments, players probably avoid / severely limit playing against chess engines, to avoid this natural anti-computer bias to their play. Sure, the engines are good for checking variations and analysis, as an assistant. But not as a leader, nor as an opponent.
  - halfcat 12y ago
    Olympic sprinters do not practice racing against race cars. Maybe for a publicity stunt. But not for improvement.
- StavrosK 12y ago
  Have computers and software progressed that much since the Deep Blue days? I didn't realize that chess has pretty much been solved now. Are computers really that much better than people nowadays?
  - Bootvis 12y ago
    The computer is incredibly strong and there is no interest for a Man again Computer match but I wouldn't say it's close to being solved or that it's impossible to make further progress for three reasons: 1) Computer vs Computer can still go both ways so clearly the one chess engines make mistake others can exploit; 2) Some sequences are still hard find for computers, e.g. [1]. The long king walk by White is beyond the horizon of the computer. 3) Computer + Man against Computer + Man correspondence matches do benefit from human intervention. [1]: http://www.chessgames.com/perl/chessgame?gid=1124533&kpage=5 http://www.chessgames.com/perl/chessgame?gid=1124533&kpage=5
  - Isofarro 12y ago
    Stockfish has been available for quite a while, and over the last few years has been recognised as being the strongest open-source chess engine. It's fantastic to see it take on the best commercial engines toe-to-toe and come out on top. These ratings are computer ratings, and because they are playing very much isolates (computers vs other computers), without enough external human encounters, the ratings are consistent within the computer sphere, but not really like-for-like with human rating lists. A lot has happened since Deep Blue - that was just a 6 game match, which really only proved that machines can play near a grandmaster level, but without nerves. And at times that is good enough to beat a world champion in match conditions, because the human is more susceptible to the psychological events. Since Deeper Blue we've seen the emergence of chess engines on personal computers (rather than racks and racks of RS6000 mid-range servers), so Chess Genius, Fritz, Shredder, Junior, Rybka, Houdini and now Stockfish pushing chess engines ahead. The Chess Genius - Junior levels show chess engines capable of matching grandmasters at blitz/active-play speeds, but still struggling at slower time controls. Rybka onwards show a computer engine where elite grandmasters are seriously challenged. Chess hasn't been solved. Just watching Magnus Carlsen's play shows we are expanding chess knowledge incrementally one game at a time. Yes, progress has slowed down, but the incremental improvement are still there. Chess technique is refined to such a super high degree, but still far short of "solving chess". Perhaps this current generation can beat grandmasters when psychological factors are minimised, it's hard to tell. It's certainly not as clear cut as a Porsche vs Usain Bolt over 100 meters.
- _flag 12y ago
  You can watch Magnus Carlsen play a chess program on his phone here: https://www.youtube.com/watch?v=pNvVWeHZG00 https://www.youtube.com/watch?v=pNvVWeHZG00 As you can tell from his commentary, he thinks the machine is rather stupid (and he is winning after the opening) but the machine is a much better calculator than he is and when the situation becomes more concrete he has to force a draw. Of course, the computer on his phone is considerably worse than the best chess engines, but top chess players generally consider computers to be excellent calculators but dumb in terms of general strategy.
  - ars 12y ago
    > but top chess players generally consider computers to be excellent calculators but dumb in terms of general strategy. Is computer assisted chess a thing? Perhaps with standardized hardware, but any software. In chess I'm good at strategy but terrible at calculating and I miss obvious stuff all the time in my fight for strategy. I always thought I'd do great with computer assistance to look for the obvious stuff, and me telling the computer the long term strategy.
    - Bootvis 12y ago
      It is. See Centaur chess[1] and correspondence chess. In correspondence chess people don't use standardized hardware (people will probably cheat anyway) but in Centaur chess this can be done. [1]: http://en.wikipedia.org/wiki/Advanced_Chess http://en.wikipedia.org/wiki/Advanced_Chess
    - deleted 12y ago
      [deleted]
    - _flag 12y ago
      > Is computer assisted chess a thing? It was called Advanced Chess [1], now it's sometimes referred to as Centaur Chess. You can find online servers to play this kind of thing but it's mostly just a sort of curiosity. Computer-human pairs are well known to be better than just computers at chess though. 1. http://en.wikipedia.org/wiki/Advanced_Chess http://en.wikipedia.org/wiki/Advanced_Chess
    - 12y ago
- modernerd 12y ago
  “I find playing against computers depressing. I don’t like losing. I also think it's not so useful practise. Even though computers have become more human in their style ... it doesn't help you too much in preparation for playing humans (which are still my main opponents).” — Magnus Carlsen, in an interview with Peter Thiel at the Churchill Club: https://www.youtube.com/watch?v=ZBnSU-LX1ss&t=23m51s https://www.youtube.com/watch?v=ZBnSU-LX1ss&t=23m51s
kristopolous 12y ago
note that it's just a chess "engine" and doesn't come with a human-usable front-end. But it's not hard to get the thing to use xboard in linux. This is what i use to invoke it: xboard -fUCI -fcp stockfish -sUCI -scp stockfish& it's very challenging - and in my experience, will unfortunately peg a core unless you explicitly pause the front end (the "P" button in between the two arrows in the upper right)
- y4mi 12y ago
  what do you mean? just click play a game http://en.lichess.org/ http://en.lichess.org/
  - kristopolous 12y ago
    Yes that way also works! But the xboard way allows me to modify the engine, get debug output, try various different weights for pieces, put two engines against each other to see how specific features affect game play... It helps me demystify such sophisticated software.
    - y4mi 12y ago
      heh, did you ever upload a match between two engines to the site and let the online engine analyze it?
      - toxik 12y ago
        Very interesting idea!
arek2 12y ago
I don't think that the present situation, when the top chess playing program is free and open-source, is good for innovation. I estimate that it would take me six months of work to get to the top-20 in the world, and I don't see how I can justify that work to myself.
- DanBC 12y ago
  I did not downvote you. Your comment seems either ignorant of the amount of work involved, or very arrogant about your skill, but does not provide any information for me to judge. Why do you think you could beat even rybka with just six months of work?
  - arek2 12y ago
    Rybka is better than top-20, so I did not claim that. I was a half-pro ~10 years ago. I won the championship of my country. I wrote an M.Sc. thesis on evaluation tuning. If I'd use the standard approach that I already know, 6 months is a conservative estimate. The question is - what for?
    - hollerith 12y ago
      >The question is - what for? Uh, to win prize money?
- deutronium 12y ago
  Seems like an idle boast to me. Also why isn't open source good for innovation?
anatoly 12y ago
Can anyone comment on what makes stockfish different from other chess engines? If I'm curious about state of the art in computer chess, is it worthwhile to study its source? What interesting ideas should I expect to see there beyond what I vaguely know to be the standard approach from introductory AI courses, i.e. some sort of alpha-beta pruning search?
- onestone 12y ago
  What sets Stockfish apart is its open testing framework. Many volunteers allocate CPU time to run test games, checking if a pull request actually adds Elo rating points. Also, its open development model is a big advantage compared to closed-source chess engines, which usually just have 1-2 developers. The ideas used are mostly the same in all top engines, the advantage of Stockfish comes from tuning/testing/polishing/tweaks. An year ago Stockfish was near the top, but not quite there. Since then its development model became more open, and the open testing framework was introduced. The rate it's been adding Elo rating points since then has been amazing.
- ekr 12y ago
  I'm not familiar with it, but I'd assume most of them use the same mini-max, with alpha-beta prunning, or some similar algorithm, with small improvements, especially in the heuristic state evaluation function. You'd learn more by studying a book on AI, and maybe trying to implement an engine yourself, iMO. The difference in score between the top engines is not that big.
- arek2 12y ago
  I have not seen the Stockfish source, but my impression is that computer chess was going round in circles in the last 10 years, and it's still good old alpha-beta search with more refined heuristics. The biggest value in studying computer chess for a programmer is IMO in seeing all the different performance optimization tricks.
  - halfcat 12y ago
    Performance optimization, while it is not neglected, is not what separates chess engines. You don't win an exponential race with linear advantages.
saykou 12y ago
Aside from the topic, the lichess.org captcha on inscription is a chess challenge? I advise against, that is one thing that computers are better than us, and goes everything what a captcha should be xD
thejteam 12y ago
What I would love to see is work on chess engines having better strength levels for amateurs. For me at least there is a line where everything below a certain level I can beat 95 percent of the time and everything at or above that level I lose to 95 percent of the time.
- notacoward 12y ago
  One of my long-term goals is to write a chess program that plays more like a lower-level human. Every program I've played seems to have the same pattern: play like a master for ten moves, then make an insultingly obvious blunder to let me catch up. Lather, rinse, repeat. It's neither realistic nor fun. Might as well play with a handicap. Real players at each level tend to have characteristic weaknesses that are expressible in terms of the same factors used in a program's evaluation function. Consider some of the following, which players at a certain level will exhibit and then get past as they improve. * Bringing the queen out too early. * Missing pins and discoveries. * Failing to contest the center. * Creating bad bishops. * Bad pawn structure. * Failing to use the king effectively in the endgame. These are all quantifiable. They could all be used to create a more realistic and satisfying opponent at 1200, 1500, 1800, etc. All it takes is some basic machine learning applied to a corpus of lower-level games, and a way to plug the discovered patterns into the playing engine.
- Isofarro 12y ago
  Shredder (on iOS and Android, at least) has the capacity to tailor it's playing strength based on your success with it. It tries to match your own playing strength to give you a more interesting game. As you win it gets stronger, as you lose it plays weaker.
- Steko 12y ago
  Pole Position mode. The AI quality ramps up when losing and down when winning.
- zem 12y ago
  that's a big problem in scrabble AIs too - there are heuristics for how to dial down the difficulty level, but they mostly involve things like finding the list of best possible moves, and then randomly picking a suboptimal one with how often you do it and how far down the list you go weighted by the handicap level, or constraining the dictionary, or not using lookahead. the problem is that it doesn't always model how actual low-rated scrabble players play, and so does not really make for a satisfying opponent - every now and then you hit a "well, that was stupid" move that just feels implausible for even a weaker human player to make. the other problem is that the devs of the current top scrabble engines, quackle and elise, are (naturally) focused on getting better and better at playing, not on plausible ways to play badly. it's something i keep meaning to work on when i have some spare time; i have a few ideas, but nothing i've had the time to explore properly.
DanBC 12y ago
Are chess engines still trying to play against humans in an interesting way? (I understand they beat human players, but that people feel computers play in dull ways). Is there a Turing Test for computer chess, where humans and computers play each other and they, and commentators, analyse the play, but no-one knows who is a computer or human until after the commentary is published? And if we ignore humans are people playing computers against other computers for some kind of machine learning play? And how optimized for speed is the software? Do they really crunch out all performance they can? (Sorry for the barrage of questions but I don't know enough about this space to do efficient websearches).
- cven714 12y ago
  > Are chess engines still trying to play against humans in an interesting way? (I understand they beat human players, but that people feel computers play in dull ways). Depends on what you mean by dull. Computers play so well that it tends to be a complete mop-up regardless of any "anti-computer" strategies people may try. The dull aspect is the one-sidedness. What's hard to do is make computers play weakly in a human-like way. Lobotomized computers tend to make very inhuman blunders. > Is there a Turing Test for computer chess, where humans and computers play each other and they, and commentators, analyse the play, but no-one knows who is a computer or human until after the commentary is published? Not that I've ever seen, though it still wouldn't be much of a challenge. Computer moves are pretty easy to spot--generally unintuitive or seemingly paradoxical moves that have a very concrete justification. Especially, as I said above, if they were set to play at a weaker level. > And if we ignore humans are people playing computers against other computers for some kind of machine learning play? I'll let others answer this, but it would surprise me if nobody was. Still, improving evaluation heuristics and analysis efficiency seems to be yielding better results than just machine learning. > And how optimized for speed is the software? Do they really crunch out all performance they can? Yes. The more positions you can examine per second, the deeper your search tree can go, and the better your evaluations, etc.
- omaranto 12y ago
  > I understand they beat human players, but that people feel computers play in dull ways. Without knowing anything about chess, I'd say that makes humans sounds like sore losers. I wonder if this will lead to having tournaments where you don't get points for winning matches but rather get judged on style by a panel of (human) judges...
alien3d 12y ago
Just try havin fun.. http://en.lichess.org/SMVJP07p http://en.lichess.org/SMVJP07p The strongest player.. me kinda noobs..
basic_stat 12y ago
> During the final event, after playing 64 games against Komodo, Stockfish won with the score of 35½-28½. No doubt is further allowed: Stockfish is the best chess player ever! From a statistical point of view, this isn't actually significant, despite the fact that draws help reduce the variance. 45 of those games are draws, leaving a 13-6 score in favor of Stockfish. Considering a null hypothesis of a binomial distribution with n=19 and equal chance of winning, the two-sided p-value for that score is 0.115. Unless you already have strong evidence that Stockfish is better than Komodo, you shouldn't conclude anything about which one is best.
- eterm 12y ago
  But it fundamentally _isn't_ binomial across 19 games, because of draws. You can't just ignore draws from the analysis, to do so is terrible application of statistics.
  - jmount 12y ago
    The first analysis isn't terrible. It gets the important points right (that the draws are not evidence of difference in skill) and moves on to the remaining evidence (difference in wins to losses). The draws are at best evidence towards equality (not against it). Allow them to vary and the likelihood of seeing a difference of 9 wins in 64 games with 45 draws moves up to 0.13 or 13% (when we assume the two players are identical, an appropriate null hypothesis) (even less significant). So in about one tournament in 8 you would expect this much of a lead, even if it was one algorithm playing itself. So from one tournament we say it is likely the one algorithm is in fact better, but it doesn't rise to the standard of being statistically significant. <code> # R code to empirically estimate two-sided probablity of # seeing a lead of 9 games when 64 games are played # and the assumed probability of a draw is 45/64 # with the null assumption win/loss odds are equal simulate <- function(nplay,ndraw) { sample(c('w','d','l'),size=nplay,replace=TRUE, prob=c((nplay-ndraw)/2,ndraw,(nplay-ndraw)/2)/nplay) } wldiff <- function(v) { abs(sum(v=='w')-sum(v=='l')) } set.seed(350920) stats <- replicate(10000,wldiff(simulate(64,45))) print(sum(stats>=13-6)/length(stats)) ## [1] 0.1341 </code> (it is weird that somebody, not me, created a throw-away account to make the original comment. likely they are involved in chess development, or know how quickly stat discussions go sideways)
jtth 12y ago
(Side note: what the hell is that website trying to do to my computer? It's trying to open so many connections on weird ports!)
- showerst 12y ago
  Looks like they're using a bunch of websockets for something. Probably that live player count up in the navigation.
- ornicar 12y ago
  lichess opens only one websocket connection, on a port between 9021 to 9029. That port range prevents a weird behavior on firefox when the connection is interrupted.
asdfologist 12y ago
This is extremely impressive - it's cool that talented programmers are pushing the limits of computer science to advance the state of the art of chess engines. However, I also wish that there were comparable efforts to create AIs that train humans. Basically, figure out a way to systematically, efficiently, and scalably train amateurs into masters. That IMO would be absolutely amazing (and something I'd gladly pay for).
- mnemonicsloth 12y ago
  Are you sure you wouldn't rather learn, say, mathematical physics or computer science? The neurological topiary for chess is strictly harder, because chess has fewer analogues in day-to-day experience.
  - asdfologist 12y ago
    Chess is a fun hobby, and I feel it gets more fun the better I get. That's a good enough justification for me.
  - cwp 12y ago
    It's a place to start. It's not a huge leap from modelling the game of chess to modelling the skill of chess-playing. Stockfish already offers post-game analysis. Imagine extending that to analyze a players entire career, and offer a series of problems, games against specially-constructed opponents, analysis of relevant historical games etc, all of which are designed to help the player improve. Given Siri, Google Now, Watson and so on, we're probably not far from being able to have a meaningful natural-language conversation with a computer on a narrow subject like chess. One could imagine this kind of thing being extended to teaching other, similarly focussed skills, at least for beginners. Piano. Tennis. Rock climbing. Maybe even things like soccer and basketball. But that kind of teaching-based-on-deep-analysis is a long way off for subjects like physics or electrical engineering. Computers can't do physics, much less evaluate human physicists. The best we can do in these areas is something like the Khan Academy, where computers present "content" created by humans, administer standardized tests designed by humans and present the results to humans for interpretation. So yeah, teaching chess in a really sophisticated way isn't all that useful in the sense that physics or EE are useful. But really, if we could teach computers to understand physics better than people do, we'd use them to make breakthroughs in physics, and that would be a much bigger deal than being able to teach physics more effectively. On the other hand, we don't play chess or tennis or piano because they're useful, so expert AI teachers for these subjects would be really valuable.
pk2200 12y ago
Stockfish 5 was released yesterday: http://stockfishchess.org/ http://stockfishchess.org/ As mentioned elsewhere in this thread, Stockfish is just an engine - you must install a GUI separately. XBoard is well known, but there are better alternatives: http://chessx.sourceforge.net/ http://chessx.sourceforge.net/ http://scidvspc.sourceforge.net/ http://scidvspc.sourceforge.net/
beefman 12y ago
TCEC is cool but has less statistical power than many ratings lists out there, which show that Houdini, Komodo and Stockfish are very closely matched, with Houdini having a slight edge at long time controls and a moderate edge at quick time controls. Stockfish does release more frequently and I'm not sure which version competed in TCEC, but until the lists catch up this article is fluff.
- onestone 12y ago
  Houdini had a slight edge against Stockfish DD, which is a few months old. The latest Stockfish version (v5, very similar to the version that competed in the TCEC Superfinal) is ~60 Elo stronger, and demolishes Houdini at all time controls. It should soon start hitting the more conservative ratinglists.
  - beefman 12y ago
    "Demolishes"? Hardly. It is so far tied with H4 on IPON. Even if it winds up on top, it will not be by a large margin.
runeks 12y ago
Has anyone else watched http://en.lichess.org/tv http://en.lichess.org/tv? If this is two humans playing against each other in real-time, that is really impressive! It's so mind-boggling fast (mind you I'm not a chess player).
- varg 12y ago
  Many games are played 'bullet' style - i.e. 60s each for the whole game.
- doughj3 12y ago
  ChessNetwork on YouTube[1] has many videos of blitz chess tournaments where he provides commentary as he plays. Very interesting to watch! [1] https://www.youtube.com/watch?v=7YWYS209ydE https://www.youtube.com/watch?v=7YWYS209ydE
  - electrotype 12y ago
    Wow, very cool! Can someone explain what's going on at 20:24 : http://youtu.be/7YWYS209ydE?t=20m20s http://youtu.be/7YWYS209ydE?t=20m20s ? He forks the king and the queen, then his opponent moved his queen!?! This is not a legal move right?
    - doughj3 12y ago
      Had to re-watch a couple times: on that move, white had already moved pawn from g2 to g3, the knight from d6 to e8 is a pre-move that happens very quickly (you can see the red square); while white was setting up that pre-move, black moved the queen. Does that clarify or did I miss what you are asking?
      - electrotype 12y ago
        Got it, thanks!
  - runeks 12y ago
    That's pretty cool! Can anyone inform an amateur like me what happens at 2:14 (https://www.youtube.com/watch?v=7YWYS209ydE#t=134 https://www.youtube.com/watch?v=7YWYS209ydE#t=134) and before that as well. It seems to me that black captures his own pawn using his knight. Is that legal? EDIT: Ah, I see. White pawn captures black pawn, and the knight captures the white pawn. I just happens so fast I didn't see the white move.
icambron 12y ago
I always thought of the heuristics for evaluating a chess position as the really hard part of building a chess engine; i.e. how do you capture all of the positional subtleties in a number to feed into minimax? But looking at the source, it's not really that complicated [1]. Can someone who knows more than me comment on that? Is it that the innovations are elsewhere? That good chess really can be boiled down to < 1000 LOC? That the numbers in this heuristic are just super expertly tuned? [1] https://github.com/mcostalba/Stockfish/blob/master/src/evaluate.cpp https://github.com/mcostalba/Stockfish/blob/master/src/evalu...
- glinscott 12y ago
  Nearly all the evaluation terms in Stockfish have been extensively tuned. Joona Kiiski used the SPSA[1] algorithm on a lot of them. Others have been hand-tuned using tens of thousands of games per attempt on fishtest [2]. Fishtest actually just recently got support for running SPSA tuning as well. There is also a strong bias towards simplification, so if an evaluation feature is not proven to be an improvement it will be removed. Over the last few Stockfish versions, the # of lines has actually decreased in each version. [1] http://en.wikipedia.org/wiki/Simultaneous_perturbation_stochastic_approximation http://en.wikipedia.org/wiki/Simultaneous_perturbation_stoch... [2] http://tests.stockfishchess.org/tests http://tests.stockfishchess.org/tests
  - icambron 12y ago
    That's helpful, thanks.
- deleted 12y ago
  [deleted]
- halfcat 12y ago
  What separates one chess engine from another is its ability to efficiently navigate and evaluate the game tree. Stockfish effectively discards probably 95% or more of the nodes in any position. Easier said than done, better not discard the best move in that 95%. Evaluation is relatively simple, a fancy lookup table. They say love covers a multitude of sins. Well, searching one move deeper than your opponent covers a multitude of static positional evaluation mistakes. Searching deeper can figure out the complexity of the position better than anything that can be statically evaluated. That's where virtually all improvements exist today, figuring out which moves can be safely discarded.
rmbe 12y ago
lichess doesn't seem to be giving it enough juice to perform at its stated levels at the moment. On my first try I managed to draw the highest AI level, rated at 2510, while my rating is under 2000 irl. (I was unable to find an "offer draw" button so relied on the 50-move rule) Against stockfish running on my PC that would be impossible. http://en.lichess.org/reDfuSvI http://en.lichess.org/reDfuSvI
- ornicar 12y ago
  Have you actually read the article? It's all explained http://en.lichess.org/blog/U4mtoEQAAEEAgZRL/strongest-chess-player-ever http://en.lichess.org/blog/U4mtoEQAAEEAgZRL/strongest-chess-...
  - rmbe 12y ago
    I did. I was complaining about the rated strength of the AI levels, they're off by a wide margin.
meowface 12y ago
I'm rather surprised at how relatively simple and small the codebase is: https://github.com/mcostalba/Stockfish/tree/master/src https://github.com/mcostalba/Stockfish/tree/master/src
- QuantumChaos 12y ago
  Especially interesting to see everything hardcoded into C++. Quite different from most modern codebases that use a mix of languages, configuration files, etc.
arikrak 12y ago
That site's pretty impressive. They're open source and give away for free what Chess.com charges for.

170 points by gren 12y ago

EGreg 12y ago

I bet it would be unnerving for someone like Kasparov or Magnus Carlsen to play this program, where it would have 1 minute on the clock and they could have the whole day. It would make many of the moves in under a second and they'd be better than the grandmasters' moves!

radicalbyte 12y ago
It wouldn't surprise me if the top players regularly played against the best algorithms as part of their training.
- _flag 12y ago
  It's actually not that common. Players will use chess engines to analyze positions, but they will rarely play against them because it doesn't provide realistic practice for an actual chess match. For example, it may be reasonable to play an aggressive variation against an opponent because you think they might have difficulty finding a response in time pressure, but a computer can make precise calculations in any situation, and so such a strategy almost always backfires. What's more, chess is often abstracted at higher levels in terms of things like long term plans ("I want to put pressure on the c7 pawn and control c6") instead of concrete material gains, which allows players to make progress even in positions where there are no direct threats and no sensible exchanges of pieces. Computers when faced with such situations will know that the position is objectively drawn and so will just shuffle their pieces around aimlessly, and playing against this kind of thing is not very good practice for actual human opponents who will try to find ways to beat you anyway.
- Isofarro 12y ago
  Only realistic if they were preparing for some sort of Man vs Computer tournament. Computers don't play like humans, and they are consistent in their weaknesses as well as their strengths. As players are more exposed to playing against various chess programs, they learn what positions computers are not so good at (long-range strategical themes, beyond the typical move horizon). A local optimisation, of sorts. So players start adopting an anti-computer style of play, which requires a different mind set to pull off. That in turn affects their natural style of play. So I argue the opposite, for preparing for top-flight tournaments, players probably avoid / severely limit playing against chess engines, to avoid this natural anti-computer bias to their play. Sure, the engines are good for checking variations and analysis, as an assistant. But not as a leader, nor as an opponent.
- halfcat 12y ago
  Olympic sprinters do not practice racing against race cars. Maybe for a publicity stunt. But not for improvement.
StavrosK 12y ago
Have computers and software progressed that much since the Deep Blue days? I didn't realize that chess has pretty much been solved now. Are computers really that much better than people nowadays?
- Bootvis 12y ago
  The computer is incredibly strong and there is no interest for a Man again Computer match but I wouldn't say it's close to being solved or that it's impossible to make further progress for three reasons: 1) Computer vs Computer can still go both ways so clearly the one chess engines make mistake others can exploit; 2) Some sequences are still hard find for computers, e.g. [1]. The long king walk by White is beyond the horizon of the computer. 3) Computer + Man against Computer + Man correspondence matches do benefit from human intervention. [1]: http://www.chessgames.com/perl/chessgame?gid=1124533&kpage=5 http://www.chessgames.com/perl/chessgame?gid=1124533&kpage=5
- Isofarro 12y ago
  Stockfish has been available for quite a while, and over the last few years has been recognised as being the strongest open-source chess engine. It's fantastic to see it take on the best commercial engines toe-to-toe and come out on top. These ratings are computer ratings, and because they are playing very much isolates (computers vs other computers), without enough external human encounters, the ratings are consistent within the computer sphere, but not really like-for-like with human rating lists. A lot has happened since Deep Blue - that was just a 6 game match, which really only proved that machines can play near a grandmaster level, but without nerves. And at times that is good enough to beat a world champion in match conditions, because the human is more susceptible to the psychological events. Since Deeper Blue we've seen the emergence of chess engines on personal computers (rather than racks and racks of RS6000 mid-range servers), so Chess Genius, Fritz, Shredder, Junior, Rybka, Houdini and now Stockfish pushing chess engines ahead. The Chess Genius - Junior levels show chess engines capable of matching grandmasters at blitz/active-play speeds, but still struggling at slower time controls. Rybka onwards show a computer engine where elite grandmasters are seriously challenged. Chess hasn't been solved. Just watching Magnus Carlsen's play shows we are expanding chess knowledge incrementally one game at a time. Yes, progress has slowed down, but the incremental improvement are still there. Chess technique is refined to such a super high degree, but still far short of "solving chess". Perhaps this current generation can beat grandmasters when psychological factors are minimised, it's hard to tell. It's certainly not as clear cut as a Porsche vs Usain Bolt over 100 meters.
_flag 12y ago
You can watch Magnus Carlsen play a chess program on his phone here: https://www.youtube.com/watch?v=pNvVWeHZG00 https://www.youtube.com/watch?v=pNvVWeHZG00 As you can tell from his commentary, he thinks the machine is rather stupid (and he is winning after the opening) but the machine is a much better calculator than he is and when the situation becomes more concrete he has to force a draw. Of course, the computer on his phone is considerably worse than the best chess engines, but top chess players generally consider computers to be excellent calculators but dumb in terms of general strategy.
- ars 12y ago
  > but top chess players generally consider computers to be excellent calculators but dumb in terms of general strategy. Is computer assisted chess a thing? Perhaps with standardized hardware, but any software. In chess I'm good at strategy but terrible at calculating and I miss obvious stuff all the time in my fight for strategy. I always thought I'd do great with computer assistance to look for the obvious stuff, and me telling the computer the long term strategy.
  - Bootvis 12y ago
    It is. See Centaur chess[1] and correspondence chess. In correspondence chess people don't use standardized hardware (people will probably cheat anyway) but in Centaur chess this can be done. [1]: http://en.wikipedia.org/wiki/Advanced_Chess http://en.wikipedia.org/wiki/Advanced_Chess
  - deleted 12y ago
    [deleted]
  - _flag 12y ago
    > Is computer assisted chess a thing? It was called Advanced Chess [1], now it's sometimes referred to as Centaur Chess. You can find online servers to play this kind of thing but it's mostly just a sort of curiosity. Computer-human pairs are well known to be better than just computers at chess though. 1. http://en.wikipedia.org/wiki/Advanced_Chess http://en.wikipedia.org/wiki/Advanced_Chess
  - 12y ago
modernerd 12y ago
“I find playing against computers depressing. I don’t like losing. I also think it's not so useful practise. Even though computers have become more human in their style ... it doesn't help you too much in preparation for playing humans (which are still my main opponents).” — Magnus Carlsen, in an interview with Peter Thiel at the Churchill Club: https://www.youtube.com/watch?v=ZBnSU-LX1ss&t=23m51s https://www.youtube.com/watch?v=ZBnSU-LX1ss&t=23m51s

kristopolous 12y ago

note that it's just a chess "engine" and doesn't come with a human-usable front-end. But it's not hard to get the thing to use xboard in linux. This is what i use to invoke it: xboard -fUCI -fcp stockfish -sUCI -scp stockfish& it's very challenging - and in my experience, will unfortunately peg a core unless you explicitly pause the front end (the "P" button in between the two arrows in the upper right)

y4mi 12y ago
what do you mean? just click play a game http://en.lichess.org/ http://en.lichess.org/
- kristopolous 12y ago
  Yes that way also works! But the xboard way allows me to modify the engine, get debug output, try various different weights for pieces, put two engines against each other to see how specific features affect game play... It helps me demystify such sophisticated software.
  - y4mi 12y ago
    heh, did you ever upload a match between two engines to the site and let the online engine analyze it?
    - toxik 12y ago
      Very interesting idea!

arek2 12y ago

I don't think that the present situation, when the top chess playing program is free and open-source, is good for innovation. I estimate that it would take me six months of work to get to the top-20 in the world, and I don't see how I can justify that work to myself.

DanBC 12y ago
I did not downvote you. Your comment seems either ignorant of the amount of work involved, or very arrogant about your skill, but does not provide any information for me to judge. Why do you think you could beat even rybka with just six months of work?
- arek2 12y ago
  Rybka is better than top-20, so I did not claim that. I was a half-pro ~10 years ago. I won the championship of my country. I wrote an M.Sc. thesis on evaluation tuning. If I'd use the standard approach that I already know, 6 months is a conservative estimate. The question is - what for?
  - hollerith 12y ago
    >The question is - what for? Uh, to win prize money?
deutronium 12y ago
Seems like an idle boast to me. Also why isn't open source good for innovation?

anatoly 12y ago

Can anyone comment on what makes stockfish different from other chess engines? If I'm curious about state of the art in computer chess, is it worthwhile to study its source? What interesting ideas should I expect to see there beyond what I vaguely know to be the standard approach from introductory AI courses, i.e. some sort of alpha-beta pruning search?

onestone 12y ago
What sets Stockfish apart is its open testing framework. Many volunteers allocate CPU time to run test games, checking if a pull request actually adds Elo rating points. Also, its open development model is a big advantage compared to closed-source chess engines, which usually just have 1-2 developers. The ideas used are mostly the same in all top engines, the advantage of Stockfish comes from tuning/testing/polishing/tweaks. An year ago Stockfish was near the top, but not quite there. Since then its development model became more open, and the open testing framework was introduced. The rate it's been adding Elo rating points since then has been amazing.
ekr 12y ago
I'm not familiar with it, but I'd assume most of them use the same mini-max, with alpha-beta prunning, or some similar algorithm, with small improvements, especially in the heuristic state evaluation function. You'd learn more by studying a book on AI, and maybe trying to implement an engine yourself, iMO. The difference in score between the top engines is not that big.
arek2 12y ago
I have not seen the Stockfish source, but my impression is that computer chess was going round in circles in the last 10 years, and it's still good old alpha-beta search with more refined heuristics. The biggest value in studying computer chess for a programmer is IMO in seeing all the different performance optimization tricks.
- halfcat 12y ago
  Performance optimization, while it is not neglected, is not what separates chess engines. You don't win an exponential race with linear advantages.

saykou 12y ago

Aside from the topic, the lichess.org captcha on inscription is a chess challenge? I advise against, that is one thing that computers are better than us, and goes everything what a captcha should be xD

thejteam 12y ago

What I would love to see is work on chess engines having better strength levels for amateurs. For me at least there is a line where everything below a certain level I can beat 95 percent of the time and everything at or above that level I lose to 95 percent of the time.

notacoward 12y ago
One of my long-term goals is to write a chess program that plays more like a lower-level human. Every program I've played seems to have the same pattern: play like a master for ten moves, then make an insultingly obvious blunder to let me catch up. Lather, rinse, repeat. It's neither realistic nor fun. Might as well play with a handicap. Real players at each level tend to have characteristic weaknesses that are expressible in terms of the same factors used in a program's evaluation function. Consider some of the following, which players at a certain level will exhibit and then get past as they improve. * Bringing the queen out too early. * Missing pins and discoveries. * Failing to contest the center. * Creating bad bishops. * Bad pawn structure. * Failing to use the king effectively in the endgame. These are all quantifiable. They could all be used to create a more realistic and satisfying opponent at 1200, 1500, 1800, etc. All it takes is some basic machine learning applied to a corpus of lower-level games, and a way to plug the discovered patterns into the playing engine.
Isofarro 12y ago
Shredder (on iOS and Android, at least) has the capacity to tailor it's playing strength based on your success with it. It tries to match your own playing strength to give you a more interesting game. As you win it gets stronger, as you lose it plays weaker.
Steko 12y ago
Pole Position mode. The AI quality ramps up when losing and down when winning.
zem 12y ago
that's a big problem in scrabble AIs too - there are heuristics for how to dial down the difficulty level, but they mostly involve things like finding the list of best possible moves, and then randomly picking a suboptimal one with how often you do it and how far down the list you go weighted by the handicap level, or constraining the dictionary, or not using lookahead. the problem is that it doesn't always model how actual low-rated scrabble players play, and so does not really make for a satisfying opponent - every now and then you hit a "well, that was stupid" move that just feels implausible for even a weaker human player to make. the other problem is that the devs of the current top scrabble engines, quackle and elise, are (naturally) focused on getting better and better at playing, not on plausible ways to play badly. it's something i keep meaning to work on when i have some spare time; i have a few ideas, but nothing i've had the time to explore properly.

DanBC 12y ago

Are chess engines still trying to play against humans in an interesting way? (I understand they beat human players, but that people feel computers play in dull ways). Is there a Turing Test for computer chess, where humans and computers play each other and they, and commentators, analyse the play, but no-one knows who is a computer or human until after the commentary is published? And if we ignore humans are people playing computers against other computers for some kind of machine learning play? And how optimized for speed is the software? Do they really crunch out all performance they can? (Sorry for the barrage of questions but I don't know enough about this space to do efficient websearches).

cven714 12y ago
> Are chess engines still trying to play against humans in an interesting way? (I understand they beat human players, but that people feel computers play in dull ways). Depends on what you mean by dull. Computers play so well that it tends to be a complete mop-up regardless of any "anti-computer" strategies people may try. The dull aspect is the one-sidedness. What's hard to do is make computers play weakly in a human-like way. Lobotomized computers tend to make very inhuman blunders. > Is there a Turing Test for computer chess, where humans and computers play each other and they, and commentators, analyse the play, but no-one knows who is a computer or human until after the commentary is published? Not that I've ever seen, though it still wouldn't be much of a challenge. Computer moves are pretty easy to spot--generally unintuitive or seemingly paradoxical moves that have a very concrete justification. Especially, as I said above, if they were set to play at a weaker level. > And if we ignore humans are people playing computers against other computers for some kind of machine learning play? I'll let others answer this, but it would surprise me if nobody was. Still, improving evaluation heuristics and analysis efficiency seems to be yielding better results than just machine learning. > And how optimized for speed is the software? Do they really crunch out all performance they can? Yes. The more positions you can examine per second, the deeper your search tree can go, and the better your evaluations, etc.
omaranto 12y ago
> I understand they beat human players, but that people feel computers play in dull ways. Without knowing anything about chess, I'd say that makes humans sounds like sore losers. I wonder if this will lead to having tournaments where you don't get points for winning matches but rather get judged on style by a panel of (human) judges...

alien3d 12y ago

Just try havin fun.. http://en.lichess.org/SMVJP07p http://en.lichess.org/SMVJP07p The strongest player.. me kinda noobs..

basic_stat 12y ago

> During the final event, after playing 64 games against Komodo, Stockfish won with the score of 35½-28½. No doubt is further allowed: Stockfish is the best chess player ever! From a statistical point of view, this isn't actually significant, despite the fact that draws help reduce the variance. 45 of those games are draws, leaving a 13-6 score in favor of Stockfish. Considering a null hypothesis of a binomial distribution with n=19 and equal chance of winning, the two-sided p-value for that score is 0.115. Unless you already have strong evidence that Stockfish is better than Komodo, you shouldn't conclude anything about which one is best.

eterm 12y ago
But it fundamentally _isn't_ binomial across 19 games, because of draws. You can't just ignore draws from the analysis, to do so is terrible application of statistics.
- jmount 12y ago
  The first analysis isn't terrible. It gets the important points right (that the draws are not evidence of difference in skill) and moves on to the remaining evidence (difference in wins to losses). The draws are at best evidence towards equality (not against it). Allow them to vary and the likelihood of seeing a difference of 9 wins in 64 games with 45 draws moves up to 0.13 or 13% (when we assume the two players are identical, an appropriate null hypothesis) (even less significant). So in about one tournament in 8 you would expect this much of a lead, even if it was one algorithm playing itself. So from one tournament we say it is likely the one algorithm is in fact better, but it doesn't rise to the standard of being statistically significant. <code> # R code to empirically estimate two-sided probablity of # seeing a lead of 9 games when 64 games are played # and the assumed probability of a draw is 45/64 # with the null assumption win/loss odds are equal simulate <- function(nplay,ndraw) { sample(c('w','d','l'),size=nplay,replace=TRUE, prob=c((nplay-ndraw)/2,ndraw,(nplay-ndraw)/2)/nplay) } wldiff <- function(v) { abs(sum(v=='w')-sum(v=='l')) } set.seed(350920) stats <- replicate(10000,wldiff(simulate(64,45))) print(sum(stats>=13-6)/length(stats)) ## [1] 0.1341 </code> (it is weird that somebody, not me, created a throw-away account to make the original comment. likely they are involved in chess development, or know how quickly stat discussions go sideways)

jtth 12y ago

(Side note: what the hell is that website trying to do to my computer? It's trying to open so many connections on weird ports!)

showerst 12y ago
Looks like they're using a bunch of websockets for something. Probably that live player count up in the navigation.
ornicar 12y ago
lichess opens only one websocket connection, on a port between 9021 to 9029. That port range prevents a weird behavior on firefox when the connection is interrupted.

asdfologist 12y ago

This is extremely impressive - it's cool that talented programmers are pushing the limits of computer science to advance the state of the art of chess engines. However, I also wish that there were comparable efforts to create AIs that train humans. Basically, figure out a way to systematically, efficiently, and scalably train amateurs into masters. That IMO would be absolutely amazing (and something I'd gladly pay for).

mnemonicsloth 12y ago
Are you sure you wouldn't rather learn, say, mathematical physics or computer science? The neurological topiary for chess is strictly harder, because chess has fewer analogues in day-to-day experience.
- asdfologist 12y ago
  Chess is a fun hobby, and I feel it gets more fun the better I get. That's a good enough justification for me.
- cwp 12y ago
  It's a place to start. It's not a huge leap from modelling the game of chess to modelling the skill of chess-playing. Stockfish already offers post-game analysis. Imagine extending that to analyze a players entire career, and offer a series of problems, games against specially-constructed opponents, analysis of relevant historical games etc, all of which are designed to help the player improve. Given Siri, Google Now, Watson and so on, we're probably not far from being able to have a meaningful natural-language conversation with a computer on a narrow subject like chess. One could imagine this kind of thing being extended to teaching other, similarly focussed skills, at least for beginners. Piano. Tennis. Rock climbing. Maybe even things like soccer and basketball. But that kind of teaching-based-on-deep-analysis is a long way off for subjects like physics or electrical engineering. Computers can't do physics, much less evaluate human physicists. The best we can do in these areas is something like the Khan Academy, where computers present "content" created by humans, administer standardized tests designed by humans and present the results to humans for interpretation. So yeah, teaching chess in a really sophisticated way isn't all that useful in the sense that physics or EE are useful. But really, if we could teach computers to understand physics better than people do, we'd use them to make breakthroughs in physics, and that would be a much bigger deal than being able to teach physics more effectively. On the other hand, we don't play chess or tennis or piano because they're useful, so expert AI teachers for these subjects would be really valuable.

pk2200 12y ago

Stockfish 5 was released yesterday: http://stockfishchess.org/ http://stockfishchess.org/ As mentioned elsewhere in this thread, Stockfish is just an engine - you must install a GUI separately. XBoard is well known, but there are better alternatives: http://chessx.sourceforge.net/ http://chessx.sourceforge.net/ http://scidvspc.sourceforge.net/ http://scidvspc.sourceforge.net/

beefman 12y ago

TCEC is cool but has less statistical power than many ratings lists out there, which show that Houdini, Komodo and Stockfish are very closely matched, with Houdini having a slight edge at long time controls and a moderate edge at quick time controls. Stockfish does release more frequently and I'm not sure which version competed in TCEC, but until the lists catch up this article is fluff.

onestone 12y ago
Houdini had a slight edge against Stockfish DD, which is a few months old. The latest Stockfish version (v5, very similar to the version that competed in the TCEC Superfinal) is ~60 Elo stronger, and demolishes Houdini at all time controls. It should soon start hitting the more conservative ratinglists.
- beefman 12y ago
  "Demolishes"? Hardly. It is so far tied with H4 on IPON. Even if it winds up on top, it will not be by a large margin.

runeks 12y ago

Has anyone else watched http://en.lichess.org/tv http://en.lichess.org/tv? If this is two humans playing against each other in real-time, that is really impressive! It's so mind-boggling fast (mind you I'm not a chess player).

varg 12y ago
Many games are played 'bullet' style - i.e. 60s each for the whole game.
doughj3 12y ago
ChessNetwork on YouTube[1] has many videos of blitz chess tournaments where he provides commentary as he plays. Very interesting to watch! [1] https://www.youtube.com/watch?v=7YWYS209ydE https://www.youtube.com/watch?v=7YWYS209ydE
- electrotype 12y ago
  Wow, very cool! Can someone explain what's going on at 20:24 : http://youtu.be/7YWYS209ydE?t=20m20s http://youtu.be/7YWYS209ydE?t=20m20s ? He forks the king and the queen, then his opponent moved his queen!?! This is not a legal move right?
  - doughj3 12y ago
    Had to re-watch a couple times: on that move, white had already moved pawn from g2 to g3, the knight from d6 to e8 is a pre-move that happens very quickly (you can see the red square); while white was setting up that pre-move, black moved the queen. Does that clarify or did I miss what you are asking?
    - electrotype 12y ago
      Got it, thanks!
- runeks 12y ago
  That's pretty cool! Can anyone inform an amateur like me what happens at 2:14 (https://www.youtube.com/watch?v=7YWYS209ydE#t=134 https://www.youtube.com/watch?v=7YWYS209ydE#t=134) and before that as well. It seems to me that black captures his own pawn using his knight. Is that legal? EDIT: Ah, I see. White pawn captures black pawn, and the knight captures the white pawn. I just happens so fast I didn't see the white move.

icambron 12y ago

I always thought of the heuristics for evaluating a chess position as the really hard part of building a chess engine; i.e. how do you capture all of the positional subtleties in a number to feed into minimax? But looking at the source, it's not really that complicated [1]. Can someone who knows more than me comment on that? Is it that the innovations are elsewhere? That good chess really can be boiled down to < 1000 LOC? That the numbers in this heuristic are just super expertly tuned? [1] https://github.com/mcostalba/Stockfish/blob/master/src/evaluate.cpp https://github.com/mcostalba/Stockfish/blob/master/src/evalu...

glinscott 12y ago
Nearly all the evaluation terms in Stockfish have been extensively tuned. Joona Kiiski used the SPSA[1] algorithm on a lot of them. Others have been hand-tuned using tens of thousands of games per attempt on fishtest [2]. Fishtest actually just recently got support for running SPSA tuning as well. There is also a strong bias towards simplification, so if an evaluation feature is not proven to be an improvement it will be removed. Over the last few Stockfish versions, the # of lines has actually decreased in each version. [1] http://en.wikipedia.org/wiki/Simultaneous_perturbation_stochastic_approximation http://en.wikipedia.org/wiki/Simultaneous_perturbation_stoch... [2] http://tests.stockfishchess.org/tests http://tests.stockfishchess.org/tests
- icambron 12y ago
  That's helpful, thanks.
deleted 12y ago
[deleted]
halfcat 12y ago
What separates one chess engine from another is its ability to efficiently navigate and evaluate the game tree. Stockfish effectively discards probably 95% or more of the nodes in any position. Easier said than done, better not discard the best move in that 95%. Evaluation is relatively simple, a fancy lookup table. They say love covers a multitude of sins. Well, searching one move deeper than your opponent covers a multitude of static positional evaluation mistakes. Searching deeper can figure out the complexity of the position better than anything that can be statically evaluated. That's where virtually all improvements exist today, figuring out which moves can be safely discarded.

rmbe 12y ago

lichess doesn't seem to be giving it enough juice to perform at its stated levels at the moment. On my first try I managed to draw the highest AI level, rated at 2510, while my rating is under 2000 irl. (I was unable to find an "offer draw" button so relied on the 50-move rule) Against stockfish running on my PC that would be impossible. http://en.lichess.org/reDfuSvI http://en.lichess.org/reDfuSvI

ornicar 12y ago
Have you actually read the article? It's all explained http://en.lichess.org/blog/U4mtoEQAAEEAgZRL/strongest-chess-player-ever http://en.lichess.org/blog/U4mtoEQAAEEAgZRL/strongest-chess-...
- rmbe 12y ago
  I did. I was complaining about the rated strength of the AI levels, they're off by a wide margin.

meowface 12y ago

I'm rather surprised at how relatively simple and small the codebase is: https://github.com/mcostalba/Stockfish/tree/master/src https://github.com/mcostalba/Stockfish/tree/master/src

QuantumChaos 12y ago
Especially interesting to see everything hardcoded into C++. Quite different from most modern codebases that use a mix of languages, configuration files, etc.

arikrak 12y ago

That site's pretty impressive. They're open source and give away for free what Chess.com charges for.