5 ms·
My understanding is that AlphaZero only really existed for a year or two; there's no objective way to compare it at the moment. Leela Zero tried to open-source
by gwd 9d ago
My understanding is that AlphaZero only really existed for a year or two; there's no objective way to compare it at the moment.
Leela Zero tried to open-source that work, but Stockfish incorporated a number of improvements from AlphaZero, including a neural network and a different search method, and consistently beats Leela Zero.
I have a book, "Game Changer", in which a chess expert calls out several instances where AlphaZero made moves surprising at the time; situations where all chess engines rated things one way and AlphaZero rated them differently. When I enter them into Stockfish now, it usually rates things more similarly to the way AlphaZero did, and often chooses the move chosen by AlphaZero.
The only real test of course would be to dig up AlphaZero and run it again; but I think based on the evidence we have, Stockfish of 2026 would probably trounce AlphaZero of 2018 with equivalent compute available.
- sinuhe69 9d agoI guess someone can twit Hassabis and ask him :D He certainly has access.
- NooneAtAll3 9d agowho is Hassabis?
- xenocratus 9d agoDemis Hassabis [1], co-founder of DeepMind [1] https://en.wikipedia.org/wiki/Demis_Hassabis https://en.wikipedia.org/wiki/Demis_Hassabis
- kzrdude 9d agoIsn't one problem that it's hard to determine what equivalent compute is, for CPU search vs a neural net based engine like AZ or Leela?
- gwd 9d agoOne way would be to calculate a cost per game, factoring in both electricity and an amortized cost of the hardware, maybe having a penalty too for extra time run (e.g., if focusing only on hardware depreciation and electricity, 1 minute of TPU would translate to 2 weeks of CPU, that 2 weeks of waiting still costs you something). Obviously this isn't stable, as relative prices of GPUs and memory shift over time, and it's somewhat sensitive to setup; but done right it's probably more "what a user actually wants to know", in terms of what it would take to get equivalent performance.
- zarzavat 8d agoIt's more fundamental than that. AlphaZero is a shallower search with a heavier evaluation function. Stockfish is a deeper search with a lighter evaluation function. In chess, depth usually wins because of how narrow the search tree is compared e.g. to Go.
- kzrdude 8d agoInteresting, and if you don't mind, where do we put humans (and superhumans like Magnus Carlsen)? I think they have a heavy evaluation function and do shallower search.
- FergusArgyll 8d agoWay way shallower search. A human doesn't consider more than a few candidate moves. But they (we?) build up much better intuitions and heuristics
- dllu 9d agoThe Stockfish NNUE is completely unrelated to AlphaZero.
- gwd 8d agoIt's a neural network rather than a bunch of hard-coded rules. That turns out to make a big difference. Actually, there's this interesting snippet from the release page: > These techniques have been applied to hundreds of billions of training positions, all of which have been consistently rescored using a strong Leela net. So Stockfish's neural network evaluator is actually trained using Leela Zero.
- adamt 8d ago"Completely unrelated" is not quite true. Stockfish current NNUE models are trained on LC0 training data. LC0 is pretty much an open-source community replication of the ideas from AlphaZero.
- dllu 8d agoI stand corrected. However, fundamentally, the idea of "tiny CPU-only neural network" combined with traditional alpha-beta search is substantially different from "big GPU network" combined with Monte Carlo Tree Search. And historically the NNUE came from a 2018 idea for shogi engines rather than from AlphaZero.
- VulgarExigency 9d agoLeela Chess Zero did surpass Stockfish for a while, until Stockfish switched its eval to NNUE. Modern Stockfish would annihilate AlphaZero.
- Isofarro 8d ago> My understanding is that AlphaZero only really existed for a year or two; It still exists, but it's private / internal, and sometimes used for a few different things. It was used by Kramnik to test the hypothesis whether no-castling chess was viable (basically chess, but disallowing castling). That was a year after DeepMind published the match they ran of AlphaZero versus Stockfish. It was still in use last year, I remember seeing some Grandmasters with interests in chess studies were invited by DeepMind to judge the beauty of chess problems composed by AlphaZero (or whatever form the thing that used to be AlphaZero is now). So it's still kicking around in the background.
- Aachen 8d ago> It was used by Kramnik to test the hypothesis whether no-castling chess was viable (basically chess, but disallowing castling). Viable in what way? That it's advantageous to never castle if an engine learns to play with that directive? Or that it still makes for a fun game with that new rule?
- ekelsen 8d agoThe hope was that it would be a slight rule tweak that would lead to fewer draws without favoring white too strongly.