5 ms·
The name of the game here is generality. For a really general agent, they are looking to have superhuman performance, not get state of the art on every individu
by dontreact 5y ago
The name of the game here is generality. For a really general agent, they are looking to have superhuman performance, not get state of the art on every individual task. Beating stockfish 8 convinces me that it would be superhuman at chess.
- remram 5y agoThey could still be honest that it's Stockfish 8, not the Stockfish everyone has. Your product having genuine value does not excuse lying about that value.
- ShamelessC 5y agoThey were? They say they use Stockfish 8 the very first time they mention it.
- hesperiidae 5y agoYup, "In chess, we evaluated PoG against Stockfish 8, level 20 [81] and AlphaZero."
- remram 5y agoFirst time they mention it is page 10: > one of the strongest and most widely-used programs is Stockfish [81]. Here's the citation, note the date: > [81] The Stockfish Development Team. Stockfish: Open source chess engine, 2021. https://stockfishchess.org/ https://stockfishchess.org/. They mention the version number only once, further down, and don't point out that it's out of date since February 2018. All other 11 mentions of it don't have the version number, like in that sentence: > In Chess, PoG(60000,10) is stronger than Stockfish using 4 threads and one second of search time.
- hesperiidae 5y ago>First time they mention it is page 10: Yeah, so it is! I guess I ran into the same weirdness as ShamelessC, since when I first Ctrl-F:ed the PDF, hit 1/11 was on page 11. Now that I try my damndest to reproduce it, I get 12 hits and the first is that one on page 10.
- Skyy93 5y agoI observed this kind of behavior in many papers nowadays. This extremely painful for research, because some better candidates could be overseen and FAANG publishs a majority in the ML-paper section. Its a mess.