7 ms·
Surprisingly, storing a game (with all moves) can take less space than encoding a single board. This is because you can effectively encode a move in a single by
by ThomasCM 3y ago
Surprisingly, storing a game (with all moves) can take less space than encoding a single board. This is because you can effectively encode a move in a single byte (as there are less than 255 moves possible in each position). Applying compression to the resulting binary string will allow you to reduce the space even more.
Check out this great blog post of Lichess for more information: https://lichess.org/blog/Wqa7GiAAAOIpBLoY/developer-update-275-improved-game-compression https://lichess.org/blog/Wqa7GiAAAOIpBLoY/developer-update-2...
And shameless plug: Using this encoding, I'm storing millions of games on https://www.chessmonitor.com/ https://www.chessmonitor.com/ There you can link your Chess.com or Lichess account and view all kind of statistics of your games.
- foota 3y agoHeh, I guess there are board states that are not possible to reach through a valid sequence of moves, but I guess otherwise it's not possible that games are more compressable by definition, since any valid board state could be represented as a sequence of moves. This does raise the question of the efficiency of reverse engineering a series of minimal moves for some board state.
- seabass-labrax 3y agoThat is a beautifully designed website. I don't think I've ever used an OAuth authentication flow as smooth as the one that your site uses to access my Lichess credentials (same as my HN name, btw, in case you want to beat me at a correspondence game). ChessMonitor is practically a work of art!
- robertlagrant 3y agoI created an account just to see that flow, and it did not disappoint! How's it this fast?
- ThomasCM 3y agoThanks for your feedback! I guess OAuth is relatively fast as I don't have a middleman (like Auth0) in there. It's just Passport.js.
- gowld 3y agoAre you saying that, to encode a valid position, ignoring the cost to encode both the starting position and the move definitions, you can encode the move sequence using less information than encoding a single position?
- ThomasCM 3y agoYes, ignoring compression each (half-)move takes up one byte. So if you want to store a sequence of 10 (half-)moves it would take only 10 bytes.
- SideQuark 3y agoAverage game length seems to be around 80 half moves though. https://chess.stackexchange.com/questions/2506/what-is-the-average-length-of-a-game-of-chess https://chess.stackexchange.com/questions/2506/what-is-the-a... I'd expect that ordering moves by popularity at each half move index, using say the above dataset, would allow you to select lower indexed values at each step, allowing a nice context based arithmetic compression to really shrink them well.
- ThomasCM 3y agoYes, in contrast to storing the board (which can be a fixed size), this depends on the length of the game of course. That said, there are some optimizations you can apply that will compress moves even further (for example the order of the moves as explained in the Lichess blog post is important). In the end it's a tradeoff between (de)serializing the game fast and reducing the size (even more).
- chipsrafferty 3y agoIt seems, then, if you wanted to efficiently store lots of positions, it would be smart to store a large list of openings, a large list of following sequences, and then your positions can be stored as a list of sequence ids.
- omoikane 3y agoChessMonitor looks very nice, I like how you can link to a particular opening: https://www.chessmonitor.com/u/XqaFNTHcR61WpiMfOhEY/games?pos=03030f&color=white https://www.chessmonitor.com/u/XqaFNTHcR61WpiMfOhEY/games?po...