5 ms·

Show HN: Chessdream – Generate realistic chess positions with AI

45 points by MereGurudev 2y ago

For no specific reason, I trained a neural network to generate random chess positions that look similar to positions from actual games (from lichess db). I also made it so you can condition it on some fixed pieces, and adjust the number of pieces. It turned out to be quite effective and I find it surprisingly fun and instructive to generate e.g. endgame positions with a certain pawn structure (set low temperature, place some pawns and position the kings, adjust number of pieces to get an endgame), and then figure out how to win vs. the computer in those positions. I hooked it up so that you can easily play vs. a computer, either via lichess analysis → continue from here, or via my own project Noctie. I’m thinking about whether I could develop it further to create position variants on a certain theme that the AI thinks I need to practice, or maybe make a PvP feature where you play a random position vs. a human. Ideas or feedback?

programjames 2y ago
It sometimes adds too many pieces: https://noctie.ai/play?newGameDialogState_customGame=true&newGameDialogState_overrideFen=3r2k1%252F1p1b1ppp%252Fp1ppnb2%252FP3p2n%252FP1B1PPPq%252F2NpB1QP%252F1P2N3%252FR2R2K1%2520w%2520-%2520-%25200%25201 https://noctie.ai/play?newGameDialogState_customGame=true&ne... I'm pretty sure black should have at most eight pawns.
- klysm 2y ago
  This feels like a perfect example of just whacking a neural net on something without actually applying and understanding of the domain and validating things.
- spacebanana7 2y ago
  I wonder whether that could be fixed by using an approach like this [1] [2], to guarantee the model output meets a schema. [1] https://news.ycombinator.com/item?id=37125118 https://news.ycombinator.com/item?id=37125118 [2] https://blog.normalcomputing.ai/posts/2023-07-27-regex-guided-generation/regex-guided-generation.html https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...
- progval 2y ago
  Or two bishops of the same color
  - cenanozen 2y ago
    That is possible with pawn promotion
- GistNoesis 2y ago
  There is something wrong with the rendering of the color of pieces : In https://chessdream.ai/?dreamId=ab86d6a2-470a-4238-9181-8b4d3013103b https://chessdream.ai/?dreamId=ab86d6a2-470a-4238-9181-8b4d3... there are two white kings, and queens. The position is correct though when viewed on Noctie. Edit : In fact, the kings and queen are neither black or white but a third color : grey Edit 2: There is probably some issue with the adjust piece count, when opening the same dreamId in multiple windows. As following the same link doesn't have the same number of pieces.
  - MereGurudev 2y ago
    Hey, you're right, adjust piece count is a local modification, so opening the same dreamId in multiple windows would let you view the position with different piece counts (and the link to Noctie uses the local state)
- MereGurudev 2y ago
  Thanks for the feedback! It's a simple modification to guarantee things like this – for this prototype I thought it interesting though to expose the "raw" output of the neural network, including crazy things like sometimes placing two kings of the same color. I even force the neural network to place more pieces than it intends, if you put the piece slider on max, to see what happens (and often the result is very bad)
abhayhegde 2y ago
This looks amazing! To generate multiple (almost) equal positions with a given set of pieces can be handy while practicing. I love that there is a way to analyze it on Lichess as well. Would it make sense to export a PGN file of the position as well? I'm not sure how to put it, but would be great to also include an option to simulate a position with a given opening. For example, 15 moves down the line of playing Queen's Gambit accepted variation with the given pieces on the board would be instructive in learning what's an equal position and how to improve middle game etc. Maybe too hard to implement, I understand. I saw your submission on r/chess also.
- MereGurudev 2y ago
  Thanks! Good idea, I could add FEN / PGN export in addition to lichess. I haven't tried conditioning the network on opening but given my training data, that should be possible. For now, you can simulate openings somewhat, by trying to fix pawns and pieces in their typical positions, but it would be better if I could do it like you suggested "QGA, 15–20 moves" etc. I might try this out!
somenameforme 2y ago
Your software (noctie) was pretty fun to play against, but it's probably no more than ~1600 ELO. Having it assess me as 2574 after beating it was nice on the ego, but a bit silly. The comparison to Lichess ratings also seems a bit off. 2574 ELO is probably much higher than 2809 Lichess unless there's been some serious rating deflation going on over there (been a while since I played on that site). I assume you're using an accuracy correlation, but those fail in lots of situations. For instance if somebody is substantially better than somebody else, they're probably going to have near 0 pawn loss, but that's only because the opponent never posed any problems to them. Style issue also tend to break these correlations. E.g. - Capablanca was much more accurate, by this metric, than Kasparov. But that's because Capablanca had an extremely solid style. In reality, he would probably not fare well against Kasparov or most modern super GMs, even though many of them are far less accurate on paper.
- MereGurudev 2y ago
  Hi, thanks for trying Noctie! If you did the rating test, Noctie tries to adapt to your strength while you're playing, so if you play at 2574 level for a while, eventually Noctie will also play at that level. Since Noctie has no idea about your rating when the game starts, it might be that it took some time for the rating to adapt and therefore you found that the AI played much weaker. The max strength of the AI is about 2700–2800 FIDE (level "Queen 4" inside the app if you have an account). Per this site, https://chessgoals.com/rating-comparison/#lichessotb https://chessgoals.com/rating-comparison/#lichessotb, 2809 Lichess is equal to around 2550 FIDE so if that's your lichess rating (wow btw!) maybe Noctie wasn't so far off. (EDIT: Ah I see, that's not your rating, sorry) Obviously, you might get a different result next time – one game is very little information to make an accurate estimation off of, especially when the AI has to adapt it's playing strength as we go. I don't use accuracy for the rating estimation BTW, I use custom neural networks that observe patterns in how humans at various rating levels play chess.
  - somenameforme 2y ago
    Interesting! I just played it again and can definitely see what you mean. But it keeps running into the same issue. It ends up giving itself lost positions early on which it's not really capable of defending. I was about to ask why you didn't go the other way (strong at first then gradually handicapping) but on the other hand I've never had anywhere near this much fun playing a bot, and maybe this is part of the reason why? Well another obvious factor is that it plays in an extremely human-like fashion. I'm a relatively strong player and have been the reason for plenty of (C) labels but I would never, in a million years, think I was playing a bot here. Anyhow, awesome job.
eertami 2y ago
Cool idea. Since you asked for feedback, I found it really hard to discern the different piece colours, especially when there are white pieces on dark squares and dark pieces on white squares. The contrast ratio for these looks to be around 1.58:1, I had to really focus to pick out which is which.
- MereGurudev 2y ago
  Thanks! Ah, sorry for that. Just to clarify, do you mean for the semi-transparent pieces (the generated ones without "locking in") or even the manually placed pieces?
  - impendia 2y ago
    I tried it out, and also couldn't distinguish the colors. > Just to clarify, do you mean for the semi-transparent pieces (the generated ones without "locking in") or even the manually placed pieces? Are these visually distinguished somehow? Are there four different colors of pieces? The piece color also seems to be different depending on the background square color, and I wasn't able to distinguish what is what.
    - MereGurudev 2y ago
      Yes, they're distinguished: the pieces you place there manually ("fixed pieces" are opaque and thus should have the same color, off-white or dark-grey, no matter the background square. The pieces the AI adds when you press generate, are translucent (and as such will be affected by the background color). Then you can press "Lock in" to convert the generated pieces to fixed pieces, in which case they will become opaque. I realize that for anyone less used to the color scheme than me, this distinction is problematic since there will be no less than six different piece colors (white, black, translucent-white-on-light, translucent-white-on-dark, translucent-black-on-light, translucent-black-on-dark)

45 points by MereGurudev 2y ago

programjames 2y ago

It sometimes adds too many pieces: https://noctie.ai/play?newGameDialogState_customGame=true&newGameDialogState_overrideFen=3r2k1%252F1p1b1ppp%252Fp1ppnb2%252FP3p2n%252FP1B1PPPq%252F2NpB1QP%252F1P2N3%252FR2R2K1%2520w%2520-%2520-%25200%25201 https://noctie.ai/play?newGameDialogState_customGame=true&ne... I'm pretty sure black should have at most eight pawns.

klysm 2y ago
This feels like a perfect example of just whacking a neural net on something without actually applying and understanding of the domain and validating things.
spacebanana7 2y ago
I wonder whether that could be fixed by using an approach like this [1] [2], to guarantee the model output meets a schema. [1] https://news.ycombinator.com/item?id=37125118 https://news.ycombinator.com/item?id=37125118 [2] https://blog.normalcomputing.ai/posts/2023-07-27-regex-guided-generation/regex-guided-generation.html https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...
progval 2y ago
Or two bishops of the same color
- cenanozen 2y ago
  That is possible with pawn promotion
GistNoesis 2y ago
There is something wrong with the rendering of the color of pieces : In https://chessdream.ai/?dreamId=ab86d6a2-470a-4238-9181-8b4d3013103b https://chessdream.ai/?dreamId=ab86d6a2-470a-4238-9181-8b4d3... there are two white kings, and queens. The position is correct though when viewed on Noctie. Edit : In fact, the kings and queen are neither black or white but a third color : grey Edit 2: There is probably some issue with the adjust piece count, when opening the same dreamId in multiple windows. As following the same link doesn't have the same number of pieces.
- MereGurudev 2y ago
  Hey, you're right, adjust piece count is a local modification, so opening the same dreamId in multiple windows would let you view the position with different piece counts (and the link to Noctie uses the local state)
MereGurudev 2y ago
Thanks for the feedback! It's a simple modification to guarantee things like this – for this prototype I thought it interesting though to expose the "raw" output of the neural network, including crazy things like sometimes placing two kings of the same color. I even force the neural network to place more pieces than it intends, if you put the piece slider on max, to see what happens (and often the result is very bad)

abhayhegde 2y ago

This looks amazing! To generate multiple (almost) equal positions with a given set of pieces can be handy while practicing. I love that there is a way to analyze it on Lichess as well. Would it make sense to export a PGN file of the position as well? I'm not sure how to put it, but would be great to also include an option to simulate a position with a given opening. For example, 15 moves down the line of playing Queen's Gambit accepted variation with the given pieces on the board would be instructive in learning what's an equal position and how to improve middle game etc. Maybe too hard to implement, I understand. I saw your submission on r/chess also.

MereGurudev 2y ago
Thanks! Good idea, I could add FEN / PGN export in addition to lichess. I haven't tried conditioning the network on opening but given my training data, that should be possible. For now, you can simulate openings somewhat, by trying to fix pawns and pieces in their typical positions, but it would be better if I could do it like you suggested "QGA, 15–20 moves" etc. I might try this out!

somenameforme 2y ago

Your software (noctie) was pretty fun to play against, but it's probably no more than ~1600 ELO. Having it assess me as 2574 after beating it was nice on the ego, but a bit silly. The comparison to Lichess ratings also seems a bit off. 2574 ELO is probably much higher than 2809 Lichess unless there's been some serious rating deflation going on over there (been a while since I played on that site). I assume you're using an accuracy correlation, but those fail in lots of situations. For instance if somebody is substantially better than somebody else, they're probably going to have near 0 pawn loss, but that's only because the opponent never posed any problems to them. Style issue also tend to break these correlations. E.g. - Capablanca was much more accurate, by this metric, than Kasparov. But that's because Capablanca had an extremely solid style. In reality, he would probably not fare well against Kasparov or most modern super GMs, even though many of them are far less accurate on paper.

MereGurudev 2y ago
Hi, thanks for trying Noctie! If you did the rating test, Noctie tries to adapt to your strength while you're playing, so if you play at 2574 level for a while, eventually Noctie will also play at that level. Since Noctie has no idea about your rating when the game starts, it might be that it took some time for the rating to adapt and therefore you found that the AI played much weaker. The max strength of the AI is about 2700–2800 FIDE (level "Queen 4" inside the app if you have an account). Per this site, https://chessgoals.com/rating-comparison/#lichessotb https://chessgoals.com/rating-comparison/#lichessotb, 2809 Lichess is equal to around 2550 FIDE so if that's your lichess rating (wow btw!) maybe Noctie wasn't so far off. (EDIT: Ah I see, that's not your rating, sorry) Obviously, you might get a different result next time – one game is very little information to make an accurate estimation off of, especially when the AI has to adapt it's playing strength as we go. I don't use accuracy for the rating estimation BTW, I use custom neural networks that observe patterns in how humans at various rating levels play chess.
- somenameforme 2y ago
  Interesting! I just played it again and can definitely see what you mean. But it keeps running into the same issue. It ends up giving itself lost positions early on which it's not really capable of defending. I was about to ask why you didn't go the other way (strong at first then gradually handicapping) but on the other hand I've never had anywhere near this much fun playing a bot, and maybe this is part of the reason why? Well another obvious factor is that it plays in an extremely human-like fashion. I'm a relatively strong player and have been the reason for plenty of (C) labels but I would never, in a million years, think I was playing a bot here. Anyhow, awesome job.

eertami 2y ago

Cool idea. Since you asked for feedback, I found it really hard to discern the different piece colours, especially when there are white pieces on dark squares and dark pieces on white squares. The contrast ratio for these looks to be around 1.58:1, I had to really focus to pick out which is which.

MereGurudev 2y ago
Thanks! Ah, sorry for that. Just to clarify, do you mean for the semi-transparent pieces (the generated ones without "locking in") or even the manually placed pieces?
- impendia 2y ago
  I tried it out, and also couldn't distinguish the colors. > Just to clarify, do you mean for the semi-transparent pieces (the generated ones without "locking in") or even the manually placed pieces? Are these visually distinguished somehow? Are there four different colors of pieces? The piece color also seems to be different depending on the background square color, and I wasn't able to distinguish what is what.
  - MereGurudev 2y ago
    Yes, they're distinguished: the pieces you place there manually ("fixed pieces" are opaque and thus should have the same color, off-white or dark-grey, no matter the background square. The pieces the AI adds when you press generate, are translucent (and as such will be affected by the background color). Then you can press "Lock in" to convert the generated pieces to fixed pieces, in which case they will become opaque. I realize that for anyone less used to the color scheme than me, this distinction is problematic since there will be no less than six different piece colors (white, black, translucent-white-on-light, translucent-white-on-dark, translucent-black-on-light, translucent-black-on-dark)