cbutner

4 ms·

1.
by cbutner 5y ago
The commentary net inspects the final state of the engine net, but not internal layers. Deeper introspection is a really important goal, but by the time you make serious progress there, chess is the least of your worries. I do really like t
2.
by cbutner 5y ago
Yes! I haven't done as much testing with GPU, but did validate running with 4x V100s. You just need to adjust the "search_threads" option to the number of GPUs, but set it to at least 2. Installation for GPU is covered here:
3.
by cbutner 5y ago
It loses some games to Stockfish 13 and 14, and Lc0 - rarely at slow time control, and more often at blitz and bullet (actually, it has losses all the way down to Stockfish 9 in blitz). Partly because of the way it tries to search more wide
4.
by cbutner 5y ago
It was partly because I was looking to improve self-play and training tractability on a home desktop with 1 GPU (complete failure), and partly to learn about everything from scratch. I would be interested to see how strong it is with the sa
5.
by cbutner 5y ago
That's a good idea. A bigger problem than time-slicing is probably GPU/TPU device ownership issues and GPU/TPU memory usage with multiple games going in parallel. There may be some ways to multiplex it intelligently though. C
6.
by cbutner 5y ago
I think it always helps to have a project to apply things to as you're learning something, even if it means coming up with something small. While preparing, I found it helpful to read for at least an hour each morning, and then divided
7.
by cbutner 5y ago
The engine itself is in C++, but it calls in to TensorFlow via Python as a portability/distribution vs. performance trade-off. Next steps could be using one of Lc0's backends for GPU scenarios, or taking the other side of the trad
8.
by cbutner 5y ago
SentiMATE[1] looks at one of the reverse problems in a way - training an engine on commentary data - although it's not exactly what you're talking about. I think this line of thinking could eventually lead to automated metrics for
9.
by cbutner 5y ago
It does train on variations too, given the scarcity of data available, so that can hurt accuracy, mood, etc.
10.
by cbutner 5y ago
The original hope was for this to be a third head on top of the AlphaZero model, but I couldn't think of a way to generate commentary during self-play (such that it would gradually improve), and trying to rotate supervised commentary t
11.
by cbutner 5y ago
It is using a full-sized transformer decoder, trained on about 1 million data samples, but with far fewer neural network parameters and training samples than GPT-2 or GPT-3.
12.
by cbutner 5y ago
Yes, in this case as long as they still agree when it actually names people, I don't think it would be too difficult. There may be factors I'm not considering though. Harder would be more general models like GPT-2 and GPT-3.
13.
by cbutner 5y ago
I agree with what you're saying. On the flip side, there are multiple systems (Elo, Glicko), anchors, playing pools, etc. in use around the place, and FIDE and CCRL are offset by around 80 magnitude I heard, compared to about 600-700 d
14.
by cbutner 5y ago
Haha - I dislike how much of a black box it is, despite the statistical basis (for example, the back and forth on batch normalization rationale). But lots of interesting problems and tech to dig into.
15.
by cbutner 5y ago
Yes, each bot uses a v3-8 Cloud TPU VM, and tries to be constantly playing a game. The search tree is also very memory-hungry. And right now it's also using the Python API for TensorFlow, which is likely wasting a lot of potential. Lot
16.
by cbutner 5y ago
It does tend to name-drop: often famous names, but also just "Jeff". And if you spice up the commentary sampling parameters, it gets even more inventive, making up names, and saying that "the rook is pinning Fischer against t
17.
by cbutner 5y ago
Sometimes it seems really accurate (like the cherry-picked GIF in the overview docs) and sometimes really off. I think for the most part, it knows more than it lets on, but finding the right sampling methods (or better yet, generalized sear
18.
by cbutner 5y ago
Yeah, that's a massive problem with the natural language domain all across machine learning. Unfortunately it's very difficult to track down training data for chess commentary in the first place, let alone trim down biases. For re
19.
by cbutner 5y ago
Yeah the auto-linking is just Lichess doing its best, even when the bot's talking nonsense sometimes. And thank you!
20.
by cbutner 5y ago
Oh, that message is a little backwards, but the main bot only accepts challenges from 1+0 or 0+1 up to 15+10 time control. You can challenge https://lichess.org/?user=chesscoachclassical#friend to 30+20. Unfortunately, neit
21.
by cbutner 5y ago
Thank you! I do get that itch to jump in and improve things whenever I see it lose a game, but I don't have further plans (development or commercial) in the near-term. The goal originally was to see whether I liked ML, to decide on my
22.
by cbutner 5y ago
This took about a year and a half – a little over a year coding in between experiments and training. It's a chess engine with a primary neural network just like AlphaZero or Leela Chess Zero's, but it adds on a secondary "com
23.
Show HN: ChessCoach – A neural chess engine that comments on each player's moves(chrisbutner.github.io)
506 points by cbutner 5y ago|89 comments