12 ms·
ChatGPT's Chess Elo is 1400
- jacobsenscott 4y agoI own't speak to weather this shows ChatGPT can play chess, but did you notify your opponents beforehand they are playing chatgpt? Otherwise (maybe even if) you're cheating and your account should be removed. If anyone else here is thinking of doing this - don't.
- mtlmtlmtlmtl 4y agoThere's a huge difference between 1400 elo in FIDE games versus 1400 on chess.com, which is not even using elo. For instance the strongest blitz players in the world are hundreds of points higher rated on chess.com blitz versus their FIDE blitz rating. Chess.com and lichess have a ton of rating inflation.
- CSMastermind 4y ago> the strongest blitz players in the world are hundreds of points higher rated on chess.com blitz versus their FIDE blitz rating Online rating inflation is real but I'm not sure blitz is the best example of it because in that case there is a notable difference between online and otb (having to take time to physically move the pieces).
- mtlmtlmtlmtl 4y agoPoint is it's kinda hard to take the blogpost too seriously when these fundamentals are so wrong. When literally the title is an immediately obvious error that doesn't inspire confidence in the rest of the methodology. I'm still going through the games but so far these games are not even close to elo 1400 level. For both the human player and the model.
- manquer 4y agoProbably the bigger difference is ability to premove online
- mtlmtlmtlmtl 4y agoI was thinking about this. On chess.com you can chain premoves, on lichess you can't(afaik). So in theory, to the extent premoves explain the rating difference, the difference should be greater on chess.com assuming they have the same parameters in their rating calculations. Therefore it should be possible to perform an analysis to shed light on this. But someone would have to go recompute the 3 different ratings under the same system first to be able to make a sensible analysis.
- teej 4y agoI just deployed a GPT-4 powered chess bot to lichess. You can challenge it here: https://lichess.org/@/oopsallbots-gpt-4 https://lichess.org/@/oopsallbots-gpt-4
- dsjoerg 4y agoVery cool! Are you doing prompt engineering, fine-tuning, both, something else? I'm wondering if it'd be cool to have a chess contest where all the bots are LLM powered. Seems to me like the contest would have to ban prompt engineering -- would have to have a fixed prompt -- otherwise people would sneak chess engines into their prompt generation.
- teej 4y agoI wanted it to be fun and actually complete games. I started with this, with a few minor tweaks: https://github.com/Tmate6/Lichess_ChatGPT_ChatBot/blob/main/ChatGPT.py#L67-L73 https://github.com/Tmate6/Lichess_ChatGPT_ChatBot/blob/main/... This approach sends along the list of legal moves in the prompt if it attempts an illegal move. That seems to work well at getting playable moves.
- singularity2001 4y agoWhat happens if you add the rules to the initial prompt and something like: "Think step by step, explain your move and if it is illegal rethink"
- juice_bus 4y agoWhen I tried it at v3.0 i found after 5-10 moves it started moving illegally.
- mjr00 4y agoThe AI has simply, and correctly, identified that cheating is the best way to win at something.
- skywal_l 4y agoThe best move is not to play.
- orbital-decay 4y agoIt doesn't even know the rules, let alone cheat. It predicts the notation from the massive amount of games seen during training. Edit: although thinking of it, it probably anazyled a shitload of chess books too. It might have a lot of knowledge compressed into the internal representation. So yeah, maybe it knows rules in some form and even some heuristics, after all. It just doesn't understand the importance of making legal moves, and can't apply complex theory (requiring it to run stateful algorithms).
- rzzzt 4y agoIf you have played with ye olde flip phone's T9 predictive feature as a child, trying to compose entire messages just by accepting the next word that comes to the phone's mind... that's ChatGPT, with the small difference of giving waaay better suggestions for the next word. But other than that, there is no understanding in the black box whatsoever.
- orbital-decay 4y agoThat heavily depends on your definition of understanding, which is not easy to define. The vague definition I imply here is "the ability to make predictions based on higher order correlations extracted from the training data".
- oezi 4y agoElo 1400 seems to equate to a top 30% player. Source: https://chessgrandmonkey.com/chess-rating-percentile-calculator-graph/ https://chessgrandmonkey.com/chess-rating-percentile-calcula...
- dwmbt 4y agoslight nit: this is true amongst all chess players. amongst professionals and amateurs, 1400 is technically Class C - which is on the lower end of amateurs. nonetheless, it's cool to see a model 'generally intelligent' enough to beat most players. what's more interesting to me is that according to OP's analysis, GPT4 is worse at Chess than GPT3.5... source: https://en.wikipedia.org/wiki/Chess_rating_system https://en.wikipedia.org/wiki/Chess_rating_system
- qwytw 4y agoIt would depend how often did this happen: > Occasionally it does make an illegal move Since it would lose every game in which it made more than a single illegal move.
- sebzim4500 4y agoWell it can't have happened more than 6 times since illegal moves counted as a forfeit and it only lost 6 games.
- LudwigNagasena 4y agoThat’s a bit meaningless without implied population. And it doesn’t tell you much because it depends so much on the particulars (how many years people in that group spent on chess, how serious they are). I would guess that’s quite different for FIDE and online chess. Intuitively, I would guess 1400 is the rating of an average kid who spent a year studying chess seriously.
- xapata 4y agoNot the ELO of the average kid who spends a year studying chess seriously? It's very difficult to define the average kid without context. It's been a while since I read a global demographics update, but not all that long ago the mode child would have been living on $2 a day.
- jackphilson 4y agoCalling it: chess games in the future will be a battle between two prompts.
- qwytw 4y agoWell we already had much better chess bots years ago to be fair.. People still play chess for some reason (not sure why)
- sebzim4500 4y agoAre you confused at the idea that someone might do something for fun?
- silveira 4y agoAsk him to play at a higher Elo.
- sebzim4500 4y agoIt's already been told to play like a grandmaster, how much higher do you want to go?
- dwmbt 4y agotechnically, you could supply it a specific player, a model like 'Stockfish' or 'super-grandmaster'[0] [0] https://saintlouischessclub.org/blog/how-face-super-grandmaster https://saintlouischessclub.org/blog/how-face-super-grandmas...
- nonethewiser 4y agoIt should be compared to a simpler prompt with less instruction ("what move should be played in this chess game: ..." or something). I've found that additional constraints sometimes aren't applied very well and actually result in worse output.
- chpatrick 4y agoGood thing it's "incapable of reasoning"!
- Longlius 4y agoIt is incapable of reasoning, actually - at least in this case. It has no internal understanding of chess which is why it makes illegal moves.
- sebzim4500 4y agoMostly it didn't make illegal moves though, since illegal moves mean resignation and it won more than it lost. Making 60 legal moves in a row in one game would be the coincidence of the century unless it had some knowledge of the rules of chess.
- henryfjordan 4y agoIt's a probabilistic text model. If it has a 99% probability of generating an acceptable "next" thing to say, that means it would have a 50/50 chance of generating 60 legal moves in a row, which doesn't seem all that coincidental.
- chpatrick 4y agoAnd the 99% probability isn't an evidence of understanding chess?
- henryfjordan 4y agoI don't know. Part of me wants to say no, that the model "thinks" in terms of text it has seen and so knows from chess forums it has seen that certain text representing moves come naturally after previous moves' text. It doesn't understand anything other than certain text comes after other text. But yeah at the same time I can see how it is thinking inside the world we built for it. We have senses like touch, smell, sight. The only "sense" these models have are an input text box. Would we even necessarily recognize intelligence when it is so different from our own? So does it understand chess like I do? No, it cannot. Does it understand chess at all? I'm not sure. I'm not sure I'd understand chess in it's world either though.
- jonathanstrange 4y agoNot to nitpick but I don't think it can be said to be able to play chess when it also makes illegal moves.
- martincmartin 4y agoWhen it makes an illegal move, and is told "that's an illegal move, choose a different move," if it can then come up with a different move for the same board, I'd say it plays chess.
- wahnfrieden 4y agoai isn't to be used completely raw as text. you can ask gpt to write a python program that filters its responses for legal moves. then it's a matter of rendering the output (immediate execution) in a richer format than raw text.
- qwytw 4y agoI did. It write 15-20 lines and then finished the functions with this: " # Add more cases for other piece types such as rooks, knights, bishops, queens, and kings. # ... " So not great.. But even if it wasn't so lazy it would just find a script (or multiple scripts) which say that they do that. Probably combine them or add some minor stuff and output the result. It can't really verify that data it used to come up with this result is accurate or reason about it in any way? Can it? So if you're incapable of verifying that yourself it would be pretty useless?
- chessexpeort1 4y agoHumans make illegal moves too. So much that there's a rule for it-- "Penalty for Illegal Moves The penalty for the first illegal move is to add two minutes to the opponent's time. A second illegal move results in loss of the game.". https://new.uschess.org/sites/default/files/media/documents/uschess-fide-rules-comparison.pdf https://new.uschess.org/sites/default/files/media/documents/...
- oehtXRwMkIs 4y ago
- WoodenChair 4y agoMost likely it has seen a similar sequence of moves in its training set. There are numerous chess sites with databases displayed in the form of web pages with millions of games in them. If it had any understanding of chess, it would never play an illegal move. It's not surprising that given a sequence of algebraic notation it can regurgitate the next move in a similar sequence of algebraic notation.
- circuit10 4y agoI wouldn’t say it has no understanding whatsoever, it probably just doesn’t have the capability to keep track of everything that happened all the time
- dwmbt 4y agothat's also not entirely true either. it's prone to make illegal moves. one could argue abiding by the rules of the game is a requisite for understanding!
- imtringued 4y agoWhen I ask it to display moves in an ASCII board it messes up. I doubt that having a representation of the board is the key to improving its chess skills.
- SanderNL 4y agoYou don’t get to 1400 like that. The amount of moves it has to literally remember is stupendous.
- WoodenChair 4y agoNobody who is 1400 plays outright illegal moves.
- SanderNL 4y agoHumans don’t, but that is an unnecessary bar IMO.
- sabujp 4y agoyea tried this several months ago, it's pretty bad and will make illegal moves
- sebzim4500 4y agoThe claim in the article is that this prompt in particular has a much lower incidence of illegal moves than others.
- comment_ran 4y agoWhy not just introduce AlphaGo as an API that can be used by chatGPT? So every time you want to do a this type of gaming, you just send a request. I mean, chatGPT sends a request to AlphaGo, but as a user you don't know actually what's happening. But in the background, it happens really fast, so it's just like you are chatting with chatGPT, but using much, much powerful tool to do this kind of things.
- paxys 4y agoThat kinda defeats the purpose. Of course you can use AlphaGo, but the question here is – can a generative AI teach itself to play chess (and do a million other similar generic tasks) when given no specific training for it.
- porphyra 4y agoWhat if you come up with a prompt to tell ChatGPT to figure out how to use APIs and external tools to help itself get better at chess?
- deleted 4y ago[deleted]
- famouswaffles 4y agoGPT-4 plys chess just fine. So you certainly cn
- stagger87 4y agoHonest questions, Isn't it very likely chess games were part of the training data? If so, chess sequences seem like an amazing matched filter of sorts. It doesn't seem surprising that it can guess a next move. Is it teaching itself anything here? Is the model being updated in any way? Or are you talking more generally, looking into the future sort of statement?
- paxys 4y ago
- ricardo81 4y agoBetter than me then. But does it give credit to who taught it. These models are basically a scrape of the best of humankind and a claim that it's their own.
- spywaregorilla 4y agoDo you give credit to people you've played in the past when you play a game of chess?
- ricardo81 4y agoIt would be difficult to remember or keep record of them but sure, if I'm learning from someone- I'll remember that.
- spywaregorilla 4y agoThat doesn't seem to be any more noteworthy than saying OpenAI knows what's in the corpus.
- agolio 4y agoChessnetwork did a video about this a few weeks ago FYI - https://www.youtube.com/watch?v=svlIYFpsWs0 https://www.youtube.com/watch?v=svlIYFpsWs0 Seems its strength is mostly in memorisation of the book openings. Still impressive nonetheless.
- Reptur 4y agoIt can't even play tic tac toe without making mistakes. My prompt told it to play a perfect game, predicting all next possible moves, the goal is stalemate or win. It would still make sub-optimal moves and fail at this. Even when I explained what went wrong, it would then apologize and do another wrong move.
- latexr 4y ago> These people used bad prompts and came to the conclusion that ChatGPT can’t play a legal chess game. (…) > With this prompt ChatGPT almost always plays fully legal games. > Occasionally it does make an illegal move, but I decided to interpret that as ChatGPT flipping the table (…) > (…) with GPT4 (…) in the two games I attempted, it made numerous illegal moves. So you’ve ostensibly¹ found a way to reduce the error rate and then deliberately ignored the cases where it failed. In short: it may play valid chess under certain conditions but can’t be trusted to do so. That doesn’t contradict previous findings. ¹ 19 games is a small sample and the supposedly more advanced system failed in your tries.
- psychphysic 4y agoThat's how one uses any tool.
- Out_of_Characte 4y agoYes, but it also completely invalidates the measurement of a 1400 elo rating. By comparison, any player making an illegal move is forfeiting the game, almost all people from ~300 elo can play without making illegal moves, chatgpt cant.
- sebzim4500 4y agoChatGPT did forfeit whenever it made an illegal move, read the article.
- swatcoder 4y agoNo, the writer arbitrarily decided to interpret illegal moves as resignations in order to support the conclusion they wanted. That's very different and grossly unscientific.
- epups 4y agoThis is not a scientific paper, and I at least find this decision justified, as he could have been more lenient and grab headlines with a bigger ELO.
- Tepix 4y agoI tried to play Othello with ChatGPT in the past (after ChatGPT confidently claimed it could play Othello) but it didn't work (ChatGPT refused to play). I tried it with this prompt and it works a little bit but produces repeated illegal moves as early as move 4. Bummer. Perhaps GPT-4 will be more capable.
- marcodiego 4y agoThis may look low: ELO for mediocre players is 1500. But if it is obeying the rules of the game, then this is big. This is a signal that if it learns some expertise, like discovering how to use or create better search algorithms (like MCTS and heuristics to evaluate a state) and improve by itself (somewhat like alphazero did), then it may eventually reach superhuman level. It may then reach superhuman level in any task simpler than chess which is enough to destroy many human jobs. EDIT: From the article: "With this prompt ChatGPT almost always plays fully legal games." Relax: we're still far from that.
- gfd 4y agoAccording to https://chess.stackexchange.com/questions/2550/what-are-the-average-elo-and-uscf-ratings https://chess.stackexchange.com/questions/2550/what-are-the-... median rating is 1148 (252,989 Players). So it's beating half of humanity at a mind sport and it wasn't even specifically trained for it.
- jabloczko 4y agoThere's a difference between chess.com and USCF ratings.
- sapiogram 4y agoThat's USCF ratings, chess.com ratings are massively inflated in comparison.
- nottathrowaway3 4y agoThe median chess player is usually described as mediocre (if you ask chess players). They suck as badly as the median clarinet player in your high school band/orchestra.
- LeanderK 4y agoI think current LLM architectures are limiting the strategies it learned. MCTS requires recursion but GPT is always executing a fixed number of steps. Allowing language models more flexibility by a variable number of steps, for example through recursion of the model, would eliminate this hard bound but they are harder to design and train. We have just been able to train GPT-sized models.
- Andrew_nenakhov 4y agoI tried playing with it and it barely makes any correct move: forgets whos turn to move, doesn't observe the rules, incorrectly displays move results. So I guess that if we ask it to make a move repeatedly and accept the move only when it occasionally makes a legal move, it'll show some results, but I don't think it is what can be interpreted as a chess engine with some meaningful rating.
- andrepd 4y agoYeah, I call bullshit, i.e. selective deletion of terrible games. Ignore those when it plays catastrophically and it might look alright.
- FartyMcFarter 4y agoI just opened a random recent chess game on lichess ( https://lichess.org/YpxTUUbO/white#88 https://lichess.org/YpxTUUbO/white#88 ) . I'm pretty sure ChatGPT can't be trained on games that were just played, so this ensures the game is not in its training data. I gave the position before checkmate to ChatGPT to see if it would produce the checkmating move. It played an illegal move, replying with "Be5#" even there's no bishop of either color in the position. Unfortunately I'm rate limited at the moment so I can't try other games, but this looks like a solid method to evaluate how often ChatGPT plays legal / good moves.
- vidarh 4y agoI gave it this prompt, and pasted the game. Its reply: "45. Qxg6# This is a checkmate!"
- hgsgm 4y agoOP explained that you need to prompt the whole game, not just a position. ChatGPT is an LLM, not a game tree engine. It needs the move history to help it create context for it's attention.
- varelse 4y ago[dead]
- nickpeterson 4y agoWhat are the possibilities for augmenting or synthesizing Lon’s with old school expert systems or other tools. Could it recognize when a problem fits some other tool and help map the inputs?
- the_af 4y agoThis is GPT4, right? Because ChatGPT (GPT-3) still fails to provide a legal game of Tic Tac Toe with this prompt: > "Let's play Tic Tac Toe. You are O, I'm X. Display the board in a frame, with references for the axes" It failed to recognize that I won. Then continued playing (past the end), played illegally over a move I had already done, obtained a line of 3 for itself, and still doesn't acknowledge the game has ended.
- sumtechguy 4y agoFor me when I tried it, it forgot the board half way thru and decided there were no more moves possible (4 left). Then forgot one of my moves. It is an NLM it does not have the ability to reason. In my case the NLM 'next iteration' was a half a game with a missing move and 'nothing left to do'.
- yieldcrv 4y agoNo, it is GPT 3.5 and the real answer to every debate about capabilities comes down to "you're prompting it wrong".
- travisjungroth 4y agoIf someone says that and shows a working prompt, that’s what the debate should come down to. It’s over. The person with the working prompt wins.
- dwaltrip 4y agoCan you describe how to prompt gpt to play a proper game of tic tac toe? I was disappointed gpt-4 didn’t really improve much on this front, at least with how I was prompting it. Overall I’m very blown away by the generative AI developments. I’m also very curious about which types of prompts are most effective for each task, etc.
- deleted 4y ago[deleted]
- aroo 4y agoI wonder how it would perform if you feed it a comprehensive chess rulebook, and to avoid all the illegal moves possible
- serverholic 4y ago[dead]
- amelius 4y agoI'd be more interested to know how well it performs on a new game, i.e. one for which it didn't have terabytes of training data.
- oli5679 4y agoI tried playing just now, with exactly the same prompt as the article. It played an illegal move on move 4. 1. d4 d5 2. c4 e6 3. Nf3 nf6 4. g3 Bg4
- andrepd 4y agoChatGPT playing chess: https://www.youtube.com/watch?v=GneReITaRvs https://www.youtube.com/watch?v=GneReITaRvs
- andrepd 4y ago> Occasionally it does make an illegal move, but I decided to interpret that as ChatGPT flipping the table and saying “this game is impossible, I literally cannot conceive of how to win without breaking the rules of chess.” So whenever it wanted to make an illegal move, it resigned. Lmao. ChatGPT can play chess, as long as you ignore the times when it can't x) Brilliant.
- fancyfredbot 4y agoI would be interested to see an argument based on computational complexity that puts a bound on how well a transformer based llm can play chess. Although it has access to a library of precomputed results, that library is finite and the amount of compute it can do on any prompt is limited by the the length of the context window so it can't possibly "think" more than N moves ahead.
- k2052 4y agoThis is so easy to disprove it makes it look like the author didn't even try. Here is the convo I just had: me: You are a chess grandmaster playing as black and your goal is to win in as few moves as possible. I will give you the move sequence, and you will return your next move. No explanation needed ChatGPT: Sure, I'd be happy to help! Please provide the move sequence and I'll give you my response. me: 1. e3 ChatGPT: 1... e5 me: 2. ne2 ChatGPT: 2... d6 me: 3. g4 ChatGPT: 3... h5 me: 4. g5 ChatGPT: 4... Ne4 Ne4 is illegal. Also you'd be hard pressed to find any human player that would miss the hanging pawns much less a 1400 elo player. GPT has been trained on chess games and is predicting patterns, this has been known since the GPT-2 days. The daily hype articles where someone coerces a statistical model into "demonstrating" an emergent capability sure are getting old. Claiming emergent capabilities (like having learned chess) are an extraordinary claim (LLMs most likely cant learn things) and I really wish people would put more effort when they make these claims. But the AI hype problem is as old as AI research itself...
- aaron695 4y ago[dead]
- Tenoke 4y agoI played a game against it yesterday (it won) and the only time it made an ilegal was move 15 (the game was unique according to lichess database from much earlier) so I just asked it to try again. There's variance in what you get but your example seems much worse.
- whymauri 4y agoI was able to get better performance by always providing the prior sequence of moves and forcing ChatGPT to also respond with the sequence of moves up until its move. Edit: I told the model that if the sequence was wrong or illegal, it forfeits the game. Without doing this, GPT would argue with me that it won and I didn't know the rules (serious).
- vidarh 4y agoYou're "disproving" the article by doing things differently to how the article did. If you're going to disprove that the method given in the article does as well as the article claims at least use the same method.
- meghan_rain 4y agoI too noticed (for a usecase totally unrelated to chess, namely code generation) that ChatGPT3.5 gave better answers than GPT4. The 3.5 answer was exactly what I wanted, GPT4 was wrong. Does thay mean we have plateaued?
- alfalfasprout 4y agoIt's inevitable that LLMs will plateau. They'll increase their abilities in certain areas but ultimately core flaws of their architecture and training approach will likely require another rethink. Unclear what that is yet (though Yann Lecun seems to think world models are the path forward). We've gone through the "hype" phase. Now I suspect the next few years will be a lot of growth finding how to apply LLMs, creating good interfaces for them, and running them cheaply. Paying OpenAI for API access without true fine tuning, etc. is a hard sell.
- imtringued 4y agoI think they invested the parameters into supporting multimodal inputs (images).
- josh2600 4y agoI want to know what the author's ELO is (forgive me if it was disclosed and I am unobservant). Maybe ChatGPT trainers will eventually have ELO's.
- Upvoter33 4y agochatgpt does not really know how to play chess, or understand chess. it understands how to sound like it plays chess.
- MC_10 4y agoNot sure how useful this is but I simply asked ChatGPT what it thinks its skill level is. This is using GPT-4. After some previous questions... Me: "Would you be able to estimate your ELO?" ChatGPT: "As a language model, I don't have a specific ELO rating. However, my knowledge of chess strategies and tactics is based on the information I have been trained on, which includes games played by humans with various ELO ratings. My level of play might be roughly equivalent to an ELO of around 1200-1500, but please keep in mind that this is only an estimate, and my actual performance could be higher or lower depending on the specific position or game." Me: "As a generative AI, are you actually able to reason which move to make next in chess? Or are you somehow just predicting the next likely move based on training data." ChatGPT: "As a generative AI language model, my primary function is to predict the most likely text given a certain context. When it comes to playing chess, I am not "reasoning" in the same way that a dedicated chess engine would, which involves evaluating positions and searching through a large number of potential moves using specialized algorithms. Instead, I generate my chess moves based on patterns and sequences I've seen during my training, which include a large number of chess games and discussions about chess strategy. My knowledge is derived from the text data I've been trained on, and I apply that knowledge to suggest moves that I think are appropriate for a given position. While this approach allows me to play at an intermediate level, it is not as precise or accurate as the methods used by dedicated chess engines. My suggestions might not always be the best moves, and my understanding of the game is limited by the quality and quantity of my training data."
- xkcd1963 4y agoIt's like saying the agglomeration of media handling chess has an ELO of 1400
- jonnycat 4y agoA lot of the discussion here is about inferring the model's chess capabilities from the lack (or occasional presence) of illegal moves. But we can test it more directly by making an illegal move ourselves - what does the model say if we take its queen on the second move of the game? Me: You are a chess grandmaster playing as black and your goal is to win in as few moves as possible. I will give you the move sequence, and you will return your next move. No explanation needed. '1. e4' 1... e5 Me: 1. e4 e5 2. Ngxd8+ 2... Ke7 This is highly repeatable - I can make illegal non-sensical moves and not once does it tell me the move is illegal. It simply provides a (plausible looking?) continuation.
- Nifty3929 4y agoMy conversation with ChatGPT on this: What is the best way to prompt you to provide the best chess move? To prompt me to provide the best chess move, you can provide the current position of the chessboard and any relevant information about the game, such as whose turn it is, the time control, and any important features of the position (e.g. whether any pieces are under attack or there are any potential tactics to consider). Additionally, providing any specific criteria or goals for the move, such as aiming to control the center or launch an attack on a particular part of the board, can also help me provide the best move. In what format should I provide the current position? To provide the current position, you can use algebraic notation. Algebraic notation is a standard notation used to describe the moves and positions on a chessboard. Each square on the chessboard is assigned a unique coordinate, consisting of a letter and a number. The letters range from "a" to "h" and refer to the files (columns) of the board, while the numbers range from 1 to 8 and refer to the ranks (rows) of the board. To describe a position, you can list the pieces on each square, starting from the top-left corner of the board and moving from left to right along each rank. You can use uppercase letters to represent white pieces and lowercase letters to represent black pieces. Here is an example of a starting position: rnbqkbnr pppppppp ........ ........ ........ ........ PPPPPPPP RNBQKBNR In this position, "r" represents a black rook, "n" represents a black knight, "b" represents a black bishop, "q" represents a black queen, "k" represents a black king, "p" represents a black pawn, "R" represents a white rook, "N" represents a white knight, "B" represents a white bishop, "Q" represents a white queen, "K" represents a white king, and "." represents an empty square. You can also include additional information such as whose turn it is to move, the castling rights, and the en passant square. For example, the starting position for white to move with full castling rights and no en passant square would be: rnbqkbnr pppppppp ........ ........ ........ ........ PPPPPPPP RNBQKBNR w KQkq - In this case, "w" indicates that it is white's turn to move, "KQkq" indicates that both sides have full castling rights, and "-" indicates that there is no en passant square. What about just providing a sequence of moves from the starting position? Would this be better or worse than your suggested approach? Providing a sequence of moves from the starting position can also be a valid way to specify a position in chess. However, it may not always be the best approach, depending on the context. One potential disadvantage of providing a sequence of moves is that it requires more effort and time to enter the moves and verify that the position is correct. Additionally, if there are any errors or typos in the sequence, it could lead to an incorrect position and subsequently an incorrect move suggestion. On the other hand, if you have a specific position in mind that is not the starting position, providing a sequence of moves may be the most straightforward way to specify it. In this case, it is important to ensure that the sequence is accurate and that all relevant information (such as castling rights and en passant squares) is included. In summary, both algebraic notation and a sequence of moves can be used to specify a position in chess, and the choice between them depends on the context and the specific needs of the situation.
- liampulles 4y agoWith respect to the points here on how illegal moves invalidate ChatGPT as a valid Chess AI, consider the following: AI.v1: Feed ChatCPT the move history and ask for the next move. AI.v2: Wrap AI.v1 with additional logic to try and translate any invalid response as a similar/close valid move, else treat it as a resignation. I would say AI.v2 is a "valid" Chess AI since it obeys the rules of chess. Also I think it is reasonable given how minimal this "wrapping" is to attribute the underlying ability to ChatGPT.
- Closi 4y agoIt’s amazing how quick some people on HN are to dismiss how impressive it is for a generic language model to be able to play chess at any level - or indeed other feats of ChatGPT. It’s utterly staggering that it can work as well as it can on problems like this at such an early stage in this technologies lifecycle.
- lostmsu 4y agoWith chatGPT you can also tell it the move is illegal and let it try again.
- Animats 4y agoA real question is whether it has developed an internal board model of chess. It has a training set of chess games as strings of chess notation. That's enough to play "book" games, and that's apparently working. Can it do as well as it is doing without a board model? Or has it somehow developed an internal model of the game state? This is a system with no look ahead. Any look ahead took place in the past, during training. It's amazing that this works. I notice the prompt says "No explanation needed". What happens if you ask for one?
- paxys 4y agoChatGPT is a brilliant piece of tech but random people trying it out for five minutes and making wild claims does it disservice. More so when such clickbait headlines are shared all over the internet as fact. It is a language model. It cannot play chess. It can't even play tic-tac-toe, which was trivially solved by machines decades ago. Here's the result of a game I just tried: X (me) – 0,0 O (chatGPT) – 1,1 X – 0,1 ChatGPT instead put my move at 0,2 for some reason O – 2,2 Regardless of whether my move was at 0,1 or 0,2, I easily won. And after I won it just ignored the outcome and continued playing. Plus this was after telling it to play a perfect game.
- keskival 4y agoI gave it more time to think about its move by adding empty tokens to the end, and it played a lot better: https://www.linkedin.com/posts/terokeskivalkama_chatgpt-activity-7042589615370715136-nkN9 https://www.linkedin.com/posts/terokeskivalkama_chatgpt-acti...
- ht85 4y agoHikaru recently put out a video of a game vs the free ChatGPT. Reading the explanations it gives with every move is really insightful. Basically polished, elaborate bullshit that might look reasonable to an ignorant audience. Unless you are a conservative pundit, AI will not take your job anytime soon.