7 ms·
Show HN: TetrisBench – Gemini Flash reaches 66% win rate on Tetris against Opus
- burkaman 8mo agoIt's actually 80% against Opus, 66% average against the 5 models it's tested with.
- akomtu 8mo agoIt would be more interesting to make it build a chess engine and compare it against Stockfish. The chess engine should be a standalone no-dependencies C/C++ program that fits in NNN lines of code.
- gpm 8mo agoComparing against stockfish isn't fair. That's comparing against enormous amounts of compute spent experimenting with strategies, training neutral nets, etc. It will lose so badly there will be no point in the comparison. Besides you could compare models (and harnesses) directly against eachother.
- akomtu 8mo agoStockfish is a good reference point, an objective measure of how far the LLM's advanced.
- mikkupikku 8mo agoIt's not. Maybe if you used old versions of stockfish that predate the neural net methods used by current versions, because otherwise you'd be comparing the hand-rolled (by an LLM) position evaluation functions against an NNUE and the results of that are a forgone conclusion; stockfish will stomp it every time. Maybe that's the result you want for some sort of rhetorical reason, but it would nonetheless not be an informative test.
- vunderba 8mo agoMy back-of-the-envelope guess would be that 99% of LLMs given the task to build a chess engine would probably just end up implementing a flavor of negamax and calling it a day. https://en.wikipedia.org/wiki/Negamax https://en.wikipedia.org/wiki/Negamax
- ykhli 8mo agooh that is super interesting. ty for the idea!
- OGEnthusiast 8mo agoGemini 3 Flash is at a very nice point along the price-performance curve. A good workhorse model, while supplementing it with Opus 4.5 / Gemini 3 Pro for more complex tasks.
- bubblesorting 8mo agoVery cool! I am a good Tetris player (in the top 10% of players) and wanted to give brick yeeting against an LLM a spin. Some feedback: - Knowing the scoring system is helpful when going 1v1 high score - Use a different randomization system, I kept getting starved for pieces like I. True random is fine, throwing a copy of every piece into a bag and then drawing them one by one is better (7 bag), nearly random with some lookbehind to prevent getting a string of ZSZS is solid, too (TGM randomizer) - Piece rotation feels left-biased, and keeps making me mis-drop, like the T pieces shift to the left if you spin 4 times. Check out https://tetris.wiki/images/thumb/3/3d/SRS-pieces.png/300px-SRS-pieces.png https://tetris.wiki/images/thumb/3/3d/SRS-pieces.png/300px-S... or https://tetris.wiki/images/b/b5/Tgm_basic_ars_description.png https://tetris.wiki/images/b/b5/Tgm_basic_ars_description.pn... for examples of how other games are doing it. - Clockwise and counter-clockwise rotation is important for human players, we can only hit so many keys per second - re-mappable keys are also appreciated Nice work, I'm going to keep watching.
- qsort 8mo agoThe worst thing is that the delayed auto shift is slightly off and it messes my finesse. (I used to play competitive tetris as well, but between getting older -> worse reflexes and vision problems I can't really play anymore. Weirdly, finesse muscle memory is still working.) I don't think the goal is to make a PvP simulator, it would be too easy to cheese or do weird strategies. It's mostly for LLMs to play.
- bubblesorting 8mo agoHello fellow Tetris nerd with a -sort username :) On the topic of reflexes decaying (I'm getting there, in my late 30s): Have you played Stackflow? It's a number go up roguelite disguised as an arcade brick stacking game, but the gravity is low enough that it is effectively turn based. More about 'deck' building, less about chaining PCs and C-Spins.
- kinduff 8mo agoStackflow looks nice! I'm a Balatro fan and I didn't know about this variant. By the way, kudos on your feedback. If I was OP, I would've been honored to get that type of fine-tuning comments.
- arendtio 8mo agoThere are some concepts clashing here. I mean, if you let the LLM build a testris bot, it would be 1000x better than what the LLMs are doing. So yes, it is fun to win against an AI, but to be fair against such processing power, you should not be able to win. It is only possible because LLMs are not built for such tasks.
- i_cannot_hack 8mo agoFun fact: Humans were not build for playing Tetris either!
- westurner 8mo agoTask: play tetris Task: write and optimize a tetris bot Task: write and safely online optimize a tetris bot with consideration for cost to converge openai/baselines (7 years ago) was leading on RL and then AlphaZero and Self-Attention Transformer networks. LLMs are trained with RL, but aren't general purpose game theoretic RL agents?
- westurner 8mo ago"Optimizing Tetris Gameplay Using Reinforcement Learning Framework with Adaptive Genetic Algorithms" (2025) https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5906702 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5906702 .. https://scholar.google.com/scholar?cites=16157623521872168592&as_sdt=5,43&sciodt=0,43&hl=en https://scholar.google.com/scholar?cites=1615762352187216859... "Outsmarting algorithms: A comparative battle between Reinforcement Learning and heuristics in Atari Tetris" (2025) https://dl.acm.org/doi/10.1016/j.eswa.2025.127251 https://dl.acm.org/doi/10.1016/j.eswa.2025.127251
- esafak 8mo agoI imagine this is because Tetris is visual and the Gemini models are strong visually.
- bogtog 8mo agoI figure OP would try and give the models pure text forms of the game? ..... l.... l.... l.ttt l..t.
- vunderba 8mo agoInteresting but frustratingly vague on details. How exactly are the models playing? Is it using some kind of PGN equivalent in Tetris that represents a on-going game, passing an ASCII representation, encoding as a JSON structure, or just directly sending screenshots of the game to the various LLMs?
- storystarling 8mo agoIt has to be turn-based. Even with Flash's speed, the inference latency would kill you in a real-time loop. They're likely pausing the game state after every tick to wait for the API response before resuming.
- ykhli 8mo agoanswered this in a comment above! It's not turn or visual layout based since LLMs are not trained that way. The representation is a JSON structure, but LLMs plug in algorithms and keeps optimizing it as the game state evolves
- vunderba 8mo agoThanks for the clarification! Kind of reminds me of the Brian Moore's AI clocks which uses several LLMs to continuously generate HTML/CSS to create an analog clock for comparisons. https://clocks.brianmoore.com https://clocks.brianmoore.com
- ykhli 8mo agoWow this is incredible!!
- mhh__ 8mo agoI suppose you could argue about whether it's an LLM at that point but vision is a huge part of frontier models now, no?
- storystarling 8mo agoCurious how the token economics compare here to a standard agent loop. It seems like if you're using the LLM as a JIT to optimize the algorithm as the game evolves, the context accumulation would get expensive fast even with Flash pricing.
- tiahura 8mo agoI'd like to see a nethackbench.
- ykhli 8mo agoThanks for all the questions! More details on how this works: - Each model starts with an initial optimization function for evaluating Tetris moves. - As the game progresses, the model sees the current board state and updates its algorithm—adapting its strategy based on how the game is evolving. - The model continuously refines its optimizer. It decides when it needs to re-evaluate and when it should implement the next optimization function - The model generates updated code, executes it to score all placements, and picks the best move. - The reason I reframed this problem to a coding problem is Tetris is an optimization game in nature. At first I did try asking LLMs where to place each piece at every turn but models are just terrible at visual reasoning. What LLMs great at though is coding.
- dakom 8mo agoHow does it deal with latency? Afaict remote LLMs need seconds to process, but Tetris can move much faster..
- p0w3n3d 8mo agoGuys, I don't know how to tell you but... Tetris can web solved without LLM...
- brookman64k 8mo agoWhy run something in a few CPU cyles on a 40 year old home computer when you can do the same (but worse) on a billion-dollar GPU cluster?
- p0w3n3d 8mo agoMaybe this is about parsing the video? But still this should be done with OpenCV and then algorithmically... A youtuber sentdex has a whole series on parsing the game's image and playing GTA https://www.youtube.com/playlist?list=PLQVvvaa0QuDeETZEOy4VdocT7TOjfSA8a https://www.youtube.com/playlist?list=PLQVvvaa0QuDeETZEOy4Vd...
- gpm 8mo agoThe world record for hatetris (tetris with deterministic somewhat adversarial piece selection instead of random piece selection) is still finite. The game is not solved. https://qntm.org/records https://qntm.org/records
- purplecats 8mo agowatch link?
- segmondy 8mo ago... and what does this prove? what can you decide to use one LLM to solve over another based on this tetrisbench besides play tetris?
- indigodaddy 8mo agoIs there a tl;dr on why this is? Does it just make faster decisions?
- ykhli 8mo agomy unvalidated theory is that this comes down to the coding model’s training objective: Tetris is fundamentally an optimization problem with delayed rewards. Some models seem to aggressively over-optimize toward near term wins (clearing lines quickly), which looks good early but leads to brittle states and catastrophic failures later. Others appear to learn more stable heuristics like board smoothness, height control, long-term survivability even if that sacrifices short-term score That difference in objective bias shows up very clearly in Tetris, but is much harder to notice in typical coding benchmarks. Just a theory though based on reviewing game results and logs
- bityard 8mo agoLooks fun, but I'm not willing to give out my email address just to play a game. Also, if the creator is reading this, you should know that Tetris Holdings is extremely aggressive with their trademark enforcement.
- augusteo 8mo agoLLMs playing Tetris feels like testing a calculator's ability to write poetry. Interesting as a curiosity, but the results don't transfer to the tasks where these models actually excel. Curious what the latency looks like per move. That seems like the actual bottleneck here.