11 ms·
This report provides a detailed background of Hans' potential cheating, and detailed breakdowns of certain aspects of chess.com's cheat-detection methodology, i
by erdevs 4y ago
This report provides a detailed background of Hans' potential cheating, and detailed breakdowns of certain aspects of chess.com's cheat-detection methodology, including previously unknown (or little known) methods such as window focus change event monitoring and post-focus-change move analysis.[1]
The report also reveals Niemann's engine move correlations alongside over two dozen chess Grandmasters who have admitted to cheating on chess.com. The fact that online cheating is so widespread even among top chess players is certainly news to many, including me. Perhaps it is a good thing that this scandal is highlighting the issue, and given how widespread cheating may be, perhaps chess tournaments both online and physical need to take cheating much more seriously than they apparently have been.
There is also an interesting analysis of Hans' rating improvement history, his over the board tournament performance and key game analysis, and a rundown of key moments in his game against Carlsen in the Sinquefield cup. Each raises concerns.
Chess.com's report also makes it clear that Niemann lied outright about his history of cheating in post-Sinquefield interviews, as he admits in communications with chess.com Fairplay staff to much broader cheating.
All in all, the report raises many concerns and it seems reasonable for the chess community to demand much higher standards of cheat prevention and detection across competitive venues. How long might cheating issues have gone on merely rumored vs fully investigated or acted upon, had this intrigue not developed due to Carlsen's withdrawal from Sinquefield '22?
[1]Tangentially, this induces an obvious concern about cheat and cheat-detection arms races. A clever cheater might scrutinize this report and refine their cheating plan. For example, they might recognize the need to use a second device (such as a phone) to cheat. They might use the data corpus presented in this report to establish limits on how often they use chess engine moves per game, and they might manage their ratings progress over time carefully, so as to stay in acceptable ranges of engine move correlation, rate of improvement, etc.
- htrqball 4y ago
- altairprime 4y agoHi, welcome to HN. Speaking for all of us long-term users, I think we would very much enjoy that 72-page report that you've offered to compile. If you post a submission linking to it, let us know. (Especially interesting would be a meta-analysis of the types of lying and so on!)
- deleted 4y ago[deleted]
- erdevs 4y ago> I find the decsription "lied outright" unfitting Hmm.. if time allows in the next hour or so, I will find his exact quotes from the interview and will definitely edit this out if it is unfair. But my recollection is that Hans stated explicitly that he had never cheated in a serious tournament since he was 12, that he only cheated thereafter in "unrated" games, that he only cheated twice overall, and that he hadn't cheated at all in the ~2.5 years (since he was 16) preceding his game against Carlsen in Sinquefield this year. From the report, chess.com is saying any or all of those statements are outright lies. But it's good to call this out and it sucks to unfairly malign anyone so I'll follow-up if other obligations allow me to scrub through the post-game interviews videos quickly. > Expecting a perfect verbal 72 page report from him under these circumstances fells pretty cruel. To be equally fair, 'a perfect verbal 72 page report from him' is not at all what I stated or implied as a standard, and saying so is a strawman.
- nwienert 4y ago> that he only cheated twice overall I don’t think he claimed “twice overall” as in “twice exactly”, to be fair, just within a period of time.
- camjohnson26 4y agoHe specifically said he never cheated for prize money or while streaming and couldn’t imagine doing so, both statements were false.
- deleted 4y ago[deleted]
- b4je7d7wb 4y agoProbably false
- erdevs 4y agoYes, this is good clarification, thank you!
- 4y ago
- tomcam 4y ago> Shall we compile a 72 page report on Silicon Valley CEOs who have lied? I am not even a little averse to that
- zucgkr 4y agoWe could start with examining the fact that Dlugy was promised privacy, but then chess.com leaked the correspondence to vice.com. Or why exactly the two persons Carlsen mentioned were exposed, but not the other GMs who cheated.
- deleted 4y ago[deleted]
- rfrey 4y ago> Shall we compile a 72 page report on Silicon Valley CEOs who have lied? Hopefully you mean, "a 72 page report"... each.
- notfed 4y ago> He seemed a bit on the spectrum and oversharing, literally hyperventilating and not talking in coherent sentences. Yeah, not really. He's 19 and is a non-native English speaker. He understands that he's under siege and has to show some vulnerability to gain trust in listeners. IMO, the way he spoke was completely in accordance with all of this, while also remaining coherent.
- kentor 4y agoHe's a native speaker, just spent too much time around people with broken English recently.
- runnerup 4y agoHe’s born in San Francisco, very classic white American native. While he did spend ages 7-10 in the Netherlands, he went to high school in Connecticut and New York. His uncharacteristic accent is from spending the last few years studying chess in Europe.
- computerdork 4y agoIt's a good point, I do also think that people (especially on social media) are too quick these days to jump all over any minor mistake a person makes. But (in my humble opinion), if the report is correct, then Niemann went waay too far, beyond a little lying and cheating to possibly raising himself at the highest levels of chess (and winning actual money). Yeah, as you probably saw, it says he cheated online an estimated 100 times, some for actual prize events (amongst other implications). Yeah, personally, I do believe the report as it does seem to match a lot of other online analysis of his game history (this isn't the greatest video, but it is compelling: https://www.youtube.com/watch?v=qjtbXxA8Fcc https://www.youtube.com/watch?v=qjtbXxA8Fcc). And sure, it does seem cruel to call him out in an interview, but so is cheating to get to the top. Yeah, someone who cheated on this scale has robbed those players of the recognition and prize money they worked so hard to deserve. Honest players who have probably devoted way more time than Niemann had (the two years of study you mentioned is impressive, but maybe others have studied like this for decades??)
- vkou 4y agoI can't speak for SV, but I think a 72-page report from every confessed past fraudster that's engaged in what seems to be fraud-adjacent activity would do wonders in clearing the crap out of the crypto scam space.
- berkut 4y agoDoes engine correlation actually prove anything though? Some of the 'statistical analysis' that has been posted on twitter regarding it in the last week has been against hundreds of engines, so 'engine correlation' seems to mean "the move made matched against at least one engine that would have made that move" I think?
- gbear605 4y agoChess.com explicitly states in the report that that sort of methodology does “not meet our standard” for cheating detection. If they don’t feel comfortable using it, I certainly don’t.
- erdevs 4y agoI don't think engine correlation necessarily proves anything, on its own. It's worth remembering, though, that chess.com's report a) presents more than merely raw engine correlation, and b) its correlations do not seem to match against hundreds of engines. But even with all the evidence presented, "proof" is a tricky thing. To what standard would we be trying to prove a claim? Does this report prove beyond all reasonable doubt that Niemann cheated? I'd say no, but others may disagree. How about to a preponderance of evidence? Perhaps. But even that is hard to say when no one has yet presented a rigorous defense or set of counterpoints. In any case, my post wasn't meant to say that Niemann cheated per se. I have no idea, and chess.com themselves may not be able to actually prove whether he did. But I found the report interesting, even beyond the current issue surrounding Niemann and speaking to potential cheating in high-level chess more broadly, and if you re-read my post, I tried not to state anything definitive about whether Hans actually cheated or not.
- bsder 4y ago1) You can look at the "strength" of individual moves. Someone who plays at 2000-level normally but magically coughs up 2600-level moves when in trouble is probably cheating (watch some of the live chess streamers--you'll regularly see this in real time). Computers are quite good at estimating the strength of a move after the fact. 2) Quite often there are certain "play lines" that computers will play that humans simply can't find over the board. For example, a computer can take a defensive "play line" that is littered with traps with only a single non-losing path for 30+ moves and work it out really quickly (there is only one non-losing path to take so it prunes the search space mega fast) and play it perfectly. A human playing such a line is almost always cheating--humans simply can't run those kinds of lines in real time. If you look at computers analyzing even the highest end games, you see the humans making quite a few mistakes that the computers will spot and take advantage of immediately. Someone who walks down these kinds of paths regularly is a statistical anomaly. That having been said, given the current crop of computer-trained chess kids, it IS possible that we'll grow a prodigy that can run those kinds of lines. However, it doesn't seem like that person exists, yet.
- Dalewyn 4y ago>The fact that online cheating is so widespread even among top chess players is certainly news to many, including me. Not to me. When the stakes of winning are so damn valuable, sometimes literally so, it would be patently stupid to not cheat and increase your odds of winning. Remember: Cheating is only a problem if you're caught. If you're never caught, cheating is completely legitimate. These aren't games of pleasure, played for the sake of playing chess. These are games played for the sake of winning. Prize money, ELO, fame and renown, etc. When the only thing that matters is winning, you gain nothing by playing honestly.
- lvass 4y agoGames are just games, they don't have to mean the same thing to everyone. The way you view them tells more about youself than about the games.
- Dalewyn 4y agoThe question is one of incentives and motivations. Am I playing chess because I love chess? To have fun? In that case, cheating is counter to my goals (unless I enjoy the act of cheating itelf, but that's getting besides the point). On the other hand, is chess simply a means to an end? Am I playing chess because I want to win something? If so, cheating is but one of many possible decisions I can undertake to increase my odds of winning. Where professional games are concerned, where money and fame are at stake if you lose, cheating is downright inevitable and certainly not something to be surprised about.
- lvass 4y agoI don't disagree at all. But there are people who just want to play the game by the rules. There are people who won't cheat for moral reasons. There are people who lack competence to cheat. There are people who are risk averse. Cheating should be expected, but you were implying every single person has more incentive to cheat than not, which just isn't true. You could even argue most people are in fact cheating and I might even agree with you, but without evidence it's just slandering, something even the world champion should be reprehended on.
- maegul 4y ago> [1]Tangentially, this induces an obvious concern about cheat and cheat-detection arms races. A clever cheater might scrutinize this report and refine their cheating plan. For example, they might recognize the need to use a second device (such as a phone) to cheat. They might use the data corpus presented in this report to establish limits on how often they use chess engine moves per game, and they might manage their ratings progress over time carefully, so as to stay in acceptable ranges of engine move correlation, rate of improvement, etc. Important point I'd say. I really can't shake my personal feeling that chess as a sport is just simply dead, especially as an online e-sport. Especially in combination with the possibility that there's plenty more cheating that they're not catching/detecting. Taking this report along with Hans's admission to only cheating twice, for example, and accepting Chess.com's assessment as accurate, it would seem that Hans's mistake was to confess only to the instances of cheating that he thought had been caught. Which indicates the mentality and experience involved where the actual game is not getting caught and many, just as Hans was before beating Magnus, are playing it successfully.
- ninethirty 4y agoMakes me wonder how many people are running game-theory-optimal poker bots and raking in fortunes.
- rrrrrrrrrrrryan 4y agoI don't know if a game-theory optimal poker bot would work well - You'd really want a battle-tested psychologically optimal bot that can convincingly play like an amateur, except when it really counts, to give your victims a false sense of confidence. A hustler-bot, essentially. I'm sure it'd be highly illegal.
- remarkEon 4y agoI guess I’m not as cynical. I picked up chess again when I was deployed (I was on the chess team in high school) and have been closely following this drama, and my sense is that perhaps we all admit that online chess is dead, but a return to analog chess is possible and, perhaps, preferable. I know this begs the question of defeating cheats in irl play, but shouldn’t that be an easier problem to solve?
- cowvin 4y agoOne of the most interesting (but inconclusive) points they found was how Hans' evaluated strength dropped after they introduced the 15 minute broadcast delay.
- krick 4y agoExcept, introducing the 15 minute broadcast delay just happens to coincide with Magnus scandalous withdrawal, Hans being especially carefully searched and being stared at by everyone (that is, if we trust how it was evaluated over a couple of games in the first place). So, again, even though some statement from chess.com was expected, this one probably does more harm than good. Heavy implications, winking and chuckling, but nothing that would allow one to close the case (quite naturally). All of this historical progress evaluation was done by people a month ago already, and did achieve as much as this one. This one is "official" though, so pours another barrel of fuel into the fire. That said, I wonder why 15 minute (or even more) delay isn't standard in events like that. Seems like the least you could do, given how questionable chess in 2022 is in the first place.
- 6510 4y agoI didn't look at the chess.com thing since the historic progress is eclipsed by a streak of 5 important games where he simply scores 100%. Everything else is irrelevant. GM's usually have zero games at that level. Dropping to his usual level after that doesn't average it out to human level. lol Someone should organize that running joke: naked chess tournament.
- sigmaml 4y ago> Tangentially, this induces an obvious concern about cheat and cheat-detection arms races. This exists in every domain, and is - perhaps - inevitable. Look at what SEO has done to web search.
- 72deluxe 4y agoThe one thing that baffles me with SEO is how it's just guesswork yet is a massive business. It's like promising to someone that you'll get their name listed sooner in the phonebook without any control over the phonebook itself, and then that person pays you to do it. It's no different, since we have no control over Google's ranking mechanism and they won't explicitly tell you what the algorithm is (and change the rules daily), so it's just guesswork. Baffling an entire colossal "industry" is built from guesswork.
- krageon 4y agoIt's chess, it's not heart surgery exams. The cheating sucks, but anti-cheat measures are infinitely worse. There is no better way to suck the fun out of something than demanding you submit to a patdown and metal scan or a rootkit before you can participate.
- WA 4y agoIt's just soccer, bicycle racing, weight lifting, sprinting, marathon running, ..., no need to submit to a urine and blood test, if you want to compete in the olympics. That sucks the fun out of it. /s
- krageon 4y agoWith respect, in those examples the extreme amount of money to be made with the first place motivates the cheating. The anti-cheat for such competitions only barely works, as top competitors do as many (possibly dangerous) things they can to come out on top. As long as they are undetectable. We should approach a disincentive to cheating in a different way, so it is just a few lemons as it is in chess.
- res0nat0r 4y agoThey say they estimate that less than 0.14% of players on chess.com ever cheat, so it may be less than everyone assumes just due to this current drama.
- tarentel 4y agoThis number seems wildly low to me. I've been playing on chess.com for 2 years and have run into ~30 cheaters. I know they were cheating because chess.com told me they were. I've suspected a handful more of cheating who may not be as they were never banned. Either way this number doesn't seem correct.
- johnaspden 4y agoThey also claim 100 million subscribers, so that's 140,000 cheats. Since if you're cheating you probably do quite well (I pity the engine that could lose to me!), that's likely most of the high-rated players on their site. And actually this tallies quite well with my experience of amateur physical sport, where at the lower levels everyone's honest and sporting and it's all great fun, but as soon as you get to the point where people are putting their heart and soul into the game, everyone's cheating, everyone knows everyone else is cheating, and anyone who's not cheating might as well give up and go home, as far as winning things is concerned. Once you put money on it, and you're playing against people you're never likely to meet in real life, Jesus I can't imagine what it's like.
- bryanrasmussen 4y ago> window focus change event monitoring if you have ADHD you are naturally a cheater! or if you have some important reason to check your email periodically. same applies to online tests of various sorts.
- hdjjhhvvhga 4y ago> The fact that online cheating is so widespread even among top chess players is certainly news to many, including me. A few weeks ago my daughter very proudly announced that she managed to "outsmart" (=hack) her online test app used by her class. I was shocked and asked why she would do that, she's smart enough to get an A without that. She seemed obviously puzzled by my question and my lack of enthusiasm about her "achievement" she was so proud of (it involved some JS modifications). I guess is it's just another kind of thrill.
- Bakary 4y agoShe should be proud in the sense that real world problem solving and valuable technical skills is a lot more important than any school test. That said she will have to learn not to let arrogance and risk-taking be her downfall.
- bombcar 4y agoDepending on what she had to do to hack it, I’d be proud, and then try to explain to her why even if we can do it we shouldn’t. When you’re bored you try to learn how the system works - it’s a valuable skill.
- borbulon 4y ago> A few weeks ago my daughter very proudly announced that she managed to "outsmart" (=hack) her online test app used by her class. I was shocked and asked why she would do that, she's smart enough to get an A without that. She seemed obviously puzzled by my question and my lack of enthusiasm about her "achievement" she was so proud of (it involved some JS modifications). I wonder if maybe it was the chance to solve a different challenge than the one before her (the test), which if she could already get an A maybe wasn't the right challenge?
- lvl102 4y agoTracking window focus change is not a good deterrent.
- Maursault 4y ago> two dozen chess Grandmasters who have admitted to cheating on chess.com. The fact that online cheating is so widespread chess.com globally has more than 93M members. There is cheating, but a dozen admissions can't be accurately described as "so widespread." Research has shown fewer than 0.02% cheat. While this is mildly shocking, your statement based on 12 grandmaster admissions is a sweeping generalization.
- daveguy 4y agoTwo dozen is 24, not 12. This is how many grandmasters admitted cheating, not how many players globally have admitted it. As of 2021 there were 1,315 active grandmasters. [0] 24/1,315 = ~1.8% of grandmasters admit cheating (give or take a few depending on how many play online chess or are still active). In my opinion, that is a serious problem. [0] https://chessdelta.com/how-many-chess-grandmasters-are-there/ https://chessdelta.com/how-many-chess-grandmasters-are-there...
- Maursault 4y agoEven with my careless error, the statement in question is still a sweeping generalization. > As of 2021 This is a tough number to track down, and you've limited the count by "active," in 2021, but the number is rapidly increasing. According to the FIFA Database as of a few moments ago, there are 1771 chess grandmasters.[1] 24/1771 < 1.4% > In my opinion, that is a serious problem. Even assuming 1.8% of chess grandmasters are cheaters, this means that 98.2% of them are not. If you tested a 98.2% of a perfect score on a test, would you really think your grade was a serious problem? If you had the chance to retake the test for a replacement score, either better or worse, would you? [1] https://ratings.fide.com/advaction.phtml?idcode=&name=&title=g&other_title=&country=%25&sex=&srating=0&erating=3000&birthday=&radio=rating&line=desc https://ratings.fide.com/advaction.phtml?idcode=&name=&title...
- daveguy 4y agoThat's how many grandmasters there are total (1771). I was quoting the number active, hence the hedge of give or take current active / cheating. Yes, I think cheating more than a fraction of a percent as you originally posited is a detriment to competition.
- jsmith45 4y agoWhat seems really odd is that they don't have any data accusing him of cheating after his initial 2020 ban/new account creation. They conclude that while there are several unusual things about his OTB play, there is not enough for them to conclude he probably cheated. Basically the biggest takeaway here is that he definitely publicly lied about the extent of his cheating prior to his original ban.