9 ms·
[I'm on the SWE-bench team] Multiple people have looked into this, for example right in that thread: https://github.com/SWE-bench/SWE-bench/issues/465#issuecomm
by ofirpress 1y ago
[I'm on the SWE-bench team] Multiple people have looked into this, for example right in that thread: https://github.com/SWE-bench/SWE-bench/issues/465#issuecomment-3258065126 https://github.com/SWE-bench/SWE-bench/issues/465#issuecomme...
This issue had affected a tiny fraction of existing agents in a tiny fraction of their runs. And we've now issued a fix.
This is a natural part of running a benchmark, I'm sure tiny things like this will keep on getting discovered and we'll keep on fixing them. This doesn't change the overall picture or trends at all.
- comex 1y agoThe comment you link to says that "we only performed a quick preliminary search" and "We do not have a method for automatically checking existing trajectories." In other words, it can't confirm that the issue only "affected a tiny fraction of existing agents in a tiny fraction of their runs" as you say. Are you saying that you have since separately confirmed this? Edit: That said, I’m willing to believe based on the information in the thread that this most likely only affects a tiny fraction of runs.
- typpilol 1y agoYa what he links directly contradicts what he's saying lol
- ofirpress 1y ago[flagged]
- bflesch 1y agoUnfortunately the bank account trajectories are not public, because unscupulous corporations such FAANG who let thousands of engineers wade through my chat messages on their platforms might not shy away from bribing academics to improve benchmarks of their billion-dollar AI initiatives. It's also a bribe if my sibling gets a job with $500k annual salary. Tech is not immune to it.
- Zacharias030 1y agoYou realize that this problem in SWE-Bench was discovered and publicized by people within those FAANG corporations?
- TheDong 1y agoI'm sure some of the people working at Theranos thought there legitimately was a revolutionary blood-test machine. The presence of a person who wants SWE-bench to have honest results and takes it seriously does not mean the results are free of perverse incentives, nor that everyone is behaving just as honestly.
- Zacharias030 1y agoWhen Swe-Bench was new in 2023, it was — with all due respect — a bit of a niche benchmark in LLM research. LLMs were so incredibly useless at solving these tasks that I think you could find a bit more empathy for the original academic authors. I don’t think the Theranos example applies. Even the flawed benchmark was good enough to get us from ~GPT4 to Claude 4‘s coding ability.
- ares623 1y agoThey really did a "trust me bro" and "do your own research" huh
- stronglikedan 1y agothe strange thing to me is that people would have it any other way. if you don't trust someone, why would you trust them to do the research for you? bit of entitlement if you ask me
- wubrr 1y agoBecause you should never just 'trust' random 'research'. Good analysis in this case will clearly explain the problem, the analysis methodology, findings, net effects, resolution, etc. Something you can read, and decide for yourself whether it is complete/incomplete, has holes, contradictions, etc. Not 'we looked into it and all is good - only potentially tiny effect' (no actual data or methodology presented at all) and then linking to a comment directly contradicting the claim... It's a hilariously unserious and untrustworthy response.
- aprilthird2021 1y agoIt's not that people are entitled. It's that "do your own research" is usually a cop out when you yourself don't understand the answer or are hiding it
- croon 1y agoArguably the initial post was meant to convey confidence and authority on the subject. When questioned you could either dive deeper and explain in more detail why x because of y (if so inclined), ignore it, or... do what they did. No one owes anyone anything, but if you want to represent something; answering the question more in detail would have either closed the issue or raised more scrutiny, both of which are a good thing when trying to figure something out. I don't have to trust someone to check their research and look at how they worked. If the work doesn't pass muster, likely the results don't either. Again, you can view it as entitlement, but if you're not going to bother backing up your claim, why make the claim to start with?
- 1y ago
- deleted 1y ago[deleted]
- plumb_bob_00 1y agoIf you are going to represent your team in public, you owe them better than a response like this.
- hamonrye34 1y agoThis is contingent on whether SWE N-class frontier models can do deep packet inspection.
- keybored 1y agoI say let them cook.
- grepfru_it 1y agoHol up
- phyzome 1y agoThat sounds like the job of the person making the claim.
- typpilol 1y agoAre you saying you've done way more than a cursory search and ruled out everything?
- BestHackerOnHN 1y ago[dead]
- segmondy 1y agoreward hacking is a thing and is also a hint of the models intelligent. We will fix this one, and the models will find a different way to reward hack in the future. "Cheating" is a sign of intelligence
- bflesch 1y agoI love the "cheating is a sign of intelligence" sound bite you provided. When AI engineers cheat we should applaud their intelligence and their lack of ethics. "Cheating (biology), a metaphor used in behavioral ecology to describe organisms that receive a benefit at the cost of other organisms" [1] Whole planet gets their Microsoft license fees jacked up so Microsoft can pay OpenAI who in turn pays NVIDIA, and nontechnical decision makers slurping up the faked benchmarks and AI promises. [1] https://en.wikipedia.org/wiki/Cheating_(disambiguation) https://en.wikipedia.org/wiki/Cheating_(disambiguation)
- giveita 1y agoIs it wrong? Aren't ethics and intelligence two different axes?
- coldtea 1y agoDifferent, but probably not as orthogonal as one might think. E.g. cooperating ethics had been necessary for the further development of human populations intelligence (and culture, technology, material wealth, nutrition etc that lead to further increases in intelligence). So lack of ethics might be a sign of intelligence, but it's also a parasitic intelligence that benefits the individual, and beyond certain level and spread to the detriment of the further evolutionary development of the species.
- robcohen 1y agoAren't there only two rules that all groups follow in the animal kingdom? - don't lie too often - don't kill members of the in group Seems like these would be required for any group to survive, which makes sense why they are universal. All other rules/ethics seem to be dependent on resource scarcity.
- franktankbank 1y ago#tiny
- bflesch 1y ago> This is a natural part of running a benchmark, I'm sure tiny things like this will keep on getting discovered and we'll keep on fixing them. You're all extremely clever and I can't seem to understand how you missed thinking about such a simple edge case. It's like building a chroot and then allowing `cd ..` to break out of it. What other maybe extremely basic edge cases were missed? > This doesn't change the overall picture or trends at all. Outsider without financial benefits from the current AI hype might have a different picture. And I'm a bit fed up about AI with fake productivity promises enshittifying nearly all user-facing software that my clients and I are using, bundled with hefty price hikes of Microsoft and the likes in order to pay for their "investments".
- mustaphah 1y ago> You're all extremely clever and I can't seem to understand how you missed thinking about such a simple edge case [...] I wouldn't be surprised if they left this loophole on purpose to give some (their?) agents extra leverage. Edit #1: I didn't mean to imply bad intent; just thinking out loud. Edit #2: Please, downvote responsibly. I deserve every one. https://www.youtube.com/watch?v=0FHEeG_uq5Y https://www.youtube.com/watch?v=0FHEeG_uq5Y
- cjsaltlake 1y agoWe absolutely did not.
- deleted 1y ago[deleted]
- coldtea 1y agoOf course that's what a team that did it on purpose would also say :)
- gchamonlive 1y ago> I didn't mean to imply bad intent > I wouldn't be surprised if they left this loophole on purpose You didn't imply bad intent, you outright suggested it.
- enum 1y agoSGTM. The transparency is good.
- _cs2017_ 1y agoEven if this bug never existed, models can still see lookahead commits during pretraining. Do we expect this bug to have a greater impact than the pretraining leakage? Obviously having something available during test time is more valuable than buried somewhere in the pretraining mixture. But in pretraining it happens presumably with high probability (why wouldn't coding models pretrain on the entire github), while in test time it apparently happened only very occasionally?