11 ms·
Part of my code makes Copilot crash
- uhtred 4y agoIf you told me 10 years ago that gender would be such a hot topic in 2022 I'd have thought you were crazy.
- coolspot 4y agoEverything about 2020-2022 is unreal
- coffee_beqn 4y ago
- the_doctah 4y agoI hope this is satire.
- nonethewiser 4y agoWhy is it a hot topic? There are a range of opinions. It's a manageable little fire. Thats fine. Except some people want to punish others for their opinions. That is the gasoline. And Microsoft is selling gas cans.
- throwaway290 4y agoNow if only someone could figure out a magic word that would stop Copilot from being trained on my code.
- deleted 4y ago[deleted]
- staticassertion 4y agoContent filters on ML feel so silly. I assume the goal is to avoid bad press? Because the... "attack" would be someone generating offensive material, which they could just write themselves, not to mention I have serious doubts that any filter is going to be a serious barrier. For images/ video I can see merit, ex: using that nudity inference project on images of children, but text seems particularly pointless.
- Bolkan 4y ago
- ace2358 4y agoI guess they’re trying to avoid the twitter AI bit incident. https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...
- RajT88 4y agoIt is for certain that. File under not "Why we can't have nice things", but "downstream effects of why we can't have nice things".
- brew-hacker 4y agoThe only reasonable content filters on these sort of models would be something that could have legal repercussions. This is absolutely silly. Solid work GitHub team!
- evrydayhustling 4y agoImagine that you had a co-worker who seemed totally normal 90% of the time... But about once a week, someone would bring up a topic that made them go full nazi or attempt to seduce their coworker. That's where we are with LLM-based generative text. It's not (just) about PR, it's about putting guardrails around the many many many circumstances the tech can do harm or just seem ignorant.
- richardfey 4y agoImagine having a coworker like that.. But he's fully remote, and basically generated in real time by AI (appearance on video, voice etc). Maybe that's where we're going? :) then humans would be hired to occasionally pop in and pass some heavier scrutiny.
- CoastalCoder 4y ago> Imagine that you had a co-worker who seemed totally normal 90% of the time... But about once a week, someone would bring up a topic that made them go full nazi or attempt to seduce their coworker. This is my mental image of how company happy-hour-Fridays play out. It's one of the reasons I don't drink. [And if you're curious, in fact I'm not fun at parties ;) ]
- deleted 4y ago[deleted]
- betwixthewires 4y agoI hope to god that one day we will all see this nonsense for what it is: absurdly hilarious.
- drewcoo 4y ago
- smegsicle 4y ago
- MarkSweep 4y agoThere is a whole Wikipedia article for it!: https://en.wikipedia.org/wiki/Postgenderism https://en.wikipedia.org/wiki/Postgenderism
- rayiner 4y agoI wish “the discourse” would learn to acknowledge the difference between post-genderist ideas, and basic rights for transgender people. The latter is (1) already the law of the land, and (2) not politically controversial: https://www.kff.org/other/press-release/poll-large-majorities-including-republicans-oppose-discrimination-against-lesbian-gay-bisexual-and-transgender-people-by-employers-and-health-care-providers/ https://www.kff.org/other/press-release/poll-large-majoritie... (“Large majorities of Americans think it should be illegal for either employers or health care providers to discriminate against people because they are lesbian, gay or bisexual, or transgender, a new KFF poll finds. This includes large majorities of Republicans, independents and Democrats.”) See also: https://www.hrc.org/press-releases/icymi-bidens-bostock-eo-rated-most-popular https://www.hrc.org/press-releases/icymi-bidens-bostock-eo-r... (83% of the public, including 63% of Republicans support Biden’s executive order protecting trans rights). At this stage, attempting to use trans rights as a vehicle for broader changes to society’s notions of gender harms rather than helps trans people. It takes something that has broad support and is increasingly uncontroversial, and lashes it to a broader intellectual project that’s alienating and unpopular.
- jan_Inkepa 4y agoI encountered this when writing some scripts for Latin-language text processing (which dealt with grammatical gender). Thankfully the Latin-native term 'genus' passed the Copilot smell-test and I could continue with my work. I found it pretty amusing.
- sergiomattei 4y agoI don’t understand, there’s no news here. It’s a comment from a third party speculating over what causes the crash.
- EddySchauHai 4y agoIt seems pretty reproducible. I can’t use copilot but if anyone can reproduce it here that’d be cool. Anyhow, assuming this is reproducible and they do have filters to stop certain words giving predictions it leads that they’re trying to avoid the racist Twitter AI incident happening to them. I find that pretty funny :)
- thakoppno 4y agoit’s an intriguing guess that is at least plausible and hits a bunch of zeitgeist levers too.
- alephxyz 4y agoYeah I call BS. The "word filter" answer was selected as the valid answer by a third party (not OP).That's what the OP replied to another comment : > Heargo 24 days ago > Thanks, I'll try as soon as I get the problem again (somehow it's not bugged anymore...). Looks like it was just a temporary issue with no evidence that's it's due to a word filter.
- moyix 4y agoFYI, there is in fact a bad word filter in GitHub Copilot. When it was first released, the list was stored client-side in obfuscated form and I had a lot of fun decoding it: https://twitter.com/moyix/status/1433254293352730628 https://twitter.com/moyix/status/1433254293352730628 The Register wrote about it too: https://www.theregister.com/2021/09/02/github_copilot_banned_words_cracked/ https://www.theregister.com/2021/09/02/github_copilot_banned... They have since moved the bad word list server-side to prevent people from figuring out what's on it, but it's still there. This is easy to verify, just ask it to complete something that would include a banned word; my favorite here is "Israel", and it will just sit there and refuse to complete, either via inline suggestions or in the sidebar view that gives you 10 choices: https://i.imgur.com/O97YwKc.png https://i.imgur.com/O97YwKc.png This was what I managed to decode of the list (in ROT13 form to prevent accidental offense): https://moyix.net/~moyix/copilot_slurs_rot13.txt https://moyix.net/~moyix/copilot_slurs_rot13.txt No doubt they've added and removed some things since then.
- fuarno-eeple 4y ago
- stolen_biscuit 4y agoCan we get a source for that? Because at the moment, it's just a comment made by a person on the internet with nothing backing it up...
- bobsmooth 4y agoThe code's right there. Anyone want to try it out?
- deleted 4y ago[deleted]
- diego 4y agoI just tried Copilot with VS Code and python for the first time. If I define a function with some parameter name, I get suggestions as I type the body. I change the parameter name to gender, no suggestions. I change one letter in the parameter name (gendes, gander), I get suggestions again. There clearly is some code that gets activated when it sees the word "gender".
- sergiomattei 4y agoIt’s interesting how unsubstantiated allegations are getting so much attention, especially on a site with such high quality discussion.
- arikr 4y agoMuch more likely: the upvotes are because many people have frustration at this type of thing, and limited places to channel their frustration, so when they see a post like this, they upvote it to express their frustration. Or maybe that’s just me..
- sergiomattei 4y agoWhat type of thing? Asking for genuine curiosity.
- nomilk 4y ago> Copilot crash because the word “gender” A metaphor for our times.
- magicalist 4y ago>> Copilot crash because the word “gender” > A metaphor for our times. Social media amplifies an innocuous, extremely low stakes occurrence into a heated discussion because it happened to misstate the facts (nothing is crashing here) and focus on a hot button keyword ("gender" is only one of many blocked words)?
- tom_ 4y agoI worked on a video game in the late 2000s, and one of the bits of code I did was the code for filling the seats in the stadium with people. One of the artists cobbled together like 5 low poly man models and 5 low poly woman models, and you could just about tell the difference, and I put some code in there to ensure the genders were evenly distributed. (The 2 genders, I mean. Man, and woman.) Looking back, I don't even know why I made it an enum, rather than a 1-bit bitfield called is_woman - but in the end I was glad I didn't, because the art director moaned a bit about the clothing colour distribution, and somebody asked if we could have some mascots, and there were some complaints about the unreasonable number of interesting hats. And, so, long story short, by the time we were done, we had 18 genders based on clothing colour and type of hat, 2 genders for mascot (naturally: tall, and squat), and a table to control the relative distributions. Once we got to 5 genders I tried to change the enum name to Type - but we had this data-driven reflection system that integrated with various parts of the art pipeline, and once your enum had a name, that was pretty much that. You were stuck with it. Is that a metaphor for our times too? I don't know. My own view is that sometimes stuff just happens, and you can't read too much into it.
- onionisafruit 4y agoI would love to think msft blocks gender because your code somehow made it into the training data and somebody was confused seeing “squat” as a gender.
- MengerSponge 4y ago
- ttpphd 4y agoCrash the cistem!
- thakoppno 4y agogsender might work here
- joe_the_user 4y agoSo large language model are great on but have undesirable result occasionally. Hand coded scripts are added to remove the undesirable outcomes but still produce other problems - crashed but less often. More and more things are going to be filtered through large language model apps and the possibilities for cascading failures will be even more interesting than what exists presently.
- muglug 4y agoThe large language models already know too much. I was able to get GPT-3 to spit out reasonably accurate biographies for a couple of composers I know. GPT-3 could go even further — one of my composer friends has a reasonably rare first name, and when given the prompt "There once was a man named $first_name", GPT-3 responded with a number of limericks tailored to his particular set of skills.
- filoeleven 4y agoThere once was a man named $first_name, Who never accepted the blame. He went on a bender, And talked about gender [INFO] [default] [2022-07-10T07:59:07.641Z] [fetchCompletions] engine https://copilot-proxy.githubusercontent.com/v1/engines/copilot-codex
- nonethewiser 4y agoThat simply restates what people are taking issue with.
- antiquark 4y ago
- sn0w_crash 4y agoTheQuartering needs to hear this
- nonethewiser 4y agoWent woke, code broke.
- djbusby 4y agoThis thread needs a call to Rule 14: do not feed trolls. The bugs apparent trigger word is close to hot-button poli-sci issue. Can we please focus on the Technology.
- CoastalCoder 4y ago> The bugs apparent trigger word is close to hot-button poli-sci issue. Can we please focus on the Technology. I totally agree that this story has a high risk of flamewars. But it definitely has heavy Technology component, too.
- krapp 4y ago
- nonethewiser 4y agoNot sure what you mean. The tech is caving to politics. People dont like it.
- scarface74 4y agoI belong to a local Atlanta Slack channel - tech404 - that for the longest had an official bot that would always respond with the waving hand emoji (HN doesn’t support emojis) if you ever said the word “guys”. Even in private channels.
- LAC-Tech 4y agoThe funniest one of these was the python IRC channel, which had (has?) a policy of not allowing the word "lol". I'm pretty sure a bot would swoop in and say something like "NO LOL" which ironically only encourage more LOL.
- int_19h 4y agoAre there some specific Unicode ranges that HN filters out? I recall being able to use other alphabets and various special symbols with no issue.
- subjectsigma 4y agoI wouldn't be entirely surprised if something like this was intentional, or that they intentionally filtered the word "gender" and an unintentional side effect was the program crashing. You literally can't make any statements about gender, no matter how benign, without pissing at least a few of your users off.
- nonethewiser 4y agoThe problem is giving a shit about such users.
- duxup 4y agoHelp me out here, is the answer the official answer?
- anothermoron 4y agoThe answer was selected by Dave Cheney from Github https://github.com/davecheney https://github.com/davecheney. You can see it in the original link to the discussion: Answer selected by davecheney
- TheSpiciestDev 4y agoWhat was that bot that MSFT stood up on Twitter that trolls and memers fed to turn alt-right? I know they eventually took it down and that it stirred up a lot of controversy. I would not be surprised if someone found some Copilot output stemming from "gender" and reported to MSFT/GitHub for them to simply short circuit or "break" after finding certain keywords.
- Thorentis 4y agoYeah they probably found something like: assert gender in ["male", "female"]. If this is enough to trigger a backlash then maybe we deserve whatever fate has in store for us.
- nonethewiser 4y agoBut "we" and "the backlashers" are not one group.
- nevster 4y agoTay AI
- npteljes 4y agoThis was it: https://en.wikipedia.org/wiki/Tay_(bot) https://en.wikipedia.org/wiki/Tay_(bot)
- Thorentis 4y ago
- koshergweilo 4y agoMan so many black, gay, transgender senators and congresspeople. Oh wait...
- nonethewiser 4y agoYou can criticize congress.
- koshergweilo 4y agoYou can criticize anyone. I wasn't making a comment on the criticizing part, I was commenting on the ~power~ ruling over you part
- eyelidlessness 4y agoHow is this relevant? Or, who can you not criticize because of this?
- nonethewiser 4y ago
- pseudalopex 4y agoI see lots of criticism of trans people.
- eyelidlessness 4y agoOkay, you have permission to criticize me. Granted I’m just one non-binary shmoe and not a representative of the trans community at all, but I give you 100% authorization to criticize me. Get all your jollies out, I won’t even object.
- 4y ago
- eric4smith 4y agoBesides the absurdity of the code crashing because of the word "gender". My problem and curiosity with all of this is... "What was going on in the head of the person writing the parser?" I mean, were they thinking that if someone is writing code, let's say, for a gender dropdown and it was only ["male", "female"], it would try to suggest to us to add 26 more genders instead (and worse, suggest a list of genders to add)? Would the intention be to correct us and popup a message saying "We suggest you add more genders so as not to displease the users of your product"?? What was going on in that person's head who is trying to do all of this? What was their thought process? What were they trying to accomplish around gender? Was it the programmer, or some product manager that insisted on some kind of "copilot adjustment" for this because of a personal political viewpoint or just for GitHub being more woke? That's the most troubling aspect to this. I hope to Jesus Christ it was just a mistake.
- Thorentis 4y agoMaybe the one saving grace in all this, is that the AI singularity will never happen thanks to wokeness.
- c3534l 4y agoPerhaps it was not "do I think this is reasonable," but "is acting in good faith enough to keep me out of trouble."
- ronsor 4y agoRegardless of what Copilot suggested for "gender", it would've offended someone, and I think that's what Microsoft wants to avoid. Not even woke so much as it is trying to avoid potential controversies.
- eric4smith 4y agoYou would think they would just avoid adjusting this, right?
- mlyle 4y ago
- Thorentis 4y agoWhat is Github worried about? That Copilot might suggest some code that checks for a "gender" variable being only one of two values? Utterly absurd that we've now reached this point. I already had plenty of reasons to boycott Copilot, now I have another one.
- mcphage 4y ago> What is Github worried about? That Copilot might suggest some code that checks for a "gender" variable being only one of two values? Perhaps Github is worried about a backlash if it suggests code that allows for more than 2 values.
- creato 4y agoOr the backlash if it suggests code that only allows for two values.
- mcphage 4y agoThat's what Thorentis was suggesting in the first place. Judging by the threads here, I'd wager the backlash would be much stronger if it suggested more than 2.
- jfoster 4y agoThe backlash they ought to be worried about is the one from their customers when it refuses to operate due to an ongoing battle between opposing groups of extremists.
- moyix 4y agoYep, I noticed this last year when they still stored the list client-side and had great fun reverse engineering it: https://twitter.com/moyix/status/1433254293352730628 https://twitter.com/moyix/status/1433254293352730628
- throwaway290 4y agoInteresting, so it might not be the specific token "gender" but rather blocked words ("man" or "woman") that appear in suggestions will suppress Copilot. And presumably another token that like "communist" might do the same...
- moyix 4y agoIt suppresses output for bad words in both the prompt (your code) and the suggestions.
- tgsovlerkhgsel 4y agoThe list (https://moyix.net/~moyix/copilot_slurs_rot13.txt https://moyix.net/~moyix/copilot_slurs_rot13.txt, rot13 encoded and linked from this tweet https://twitter.com/moyix/status/1433479083376140296 https://twitter.com/moyix/status/1433479083376140296) indeed contains the word "gender".
- avian 4y agoThey fixed Copilot returning verbatim snippet of Quake source code by just blacklisting a word! How can they still pretend Copilot is not just copyright washing other people's code? https://twitter.com/moyix/status/1433261377125326851 https://twitter.com/moyix/status/1433261377125326851
- Ycombigatorz 4y ago
- LAC-Tech 4y agoAren't we missing the forest for the trees here? We're zeroing in on how silly is it for copilot to trigger its content filter on the word "gender". To me the real issue is that copilot has a content filter in the first place. It's unwelcome and unnecessary.
- camdenlock 4y agoThere’s a zealous push by a small but extremely vocal fringe to impose their very particular worldview onto emerging AI/ML models like this. They refer to it as “eliminating bias”, but it’s really just an attempt to mold these new technologies into conformance with one very specific set of ideological commitments. Proponents view it as some kind of obvious universal good, and are confused when anyone else is appalled by the blind foolishness of it all.
- Traubenfuchs 4y agoWhat a horrible time for ML/AI to explode. This is completely unlike the time of usenet and internet wild west. It‘s a whole new world of technology, castrated from the start.
- mlyle 4y ago> They refer to it as “eliminating bias”, I don't think, e.g. being able to handle black faces correctly is some sort of massive ideological commitment. So let's not pretend that the entire concern of bias in AI is irrelevant, no matter where you stand on gender. > conformance with one very specific set of ideological commitments You know-- let's just talk about basic respect and dignity: if someone strongly wants to be referred to in a particular way, the polite response is to respect their wishes. If there's a lot of people in this category, it makes sense for your system to address it. If you instead build your system in a way that you don't achieve this, you're being rude. If you use old training data and refer to people as a "Mongoloid" as a result-- don't be surprised that people are offended. Ditto, if you use old training data about gender that doesn't match many peoples' current expectations.
- 4y ago
- neonsunset 4y agoCommenters making bad-faith arguments in this discussion are the reason we can’t have nice things.
- the_doctah 4y agoKind of like making vague blanket statements with no examples.
- nonethewiser 4y agoSuch as?
- davesque 4y agoHas it been somehow confirmed that this was the cause of the issue or was it just that one guy's speculation? I don't see anything that confirmed this as the cause. Am I missing something in the linked content?
- btbuildem 4y agoThat's silly. So can I put "gender" as the first line in my code to stop copilot from ingesting it altogether? Are there any other break-words? Master, slave, Carlin's seven words, etc?
- tgsovlerkhgsel 4y agoAn earlier version of the list that someone found (see https://news.ycombinator.com/context?id=32339001 https://news.ycombinator.com/context?id=32339001 for links) does contain "gender", "slavery" and "master race" but not "master" and "slave" itself, ironically.
- nonethewiser 4y agoIronically, ignoring the actual usages of "master race" only cements its negative meaning. 95% of its modern usage is to claim PC elitism. It could be neutered if we let it.
- akomtu 4y ago#!gender
- rgoulter 4y ago> So can I put "gender" as the first line in my code to stop copilot from ingesting it altogether This means one solution for those worried about copilot laundering around code licenses is to put a statement like "for more details check the man page" at the end of each docstring.
- hda2 4y agoYesterday's timely announcement about an open source competitor to copilot that doesn't suffer from this absurdity: https://news.ycombinator.com/item?id=32327711 https://news.ycombinator.com/item?id=32327711
- jcuenod 4y agoI encountered this some time ago because I was working with grammatical gender. Unlike many of these comments, though, I do not take exception to it. Bias in ML is well established, and it's okay if, when we don't have solutions, we just disable it. If your autocomplete was capable of spitting out suggestions that made you feel isolated or kept poking you in the eye about aspects of your identity, you might feel a bit better about the creators having thought about that and taken steps to avoid it happening.
- nonethewiser 4y ago
- Banana699 4y ago"Reducing Bias" is a really strange way to put it, considering that bias usually means delibeaterly ignoring or contradicting aspects of reality/data (the classic example in ML textbooks is fitting a straight line to non-linear data), which is what Copilot is quite literally doing here. Gender is, in actual material fact, binary, and extremely strongly correlated with sex. Building a crimestop into an ML model is just teaching the machine human biases and delusions.
- fny 4y agoSo I've just tested it, and I can confirm, yes, copilot refuses to give suggestions related to gender. Now I know a lot of people are calling this absurd, but looking more closely, there are two PR nightmare scenarios. 1. Copilot makes a suggestion that implies gender is binary, a certain community explodes with anger and an entire news hype cycle starts about how Microsoft is enforcing views on gender with code. 2. Copilot makes a suggestion that implies gender is nonbinary, a certain community explodes with anger and an entire news hype cycle starts... You can't win... so why not plea the fifth? To all those claiming this is an example of "wokeism", remember the proper response from an individual who believes in nonbinary gender would be to offer suggestions of the sort. There is no advocacy here. Mums the word.
- philipswood 4y agoChoosing door 3 unfortunately leads to ... A certain community explodes with anger since their machine learning dev-tooling is closed and has arbitrary restrictions. If you try to please everybody, someone won't like it.
- woojoo666 4y agoUnfortunately the people that care (like HN people) are less likely to spend time organizing protests and riling up an internet mob
- xupybd 4y agoI'd argue it's fortunate. While it does seem like a good idea to get out there and promote your sides view of thing, I suspect the best option is to excel in your life and rise to influence. My hope is that the people here, at least the level headed ones, will rise to positions of influence and not the people rioting at every chance.
- xupybd 4y agoThere is a safe version of gender. Grammatical gender is, for now, binary and as far as I'm aware not offensive to most. But I agree you can't avoid offending people. The world is nuts everything is offensive to someone.
- politician 4y agoThere’s no reason to be surprised that elements within GitHub have an agenda. They’ve been clear about it since changing support for git’s master branch to main and then gaslighting the portion of community that doesn’t use the terminal about it. Now I’ve got Gen-Z developers that are confused and upset when `git init` does what it’s always done. GitHub, Microsoft ownership notwithstanding, was always going to inject its employees’ politics into Copilot.
- deleted 4y ago[deleted]
- Tree1993 4y agoSomeone changed the title from Copilot crash because the word “gender” to Part of my code makes Copilot crash
- dang 4y agoI changed it because of HN's rule on titles: "Please use the original title, unless it is misleading or linkbait; don't editorialize." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- Tree1993 4y agoSorry, I didn't notice the guidelines. Thank you.
- gloosx 4y agoSo does it filter out "sex" too?
- tzekid 4y agoCopilot's too useful for me to "boycot" right now, so the only alternative is using slang for the blacklisted words ... Anyone have any good recommendations for Copilot alternatives?
- potatototoo99 4y agoAmericans.
- tpoacher 4y agoBug as feature. My code from now on will be protected against copilot by looking like this: function genderPrintResult (GenderBool) if GenderBool: print "Yes" else: print "No" GenderMyVar = rand(10); GenderThreshhold = 5; genderPrintResult( GenderMyVar > GenderThreshold)
- leetrout 4y agoThis is in the FAQ: Does GitHub Copilot produce offensive outputs? GitHub Copilot includes filters to block offensive language in the prompts and to avoid synthesizing suggestions in sensitive contexts. We continue to work on improving the filter system to more intelligently detect and remove offensive outputs. However, due to the novel space of code safety, GitHub Copilot may sometimes produce undesired output. If you see offensive outputs, please report them directly to copilot-safety@github.com so that we can improve our safeguards. GitHub takes this challenge very seriously and we are committed to addressing it.
- wseqyrku 4y agoIt's baffling how the majority of commenters think this is about fighting discrimination.
- flippinburgers 4y agoNow try the word "mother".