5 ms·

What amazes me is how predictable(?) all of the recent issues were. Don't get me wrong, the folks behind Copilot are clearly, without any doubt smart, creative

What amazes me is how predictable(?) all of the recent issues were. Don't get me wrong, the folks behind Copilot are clearly, without any doubt smart, creative, and capable. But then... None of these issues (reproducing licensed code ad verbatim, non-compiling code, getting semantics wrong, and now this) are 0.01% edge cases that take specialized knowledge to see or trigger. I remember some of them being called days ago in the initial HN thread by people who haven't had beta access. I really wonder how this announcement/rollout looked like on the management side of things. Because a) these shortcomings must have been known beforehand and b) backlash from people who feel threatened for their jobs/"stolen" of their open source work was (I guess) foreseeable? I've already read calls to abandon GitHub for competitors; this can hardly have been an acceptable outcome here. Nevertheless, Copilot is still one of the most innovative and interesting products I've seen in a while.

callamdelaney 5y ago
There are things that already do what Copilot does, eg Kite so it's hardly innovative imo.
- nicce 5y ago
  Kite does only fraction what Copilot is currently doing. It is great at suggesting function names and parameters, but it does not really suggest complete code or generate somewhat new code.
syshum 5y ago
I dont know if that true. Just because you are "smart, creative, and capable" does not mean you can predict every possible outcome or be incapable of missing the obvious I have been on both sides of that, where I have had to point out obvious flaws in an idea to very smart people, and have had clearly obvious flaw pointed out to me in one of my idea's... I think it is completely possible that some or even all of the issues co-pilot is facing were unknown at the time of release, even if they are obvious to some
- tyingq 5y ago
  Though they could have proofed the small number of handpicked examples on copilot.github.com to see that they compiled/didn't blow up on first run. Or one further, that they did what they were supposed to, in a somewhat reasonable way.
rattray 5y ago
We should keep in mind that the product is still in beta / technical preview.
- alkonaut 5y ago
  This is a good point. There is a lot of outrage now, but the product when finished might have every single wrinkle removed. This one, for example, seems it should be pretty easy to fix. You could even make a hack that replaces ALL sufficiently long and sufficiently random strings with garbage/zeroes, at the point of recall. The difference from the case of regurgitating GPL sources is that the information that it looks like an API key can be deducedd from the output of copilot, so you don't need to track it through the system like you would with a system of attribution.
  - skywhopper 5y ago
    How do you tell a “long and random string” from a base64 encoded PNG file or embedded script or…
    - rattray 5y ago
      Should any of those be autocompleted?
    - alkonaut 5y ago
      You don't. The logic is unchanged if the data changes. A snippet of code would be unchanged, apart from the data. // Add an arrow icon var arrow_icon = base64decode("00000000000000000..."); add_image(arrow_icon); That is: the prerequisite for this approach being viable is if one assumes that "code" and "data" are distinct, and that data can be seen as irrelevant placeholders. That is: in the example above I was after the code to add the icon, not the icon payload itself. There are obvious bordeline cases like large numeric constants that are actually core part of the logic. E.g. a method that multiplies by Pi with 14 digits wouldn't work very well if they were replaced by zeroes. So most likely numerical constants would need to be left alone.
kalium-xyz 5y ago
Is it more innovative than for example: tab9?
Iv 5y ago
That's the norm for a Microsoft product. Sell something full of holes, deal with it only when it starts posing an existential threat to the product
Olreich 5y ago
I’d be very surprised if management at the least didn’t have their heads in the sand about the potential failure mode. There are often “must deliver” dates at large companies because someone made a promise about a deadline and now heads will roll for missing it whether anyone actually cares or not. So long as middle management thinks the C suite is watching them, they are desperate to meet quota. Hilariously, this results in stuff like Copilot getting released to great big legal problems. Only then does the C suite actually notice the project and get upset that it is a legal nightmare for them. I think the real secret to winning in big tech is that your job is just to keep your head down and keep the money rolling in without causing headaches for higher ups. Increase sales, make customers happy enough to keep paying, maybe release a cool product. But more importantly, don’t cause a major outage, burn the PR team, or get caught up in a legal kerfuffle.
- rorykoehler 5y ago
  You make a good case for innovation through acquisitions rather than in-house development. Once the derisking aspect is factored in acquisitions suddenly look a lot more attractive.
rorykoehler 5y ago
I saw this type of thing coming a mile away and left GitHub as soon as they were bought by Microsoft. TBH even despite my inherent distrust of Microsoft this is way beyond the hypotheticals I had in mind when I deleted all content from my GitHub account. Now I’m worried about VSCode as another potential vulnerability vector. Has anyone done a recent independent audit of what is sent across the wire to Microsoft from VSCode?
- belter 5y ago
  Well they have Telemetry enabled by default so you should disable it: https://code.visualstudio.com/docs/getstarted/telemetry#:~:text=enableTelemetry%20user%20setting%20to%20false,from%20VS%20Code%20going%20forward https://code.visualstudio.com/docs/getstarted/telemetry#:~:t.... Maybe something else still goes over the wire...
visarga 5y ago
Yes, it looks like unfinished work. They could have: - implemented plagiarism detection to attribute code to its source (where possible), then present the result together with the link. This makes Copilot same with Googling your answer and then copy-pasting the code. You are fully responsible - implemented some regexes to filter out secrets, or even better, change the secrets to random values in the training data - implemented a robots.txt like system so people have a method to ban the Copilot spider from their code If they did these things before release it would have been so much better. But they are simple fixes so I see no technical obstacle.

callamdelaney 5y ago

There are things that already do what Copilot does, eg Kite so it's hardly innovative imo.

nicce 5y ago
Kite does only fraction what Copilot is currently doing. It is great at suggesting function names and parameters, but it does not really suggest complete code or generate somewhat new code.

syshum 5y ago

I dont know if that true. Just because you are "smart, creative, and capable" does not mean you can predict every possible outcome or be incapable of missing the obvious I have been on both sides of that, where I have had to point out obvious flaws in an idea to very smart people, and have had clearly obvious flaw pointed out to me in one of my idea's... I think it is completely possible that some or even all of the issues co-pilot is facing were unknown at the time of release, even if they are obvious to some

tyingq 5y ago
Though they could have proofed the small number of handpicked examples on copilot.github.com to see that they compiled/didn't blow up on first run. Or one further, that they did what they were supposed to, in a somewhat reasonable way.

rattray 5y ago

We should keep in mind that the product is still in beta / technical preview.

alkonaut 5y ago
This is a good point. There is a lot of outrage now, but the product when finished might have every single wrinkle removed. This one, for example, seems it should be pretty easy to fix. You could even make a hack that replaces ALL sufficiently long and sufficiently random strings with garbage/zeroes, at the point of recall. The difference from the case of regurgitating GPL sources is that the information that it looks like an API key can be deducedd from the output of copilot, so you don't need to track it through the system like you would with a system of attribution.
- skywhopper 5y ago
  How do you tell a “long and random string” from a base64 encoded PNG file or embedded script or…
  - rattray 5y ago
    Should any of those be autocompleted?
  - alkonaut 5y ago
    You don't. The logic is unchanged if the data changes. A snippet of code would be unchanged, apart from the data. // Add an arrow icon var arrow_icon = base64decode("00000000000000000..."); add_image(arrow_icon); That is: the prerequisite for this approach being viable is if one assumes that "code" and "data" are distinct, and that data can be seen as irrelevant placeholders. That is: in the example above I was after the code to add the icon, not the icon payload itself. There are obvious bordeline cases like large numeric constants that are actually core part of the logic. E.g. a method that multiplies by Pi with 14 digits wouldn't work very well if they were replaced by zeroes. So most likely numerical constants would need to be left alone.

kalium-xyz 5y ago

Is it more innovative than for example: tab9?

Iv 5y ago

That's the norm for a Microsoft product. Sell something full of holes, deal with it only when it starts posing an existential threat to the product

Olreich 5y ago

I’d be very surprised if management at the least didn’t have their heads in the sand about the potential failure mode. There are often “must deliver” dates at large companies because someone made a promise about a deadline and now heads will roll for missing it whether anyone actually cares or not. So long as middle management thinks the C suite is watching them, they are desperate to meet quota. Hilariously, this results in stuff like Copilot getting released to great big legal problems. Only then does the C suite actually notice the project and get upset that it is a legal nightmare for them. I think the real secret to winning in big tech is that your job is just to keep your head down and keep the money rolling in without causing headaches for higher ups. Increase sales, make customers happy enough to keep paying, maybe release a cool product. But more importantly, don’t cause a major outage, burn the PR team, or get caught up in a legal kerfuffle.

rorykoehler 5y ago
You make a good case for innovation through acquisitions rather than in-house development. Once the derisking aspect is factored in acquisitions suddenly look a lot more attractive.

rorykoehler 5y ago

I saw this type of thing coming a mile away and left GitHub as soon as they were bought by Microsoft. TBH even despite my inherent distrust of Microsoft this is way beyond the hypotheticals I had in mind when I deleted all content from my GitHub account. Now I’m worried about VSCode as another potential vulnerability vector. Has anyone done a recent independent audit of what is sent across the wire to Microsoft from VSCode?

belter 5y ago
Well they have Telemetry enabled by default so you should disable it: https://code.visualstudio.com/docs/getstarted/telemetry#:~:text=enableTelemetry%20user%20setting%20to%20false,from%20VS%20Code%20going%20forward https://code.visualstudio.com/docs/getstarted/telemetry#:~:t.... Maybe something else still goes over the wire...

visarga 5y ago

Yes, it looks like unfinished work. They could have: - implemented plagiarism detection to attribute code to its source (where possible), then present the result together with the link. This makes Copilot same with Googling your answer and then copy-pasting the code. You are fully responsible - implemented some regexes to filter out secrets, or even better, change the secrets to random values in the training data - implemented a robots.txt like system so people have a method to ban the Copilot spider from their code If they did these things before release it would have been so much better. But they are simple fixes so I see no technical obstacle.