5 ms·

> The biggest issue for me is that it increases the attack surface What's your threat model?

sangnoir 2y ago
> What's your threat model? Why do you ask, will you help with designing a mitigation plan? I'll humor you: It's a turnkey gadget for sniffing/exfiltrating the output of any open iTerm2 shell.
- dcow 2y ago
  Because you’re already using other software that has LLM integration. What specifically about this iterm2 impl makes the threat more real??
  - crtasm 2y ago
    How do you know what other software they are using?
    - dcow 2y ago
      Is someone not using GitHub these days? Or web search? Or macOS? Or Windows?
      - spartanatreyu 2y ago
        I use GitHub, I don't use its copilot. I use web search, I don't use LLM websites. I use MacOS, I don't use Siri. I use Windows, I don't use Cortana/Copilot. ------------------------------------------ I don't want LLMs to parrot back code from other projects without understanding what that code does and what my code does. I don't want it to parrot back irrelevant slop. And I especially don't want it to parrot: rm -rf $BUILDDIR/ && ./build-project.sh and just hallucinate the assumption that $BUILDDIR is already defined.
        dcow 2y ago
        But GitHub doesn't ship copilot as a separate binary. So the threat vector of “AI has no place in my VCS get it out it increases the surface area” is there. So it’s okay for github to have copilot but not iterm2 to have codesierge? Doesn't add up.
        spartanatreyu 2y ago
        Github isn't a binary, it's a repo host. Github can hallucinate whatever it wants, it's not going to brick my computer. A terminal on the other hand...
        dcow 2y ago
        The point here is about compliance. I agree it’d be stupid to pipe the output of an LLM to a terminal’s command line. But people are saying they can’t use iterm2 now because compliance says no AI and having an mdm-secure way to disable the functionality is not enough because _there could be a bug_ or something. Yet they’re checking commits, in presumably the same compliance regime, into other software with AI features.
        spartanatreyu 2y ago
        Github doesn't come with Copilot, even on the enterprise plan. You have to explicitly pay for it and add it to your repo.
  - sangnoir 2y ago
    > Because you’re already using other software that has LLM integration Oh really, which software would that be? And which other LLM-enabled software connects production environments or has access to auth credentials/tokens?

> What's your threat model? Why do you ask, will you help with designing a mitigation plan? I'll humor you: It's a turnkey gadget for sniffing/exfiltrating the output of any open iTerm2 shell.

dcow 2y ago
Because you’re already using other software that has LLM integration. What specifically about this iterm2 impl makes the threat more real??
- crtasm 2y ago
  How do you know what other software they are using?
  - dcow 2y ago
    Is someone not using GitHub these days? Or web search? Or macOS? Or Windows?
    - spartanatreyu 2y ago
      I use GitHub, I don't use its copilot. I use web search, I don't use LLM websites. I use MacOS, I don't use Siri. I use Windows, I don't use Cortana/Copilot. ------------------------------------------ I don't want LLMs to parrot back code from other projects without understanding what that code does and what my code does. I don't want it to parrot back irrelevant slop. And I especially don't want it to parrot: rm -rf $BUILDDIR/ && ./build-project.sh and just hallucinate the assumption that $BUILDDIR is already defined.
      - dcow 2y ago
        But GitHub doesn't ship copilot as a separate binary. So the threat vector of “AI has no place in my VCS get it out it increases the surface area” is there. So it’s okay for github to have copilot but not iterm2 to have codesierge? Doesn't add up.
        spartanatreyu 2y ago
        Github isn't a binary, it's a repo host. Github can hallucinate whatever it wants, it's not going to brick my computer. A terminal on the other hand...
        dcow 2y ago
        The point here is about compliance. I agree it’d be stupid to pipe the output of an LLM to a terminal’s command line. But people are saying they can’t use iterm2 now because compliance says no AI and having an mdm-secure way to disable the functionality is not enough because _there could be a bug_ or something. Yet they’re checking commits, in presumably the same compliance regime, into other software with AI features.
        spartanatreyu 2y ago
        Github doesn't come with Copilot, even on the enterprise plan. You have to explicitly pay for it and add it to your repo.
- sangnoir 2y ago
  > Because you’re already using other software that has LLM integration Oh really, which software would that be? And which other LLM-enabled software connects production environments or has access to auth credentials/tokens?