5 ms·

Callum here, I was the developer that first discovered and reported the litellm vulnerability on Tuesday. I’m sharing the transcript of what it was like figuri

by Fibonar 6mo ago

Callum here, I was the developer that first discovered and reported the litellm vulnerability on Tuesday. I’m sharing the transcript of what it was like figuring out what was going on in real time, unedited with only minor redactions. I didn’t need to recount my thought process after the fact. It’s the very same ones I wrote down to help Claude figure out what was happening. I’m an ML engineer by trade, so having Claude walk me through exactly who to contact and a step by step guide of time-critical actions felt like a game-changer for non-security researchers. I'm curious whether the security community thinks more non-specialists finding and reporting vulnerabilities like this is a net positive or a headache?

Bullhorn9268 6mo ago
Not a security researcher, but this is IMHO obviously positive that the other side of the arms race is also getting stronger, and I would argue it's stronger than on the bad guys' side, due to the best being somewhat responsible and adding guardrails. I like the presentation <3.
rgambee 6mo ago
I've heard stories lately of open source projects being inundated with vulnerability reports and PRs. But in this case, it seems like AI assistance was clearly a boon for root-causing and reporting this so quickly.
- latexr 6mo ago
  Yes, in this case. Chances are in the future others will be missed despite being reported, because the maintainers are either inundated or tired of all the other reports and can’t distinguish the real one.
dot_treo 6mo ago
Looks like we discovered it at essentially the same time, and in essentially the same way. If the pth file didn't trigger a fork-bomb like behavior, this might have stayed undiscoverd for quite a bit longer. Good thinking on asking Claude to walk you through on who to contact. I had no idea how to contact anyone related to PyPI, so I started by shooting an email to the maintainers and posting it on Hacker News. While I'm not part of the security community, I think everyone who finds something like this, should be able to report it. There is no point in gatekeeping the reporting of serious security vulnerabilities.
- deleted 6mo ago
  [deleted]
- notatallshaw 6mo ago
  > I had no idea how to contact anyone related to PyPI https://pypi.org/security/ https://pypi.org/security/: > If you've identified a security issue with a project hosted on PyPI Login to your PyPI account, then visit the project's page on PyPI. At the bottom of the sidebar, click Report project as malware.
  - 0cf8612b2e1e 6mo ago
    The existing account to report is an unfortunate obstacle. Presumably not a huge deal if you were auditing code for vulnerabilities, but still an annoyance.
    - notatallshaw 6mo ago
      The threat actor was sophisticated enough to spam GitHub issues with dozens of different accounts. I imagine they could completely overwhelm PyPI with unauthenticated reports.
- Fibonar 6mo ago
  The best part was that I didn't even mean to ask Claude who to contact! I was still in disbelief that I was one of the first people affected, so I asked for existing reports on the assumption that if it was real I definitely wasn't the first. The fork-bomb part still seems really weird to me. A pretty sophisticated payload, caught by missing a single `-S` flag in the subprocess call.
gbrindisi 6mo ago
thanks for raising the alarm and sharing this, very insightful (also beautifully presented!)
lq9AJ8yrfs 6mo ago
As a sometimes peripheral and sometimes primary program manager for vulnerability disclosure, for companies you nearly can't avoid, $0.02 follows. It's a signal vs noise thing. Most of the grief is caused by bottom feeders shoveling anything they can squint at and call a vulnerability and asking for money. Maybe once a month someone would run a free tool and blindly send snippets of the output promising the rest in exchange for payment. Or emailing the CFO and the General Counsel after being politely reminded to come back with high quality information, and then ignored until they do. Your report on the other hand was high quality. I read all the reports that came my way, and good ones were fast tracked for fixes. I'd fix or mitigate them immediately if I had a way to do so without stopping business, and I'd go to the CISO, CTO, and the corresponding engineering manager if it mattered enough for immediate response.
- thierrydamiba 6mo ago
  Isn’t this a great use of llms? Clone the repo in a sandbox and have the llm identify if the issues are real and the appropriate response based on severity level. Wouldn’t be perfect but would have caught something like this.
  - joatmon-snoo 6mo ago
    Humans + LLMs are really good at producing enough spam to overwhelm anything like this. There’s a reason curl bans LLM slop reports now.
  - lq9AJ8yrfs 6mo ago
    I don't think I've met an llm that is adversary resistant, and here are counterparties that are actively playing the field, to put it mildly. The bug bounty service providers did an adequate job of filtering out junk reports. There was a survivorship bias, some of the bogus ones that got through had an uncanny ability to twist words.
edf13 6mo ago
Good write up… I’ve found Claude in particular to be very good at this sort of thing. As for whether it’s a good thing, I’d say it’s a net positive - your own reporting of this probably saved a bigger issue! We wrote up the why/what happened on our blog twice… the second based on the LiteLLM issue: https://grith.ai/blog/litellm-compromised-trivy-attack-chain https://grith.ai/blog/litellm-compromised-trivy-attack-chain
zar1048576 6mo ago
Fantastic write-up and thanks for sharing! I'm sure we will continue to see more of these types of deep supply chain vulns. I think this is valuable for the security community. Remember that Cliff Stoll was an astrophysicist turned sysadmin for Lawrence Berkeley Labs who chased down a $0.75 accounting discrepancy to identify a foreign espionage operation.
barnas2 6mo ago
As someone who works in security, it's really neat that you were able to discover this with the help of Claude. That being said the "I just opened Cursor again which triggered the malicious package" message is a bit eye opening. Ideally the instant you suspected malware that machine should have been quarantined and your security personnel contacted.
- wisemanwillhear 6mo ago
  I get why you say this, but real life is messy and the "fog of war" makes situations far less obvious in the moment. The older I get the more I realize how much we need scrappy, can-do people who don't always follow the "rules". Knowing the "rules" and knowing that people follow the "rules" because "that's what your supposed to do" is itself an avenue for malicious actors to exploit.
  - barnas2 6mo ago
    Clear procedures are the entire point of incident response plans. You follow them because of the fact that your judgement can be compromised in the moment. They re-triggered the malware payload because they decided to just dive in and handle it on their own in the "fog of war". Which would have been avoided entirely if they'd been following the standard advice to quarantine the machine and contact security so that they can investigate properly, with the developer if necessary. Your final sentence is completely irrelevant. Blind rule adherence can be an avenue for exploit in certain scenarios, but this wasn't a case of a developer being tricked into following a bad rule. They didn't follow a real and very well justified standard practice. The takeaway is "wow, we got lucky, we should have security people to loop in for this next time" not your weird life philosophy about how rule followers are a problem.
d-cc 6mo ago
Great work guys, I'm glad you were able to catch this before it propagated further.
latexr 6mo ago
> I'm curious whether the security community thinks more non-specialists finding and reporting vulnerabilities like this is a net positive or a headache? cURL had to stop the bug bounty program because they were inundated by slop reports of vulnerabilities which don’t exist. https://github.com/curl/curl/pull/20312 https://github.com/curl/curl/pull/20312 It’s good that you found and reported something real, but that isn’t the norm. Also, from the article: > AI tooling has sped up not just the creation of malware but also the detection. That’s an awful tradeoff. Detection is not a fix.
philbitt 6mo ago
[dead]

by Fibonar 6mo ago

Bullhorn9268 6mo ago

Not a security researcher, but this is IMHO obviously positive that the other side of the arms race is also getting stronger, and I would argue it's stronger than on the bad guys' side, due to the best being somewhat responsible and adding guardrails. I like the presentation <3.

rgambee 6mo ago

I've heard stories lately of open source projects being inundated with vulnerability reports and PRs. But in this case, it seems like AI assistance was clearly a boon for root-causing and reporting this so quickly.

latexr 6mo ago
Yes, in this case. Chances are in the future others will be missed despite being reported, because the maintainers are either inundated or tired of all the other reports and can’t distinguish the real one.

dot_treo 6mo ago

Looks like we discovered it at essentially the same time, and in essentially the same way. If the pth file didn't trigger a fork-bomb like behavior, this might have stayed undiscoverd for quite a bit longer. Good thinking on asking Claude to walk you through on who to contact. I had no idea how to contact anyone related to PyPI, so I started by shooting an email to the maintainers and posting it on Hacker News. While I'm not part of the security community, I think everyone who finds something like this, should be able to report it. There is no point in gatekeeping the reporting of serious security vulnerabilities.

deleted 6mo ago
[deleted]
notatallshaw 6mo ago
> I had no idea how to contact anyone related to PyPI https://pypi.org/security/ https://pypi.org/security/: > If you've identified a security issue with a project hosted on PyPI Login to your PyPI account, then visit the project's page on PyPI. At the bottom of the sidebar, click Report project as malware.
- 0cf8612b2e1e 6mo ago
  The existing account to report is an unfortunate obstacle. Presumably not a huge deal if you were auditing code for vulnerabilities, but still an annoyance.
  - notatallshaw 6mo ago
    The threat actor was sophisticated enough to spam GitHub issues with dozens of different accounts. I imagine they could completely overwhelm PyPI with unauthenticated reports.
Fibonar 6mo ago
The best part was that I didn't even mean to ask Claude who to contact! I was still in disbelief that I was one of the first people affected, so I asked for existing reports on the assumption that if it was real I definitely wasn't the first. The fork-bomb part still seems really weird to me. A pretty sophisticated payload, caught by missing a single `-S` flag in the subprocess call.

gbrindisi 6mo ago

thanks for raising the alarm and sharing this, very insightful (also beautifully presented!)

lq9AJ8yrfs 6mo ago

As a sometimes peripheral and sometimes primary program manager for vulnerability disclosure, for companies you nearly can't avoid, $0.02 follows. It's a signal vs noise thing. Most of the grief is caused by bottom feeders shoveling anything they can squint at and call a vulnerability and asking for money. Maybe once a month someone would run a free tool and blindly send snippets of the output promising the rest in exchange for payment. Or emailing the CFO and the General Counsel after being politely reminded to come back with high quality information, and then ignored until they do. Your report on the other hand was high quality. I read all the reports that came my way, and good ones were fast tracked for fixes. I'd fix or mitigate them immediately if I had a way to do so without stopping business, and I'd go to the CISO, CTO, and the corresponding engineering manager if it mattered enough for immediate response.

thierrydamiba 6mo ago
Isn’t this a great use of llms? Clone the repo in a sandbox and have the llm identify if the issues are real and the appropriate response based on severity level. Wouldn’t be perfect but would have caught something like this.
- joatmon-snoo 6mo ago
  Humans + LLMs are really good at producing enough spam to overwhelm anything like this. There’s a reason curl bans LLM slop reports now.
- lq9AJ8yrfs 6mo ago
  I don't think I've met an llm that is adversary resistant, and here are counterparties that are actively playing the field, to put it mildly. The bug bounty service providers did an adequate job of filtering out junk reports. There was a survivorship bias, some of the bogus ones that got through had an uncanny ability to twist words.

edf13 6mo ago

Good write up… I’ve found Claude in particular to be very good at this sort of thing. As for whether it’s a good thing, I’d say it’s a net positive - your own reporting of this probably saved a bigger issue! We wrote up the why/what happened on our blog twice… the second based on the LiteLLM issue: https://grith.ai/blog/litellm-compromised-trivy-attack-chain https://grith.ai/blog/litellm-compromised-trivy-attack-chain

zar1048576 6mo ago

Fantastic write-up and thanks for sharing! I'm sure we will continue to see more of these types of deep supply chain vulns. I think this is valuable for the security community. Remember that Cliff Stoll was an astrophysicist turned sysadmin for Lawrence Berkeley Labs who chased down a $0.75 accounting discrepancy to identify a foreign espionage operation.

barnas2 6mo ago

As someone who works in security, it's really neat that you were able to discover this with the help of Claude. That being said the "I just opened Cursor again which triggered the malicious package" message is a bit eye opening. Ideally the instant you suspected malware that machine should have been quarantined and your security personnel contacted.

wisemanwillhear 6mo ago
I get why you say this, but real life is messy and the "fog of war" makes situations far less obvious in the moment. The older I get the more I realize how much we need scrappy, can-do people who don't always follow the "rules". Knowing the "rules" and knowing that people follow the "rules" because "that's what your supposed to do" is itself an avenue for malicious actors to exploit.
- barnas2 6mo ago
  Clear procedures are the entire point of incident response plans. You follow them because of the fact that your judgement can be compromised in the moment. They re-triggered the malware payload because they decided to just dive in and handle it on their own in the "fog of war". Which would have been avoided entirely if they'd been following the standard advice to quarantine the machine and contact security so that they can investigate properly, with the developer if necessary. Your final sentence is completely irrelevant. Blind rule adherence can be an avenue for exploit in certain scenarios, but this wasn't a case of a developer being tricked into following a bad rule. They didn't follow a real and very well justified standard practice. The takeaway is "wow, we got lucky, we should have security people to loop in for this next time" not your weird life philosophy about how rule followers are a problem.

d-cc 6mo ago

Great work guys, I'm glad you were able to catch this before it propagated further.

latexr 6mo ago

> I'm curious whether the security community thinks more non-specialists finding and reporting vulnerabilities like this is a net positive or a headache? cURL had to stop the bug bounty program because they were inundated by slop reports of vulnerabilities which don’t exist. https://github.com/curl/curl/pull/20312 https://github.com/curl/curl/pull/20312 It’s good that you found and reported something real, but that isn’t the norm. Also, from the article: > AI tooling has sped up not just the creation of malware but also the detection. That’s an awful tradeoff. Detection is not a fix.

philbitt 6mo ago

[dead]