8 ms·

Azure's Weakest Link? How API Connections Spill Secrets

137 points by hland 2y ago

486sx33 2y ago
So this was vulnerable ? https://azure.microsoft.com/en-us/explore/global-infrastructure/government/dod/ https://azure.microsoft.com/en-us/explore/global-infrastruct...
- DoctorOW 2y ago
  Maybe with enough traction, they'll lose out on huge contracts because of stuff like this. Seems the only way to get stuff fixed is to attach dollars to it.
hland 2y ago
Binary Security found the undocumented APIs for Azure API Connections. In this post we examine the inner workings of the Connections allowing us to escalate privileges and read secrets in backend resources for services ranging from Key Vaults, Storage Blobs, Defender ATP, to Enterprise Jira and SalesForce servers.
stirlo 2y ago
That’s a scary vulnerability. There’s no mention of the bug bounty paid out for it but I hope it was substantial.
- Daedren 2y ago
  Well at the bottom of the article, they mention that Microsoft first closed the issue as invalid, and on the second attempt they closed it as "cannot be reproduced" (after fixing it). So from that I can imply there was no payment.
  - eitland 2y ago
    I've reported a trivial way to infer details about passwords in Windows. (Ctrl-arrow in password fields in Windows 8 jumped by character group even when hidden so if a prefilled password was 123 abc.de it would stop after 3, after space (I think), after c, after dot and finally after e.) All I got was an email: that is interesting bye bye. But it was fixed in the next patch or the next after I think. So I didn't care to report the two bigger problems I found with Azure Information Protection [1][2] I thought about reporting them but decided against it. And I will continue to tell people that I don't care to do free work for MS when they won't even give me a t-shirt, a mug or even acknowledge it. Maybe if one is a security researcher it can be worth it but if you just find something interesting you'll probably be better rewarded by reddit or HN, yes, the upvotes are worthless but less so than a dismissive email. [1] one in the downloadable AIP tooling where you can easily smuggle clear text information with rock solid plausible deniability - I found it by accident after having implemented a part of a pipeline in the most obvious way I could think of. [2]: the second had to do with how one can configure SharePoint to automatically protect files with AIP on download, the only problem being if you logged in using another login sequence (sorry for the lack of details, this was before the pandemic and it was just a small part of what I was working on at the time) SharePoint would conveniently forget all about it despite all efforts by me, the security admin at the company and the expert that Microsoft sent to fix it.
    - franktankbank 2y ago
      > the expert that Microsoft sent to fix it. Ha ... ha ... ha ... ha ... did they give you the run around for several months until you dropped the issue? It's actually pretty astounding that they don't get sued for this practice. If a company is paying for support and are given illiterate noobs then that is breach of contract I would think. I would never recommend entering a contract with MSFT, they produce trash products they can't support and are more invested in their Legal team than actual product.
      - RajT88 2y ago
        I thought the same when a friend of mine reported something to Apple. I would guess it's SOP at this point across big tech, unless something is too big to ignore.
        cryptonector 2y ago
        You might have no idea how expensive providing great support to customers is when you're an vendor like Apple or a Microsoft. It's like backports, which are even more unbelievably expensive still, and those are gone industry-wide for that reason. Think of the cost of opportunity in having smart, capable, experienced staff doing support or backports instead of actual dev work. (Especially backports, which when they were done frequently they were done precisely because customers are risk-averse, so a great deal more review and testing (with a much larger test matrix) was required for backports, with attendant huge increase in cost.) That cost is enormous. But of course they do need to provide some support, and at some point some really good support for the really serious bugs, and the vendor will in time do it, but first the customer demand and pressure has to build.
        dfxm12 2y ago
        I can't speak to Apple, but wrt Microsoft, you're not appreciating just how bad support is (or even the documentation is) and you're not appreciating how much people pay for support on top of the product. I feel like I know more about M365 than anyone I talk to at MS. That's bad.
- belter 2y ago
  It's a feature not a bug: "Azure’s Security Vulnerabilities Are Out of Control" - https://www.lastweekinaws.com/blog/azures_vulnerabilities_are_quack/ https://www.lastweekinaws.com/blog/azures_vulnerabilities_ar...
  - SideburnsOfDoom 2y ago
    At least this new one seems to have been fixed within two months: 6 Jan to Feb 20th.
  - eitland 2y ago
    > Let’s start with some empathy, because let’s face it: Nobody sets out to build something insecure except maybe a cryptocurrency exchange. :-)
    - belter 2y ago
      Nobody sets out to build something insecure but if they go with Azure.... "Microsoft confirms partial loss of security log data on multiple platforms" - https://www.cybersecuritydive.com/news/microsoft-loss-security-log-data/730285/ https://www.cybersecuritydive.com/news/microsoft-loss-securi... "Microsoft called out for ‘blatantly negligent’ cybersecurity practices" - https://www.theverge.com/2023/8/3/23819237/microsoft-azure-breach-blatantly-negligent-cybersecurity-practices https://www.theverge.com/2023/8/3/23819237/microsoft-azure-b...
- IcyWindows 2y ago
  The caller still needs at least the Reader role, so it was limited to accounts that were added to the Azure subscription as only Readers. I'm glad they fixed it, but this doesn't seem too scary??
  - bradford 2y ago
    Suppose user U has read access to Subscription S, but doesn't have access to keyvault K. If user U can gain access to keyvault K via this exploit, it is scary. [Vendors/Contingent staff will often be granted read-level access to a subscription under the assumption that they won't have access to secrets, for example.] (I'm open to the possibility that I'm misunderstanding the exploit)
    - hland 2y ago
      Your take is spot on, sir.
    - p_ing 2y ago
      My reading on this is that the Reader must have read access to the API Connection in order to drive the exploit [against a secure resource they lack appropriate access to]. But a user can have Reader rights on the Subscription which does cascade down to all objects, including API Connections.
      - dh2022 2y ago
        But also the API connection seems to have secret reader permissions as per screenshot in the article… Giving secret reader permission to another resource seems to be the weak link.
        p_ing 2y ago
        The API Connection in a Logic App contains a secret in order to read/write (depending on permission) a resource. Could be a Key Vault secret, Azure App Service, Exchange Online mailbox, SharePoint Online site..., etc. The secret typically is a user account (OAuth token), but it could also be an App Id/Secret.
SideburnsOfDoom 2y ago
>The Connector for Key Vaults is maybe the one with the highest impact. Yeah, no joke. Considering how well protected Azure Key Vaults typically are, and what's in them (secrets, certificates etc) this is huge way to compromise a lot of other things. It's finding the keys to the doors.
redrove 2y ago
At this point I have close to a decade of working with Azure and AWS/GCP and I can confidently say Azure is the worst when it comes to security, objectively. Performance, "I don't like the portal", service and capacity availability, and such complaints are somewhat subjective or fixable but I deeply believe Microsoft is the most insecure of the cloud giants on a measurable level. Anyone that is serious about security should just avoid Microsoft, this has honestly been the case since the early '00s at the least.
- Mossy9 2y ago
  As someone who is greatly motivated to moving off Azure (to onprem, not to another cloud), do you know of any good collection of Azure security issues I could use as 'ammunition'? Would be greatly appreciated! UPD: note to self - this seems like a good resource https://www.cloudvulndb.org/results https://www.cloudvulndb.org/results
  - redrove 2y ago
    I have some notes somewhere but unfortunately they don't have citations, these are just some of the vulns they've had in the last couple years: • Storm-0558 Breach (2023): Chinese hackers exploited a leaked signing key from a crash dump to access U.S. government emails, affecting 60,000+ State Department communications • Azure OpenAI Service Exploitation (2024): Hackers bypassed AI guardrails using stolen credentials to generate illicit content, leading to Microsoft lawsuits against developers in Iran, UK, and Vietnam • CVE-2025-21415 (CVSS 9.9): Spoofing vulnerability in Azure AI Face Service allowed authentication bypass and privilege escalation • CVE-2023-36052: Azure CLI logging flaw exposed plaintext credentials in CI/CD pipelines, risking sensitive data leakage • Azurescape (2022): Container escape vulnerability enabled cross-tenant access in Azure Container Instances, discovered by Palo Alto Networks • ChaosDB (2022): Wiz researchers exploited CosmosDB’s Jupyter Notebook integration to access thousands of customer databases including Fortune 500 companies • Executive Account Takeover Campaign (2024): Phishing campaign compromised 500+ executive accounts via Azure collaboration tools with MFA manipulation If your company or workplace is considering migrating from cloud to on-prem or from one cloud to another, I do this professionally btw, feel free to reach out at this temporary email and we can chat: pale.pearl2178 at fastmail.com (to prevent my real email being scraped from HN).
this_steve_j 2y ago
Another reason to be worried by Microsoft’s Azure security guidelines which state “Identity is the new perimeter”. Well, the perimeter is not a gate but a cattle guard, and I am not surprised to see some wolves eating a secret and a cow swaggering into the road. Azure service APIs have always conflated the principles of “reachability from the public internet” and “anonymous access” into a single concept called “Public Access” which, for Azure KV, has 6 different public/private configuration combinations! This vulnerability report did not include the Key Vault Networking settings for “Public network access”, so more testing (but not much more) is needed to see if the proxy side door can circumvent a resource ACL or private endpoint or both.
- cryptonector 2y ago
  It's not just "identity", but "authorization". Really, what they mean is "defense in depth" minus firewalls (because the "in depth" part makes those less relevant), I think. And... that is a reasonable position... provided you get the "in depth" part right, which includes not having proxies that bypass authorization.
guardiangod 2y ago
Oh a confuse delegate vulnerability. Azure is not the only cloud provider with that oversight, let me tell you.
godelski 2y ago
> The Inherent Insecurity of API Connections I'm no security expert, but this seems like a bad take. How are APIs any less secure than any other form of interacting with a program? Nothing here is really a problem with APIs but rather a problem with access control. > anyone with Reader permissions on the connection is allowed to arbitrarily call any endpoint on the connection This is not an API issue... It feels like saying we shouldn't allow users to search a database because they might run a SQL injection to drop all the tables. Searching tables isn't the problem, not sanitizing inputs is. This is more like giving all users on your network sudo access or just doing chmod -R 777 /. My concern here is that a lot of people have the takeaway that APIs shouldn't be exposed because they create security risks. But that's not true. The API exposure isn't the risk, it is the access control. If you don't have proper access control then it really isn't going to matter if you have an API or not. But then again, we have a long history of not taking fairly basic security seriously and with decades of computing and seeing the results, I really can't figure out why. Sure, security is expensive, but bad security is far more expensive. I guess maybe the issue is I'm not much of a gambler.
- twisteriffic 2y ago
  I think you misunderstood what was meant by "API connections". In azure, they're an entity that is created to represent connectivity to some external service, usually bundled with credentials and the OpenAPI definition of the downstream service. They let you consume an external service from other azure services without having to worry about things like token refresh. The article goes into better detail on this than I can in a comment.
  - godelski 2y ago
    I did read the article and I'm not sure why this isn't about access control > it is common to not mark input (and output) as sensitive. There's 2 solutions to this: 1) Fail open: default setting is that things are not marked as sensitive and an active decision has to be taken to mark sensitive 2) Fail closed: by default things are marked sensitive and action needs to be taken to mark it as non-sensitive Another way of seeing this is that 2 is the common paradigm of "least privileges." You give users, files, services, whatever the minimal privileges required. > What I would not expect is that anyone with Reader permissions on the connection is allowed to arbitrarily call any endpoint on the connection: To me this sounds like doing `chmod -R +r /`. Or as the author puts it > all Readers on that subscription can call all GET requests defined on the connection. This is certainly an access control issue. Even if the issue is that Azure doesn't allow for more fine grained access control, it is still access control. So that's what I'm not getting. It is about having the ability to do API calls, to do GET and POST commands, it is about tokens (accounts) having more privileges than they should. What am I missing here?

137 points by hland 2y ago

So this was vulnerable ? https://azure.microsoft.com/en-us/explore/global-infrastructure/government/dod/ https://azure.microsoft.com/en-us/explore/global-infrastruct...

DoctorOW 2y ago
Maybe with enough traction, they'll lose out on huge contracts because of stuff like this. Seems the only way to get stuff fixed is to attach dollars to it.

hland 2y ago

Binary Security found the undocumented APIs for Azure API Connections. In this post we examine the inner workings of the Connections allowing us to escalate privileges and read secrets in backend resources for services ranging from Key Vaults, Storage Blobs, Defender ATP, to Enterprise Jira and SalesForce servers.

stirlo 2y ago

That’s a scary vulnerability. There’s no mention of the bug bounty paid out for it but I hope it was substantial.

Daedren 2y ago
Well at the bottom of the article, they mention that Microsoft first closed the issue as invalid, and on the second attempt they closed it as "cannot be reproduced" (after fixing it). So from that I can imply there was no payment.
- eitland 2y ago
  I've reported a trivial way to infer details about passwords in Windows. (Ctrl-arrow in password fields in Windows 8 jumped by character group even when hidden so if a prefilled password was 123 abc.de it would stop after 3, after space (I think), after c, after dot and finally after e.) All I got was an email: that is interesting bye bye. But it was fixed in the next patch or the next after I think. So I didn't care to report the two bigger problems I found with Azure Information Protection [1][2] I thought about reporting them but decided against it. And I will continue to tell people that I don't care to do free work for MS when they won't even give me a t-shirt, a mug or even acknowledge it. Maybe if one is a security researcher it can be worth it but if you just find something interesting you'll probably be better rewarded by reddit or HN, yes, the upvotes are worthless but less so than a dismissive email. [1] one in the downloadable AIP tooling where you can easily smuggle clear text information with rock solid plausible deniability - I found it by accident after having implemented a part of a pipeline in the most obvious way I could think of. [2]: the second had to do with how one can configure SharePoint to automatically protect files with AIP on download, the only problem being if you logged in using another login sequence (sorry for the lack of details, this was before the pandemic and it was just a small part of what I was working on at the time) SharePoint would conveniently forget all about it despite all efforts by me, the security admin at the company and the expert that Microsoft sent to fix it.
  - franktankbank 2y ago
    > the expert that Microsoft sent to fix it. Ha ... ha ... ha ... ha ... did they give you the run around for several months until you dropped the issue? It's actually pretty astounding that they don't get sued for this practice. If a company is paying for support and are given illiterate noobs then that is breach of contract I would think. I would never recommend entering a contract with MSFT, they produce trash products they can't support and are more invested in their Legal team than actual product.
    - RajT88 2y ago
      I thought the same when a friend of mine reported something to Apple. I would guess it's SOP at this point across big tech, unless something is too big to ignore.
      - cryptonector 2y ago
        You might have no idea how expensive providing great support to customers is when you're an vendor like Apple or a Microsoft. It's like backports, which are even more unbelievably expensive still, and those are gone industry-wide for that reason. Think of the cost of opportunity in having smart, capable, experienced staff doing support or backports instead of actual dev work. (Especially backports, which when they were done frequently they were done precisely because customers are risk-averse, so a great deal more review and testing (with a much larger test matrix) was required for backports, with attendant huge increase in cost.) That cost is enormous. But of course they do need to provide some support, and at some point some really good support for the really serious bugs, and the vendor will in time do it, but first the customer demand and pressure has to build.
        dfxm12 2y ago
        I can't speak to Apple, but wrt Microsoft, you're not appreciating just how bad support is (or even the documentation is) and you're not appreciating how much people pay for support on top of the product. I feel like I know more about M365 than anyone I talk to at MS. That's bad.
belter 2y ago
It's a feature not a bug: "Azure’s Security Vulnerabilities Are Out of Control" - https://www.lastweekinaws.com/blog/azures_vulnerabilities_are_quack/ https://www.lastweekinaws.com/blog/azures_vulnerabilities_ar...
- SideburnsOfDoom 2y ago
  At least this new one seems to have been fixed within two months: 6 Jan to Feb 20th.
- eitland 2y ago
  > Let’s start with some empathy, because let’s face it: Nobody sets out to build something insecure except maybe a cryptocurrency exchange. :-)
  - belter 2y ago
    Nobody sets out to build something insecure but if they go with Azure.... "Microsoft confirms partial loss of security log data on multiple platforms" - https://www.cybersecuritydive.com/news/microsoft-loss-security-log-data/730285/ https://www.cybersecuritydive.com/news/microsoft-loss-securi... "Microsoft called out for ‘blatantly negligent’ cybersecurity practices" - https://www.theverge.com/2023/8/3/23819237/microsoft-azure-breach-blatantly-negligent-cybersecurity-practices https://www.theverge.com/2023/8/3/23819237/microsoft-azure-b...
IcyWindows 2y ago
The caller still needs at least the Reader role, so it was limited to accounts that were added to the Azure subscription as only Readers. I'm glad they fixed it, but this doesn't seem too scary??
- bradford 2y ago
  Suppose user U has read access to Subscription S, but doesn't have access to keyvault K. If user U can gain access to keyvault K via this exploit, it is scary. [Vendors/Contingent staff will often be granted read-level access to a subscription under the assumption that they won't have access to secrets, for example.] (I'm open to the possibility that I'm misunderstanding the exploit)
  - hland 2y ago
    Your take is spot on, sir.
  - p_ing 2y ago
    My reading on this is that the Reader must have read access to the API Connection in order to drive the exploit [against a secure resource they lack appropriate access to]. But a user can have Reader rights on the Subscription which does cascade down to all objects, including API Connections.
    - dh2022 2y ago
      But also the API connection seems to have secret reader permissions as per screenshot in the article… Giving secret reader permission to another resource seems to be the weak link.
      - p_ing 2y ago
        The API Connection in a Logic App contains a secret in order to read/write (depending on permission) a resource. Could be a Key Vault secret, Azure App Service, Exchange Online mailbox, SharePoint Online site..., etc. The secret typically is a user account (OAuth token), but it could also be an App Id/Secret.

SideburnsOfDoom 2y ago

>The Connector for Key Vaults is maybe the one with the highest impact. Yeah, no joke. Considering how well protected Azure Key Vaults typically are, and what's in them (secrets, certificates etc) this is huge way to compromise a lot of other things. It's finding the keys to the doors.

redrove 2y ago

At this point I have close to a decade of working with Azure and AWS/GCP and I can confidently say Azure is the worst when it comes to security, objectively. Performance, "I don't like the portal", service and capacity availability, and such complaints are somewhat subjective or fixable but I deeply believe Microsoft is the most insecure of the cloud giants on a measurable level. Anyone that is serious about security should just avoid Microsoft, this has honestly been the case since the early '00s at the least.

Mossy9 2y ago
As someone who is greatly motivated to moving off Azure (to onprem, not to another cloud), do you know of any good collection of Azure security issues I could use as 'ammunition'? Would be greatly appreciated! UPD: note to self - this seems like a good resource https://www.cloudvulndb.org/results https://www.cloudvulndb.org/results
- redrove 2y ago
  I have some notes somewhere but unfortunately they don't have citations, these are just some of the vulns they've had in the last couple years: • Storm-0558 Breach (2023): Chinese hackers exploited a leaked signing key from a crash dump to access U.S. government emails, affecting 60,000+ State Department communications • Azure OpenAI Service Exploitation (2024): Hackers bypassed AI guardrails using stolen credentials to generate illicit content, leading to Microsoft lawsuits against developers in Iran, UK, and Vietnam • CVE-2025-21415 (CVSS 9.9): Spoofing vulnerability in Azure AI Face Service allowed authentication bypass and privilege escalation • CVE-2023-36052: Azure CLI logging flaw exposed plaintext credentials in CI/CD pipelines, risking sensitive data leakage • Azurescape (2022): Container escape vulnerability enabled cross-tenant access in Azure Container Instances, discovered by Palo Alto Networks • ChaosDB (2022): Wiz researchers exploited CosmosDB’s Jupyter Notebook integration to access thousands of customer databases including Fortune 500 companies • Executive Account Takeover Campaign (2024): Phishing campaign compromised 500+ executive accounts via Azure collaboration tools with MFA manipulation If your company or workplace is considering migrating from cloud to on-prem or from one cloud to another, I do this professionally btw, feel free to reach out at this temporary email and we can chat: pale.pearl2178 at fastmail.com (to prevent my real email being scraped from HN).

this_steve_j 2y ago

Another reason to be worried by Microsoft’s Azure security guidelines which state “Identity is the new perimeter”. Well, the perimeter is not a gate but a cattle guard, and I am not surprised to see some wolves eating a secret and a cow swaggering into the road. Azure service APIs have always conflated the principles of “reachability from the public internet” and “anonymous access” into a single concept called “Public Access” which, for Azure KV, has 6 different public/private configuration combinations! This vulnerability report did not include the Key Vault Networking settings for “Public network access”, so more testing (but not much more) is needed to see if the proxy side door can circumvent a resource ACL or private endpoint or both.

cryptonector 2y ago
It's not just "identity", but "authorization". Really, what they mean is "defense in depth" minus firewalls (because the "in depth" part makes those less relevant), I think. And... that is a reasonable position... provided you get the "in depth" part right, which includes not having proxies that bypass authorization.

guardiangod 2y ago

Oh a confuse delegate vulnerability. Azure is not the only cloud provider with that oversight, let me tell you.

godelski 2y ago

> The Inherent Insecurity of API Connections I'm no security expert, but this seems like a bad take. How are APIs any less secure than any other form of interacting with a program? Nothing here is really a problem with APIs but rather a problem with access control. > anyone with Reader permissions on the connection is allowed to arbitrarily call any endpoint on the connection This is not an API issue... It feels like saying we shouldn't allow users to search a database because they might run a SQL injection to drop all the tables. Searching tables isn't the problem, not sanitizing inputs is. This is more like giving all users on your network sudo access or just doing chmod -R 777 /. My concern here is that a lot of people have the takeaway that APIs shouldn't be exposed because they create security risks. But that's not true. The API exposure isn't the risk, it is the access control. If you don't have proper access control then it really isn't going to matter if you have an API or not. But then again, we have a long history of not taking fairly basic security seriously and with decades of computing and seeing the results, I really can't figure out why. Sure, security is expensive, but bad security is far more expensive. I guess maybe the issue is I'm not much of a gambler.

twisteriffic 2y ago
I think you misunderstood what was meant by "API connections". In azure, they're an entity that is created to represent connectivity to some external service, usually bundled with credentials and the OpenAPI definition of the downstream service. They let you consume an external service from other azure services without having to worry about things like token refresh. The article goes into better detail on this than I can in a comment.
- godelski 2y ago
  I did read the article and I'm not sure why this isn't about access control > it is common to not mark input (and output) as sensitive. There's 2 solutions to this: 1) Fail open: default setting is that things are not marked as sensitive and an active decision has to be taken to mark sensitive 2) Fail closed: by default things are marked sensitive and action needs to be taken to mark it as non-sensitive Another way of seeing this is that 2 is the common paradigm of "least privileges." You give users, files, services, whatever the minimal privileges required. > What I would not expect is that anyone with Reader permissions on the connection is allowed to arbitrarily call any endpoint on the connection: To me this sounds like doing `chmod -R +r /`. Or as the author puts it > all Readers on that subscription can call all GET requests defined on the connection. This is certainly an access control issue. Even if the issue is that Azure doesn't allow for more fine grained access control, it is still access control. So that's what I'm not getting. It is about having the ability to do API calls, to do GET and POST commands, it is about tokens (accounts) having more privileges than they should. What am I missing here?