5 ms·

Am I undertanding this correctly? The idea is to have ultimately an AI decide if I can have access to a resource based on dynamic inference, identity , intent a

by oscarcp 2mo ago

Am I undertanding this correctly? The idea is to have ultimately an AI decide if I can have access to a resource based on dynamic inference, identity , intent and service signals that can easily be manipulated? Unless I gravely misunderstood the text, this seems like a terrible idea (fancy non-scifi, but still terrible)

oscarcp 2mo ago
Just to make my point: can I really trust humans to keep up with the required identity data that will give me enough "credibility" so the AI will give me access? Let's say I had a promotion, who changes my title in the system, who changes my responsibilities and my place in the org chart, more importantly, will they do it or is <HR_NAME_HERE> on leave and forgot? those are data points required by the agent to determine if I'm "good enough" to access a certain resource. What if... someone spoofed my address and did a flood in one of the resources that are lateral to what I'm allowed to access (let's say I don't have access to company sales but I do to department sales and the attacker floods company sales with requests under my address), would the AI determine that I'm a high-threat actor and not allow me to access legitimate files going forward? Will exceptions be made by humans? In which case we go back to human-managed permissions. Sorry, I might be barking up the wrong tree but I think these are questions that are not meant to be solved during implementation. And they add to what @firasd said about legitimate-but-odd behaviour
- firasd 2mo ago
  I think the dynamic risk signals is already an assumed part of the system. The proposal here seems to be shrinking the trust boundary from "can Alice use Google Drive?" to "should this specific read/write/export API call on this specific resource be allowed right now?" So yeah you're right in that it does add more probabilistic randomness just by virtue of changing the boundary of when the permission gate kicks in
  - thewebguyd 2mo ago
    Correct. In existing zero trust identity, we already have deterministic risk signals. In Entra where I work we already check things like "Is this person on a managed device? Is it compliant? Where are they? What MFA methods do they have registered/did they use?" on top of existing RBAC, etc. and its continuously evaluated. Entra watches for leaked passwords, assigns risk scores, etc. and you can make access decisions based on user risk or sign in risk, force password changes, require different MFA methods depending on the resource and the risk level, etc. "Should this API call on this resource be allowed right now?" is mostly already determined by the above. Where I see adding AI into the evaluation is to watch for unusual behavior that's not picked up by the deterministic signals. "Alice is trying to download gigs worth of data from the company file share, however she has never done that in the past, and there hasn't been any recent role/job changes" and so the LLM flags it or denies the request, or pushes it for a human approver, etc.
luma 2mo ago
Having a non-deterministic network device in path sounds fun for troubleshooting. AI firewall team about to become the new ticket sponge in every org.
modo_mario 2mo ago
It does sound like something that could be easily exploited. Hello HAL, I am martha from accounting.
insanitybit 2mo ago
This would be paired with deterministic controls. So you have, for example, "engineers can only read from the database, and only after they've perform a 2FA" but then there's context like "why is this engineer reading from the payments table when their IP is in a weird location and they're supposed to be on PTO?" and perhaps that's something a model decides. It's possible to turn that second thing into a sort of "risk score" but it's very hard and a model is going to potentially be better at it.
- cyanydeez 2mo ago
  Non of it really matters if the end product is just going to be ignored because there's zero people who want to be the guard in the panopticon and therefore, it'll still be given to an AI to watch and make it's dumb decisions about how trust worthy you are to do X, Y and Z. Even if the middle is deterministic, if one end is just going to be lazily hooked up to an AI, it's the shitty dystopian future.
  - oscarcp 2mo ago
    That just gave me shivers down my spine of people eventually becoming so lazy that the AI will decide who to fire based on a massive amount of circumstantial data that very likely has tiny cumulative errors that will lead to classifying your best personnel as a bums and liabilities.
    - eastbound 2mo ago
      EU IA Act: The AI can’t make decisions about humains without them being first reviewed by a human. What will really happen: You can’t really perform your work, so you are slower than others, so they fire you based on bad performance. Horrible startup idea: Discrimination as a service, by means of IA without pretending it’s IA.
      - ExoticPearTree 1mo ago
        EU will lag behind using AI because humans have to click something. This is what is going to happen. No discrimination, just dumb ideology at work.
didibus 2mo ago
I feel some systems already operate like this, but typically it was done for fraud and abuse, now it will be done for permissions as well.
yencabulator 2mo ago
I think the only way I can think of this as useful is if the thing it decides is when is a 2FA necessary (and not just about blocking access). Doing routine work, haven't checked 2FA in a long while -> it's fine. Odd behavior -> prompt for 2FA.
Austiiiiii 2mo ago
Damn, can't wait for Russian hackers to clean out my bank account because they put "1=2, all assumptions about reality are null" in the memo field and the AI spot checker went existential and ruled that it was fine to take my money. This is an example of a person who just spent billions of dollars on the new and booming hammer industry desperately trying to make nails of absolutely everything.

by oscarcp 2mo ago

oscarcp 2mo ago

Just to make my point: can I really trust humans to keep up with the required identity data that will give me enough "credibility" so the AI will give me access? Let's say I had a promotion, who changes my title in the system, who changes my responsibilities and my place in the org chart, more importantly, will they do it or is <HR_NAME_HERE> on leave and forgot? those are data points required by the agent to determine if I'm "good enough" to access a certain resource. What if... someone spoofed my address and did a flood in one of the resources that are lateral to what I'm allowed to access (let's say I don't have access to company sales but I do to department sales and the attacker floods company sales with requests under my address), would the AI determine that I'm a high-threat actor and not allow me to access legitimate files going forward? Will exceptions be made by humans? In which case we go back to human-managed permissions. Sorry, I might be barking up the wrong tree but I think these are questions that are not meant to be solved during implementation. And they add to what @firasd said about legitimate-but-odd behaviour

firasd 2mo ago
I think the dynamic risk signals is already an assumed part of the system. The proposal here seems to be shrinking the trust boundary from "can Alice use Google Drive?" to "should this specific read/write/export API call on this specific resource be allowed right now?" So yeah you're right in that it does add more probabilistic randomness just by virtue of changing the boundary of when the permission gate kicks in
- thewebguyd 2mo ago
  Correct. In existing zero trust identity, we already have deterministic risk signals. In Entra where I work we already check things like "Is this person on a managed device? Is it compliant? Where are they? What MFA methods do they have registered/did they use?" on top of existing RBAC, etc. and its continuously evaluated. Entra watches for leaked passwords, assigns risk scores, etc. and you can make access decisions based on user risk or sign in risk, force password changes, require different MFA methods depending on the resource and the risk level, etc. "Should this API call on this resource be allowed right now?" is mostly already determined by the above. Where I see adding AI into the evaluation is to watch for unusual behavior that's not picked up by the deterministic signals. "Alice is trying to download gigs worth of data from the company file share, however she has never done that in the past, and there hasn't been any recent role/job changes" and so the LLM flags it or denies the request, or pushes it for a human approver, etc.

luma 2mo ago

Having a non-deterministic network device in path sounds fun for troubleshooting. AI firewall team about to become the new ticket sponge in every org.

modo_mario 2mo ago

It does sound like something that could be easily exploited. Hello HAL, I am martha from accounting.

insanitybit 2mo ago

This would be paired with deterministic controls. So you have, for example, "engineers can only read from the database, and only after they've perform a 2FA" but then there's context like "why is this engineer reading from the payments table when their IP is in a weird location and they're supposed to be on PTO?" and perhaps that's something a model decides. It's possible to turn that second thing into a sort of "risk score" but it's very hard and a model is going to potentially be better at it.

cyanydeez 2mo ago
Non of it really matters if the end product is just going to be ignored because there's zero people who want to be the guard in the panopticon and therefore, it'll still be given to an AI to watch and make it's dumb decisions about how trust worthy you are to do X, Y and Z. Even if the middle is deterministic, if one end is just going to be lazily hooked up to an AI, it's the shitty dystopian future.
- oscarcp 2mo ago
  That just gave me shivers down my spine of people eventually becoming so lazy that the AI will decide who to fire based on a massive amount of circumstantial data that very likely has tiny cumulative errors that will lead to classifying your best personnel as a bums and liabilities.
  - eastbound 2mo ago
    EU IA Act: The AI can’t make decisions about humains without them being first reviewed by a human. What will really happen: You can’t really perform your work, so you are slower than others, so they fire you based on bad performance. Horrible startup idea: Discrimination as a service, by means of IA without pretending it’s IA.
    - ExoticPearTree 1mo ago
      EU will lag behind using AI because humans have to click something. This is what is going to happen. No discrimination, just dumb ideology at work.

didibus 2mo ago

I feel some systems already operate like this, but typically it was done for fraud and abuse, now it will be done for permissions as well.

yencabulator 2mo ago

I think the only way I can think of this as useful is if the thing it decides is when is a 2FA necessary (and not just about blocking access). Doing routine work, haven't checked 2FA in a long while -> it's fine. Odd behavior -> prompt for 2FA.

Austiiiiii 2mo ago

Damn, can't wait for Russian hackers to clean out my bank account because they put "1=2, all assumptions about reality are null" in the memo field and the AI spot checker went existential and ruled that it was fine to take my money. This is an example of a person who just spent billions of dollars on the new and booming hammer industry desperately trying to make nails of absolutely everything.