23 ms·

Iamlive – Generate an IAM policy from AWS client-side monitoring

81 points by aynawn 6y ago

wincy 6y ago
I can’t really judge the usefulness of this tool as I’m not that familiar with IAM, but I’d be wary of using this in a work situation because I would be really worried I’d accidentally search for “imlive”, which is an adult cam site. Maybe I’m just being paranoid but it feels like if I unintentionally named my super useful and totally innocuous project “poorNHub”, I’d want someone to point out how that might cause problems down the line for adoption.
- krab 6y ago
  Fair point. But what would happen? Even if you accidentally opened that site, most probably nobody would have noticed. Or it would have been just a short awkward moment.
  - random5634 6y ago
    Not in me too time. One good solution - turn on adult content blocking on firewall. Saved me once or twice if you typo, missearch etc. I'm usually like huh when it blocks? Then oh...
benkoller 6y ago
This is a great approach to determine permission especially for service accounts - thanks for giving me another tool for my toolchain.
joshuanapoli 6y ago
Iamlive sounds useful for someone who has something running with wide open permissions and wants to get started with narrowing down the permissions. It wastes a lot of time to discover necessary permissions one-by-one through permission-denied errors. Tag-based policies (attribute based access control) really simplifies IAM policy management. These are more reusable, so there is permission discovery isn’t needed as often.
- gingerlime 6y ago
  could you expand on that? my familiarity is very limited with IAM (as fun as getting teeth pulled), and this sounds interesting.
  - uzakov 6y ago
    I can try to explain it in a simplified fashion: When provisioning entities (EC2 instances for example) you go through certain settings (for this example I am talking about a manual provisioning through the UI, not IaC, CLI etc) and one of the things you can do is give EC2 instance a tag. There are a number of reasons to do tagging, to name a few: To make it easier to find a particular instance, cost optimisation, see who owns it. After this you write a policy that says what user/entity can do. This policy can take advantage of the tag, to say: "Only user, who has a tag XYZ can stop this EC2 instance that has tag XYZ" Let me know if that does not make sense and I can go a level simpler
- coredog64 6y ago
  In theory, ABAC solves a lot of problems. In practice, there’s no mandate for AWS product teams to support ABAC in a consistent fashion. Assuming it doesn’t cause networking issues, IMO the best approach is to finely scope AWS accounts. They’re free, and Amazon includes some reasonable tools for working with them. The major downside is the double charge on Transit Gateway traffic.
- flurie 6y ago
  ABAC is ideally much more than just scoping small sets of policy to tags. I think the problem of trying to shoehorn ABAC into AWS's IAM system, even with something implementing proper ABAC on top of it, is that you run the risk of hitting limits on role policy attachments (hard limit of 20!). There is probably a way to juggle user function across different roles, though then there's a risk of exhausting roles or still not being able to shoehorn a minimally viable policy for some function based on fine-grained attributes.
jcims 6y ago
I’d never heard of the Client Side Monitoring tooling that this appears to be using. Is anyone in here using it to any effect?
- cddotdotslash 6y ago
  It's a "hidden" (in that it's not documented very well or advertised much by AWS) feature of the AWS SDKs. I think it was likely added for debugging purposes originally, but security tools have started to hook into it as a more local and compete version of the kinds of logs you'd see in CloudTrail. I've used it mostly for debugging and temporary monitoring of service calls. https://summitroute.com/blog/2020/05/25/client_side_monitoring/ https://summitroute.com/blog/2020/05/25/client_side_monitori...
- tyingq 6y ago
  I'm curious if the tooling could be made to work with a "default deny" policy, that is, is there enough info to generate the IAM policies when you get "permission denied"?
johnnyAghands 6y ago
Wow so just yesterday I was trying to figure out more restricted policy set for an instance someone else created. This is great!!
zackmorris 6y ago
I've dabbled in dev ops in recent months and found that setting up AWS services is relatively straightforward. Although for anyone reading this, do yourself a favor right now and learn to automate the manual steps with something like Terraform or you'll most likely never get anywhere in any reasonable amount of time. I found IAM to be an anti-pattern. I grew up on the Mac in a sandboxed, single-user environment with no console. And I've always approached roles and permissions as something that should be backend-only. So for web hosting, it should be host-only and not exposed to the customer. So to me, a service like VPC should have created a sandbox that handles IAM internally. It should have provided the customer with all AWS services exposed (even oddballs like SES) in that sandbox only (not accessible from outside) similarly to how managed hosting worked a decade ago. Basically IAM feels like hand waving to me. Few insights can be gleaned by looking at the roles and permissions by eye. So I would argue that most AIM setups as they stand now are probably insecure. The way it should probably work is, the customer would set up a series of access tests that run similarly to Postman and exercise the infrastructure the way that something like Dredd does with Swagger or API Blueprint. They would "prove" the permissions ruleset by traversing the AWS infrastructure, telling you which ones to add/remove. So conceptually, I think that the article or something similar is how IAM should have worked in the first place. Thought I'd use this opportunity to ask any dev ops people about this, because it feels like I'm missing something fundamental here. Any approaches to formally proving that IAM roles and permissions are secure would be greatly appreciated!
- uzakov 6y ago
  Hey! So to touch on a couple of points: Scott Piper is probably one of the most known AWS sec experts, check out his websites. https://summitroute.com/blog/ https://summitroute.com/blog/ "Any approaches to formally proving that IAM roles and permissions are secure would be greatly appreciated! " Are you looking for a way to see that you have set up your policies correctly?
  - zackmorris 6y ago
    Ya after writing it, I think what's going on is that I do everything declaratively now. So it's great that AWS lets us create the AWS policy files rather than making us manage permissions through code. But I want to see the infrastructure as a big spreadsheet, turn on a policy, and see the accessible services highlight. basically I need something like an acceptance/integration test that proves that my permissions work like I think they do. Without something formal like that, I can't help but feel that IAM is risky to rely on alone. To the point that, I would be wary of using it, and even view it as more of a liability than a useful tool. This is really a general conceptual issue with the abstraction of roles/permissions from a computer science perspective. My gut feeling is probably that the general authentication-based logins of the open internet and circles of trust are better mechanisms for securing services. That said, I do really appreciate that we can reference security group ids in new security group rules. Once I started chaining the references like that, and having subnet rules in a central place, things worked more smoothly. I would never use roles/permissions for new development though is what I'm trying to say (whether for web development or web hosting). I don't think it makes sense to screen access by which network was used, or the source IP of the user trying to gain access. These are somewhat antiquated notions that make sense for sysadmins, but I don't think they make sense for the vast majority of use cases that web developers encounter. We could just use the open web's CLIENT_ID=abc, CLIENT_SECRET=xyz pattern for all AWS services. But maybe there's some advantage with roles and permissions that I'm not seeing. Sorry I come off as Negative Nancy, but this issue really concerns me, and I don't see much talk about it on sites like Stack Overflow.
f430 6y ago
would it be possible for this to be hooked up to existing serverless frameworks? it would be great if something could scan the code and generate an IAM policy.
arduinomancer 6y ago
I know there’s other projects with the goal to generate policies from the live behaviour of apps but I’ve always wondered how practical that actually is. It seems difficult to make sure you actually exercise every path of your app so that it makes every API call. It makes me wonder if live monitoring or static analysis is the better approach.
ttul 6y ago
This is a analogous to the creation of a chroot jail (or, in more modern times a container) by observing a running app via pstrace and the like and noting which files it accesses. It’s a really powerful technique that can save a great deal of time for the programmer while ensuring the attack surface is as small as it can possibly be. Back in the dark ages, I wrote a tool to create chroot jails in OpenBSD so as to minimize the size of a tar ball that would need to be distributed to run an app in an embedded system. Those days, flash storage was tiny and expensive, so not only was the jail technique more secure, it also saved money.
bndw 6y ago
Will this work with Terraform? I'd love to find a way to generate an IAM policy based on Terraform manifests.
orf 6y ago
I think IAM is really, really cool. I’ve always wondered: is IAM a service under the hood? Surely not - the volume of requests would be insane and the single point of failure pretty risky. So is it a spec with some kind of shared data plane to retrieve policies? If so how would they be evaluated consistently across different languages? Producing a shared library that does this in all contexts/languages sounds improbable. But then the policies are clearly compiled to some kind of bytecode, so a common implementation would make sense. But then again it is something that would definitely benefit from centralisation.
- yandie 6y ago
  ex-AWS here. IAM is definitely a service! It's super powerful underneath the hood. However as a caller, you need multiple levels of caching underneath the hood (all built by IAM team). That's part of the reasons why you can't use IAM as an external service. IAM is also eventual consistency - that's why if you modify the policy doc it takes seconds to reflect in your service calls :) (sometimes even minutes). IAM outages can and will take down multiple AWS services. > If so how would they be evaluated consistently across different languages? Internally everything is JVM AFAIK :). If you go out of the JVM world in AWS you'll have to deal with this problem yourself, which can be painful.
  - orf 6y ago
    Thanks! That’s super interesting. There’s something really cool about building something as flexible and (presumably) fast as IAM. The caching layer must also be quite complex! But thanks for settling this for me.
DSingularity 6y ago
Google cloud got IAM right. I know of no approach to AWS IAM other than start with granting all perms and then begin to reduce privilege until things break. AWS is untenable.
- musingsole 6y ago
  I've always worked in the AWS world and have only dabbled in google cloud, but hot rocks on a sunday evening if this clear page isn't refreshing coming from AWS: https://cloud.google.com/iam/docs/permissions-reference https://cloud.google.com/iam/docs/permissions-reference
Dunedan 6y ago
I have the need to run client-side monitoring for AWS Lambda functions, but haven't managed to figure out yet what's the easiest option to do so. While I could point AWS_CSM_HOST to some external IP-address, the lack of encryption and authentication makes that pretty nonviable. Alternatively I could put the AWS Lambda functions into VPC, but I'd prefer a less invasive change to the AWS Lambda functions. Does somebody here have a better solution for that?
heybrandons 6y ago
thank you for sharing this!
WatchDog 6y ago
It's mentioned in the readme, but CSM doesn't provide any information as to the resources being accessed, just the service and action, so the best this tool can do is generate policy with wildcard resources. Not ideal, but I guess it's better than running your system with a wide open policy.
rad_gruchalski 6y ago
This can be done with more visibility using CloudWatch: https://docs.aws.amazon.com/apigateway/latest/developerguide/monitoring-cloudwatch.html https://docs.aws.amazon.com/apigateway/latest/developerguide.... It’s not limited to API gateway or EC2.

81 points by aynawn 6y ago

I can’t really judge the usefulness of this tool as I’m not that familiar with IAM, but I’d be wary of using this in a work situation because I would be really worried I’d accidentally search for “imlive”, which is an adult cam site. Maybe I’m just being paranoid but it feels like if I unintentionally named my super useful and totally innocuous project “poorNHub”, I’d want someone to point out how that might cause problems down the line for adoption.

krab 6y ago
Fair point. But what would happen? Even if you accidentally opened that site, most probably nobody would have noticed. Or it would have been just a short awkward moment.
- random5634 6y ago
  Not in me too time. One good solution - turn on adult content blocking on firewall. Saved me once or twice if you typo, missearch etc. I'm usually like huh when it blocks? Then oh...

benkoller 6y ago

This is a great approach to determine permission especially for service accounts - thanks for giving me another tool for my toolchain.

joshuanapoli 6y ago

Iamlive sounds useful for someone who has something running with wide open permissions and wants to get started with narrowing down the permissions. It wastes a lot of time to discover necessary permissions one-by-one through permission-denied errors. Tag-based policies (attribute based access control) really simplifies IAM policy management. These are more reusable, so there is permission discovery isn’t needed as often.

gingerlime 6y ago
could you expand on that? my familiarity is very limited with IAM (as fun as getting teeth pulled), and this sounds interesting.
- uzakov 6y ago
  I can try to explain it in a simplified fashion: When provisioning entities (EC2 instances for example) you go through certain settings (for this example I am talking about a manual provisioning through the UI, not IaC, CLI etc) and one of the things you can do is give EC2 instance a tag. There are a number of reasons to do tagging, to name a few: To make it easier to find a particular instance, cost optimisation, see who owns it. After this you write a policy that says what user/entity can do. This policy can take advantage of the tag, to say: "Only user, who has a tag XYZ can stop this EC2 instance that has tag XYZ" Let me know if that does not make sense and I can go a level simpler
coredog64 6y ago
In theory, ABAC solves a lot of problems. In practice, there’s no mandate for AWS product teams to support ABAC in a consistent fashion. Assuming it doesn’t cause networking issues, IMO the best approach is to finely scope AWS accounts. They’re free, and Amazon includes some reasonable tools for working with them. The major downside is the double charge on Transit Gateway traffic.
flurie 6y ago
ABAC is ideally much more than just scoping small sets of policy to tags. I think the problem of trying to shoehorn ABAC into AWS's IAM system, even with something implementing proper ABAC on top of it, is that you run the risk of hitting limits on role policy attachments (hard limit of 20!). There is probably a way to juggle user function across different roles, though then there's a risk of exhausting roles or still not being able to shoehorn a minimally viable policy for some function based on fine-grained attributes.

jcims 6y ago

I’d never heard of the Client Side Monitoring tooling that this appears to be using. Is anyone in here using it to any effect?

cddotdotslash 6y ago
It's a "hidden" (in that it's not documented very well or advertised much by AWS) feature of the AWS SDKs. I think it was likely added for debugging purposes originally, but security tools have started to hook into it as a more local and compete version of the kinds of logs you'd see in CloudTrail. I've used it mostly for debugging and temporary monitoring of service calls. https://summitroute.com/blog/2020/05/25/client_side_monitoring/ https://summitroute.com/blog/2020/05/25/client_side_monitori...
tyingq 6y ago
I'm curious if the tooling could be made to work with a "default deny" policy, that is, is there enough info to generate the IAM policies when you get "permission denied"?

johnnyAghands 6y ago

Wow so just yesterday I was trying to figure out more restricted policy set for an instance someone else created. This is great!!

zackmorris 6y ago

I've dabbled in dev ops in recent months and found that setting up AWS services is relatively straightforward. Although for anyone reading this, do yourself a favor right now and learn to automate the manual steps with something like Terraform or you'll most likely never get anywhere in any reasonable amount of time. I found IAM to be an anti-pattern. I grew up on the Mac in a sandboxed, single-user environment with no console. And I've always approached roles and permissions as something that should be backend-only. So for web hosting, it should be host-only and not exposed to the customer. So to me, a service like VPC should have created a sandbox that handles IAM internally. It should have provided the customer with all AWS services exposed (even oddballs like SES) in that sandbox only (not accessible from outside) similarly to how managed hosting worked a decade ago. Basically IAM feels like hand waving to me. Few insights can be gleaned by looking at the roles and permissions by eye. So I would argue that most AIM setups as they stand now are probably insecure. The way it should probably work is, the customer would set up a series of access tests that run similarly to Postman and exercise the infrastructure the way that something like Dredd does with Swagger or API Blueprint. They would "prove" the permissions ruleset by traversing the AWS infrastructure, telling you which ones to add/remove. So conceptually, I think that the article or something similar is how IAM should have worked in the first place. Thought I'd use this opportunity to ask any dev ops people about this, because it feels like I'm missing something fundamental here. Any approaches to formally proving that IAM roles and permissions are secure would be greatly appreciated!

uzakov 6y ago
Hey! So to touch on a couple of points: Scott Piper is probably one of the most known AWS sec experts, check out his websites. https://summitroute.com/blog/ https://summitroute.com/blog/ "Any approaches to formally proving that IAM roles and permissions are secure would be greatly appreciated! " Are you looking for a way to see that you have set up your policies correctly?
- zackmorris 6y ago
  Ya after writing it, I think what's going on is that I do everything declaratively now. So it's great that AWS lets us create the AWS policy files rather than making us manage permissions through code. But I want to see the infrastructure as a big spreadsheet, turn on a policy, and see the accessible services highlight. basically I need something like an acceptance/integration test that proves that my permissions work like I think they do. Without something formal like that, I can't help but feel that IAM is risky to rely on alone. To the point that, I would be wary of using it, and even view it as more of a liability than a useful tool. This is really a general conceptual issue with the abstraction of roles/permissions from a computer science perspective. My gut feeling is probably that the general authentication-based logins of the open internet and circles of trust are better mechanisms for securing services. That said, I do really appreciate that we can reference security group ids in new security group rules. Once I started chaining the references like that, and having subnet rules in a central place, things worked more smoothly. I would never use roles/permissions for new development though is what I'm trying to say (whether for web development or web hosting). I don't think it makes sense to screen access by which network was used, or the source IP of the user trying to gain access. These are somewhat antiquated notions that make sense for sysadmins, but I don't think they make sense for the vast majority of use cases that web developers encounter. We could just use the open web's CLIENT_ID=abc, CLIENT_SECRET=xyz pattern for all AWS services. But maybe there's some advantage with roles and permissions that I'm not seeing. Sorry I come off as Negative Nancy, but this issue really concerns me, and I don't see much talk about it on sites like Stack Overflow.

f430 6y ago

would it be possible for this to be hooked up to existing serverless frameworks? it would be great if something could scan the code and generate an IAM policy.

arduinomancer 6y ago

I know there’s other projects with the goal to generate policies from the live behaviour of apps but I’ve always wondered how practical that actually is. It seems difficult to make sure you actually exercise every path of your app so that it makes every API call. It makes me wonder if live monitoring or static analysis is the better approach.

ttul 6y ago

This is a analogous to the creation of a chroot jail (or, in more modern times a container) by observing a running app via pstrace and the like and noting which files it accesses. It’s a really powerful technique that can save a great deal of time for the programmer while ensuring the attack surface is as small as it can possibly be. Back in the dark ages, I wrote a tool to create chroot jails in OpenBSD so as to minimize the size of a tar ball that would need to be distributed to run an app in an embedded system. Those days, flash storage was tiny and expensive, so not only was the jail technique more secure, it also saved money.

bndw 6y ago

Will this work with Terraform? I'd love to find a way to generate an IAM policy based on Terraform manifests.

orf 6y ago

I think IAM is really, really cool. I’ve always wondered: is IAM a service under the hood? Surely not - the volume of requests would be insane and the single point of failure pretty risky. So is it a spec with some kind of shared data plane to retrieve policies? If so how would they be evaluated consistently across different languages? Producing a shared library that does this in all contexts/languages sounds improbable. But then the policies are clearly compiled to some kind of bytecode, so a common implementation would make sense. But then again it is something that would definitely benefit from centralisation.

yandie 6y ago
ex-AWS here. IAM is definitely a service! It's super powerful underneath the hood. However as a caller, you need multiple levels of caching underneath the hood (all built by IAM team). That's part of the reasons why you can't use IAM as an external service. IAM is also eventual consistency - that's why if you modify the policy doc it takes seconds to reflect in your service calls :) (sometimes even minutes). IAM outages can and will take down multiple AWS services. > If so how would they be evaluated consistently across different languages? Internally everything is JVM AFAIK :). If you go out of the JVM world in AWS you'll have to deal with this problem yourself, which can be painful.
- orf 6y ago
  Thanks! That’s super interesting. There’s something really cool about building something as flexible and (presumably) fast as IAM. The caching layer must also be quite complex! But thanks for settling this for me.

DSingularity 6y ago

Google cloud got IAM right. I know of no approach to AWS IAM other than start with granting all perms and then begin to reduce privilege until things break. AWS is untenable.

musingsole 6y ago
I've always worked in the AWS world and have only dabbled in google cloud, but hot rocks on a sunday evening if this clear page isn't refreshing coming from AWS: https://cloud.google.com/iam/docs/permissions-reference https://cloud.google.com/iam/docs/permissions-reference

Dunedan 6y ago

I have the need to run client-side monitoring for AWS Lambda functions, but haven't managed to figure out yet what's the easiest option to do so. While I could point AWS_CSM_HOST to some external IP-address, the lack of encryption and authentication makes that pretty nonviable. Alternatively I could put the AWS Lambda functions into VPC, but I'd prefer a less invasive change to the AWS Lambda functions. Does somebody here have a better solution for that?

heybrandons 6y ago

thank you for sharing this!

WatchDog 6y ago

It's mentioned in the readme, but CSM doesn't provide any information as to the resources being accessed, just the service and action, so the best this tool can do is generate policy with wildcard resources. Not ideal, but I guess it's better than running your system with a wide open policy.

rad_gruchalski 6y ago

This can be done with more visibility using CloudWatch: https://docs.aws.amazon.com/apigateway/latest/developerguide/monitoring-cloudwatch.html https://docs.aws.amazon.com/apigateway/latest/developerguide.... It’s not limited to API gateway or EC2.