19 ms·
Zanzibar: Google’s Consistent, Global Authorization System (2019)
- 1f60c 5y ago(2019) (maybe?)
- kyrra 5y agoThis was talked about 2 years ago on here[0]. This service was also brought up in the discussion[1] of Ory Keto, as it's based on Zanzibar. [0] https://news.ycombinator.com/item?id=20132520 https://news.ycombinator.com/item?id=20132520 [1] https://news.ycombinator.com/item?id=26738344 https://news.ycombinator.com/item?id=26738344
- themarkers 5y agoAirbnb Himeji also: https://medium.com/airbnb-engineering/himeji-a-scalable-centralized-system-for-authorization-at-airbnb-341664924574 https://medium.com/airbnb-engineering/himeji-a-scalable-cent...
- etxm 5y agoCasbin is another that’s pretty interesting I’ve been evaluating alongside Ory’s https://casbin.org/ https://casbin.org/
- znpy 5y agoIs it my impression or nowadays the emerging technology in this sense is OPA (Open Policy Agent)? It looks like a flexible system to build cross-language and cross-framework authorization systems.
- etxm 5y agoI use OPA with terraform and kubernetes, but I’m looking for something for application ACLs, where I as a resource owner can assign permissions to arbitrary subjects for a resource. Does OPA support that? If so that would be very very cool.
- rad_gruchalski 5y agoI think Ory Keto would be a better choice because it's easier to manage individual resources on an ad-hoc basis.
- andyroid 5y agoCertainly! Application and microservice authorization is probably one of the more common use cases for OPA, and there's definitely benefits in having a unified policy engine in an organization or company.
- etxm 5y agoI have only found RBAC and ABAC docs and tutorials for OPA, do you happen to know of a good source of docs for ACLs like, User A gives User B edit rights on Resource C? Update: I swear I’ve looked through the docs 20 times and I’ve never seen this use case, but of course after writing this comment I go back and immediately find what may work :-) https://www.openpolicyagent.org/docs/latest/comparison-to-other-systems/#amazon-web-services-iam https://www.openpolicyagent.org/docs/latest/comparison-to-ot...
- ramraj07 5y agoI am continuing to be amazed at how much over engineering Airbnb does for ostensibly a cleaner couch surfing broker. Like they don't actually do much even for a travel site, they have so much investment and could have easily disrupted so many different travel related Fields instead they keep over engineering software. Not sure how to feel about it (since we do kinda benefit from their busywork)
- astrange 5y agoThey have similar incentives to Uber where their main goal is to get engineers to work for them by being interesting, and it doesn't actually have to be profitable. I think Uber also writes blog posts about architecture to trick competitors into thinking it can't be done by sharding each city into one box under someone's desk.
- shric 5y agoOne of the authors is Mike Burrows -- https://en.m.wikipedia.org/wiki/Michael_Burrows https://en.m.wikipedia.org/wiki/Michael_Burrows
- gneray 5y agoI'm curious what's driving the resurgence in interest authorization infrastructure, particularly the Zanzibar paper. As founder of Oso (https://www.osohq.com/ https://www.osohq.com/), I have my own opinions, and I think this is a good thing. But would love to hear others' points of view here.
- matthewaveryusa 5y agoThe rise of the zero trust paradigm in corporate networks probably.
- tolbish 5y agoAnd it's about time
- cratermoon 5y agoPandemic times and working from home. Companies were already exposed by their employees mobile devices and by people working on public wifi networks, like catching up on email while having coffee at the neighborhood coffee house. Now with employees more-or-less permanently remote, what is the corporate network? Add to that the realization that as organizations adopt more and more SaaS offerings into their operations, the distinction between "corporate network" and "public network" vanishes. The old VPN/firewall/DMZ perimeter model was leaky anyway.
- TechBro8615 5y agoSome factors might include increasing usage of microservices, frontend SPAs, serverless, and more early startups looking to integrate with enterprises, who now have high expectations of what's possible thanks to Auth0 and the like.
- thinkharderdev 5y agoMy guess is that it is mainly driven by the increasing adoption of microservice (or just generally more distributed architectures). Doing fine-grained authorization in that type of architecture quite difficult and people are starting to realize that.
- wideareanetwork 5y agoIt’s so tempting to make some snide remark about it being cancelled.
- throwaways885 5y agoIt might be. Notice that it's only been in use for about 3 years. The difference is you don't tend to upset users due to underlying infrastructure changes.
- delroth 5y agoZanzibar has been in use for way more than 3 years. I used to work in the SRE team supporting it 7 years ago, and it already had significant users back then.
- jeffbee 5y agoRule of thumb when reading Google papers: if you start now and copy it perfectly, you'll still be at least ten years behind. With few exceptions they don't publish "industry-enabling" papers.
- Tyr42 5y agoI hope not, I just finished integrating with it at work.
- amitkanfer 5y agoGreat paper, lots of it got blended into our tech at https://build.security https://build.security
- liotier 5y agoGoogle stands on it.
- dav 5y agoI also immediately think of the Brunner book
- aaronharnly 5y agoMaybe a dumb question on standalone authorization services: does the authorization service end up having a representation for every single object in all of the rest of your datastores? (e.g. every document, every blob of storage, every user in every org). If so, does that become a chokepoint in a distributed microservice architecture? Or can that be avoided with an in-process or sidecar architecture in which a given microservice's objects are not separately referenced in auth persistence? If not, how do folks determine which objects to register with the auth service and which to handle independently?
- vinay_ys 5y agoThis is relevant only if you have zillions of objects of hundreds of types and more such types of objects are likely to emerge in future (as you launch more products/use-case). And you have billions of users and their sharing permission models are complex and likely to keep evolving in the future with more devices, concepts of groups/family etc. In such a scenario, doing access control in a safe and secure way that scales and evolves well to such a large base is itself a major undertaking. You want to decouple the access control metadata from the data blob storage itself so that they each can be optimally solved for their own unique challenges and they can evolve independently too.
- NovemberWhiskey 5y agoAs with everything, it depends on your requirements. Say your goal is to externalize just your authorization policies from your code. A simple implementation might look like an OPA sidecar to your services, with the policy itself being sourced from a separate control plane - this might be something as simple as a centrally-managed S3 bucket. The service implementation provides the attributes to OPA to allow it to evaluate the authorization policy as part of the query. e.g. which groups is this user in, what document are they accessing, is this a read, write or delete operation. If you want to externalize sourcing of the attributes as well, that becomes more complicated. Now you need your authorization framework to know that Bob is in "Accounting" or that quarter_end_results.xls is a document of type "Financial Results". You can either go push or pull for attribute sourcing. The push model is to have the relevant attribute universe delivered to each of the policy decision points, along with the policy itself. This improves static stability, as you reduce the number of real-time dependencies required for authorization queries but can be a serious data distribution and management problem - particularly if you need to be sure that data isn't going stale in some sidecar process somewhere for some reason. The pull model is to have an attribute provider that can you can query as necessary; probably backed with an attribute cache for sanity's sake. The problems are basically the opposite set - liveness is guaranteed but static stability is more complicated. The methods are not equivalent: in particular, the pull model is sufficient to answer simple authorization questions like 'can X do Y to Z?' - we pull the attributes of X, Y and Z and evaluate the authorization policy. However, if you need to answer questions like 'to which Z can X do Y?', how does that work? For simple cases you may be able to iterate over the universe of Z's asking the prior question; but it generalizes poorly.
- deleted 5y ago[deleted]
- villgax 5y agoReminds me how how Basecamp would have acted during WW2 as well. Nothing to do with business so take it outside.
- aww_dang 5y agoInteresting choice of name. https://www.researchgate.net/publication/325605315_The_1964_Zanzibar_Genocide_The_Politics_of_Denial https://www.researchgate.net/publication/325605315_The_1964_... >On the fiftieth anniversary of the atrocious killing and raping of the Arabs of Zanzibar in the wake of the 1964 revolution in the Island, this paper sought to establish that this mayhem was genocide. In light of the almost complete failure to notice this tragedy, the paper pursued critical genocide studies and hidden genocide investigations to argue that this Arab tragedy in Zanzibar has been a denied genocide. Worse still, the paper showed that this genocide is commonly ignored even in studies devoted to bring to memory of hidden genocides life.
- bruckie 5y agoHere's the story of how its name came to be: https://twitter.com/LeaKissner/status/1136691523104280576 https://twitter.com/LeaKissner/status/1136691523104280576
- headmelted 5y agoSomewhat off-topic I know, but I'd love to see this extended to some of the features that Sign in with Apple has in terms of private relay. Signing in with Google yields (at a minimum) the e-mail address to the client which means that the list of third parties that have your e-mail (and can therefore spam you at will) is increasing exponentially. It would be great if Zanzibar extended the ACLs to include privacy controls with external services. (Or I'm misunderstanding and this is only the component for internal Google authentication and not external federation for clients).
- delroth 5y agoZanzibar is an authorization system, not an authentication system.
- achalshah 5y agoNot to be confused with Uber's Zanzibar: https://github.com/uber/zanzibar https://github.com/uber/zanzibar
- pnocera 5y agoI'm just wondering if there's a one size fits all solution for authz. I spent a few days on a use case : - users have one or several roles ( these are hierarchical ) - there are some objects in the system ( hierarchical too, eg files and folders ) - there are different features available according to a user's subscription. I ended up with a 30 lines program which given a set of rules calculates who can access what in less than a millisecond. Does it worth an over-engineered mega system ?
- dboreham 5y agoYou are not wrong. And this pattern shows up everywhere. e.g. do you need a SaaS for "feature flags", since they're just an if statement? In the case of authz, the argument for separating it as a concern is that many applications can share the same scheme, and you can have specialized tools for provisioning, auditing, etc.
- OJFord 5y agoIt'd be remiss of us to let left-pad aaS [0] go unmentioned in this thread... For those in today's 'lucky 10,000'^, you're welcome. There are definitely good arguments for it, services like feature-flagging I mean, and such things are generally relatively low-cost; it's more the risk of adding a 'disappearable' dependency for anything and everything that'd put me off. (^And if you don't know about this, OMG how can you not have heard about lucky 10k?! Just kidding. [1]) [0] - http://left-pad.io/ http://left-pad.io/ [1] - https://xkcd.com/1053/ https://xkcd.com/1053/
- danans 5y ago> do you need a SaaS for "feature flags", since they're just an if statement? If you want the ability to remotely enable/disable a feature, then yes.
- ogazitt 5y agoExactly. When you cross a certain complexity threshold, it's worth separating concerns. It's true for configuration, it's true for IaC, and also for authorization policy.
- stevefan1999 5y agoIs that a Metal Gear Solid[1] reference? [1]: https://metalgear.fandom.com/wiki/Zanzibar_Land_Disturbance https://metalgear.fandom.com/wiki/Zanzibar_Land_Disturbance
- xenophonf 5y agoMuch more likely to be a reference to Brunner's classic work of dystopian fiction, which postulates that the 2010 population of Earth, projected to be around 7 billion people, could all stand shoulder to shoulder on a single island the size of Zanzibar. https://en.wikipedia.org/wiki/Stand_on_Zanzibar https://en.wikipedia.org/wiki/Stand_on_Zanzibar It's not the kind of literary allusion I'd want to make, if I were a global multinational like Google/Alphabet, but there it is.
- taftster 5y agoOne of the project authors (Lea Kissner) relates the story to the naming of the project here: https://twitter.com/LeaKissner/status/1136691523104280576 https://twitter.com/LeaKissner/status/1136691523104280576
- BashiBazouk 5y agoWell, that was unexpected. I would have gone with Stand on Zanzibar as the other poster mentioned. Also makes me feel old that they don't remember the spice channel...
- taftster 5y agoHere's a decent twitter thread (2019) with some background on the project: https://twitter.com/LeaKissner/status/1136631437514272768 https://twitter.com/LeaKissner/status/1136631437514272768
- dvdkon 5y agoI'm currently building an abstracted authorization system for PostgreSQL, and one problem I ran into were timing attacks. Granted, I only had an unoptimised prototype, but querying a table and only checking if the user has permission to read the objects after the fact led to being able to differentiate "no matching object" and "one unavailable matching object". From skimming the paper, it seems Google use this approach, why are timing attacks not a problem for them? Is it because authorization checks are so fast? Or because they make sure only to query available objects, only using Zanzibar as a final "just in case" guard?
- comboy 5y agoMaybe evening response time is some abstraction on top? It may be useful for protecting much more than just auth so it would make sense not to repeat that on every layer.
- dvdkon 5y agoI considered that, but it seems way too fragile to trust, expecially if you want to test complex relationships for authorization.
- bradstewart 5y agoWhat exactly is the attack you're worried about here? Why do attackers have direct query access to your database? What useful information can they extract from knowing there is an unauthorized object in the database?
- dvdkon 5y agoMy model attacker is a limited user that has access to an advanced search function with filtering on number inequality and/or string patterns akin to LIKE. Such an attacker could send a search query such as "id = 4829 AND cost > 1000" and measure the time that query took (over multiple executions). From the time data the attacker could then determine if object 4829 has a cost value of over 1000, gaining 1 bit of data. Through a binary search they could obtain the full value in logarithmic time. If the authorization check was fast enough (which it probably is for performance reasons anyway), this would be reduced to the attacker obtaining statistical information (roughly how many objects have cost over 1000). That might be acceptable, my problem is that a benign-looking performane problem could become a serious security problem.
- Olshansky 5y agoI can't get over the name because I definitely had a memorable experience going to Zanzibar in Toronto (https://www.yelp.ca/biz/zanzibar-toronto https://www.yelp.ca/biz/zanzibar-toronto) shortly after turning 19.
- KrishnaAnaril 5y agoWhat is the status of xacml based solutions? Anyone using it?
- ogazitt 5y agoThe ideas (attribute based access control) have stood the test of time, but the spec is archaic, and there are relatively few implementations. You can achieve alot of what XACML was intended for with a general-purpose policy engine (OPA).
- SergeAx 5y agoShould add "(2019)" to the title
- btbuilder 5y agoI’m curious about what their approach is to handle consistency with object creation and deletion in the client service. ie how do clients guarantee that the relevant ACLs are created and destroyed in Zanzibar when clients create and destroy their objects. Destroy can be done asynchronously with durable messaging but asynchronous creation of ACLs is annoying from an api consumer perspective.
- andred14 5y agoYa sorry Google I will be using as little of your services as possible. Your censorship of the truth is criminal and you will be held accountable
- merqurio 5y agoThere is an Open Source (Go) implementation of "Zanzibar" called Keto [0] that integrates with the rest of the Ory ecosystem. We are actually testing it and looks great so far. [0]: https://github.com/ory/keto https://github.com/ory/keto
- jeffbee 5y agoThis comes up every time but I think it’s worth noting that Keto provides literally none of the consistency properties of Zanzibar. All of the distributed systems homework assignments for that project have been left as TODOs.
- sitkack 5y agoHow is it not a SPOF?
- skj 5y agoIt ABSOLUTELY is a SPOF and was responsible for several high profile outages at GCP last year.
- guenthert 5y agoHmmh, auditing doesn't seem to be mentioned in that paper. I'd think that's a mandatory feature of an authorization service.
- skj 5y agoIn Google, auditing is handled separately. The availability guarantees necessary for basic authorization are far more strict than auditing. Auth fails closed, audit fails open. Anything that can be stripped out of auth should be, even if we're talking about a best effort extra rpc from the auth service. Auditing typically needs more information than auth as well, and making the auth pipe wide is a risk.
- pyuser583 5y agoWhy did they name it Zanzibar? Zanzibar is an island off the coast of East Africa known for being a place where people traded cotton for enslaved humans. Not sure the connection.