11 ms·
When REST isn't Good Enough
- mikeknoop 14y agoWe (Zapier) are in a particularly good place to comment on this. We have implemented about 70 very diverse web APIs in-house. Early on, we used client libraries because they were easy and convenient. That decision bit us, hard. (Most) client libraries are un-maintained, opaque, and difficult to extend. We spent many months ripping every client library out down to the raw requests. We are in a unique position because of how many APIs we have to support, but the overarching advise we give to people designing APIs: If you are deviating from the norm, you're doing it wrong. REST APIs are nice because you can infer how to communicate with the API out of the gate. APIs are a pain and you only introduce headaches by not doing things in a sane, conventional fashion.
- jeremyjh 14y agoCan you respond directly to some of the points raised in the article? For example, ensuring SSL certs are validated?
- mikeknoop 14y agoSure. I'll start with this: as far as client libraries go, Braintree is doing it right. Because it's the only thing they support, they are incentivized to make sure it is feature complete and bug free. SSL: It is not the onus of the vendor to ensure people are properly securing requests. I have seen horrendous things done in the name of "security" but it only adds headaches. Fix this at the client level. SSL everywhere is sufficient. By extension, vendors often introduce mechanics because they want to make the world better: http://xkcd.com/927/ http://xkcd.com/927/ Platform Support: I think the counter-argument is community client libraries, which the OP is correct: they are worse. The stability + scaling + support issues mentioned by OP can usually be solved by better API design and better architecture behind the scene. Backwards Compatibility: OP claims you never have to update your code if you use their client library. This is likely true. Good API design would dictate not breaking exposed endpoints unless there is new functionality to be had. In this case, to take advantage of any new API features, you'd still have to update your code. No cost savings are had either way.
- jeremyjh 14y ago>SSL: It is not the onus of the vendor to ensure people are properly securing requests. Two responses: first, Braintree customers may appreciate knowing that Braintree provides a safer implementation than whatever the (third)contractor that they hired to build thier web app might throw together. Second, fraud does directly impact their business, regardless of the entry point used by the attacker. Even if they don't get stuck for the money directly, they are going to lose time talking to customers and helping with investigations. They don't have to do either, but they may find that they can make some types of customer relationships profitable that would not be for a competing business that doesn't do as much hand-holding.
- hennk 14y agoI can understand that a payment gateway takes security more serious, as it can directly affect their business if clients handle SSL wrongly. Maybe a kind of chaos-monkey for API connections could help? Every now and then the API returns a wrong SSL certificate. If the client ignores this, they could notify the customer, or directly block further access until the problem is solved? :)
- alanctgardner2 14y agoThe big problem I see is that SSL requires an unbroken chain of trust all the way along. If I have client-side code connecting to my server, which is then connecting to a third party server, all of those links have to be secured. As someone else pointed out, it`s nice to make your payment processing secure, but at some point the developer also has to step up and ensure their half of the bargain is upheld. If I`m sending credit card details to my backend over HTTP for some reason, no API in the world will stop me. Even looking at Stripe; they provide a JS library so you can do client-side encryption. However, if you include that JS library on an incorrectly SSLed page, a malicious party could replace it without you knowing. Even with extensive hand-holding, if a user is going to implement SSL badly, there will be security risks. The only sure-fire solution is for the developers of SSL libraries to come together and implement sane default options, and clear documentation of how not to do things. Looking at the docs for OpenSSL, it`s impossible to easily discern what counts as a sane configuration. The same problem propagates itself into language-specific libraries, where the dev wasn`t quite sure what options to tick to begin with. And so it goes.
- debacle 14y agoWhat about some sort of client library generator, similar to how .NET languages can generate code from other .NET WSDLs?
- edanm 14y agoIs there a good book/resource on learning REST API ideas? I'm looking for a beginner's resource for someone who doesn't really grok the ideas behind REST and the motivation for REST. But also something that gets technical very quickly - I've been programming for long enough that I pick ideas up very quickly.
- michaelfairley 14y agoNot your typical book, but it's very good: http://designinghypermediaapis.com/ http://designinghypermediaapis.com/
- edanm 14y agoInteresting. I wasn't sure about buying it, but ended up purchasing. Only after realizing that it was by Steve Klabnik, a name I recognize, probably from HN. I wonder why there isn't more info on the front page, even a sample/TOC would be nice. I'll be going through it tomorrow, if I remember I'll come back to post a mini review.
- toomuchtodo 14y agoThis. After seeing the URL posted above, I clicked the link just to see if it was Steve's book. Disappointed he doesn't put his name prominently on the landing page.
- sopooneo 14y agoI'm sort of with you, though I've read quite a bit at this point and interacted with a number of APIs of varying degrees of "RESTfulness". The biggest question I wish the various tutorials would address is: why is REST a good idea? Why is this particular way of doing things better than others? And here's one thing I believe is true, that I have literally never seen in a REST tutorial: sometimes REST is not the best way to go. Sometimes an RPC architecture is better.
- BerislavLopac 14y ago
- davetron5000 14y agoThis has been my approach to internal service design as well. It's far easier to make sure people integrate properly by providing a rich, well-documented API client library than it is to rely solely on the REST-based one. REST APIs are hard to document, and require a lot of orthogonal boilerplate to even make the requests properly. Making a client library is also a great way to "dogfood" your REST API - you know a developer will write code, not HTTP calls, to interact with your service. The client API allows you to test out that code and make sure your API is well designed.
- arethuza 14y ago"require a lot of orthogonal boilerplate to even make the requests properly" What kind of boilerplate would that be? For me one of the big advantages of a RESTful interface is that they are really easy to call from cURL without much need for building up a complex context.
- e28eta 14y agoI found the translation between native code and a REST API to be what I'd call boilerplate. You need to build a URL, build the request (usually converting some model object into JSON), make the request in a non-blocking fashion, receive and parse the response into either a successful response, probably converting JSON back into a model object, or a failed response, which needs to be mapped to some native representation and passed back into the business logic. Contrast that with a client library which can do all of that for you, for example: taking a native model object and calling one of two callback functions provided, letting you concentrate on the business logic of your app. I'm not arguing that a client library is the right way to go, but I certainly understand where the grandparent is coming from RE: boilerplate code. Also, this is going to be heavily influenced by the language and libraries you're using.
- MatthewPhillips 14y agoExcept company B uses a single callback, returning an object that contains an error code when not successful. Company C uses a blocking library, and requires you to pass in your model object as a dictionary. See where I'm going with this?
- pelle 14y agoThere are several good points in the article. Particular the one about https. That said as a clojure dev I'd much rather work with a nice clojure http library like clj-http than have to use their provided java library.
- scanr 14y agoBraintree have made an interesting choice in keeping their REST API private and requiring customers use their client libraries. I think they could both have client libraries that they promote as well as a public REST API. Breaking down their reasons: Security - agree with them on this, the more they can help their users make their systems secure the better. Not sure if it should preclude a public REST API but certainly motivates for having a good client library. Platform Support - another good reason to have the client library, essentially encoding best practice in the client. I've certainly seen customers abuse features of our APIs. Again, not sure if it should mean keeping the REST API private. Certainly it's a good idea to be defensive on both the client and server (e.g. for queries that request too much data, rate limiting etc.). Backwards compatibility - Here I disagree with Braintree. I think it should be just as easy to manage backwards compatibility purely on the server side.
- dan_manges 14y agoI'm a developer at Braintree. The challenge with maintaining backwards compatibility on the server side is handling the wide variety of possible inputs into the system. It's hard to write tests that account for all of them. I've seen a couple of cases where backwards compatibility can be broken in unexpected ways. For one application that we built at Braintree, we had a client that was sending us an application/x-www-form-urlencoded POST body without the Content-Type request header. We upgraded the version of Rails that this app was using, and it broke that integration because Rails made a change where it wouldn't parse the POST body without the Content-Type header. Unfortunately, we didn't have any test cases in our test suite that made POSTs without a Content-Type. We were able to identify the issue and resolve it quickly, but it was a surprising bug. With client libraries, we can test every version against the upgraded app and know that all clients will continue to work. Are there interesting request profiling techniques that can be executed on production traffic to analyze requests? I think the challenging part of backwards compatibility is making sure unintentional use cases, that were never intended to be supported, continue to work.
- scanr 14y agoThanks for responding! And I'll update my response to the backwards compatibility to say that I see the value in having a closed set of clients that you need to test for backwards compatibility. That said, these are some of the things that I've done to help with backwards compatibility through only the serverside API: * Track production requests and use them as test cases * Build a large suite of test cases against the API * Build the API in a statically typed language (yeah, I know, contentious, I love Rails but there's something about an externally facing API that makes me want to use a statically typed language). * And then the ultimate -- build the API as an app that talks back to the business logic... essentially, it's your 'client library' but deployed on the server between the actual code and the client. Then, never change the part of it that faces the client, only the mappings on the the real business logic.
- pixie_ 14y agoI don't like REST. Making a call to a server should be like calling a function anywhere else - you have your parameters and return value. In REST the parameters are spread over 3 places, the action, the url and the post parameters themselves. It's a much better design to combine all your parameters in one place and keep things simple, for example - REST version: UPDATE /course/324234 { description: "This is a level 1 course" } Improved version: POST /course/UpdateDescription { courseId: 324234, description: "This is a level 2 course" } In this case POST is always used for api calls. Course is the namespace, UpdateDesciption is the method name, and parameters are kept all together as JSON.
- scanr 14y agoHere's where I've found REST very useful: It makes for predictable APIs for consumers and it means API producers have a checklist of things to implement to consider their interface 'complete'. It also provides a consistent way to think about how to expose an interface (or in fact, how to expose entities).
- biot 14y agoIf your course has 20 member variables, now you have 20 update methods? If you want to update multiple, the client then needs to do multiple round-trip calls to your API which is time-consuming: if it's a 350ms round-trip per call, updating all 20 fields takes 7 seconds. And unless you go out of your way to offer per-object locking (which over a stateless connection has its own challenges) you prevent users from doing atomic updates to multiple fields at once. This pushes any rollback mechanism onto the client, exactly where it should not be. With your design, if your goal is to "combine all the parameters in one place to keep things simple", you could go one step further and have: POST /course/update {courseId: 1234, field: "description", description: "hello"} Taking this to its logical extreme, you can truly combine all the parameters in one place: POST / {object: "course", operation: "update", courseId: 1234, field: "description", description: "hello"}
- pixie_ 14y agoLike any other programming language if you need to update multiple fields, you pass multiple parameters to the function. The URL should be equivalent to namespace/class/method while the parameters are just that - parameters you can pass in JSON format.
- bguthrie 14y agoNote that this isn't a critique of the REST architectural style; rather, it documents their decision to prefer supplying clients with prebuilt libraries rather than a public API that their respective language communities can use to build their own. You could read the source to any one of their client libs and infer the rough structure of their services, so it's not all that private; they're simply choosing to leave what's there undocumented. It's an interesting approach, but not without its drawbacks.
- MatthewPhillips 14y agoI have no knowledge of what BrainTree does, but if I'm picking a service with an API, I'm picking the one that is easiest to learn (preferably there is little that needs to be learned), I'm not picking one that requires me to switch languages -- or just as bad, locks me into one of a few languages I'm currently using.
- elarkin 14y agoBrainTree does payments. They have concerns about security that most apps do not. Who really cares if bugged timeout/retry logic makes your app post twice to facebook? Probably no one. Who cares if bugged timeout/retry logic makes your app bill them twice? Probably everyone. BrainTree and you may not make the same decision, and you both could be right.
- caseysoftware 14y agoAt Twilio, our REST API is fully documented but we still encourage people to use our helper libraries as much as possible. It's not because we'd like to document the API less, but because there are related things - http connections, auth, etc - that need to be handled and it's easier to have the library do them. Further, the library serves to wrap the API in the idioms appropriate for the language. This may include data structures, but also includes things like variable names. That way you can focus on your app in your chosen.. and only dig into the abstraction if you choose to.
- jeremyjh 14y agoWhy not provide a C api? This would make it easy for community language bindings to capture the benefits you describe in languages you aren't natively supporting (such as Go, Erlang or Haskell).
- outdooricon 14y agoThe platform support argument is not very persuasive here. The beauty of open source and the community is that when you combine a popular service with common platforms that are likely to be used, it is likely someone in the community will release the needed api. Why not provide some community support for the platforms, but allow the users to release an api where they know their needs better than you do? On the other hand, the security argument is quite strong. Improperly implemented SSL handshaking, especially when dealing with payment transactions like they do, has the potential to be devastating. But this is where interacting with the community that is making the api's would be key. There's a lot of value to be had when company and community development work together. They could contribute the ssl code themselves to the open source client projects.
- waffle_ss 14y agoThe title should add "by itself" at the end, as they are still using REST - just adding client libraries.
- KrisJordan 14y agoBraintree isn't the only HTTPS API a developer will consume. Rather than bury the knowledge of HTTPS best practice in the black box of a library, why not exemplify it in language-specific examples of plain-old REST API documentation? If the boilerplate for manually getting an SSL connection right in a given language is that obtuse, the "just use our library" pitch is even more compelling. All this said, I have used Braintree's libraries and they are extremely well executed. Not exposing REST API documentation externally (it surely exists internally, right?) just feels like a cop out. (An argument I'm surprised was left out: it's easier for support staff to work with customers integrating with a good library than some home rolled REST client. I can imagine this to be true, but maybe a case of premature optimization if the majority of big API players expose and document their REST APIs anyway?)
- drivebyacct2 14y agoThis headline is atrocious, this article has so incredibly little to do with REST mechanics.
- tlrobinson 14y agoIt's a shame there isn't some simple standard (de-facto or otherwise) way of documenting REST APIs such that an idiomatic client library for each language could be generated automatically. The OPTIONS method is a really underused part of HTTP and would be great for this purpose: http://zacstewart.com/2012/04/14/http-options-method.html http://zacstewart.com/2012/04/14/http-options-method.html
- grncdr 14y agoCheck out github.com/RestDoc/specification, it's a project that got started after a few people (myself and Zac included) realized we were all solving the same problem. It's been languishing in relative obscurity because most of us don't have the bandwidth to properly promote it, but more contributions are very welcome!
- olalonde 14y agoYes, there should be a WSDL for REST APIs.
- cryowaffle 14y agohave you heard of the WADL? http://en.wikipedia.org/wiki/Web_Application_Description_Language http://en.wikipedia.org/wiki/Web_Application_Description_Lan...
- pbreit 14y agoI don't really care for this. The promise of web technologies, which has finally become reality over he past few hears, is obviating the need for client libraries. This benefits. Oth providers and consumers alike. The benefits cited by the OP are not that compelling. Most (all?) other APIs allow/encourage direct access so for a large number of developers, that's a strong preference.