7 ms·

Varnish author criticizing HTTP/2.0 proposals

277 points by cgbystrom 14y ago

sciurus 14y ago
His proposal is at http://phk.freebsd.dk/misc/draft-kamp-httpbis-http-20-architecture-01.txt http://phk.freebsd.dk/misc/draft-kamp-httpbis-http-20-archit... It comes with the caveats that "Please disregard any strangeness in the boilerplate, I may not thrown all the right spells at xml2rfc, and also note that I have subsequently changed my mind on certain subjects, most notably Cookies which should simply be exterminated from HTTP/2.0, and replaced with a stable session/identity concept which does not make it possible or necessary for servers to store data on the clients."
- pdeuchler 14y ago
  If I'm interpreting that correctly, that could be a huge win for privacy
  - quotemstr 14y ago
    How so? A unique identifier is a unique identifier.
    - angelbob 14y ago
      If you don't store a leakable cookie on the client and only use a connection ID (which is autoassigned by the server and cannot be easily forged) then a lot of cross-domain evil hackery goes away.
      - danellis 14y ago
        I don't know what you mean by 'leakable' exactly, but you can already mark cookies as HttpOnly (not accessible from Javascript) and Secure (only transmitted over HTTPS).
        angelbob 14y ago
        "Leakable" in the sense that you can hand it to somebody else and they can actually use it. A cookie is leakable because the client chooses to send it, so copying it to somebody else is really bad. A server-assigned per-connection ID is not leakable unless you can spoof the IP address of the one you're sending as.
        jvdongen 14y ago
        In principle you could already achieve that by binding a session (and corresponding cookie) to the client's IP address. No big deal. Problem you may have is that some clients are behind proxy farms and can arrive with different source IP addresses within the scope of a single session. If you do not bind the 'server-assigned-per-connection ID' to an IP address they become just as 'leakable' as a session cookie.
        vidarh 14y ago
        Doing server side IP checks is already easy, but in reality it can lead to massive amounts of user complaints when their sessions keep disappearing, because as it turns out, some user segments have a lot of users coming through proxies where each request is not guaranteed to come from the same IP.
        eli 14y ago
        Or their mobile device roams from WiFi to cell data. I suspect anyone suggesting that an IP be part of the session security has never actually tried it on a large scale.
        angelbob 14y ago
        Sure. It'd be nice to have some kind of actual standard to at least make a good best guess here. Cookies are a really, really bad start.
        ay 14y ago
        Then a translation device at carrier assigns the client a different address. I heard first-hand that this is very annoying.
    - ars 14y ago
      If the browser created the unique identifier instead of the server it could create a new one as often as necessary. I think that would actually work very nicely.
      - rmc 14y ago
        Browsers can do that now with cookies, by just clearing old cookies.
    - pdeuchler 14y ago
      Of course, but the question is what that unique identifier denotes. An IP address tells people who you are, where you are, and what you do. A unique identifier ('+18El1iZRFCIiqRpfw4dJR8mXjJn2UxPrjwoRNpjSWg=' for instance) tells what you do without the who and where [1]. It's even possible advertisers could still target users based upon these identifiers, just without the background knowledge that makes these sorts of things privacy issues. [1] I'll admit, it could be argued that the what can determine the who and where
- dudenewcomb 14y ago
  Maybe he's trying to bring light to Unhosted?
comex 14y ago
Removing cookies from a protocol which is otherwise fully compatible with HTTP/1, in the sense of being able to be interposed as a proxy or substituted in the web server without breaking apps, is a terrible idea. > Cookies are, as the EU commision correctly noted, fundamentally flawed, because they store potentially sensitive information on whatever computer the user happens to use, and as a result of various abuses and incompetences, EU felt compelled to legislate a "notice and announce" policy for HTTP-cookies. > But it doesn't stop there: The information stored in cookies have potentialiiy very high value for the HTTP server, and because the server has no control over the integrity of the storage, we are now seing cookies being crypto-signed, to prevent forgeries. Anyone with a grain of skill is capable of using cookies as identifiers only; it's hard to see what cookies vs identifiers has to do with "notice and announce" or security. An explicit session mechanism could provide benefits over using cookies for the same purpose, but what exactly would removing cookies achieve other than breaking the world?
- dangrossman 14y ago
  Why must HTTP 2.x be backwards compatible with 1.x? Should SSH have not been created because you can't talk to it with a telnet client? If a new protocol offers sufficient benefits, it would be worth having to make minor changes to apps to support both. Cookies suck, from a technical and regulatory-compliance standpoint. Plus, I'll finally stop having to clear my cookies every month or so just to log in to my PayPal and American Express accounts. Both sites keep creating unique cookies on every login until there are so many that they pass their own web servers' max header length limits.
  - joahua 14y ago
    I dunno - obviously different projects diverge on required backwards compatibility (Python 2/3?), but SSH isn't called Telnet v2. For something as fundamental as HTTP the author argues changes need to be radical to drive adoption, but at the same time there's not necessarily wide-spread impetus to do so if the burden is too high. This is engineering on a 15 year time-scale, which I feel a little young (at 24) to well comprehend! It's not just apps in the web-app sense, but user agents (all the way down to embedded systems) that would need changing to take advantage of the 'sufficient benefits'. That's a pretty massive undertaking.
patrickaljord 14y ago
> See for instance how SSH replaced TELNET, REXEC, RSH, SUPDUP > Or I might add, how HTTP replaced GOPHER[3]. telnet and gopher were used by a few thousands servers only and were not consumer facing technologies (for the most part), it doesn't make sense to compare that to IPv4 and HTTP that are used by millions (billion?) of servers.
- sukuriant 14y ago
  Why not?
- bmelton 14y ago
  Sukuriant's point might be terse, but it's not invalid either. In 20 years' time, I'm pretty sure that the next iteration of us will be saying something like "When they phased out HTTP, there were only a few billion servers..." Better will always make its way in, even if there are entrenched systems running the 'old faithful' code already out there. IE6 is being phased out, yes, VERY slowly, but we're already way closer to getting it down to an irrelevant number than we would be if there weren't pushes. Will we ever get rid of it completely? Maybe not. I'm sure there's still a gopher server out there somewhere or another, and it's not that uncommon to get Telnet access to some commodity (crappy) web hosts, but SSH is pervasive and good, and we're all better off for it.
- jfoutz 14y ago
  Who won't support spdy by the end of the year? The pace of protocol obsolescence is increasing. I can't find stats about how much http traffic facebook is responsible for, but i'd bet it's at least 1% Governments and home grown enterprise apps are my guess about who's late to the party.
  - mooism2 14y ago
    El cheapo web hosting will be late to the party. TLS certificates will be a stumbling block.
- jasonwatkinspdx 14y ago
  The factors that caused slow IP4 to IP6 transition are cited as examples by the author you are criticizing. It's very clear they are aware of the scale of http deployment as well as the challenges that brings. In fact, that key point motivates most of their objections to the current standards proposals.
dhruvbird 14y ago
Excellent judge of the protocol. It's interesting that no one is trying to solve the cookie bloat problem with http.
borlak 14y ago
load balancers aren't meant to just be "HTTP routers". they can definitely be used as such for smaller applications and do a good job at it, but a real load balancer needs to be quite complex, being able to adapt to the underlying applications that make use of it. if your goal is to only route HTTP requests, then you're only solving the first step of an increasingly complicated field of computer science (namely, web applications). Cookies aren't going to go away. if you want to improve the protocol to deal with cookies better, that makes sense, but acting like they are some kind of evil on the internet that should be forgotten isn't going to work. it's a bit self-defeating to argue that some protocols failed because of failure to provide new benefits and then argue against Cookies in HTTP!
- bickfordb 14y ago
  Although I think he should further consider why many websites are using cookies at all or even cryptographically secure cookies, I think he makes a good point about caching and cookies. Websites with large amounts of traffic often move their static content to separate domains to avoid clients sending cookies headers on every request to static/forever-cacheable assets.
- nl 14y ago
  load balancers aren't meant to just be "HTTP routers". they can definitely be used as such for smaller applications and do a good job at it, but a real load balancer needs to be quite complex, being able to adapt to the underlying applications that make use of it I think Poul-Henning Kamp is fairly well qualified to discuss what load balancers do. (And yes, load balancer are fundamentally HTTP Routers. Yes, sometimes they do content manipulation etc, but all those features are add-ons to the basic use-case)
  - youngtaff 14y ago
    Perhaps Kamp is qualified to discuss what some 'load-balancers' do... The proxy developers have always had their doubts about SPDY (you can see them when @mnot first proposed it as a starting point) Terminating SPDY at the HTTP Router makes a lot of sense architecturally but I know some orgs don't terminate SSL at the load-balancer due to the licensing costs. Ultimately we need load-balancing options and someone to develop the opensource proxies (haproxy, varnish etc.) into more sophisticated offerings. Perhaps they'll be a SPDY module for Traffic Server
ajankovic 14y ago
In my opinion it is always a good idea in these kind of situations to set a goal we should strive to. I wonder what the ideal web protocol would look like. If, for example, we didn't have a burden of billions of servers and Internet reliance on HTTP/1.x protocol. What would be the most ideal solution that would suite emerging use cases for the Web? Are there any research papers on this topic?
- squidi 14y ago
  I agree with this way of thinking. phkamp has come up with some innovative ideas, but if it doesn't come easy for the rest of us there are lots of techniques to help e.g. asking, "if I was superman and could implement any solution, what would it be" http://personalexcellence.co/blog/25-brainstorming-techniques/ http://personalexcellence.co/blog/25-brainstorming-technique...
pilif 14y ago
While in general I understand where he is coming from, I believe his main argument about adoption is flawed. What do you think is more likely going to be adopted? A protocol that's not backwards compatible at all (heck, it even throws out cookies) or something that works over the existing protocol, negotiating extended support and then switching to that while continuing to work the exact same way for both old clients and the applications running behind it? See SPDY which is a candidate for becoming HTTP 2.0. People are ALREADY running that or at least eager to try it. I don't think for a second that SPDY is having the adoption problems of ipv6, SNI issues aside. Even if native sessions would be a cool feature, how many years do you believe it takes before something like that can be reliably used? We're still wary of supporting stuff that a 11 years old browser didn't support.
- nl 14y ago
  I don't think for a second that SPDY is having the adoption problems of ipv6, SNI issues aside. Google being able to modify both the client (Chrome) as well as few fairly significant server installations has kind of helped there a little bit...
  - obtu 14y ago
    The key difference with IPv6 is that SPDY is opaque to routers and only needs changes at the endpoints. IPv6 has the chicken-and-egg problem that it only becomes usable once everyone (hosters, ISPs, core networks, home networks, web frameworks, client devices) has deployed it, and most of those actors don't bother with things that bring no immediate benefit.
    - phkamp 14y ago
      You do know that when IPv6 roll-out was initially contemplated, it would happen on IPv4-convertible addresses only, until everybody had converted. That would have allowed seamless IPv6/IPv4 interop and made migration a breeze. Lacking any "must have" features, nobody could see a reason to bother. Now we have to do the migration without the benefit of seamless conversion.
NHQ 14y ago
"We have also learned that a protocol which delivers the goods can replace all competition in virtually no time." This is an argument for websockets eating protocol lunch.
rdsubhas 14y ago
(These are just my opinions as web developer. Please feel free to downvote if my expectations are wrong) While I can completely agree with the technical merits of this proposal, there are some very two-faced statements. Author begins by pointing out the painfulness of IPv4 to IPv6, says that the next HTTP upgrade should be humble. But then proceeds to kill cookies and remove all the architectural problems in HTTP. Isn't that the same what IPv6 was? Wouldn't such an approach produce the same amount of pain to the implementors (that is us, the web developers)? Any upgrade will certainly have some backward-incompatible changes. But if it is totally backward incompatible, I don't understand why it still needs to be called HTTP. Couldn't we just call it SPDY v2 instead, or some other fancy name? Cookies are a problem. But the safest way to solve that problem is in isolation. Try to come up with some separate protocol extension, see if it works out, throw it away if it doesn't. But why marry the entire future of HTTP with such a do-or-die change? I blindly agree with the author that SPDY is architecturally flawed. But why is it being advocated in such big numbers? Even Facebook (deeply at war with Google) is embracing it. It's because SPDY doesn't break existing applications. Just install mod_spdy to get started. But removing cookies? What happens to the millions of web apps deployed today, which have $COOKIE and set_cookie statements everywhere in the code? How do I branch them out and serve separate versions of the same application, one for HTTP/1.1 and another for HTTP/2.0? More doubts keep coming... Problem with SPDY compressing HTTP headers? Use SPDY only for communication over the internet. Within the server's data center, or within the client's organization - keep serving normal HTTP. There are no bandwidth problems within there. Just make Varnish and the target server speak via SPDY, that is where the real gains are. I could go on. I'm not trying to say that the author's suggestions are wrong. They are important and technically good. But the way they should be taken up and implemented, without pain to us developers, doesn't have to be HTTP/2.0. Good ideas don't need to be forced down others throats.
- phkamp 14y ago
  No. Ipv6 is Ipv4 with bigger addresses, it didn't try to solve any of the other problems of IPv4 (or attempts to solve them were killed by ISPs). One example was multihoming (having more than one ISP) serveral smart proposals were floated (anycast, nearcast etc) but they were killed by ISP's who protected a lucrative business. If Ipv6 had made multi-ISP multihoming possible without all the trouble of BGP, business would have killed to get it back in the late 1990ies. Cookies only disappear from the wire, they are trivial to simulate on your server (see my other reply here).
bifrost 14y ago
Frankly many problems addressed by these proposals are not things that should be solved in a protocol like HTTP. Do you want every HTTP request to be slowed down with a crypto token exchange and verification? unlikely. In special cases you definately want it, but all the time? Absolutely not.
- obtu 14y ago
  It may be counter-intuitive, but with the right cyphers, the crypto costs of SSL and SPDY are negligible.
  - phkamp 14y ago
    No, they are not. For one thing you have to terminate all your SSL on your loadbalancer in order to distribute the traffic. That makes SPDY a no-go for web-hotels/web-hosting where each customer has their own certificate. Second, there are perfectly valid legally mandated circumstances which forbid end-to-end privacy, from children in schools to inmates in jail and patients in psych. hospitals, not to mention corporate firewalls and the monster that looks out for classified docs not leaking out of CIA.
    - obtu 14y ago
      I was talking about performance, but: SNI addresses your first point. For the second point, whoever needs to enforce snooping already has to deal with SSL.
    - comex 14y ago
      > No, they are not. For one thing you have to terminate all your SSL on your loadbalancer in order to distribute the traffic. That makes SPDY a no-go for web-hotels/web-hosting where each customer has their own certificate. That's what SNI is for. > Second, there are perfectly valid legally mandated circumstances which forbid end-to-end privacy Then install spyware on the user's computer, or add a trusted SSL key and mitm all the things.
  - mcbridematt 14y ago
jorangreef 14y ago
"Overall, I find all three proposals are focused on solving yesteryears problems, rather than on creating a protocol that stands a chance to last us the next 20 years."
regularfry 14y ago
How protocol adoption happens, in three steps: 1. Working code. 2. Publicity. 3. Ubiquity. That's it. Kamp is making a lot of the right noises here, but he's already lost ground to SPDY just because they've shipped code. No amount of sitting round tables bashing out the finer details of a better spec will help as much as getting code written - even if it's just a placeholder for an extensible spec, as long as that placeholder does something useful.
- gioele 14y ago
  > he's already lost ground to SPDY just because they've shipped code. Is SPDY good enough now and fixable enough in the future to leverage this gained ground? As more and more people deploy SPDY, they will understand its problems and there will almost no chances to change it, differently from what Google did at the beginning (they have gone through 1 big change and many small changes in the protocol). When the authors of the main servers (varnish, apache, nginx) will start feeling its limits, will they have to keep it around just for sake of compatibility? Please note that SPDY on the server is not a requirement, just an opportunity. Nothing will change for your users if you want to remove support for SPDY in your server after you deployed it and used it for some time. There are no "http+spdy://example.org" URL around, so supporting HTTP only will always be sufficient. Maybe not as much performant as SPDY but 100% supported.
mike-cardwell 14y ago
I can think of a few organisations that benefit financially, a great amount from the fact that cookies allow them to track users. Ad companies like Google and Microsoft spring to mind. The same organisations building our major browsers... Conflict of interest? You better believe it.
- SeoxyS 14y ago
  Imagine if browsers reported a user-resettable uuid sent over on each connection open. Advertisers get the benefit of being able to track users, do it more securely since no data is ever stored on the client, and must eat the cost of actually storing the key-value pairs, instead of passing it down to users and expending massive amounts of bandwidth bouncing the data back and forth. It's not about killing cookies, it's about finding a better solution that solves the problem better. Note: I work for a startup that "benefits financially" from tracking users—a feature without which we would not have a business.
  - sirclueless 14y ago
    This already exists in spirit, in the form of "Incognito" or "Private" browsing. This doesn't solve the protocol issues of cookie bandwidth and insecure storage, but it does provide a user-resettable store of cookies.
rickmb 14y ago
After the whole IPv6 story, I'm surprised the author ignores the political dimension of designing a new protocol. As Mitch Kapor said "Architecture is Politics". It's not just about what solution is best from a technological perspective, it's about what we want our future to look like. The internet has become way more important than back when these protocols first became standard, and every time a protocol or standard is up for debate, political and commercial forces try to influence it in their favor. Some of the concepts they tried to shove into IPv6 were downright evil, and would have killed the internet as we know it. Personally, I'm relieved all that is left is a small, un-sexy improvement which albeit slowly, will eventually spread and solve the only really critical problem we have with IPv4. I really dread subjecting HTTP to that process. Although I fully agree with the author's critique of cookies for instance, the idea of replacing them with something "better" frankly scares the crap out of me. Especially when the word "identity" is being used. You just know what kind of suggestions some powerful parties will come up with if you open this up for debate, and fighting that will take up all of the energy that should be put towards improving what we already have. As techies we should learn to accept design flaws and slow adoption and look at the bigger picture of the social and political impact of technology: HTTP may be flawed, but things could be way, way worse.
- neya 14y ago
  This is a well thought, excellent comment that perfectly makes sense. Thank you. When huge corporations track you as how they are tracking you currently, most of the times, they don't get to know who you really are (except Facebook, because they know 'you'). That is, your identity. That's the only level of control you have over your anonymity. And these guys are proposing a new protocol just to remove that. Ridiculous.
  - maggit 14y ago
    I suspect you are reading too much into the word "identity". It is just a matter of identifying the endpoint, so the HTTP router can delegate requests from one user to the same backend instance in any given session. This, in turn, will give a user a consistent image of the state at the server independently of cache propagation in the backend. For example. You don't need the user's real life identity to do this, and you don't need the same identity across different sites or across different sessions. This is currently sometimes done by cookies, which makes life difficult for HTTP routers. He is proposing a mechanism to keep the identifying-part while getting rid of problems in the HTTP router layer. The way I read this, it seemed to be without introducing additional privacy concerns and in fact removing some. (Cookies can carry more than identity) There is a detailed technical discussion to be had about implementing all of this, and in this discussion any privacy concerns would become visible and open for discussion. But I think it is a leap to say that the comments in TFA would necessarily make for a world bereft of privacy ;)
moe 14y ago
I have a lot of respect for phkamp, varnish is an impressive piece of engineering. I disagree with the stab he takes at cookie-sessions here, though. He seems to ignore that sessions are not only about identity but also about state. Servers should be stateless, therefor client-sessions (crypt+signed) are usually preferable over server-sessions. Having a few more bytes of cookie-payload is normally an order of magnitude cheaper (in terms of latency) than performing the respective lookups server-side for every request. Very low bandwidth links might disagree, but that's a corner-case and with cookies we always have the choice. Removing cookies in favor of a "client-id" would effectively remove the session-pattern that has proven optimal for the vast majority of websites.
- phkamp 14y ago
  No, servers should hold _all the state_ and clients none. If I add something to my shopping basket from my mobile phone, I want to be able to add more from my browser, rather than have two shopping baskets. Servers storing stuff on the clients is just plain wrong, and it is wrong from every single angle you can view it: Its wrong from a privacy point of view, it's wrong from a cost-allocation point of view, it's wrong from an architecture point of view and it's wrong from a protocol point of view. But it was a quick hack to add to HTTP in a hurry back in the dotcom days. It must die now.
  - duckharp 14y ago
    But what about preferences for anonymous users? Store that on the server side? Append them to the URL? Both options kinda suck. Also, consider dabblet. The way it allows you to store your stuff using github is very smart IMHO.
    - phkamp 14y ago
      Store it on the server: The user-agent gives you a session-id to use as key. It may be that session-keys should tell if they are anonymous or if they represent (locally) authenticated users, but that's a very complex subject I won't claim to have a clear opinion of yet.
sophacles 14y ago
One huge benefit I think people are missing about the notion of a protocol standard session mechanism, is that we can use HTTP auth much easier, and perhaps get away from this notion of every site having to redo the login process. Browsers can handle the "remember login" settings, and logging out is as simple as tab closing. No "remember this computer" or "don't remember this computer" checkbox confusion. No random sites saying "remember me always" and requiring manual logout on a borrowed computer. It certainly helps with the password wallet concept too. Sure all that stuff has become semi-standard as it currently exists, but it is ugly, hacky, and sometimes doesn't work, and other times opens doors for hilarious malfeasance.

277 points by cgbystrom 14y ago

His proposal is at http://phk.freebsd.dk/misc/draft-kamp-httpbis-http-20-architecture-01.txt http://phk.freebsd.dk/misc/draft-kamp-httpbis-http-20-archit... It comes with the caveats that "Please disregard any strangeness in the boilerplate, I may not thrown all the right spells at xml2rfc, and also note that I have subsequently changed my mind on certain subjects, most notably Cookies which should simply be exterminated from HTTP/2.0, and replaced with a stable session/identity concept which does not make it possible or necessary for servers to store data on the clients."

pdeuchler 14y ago
If I'm interpreting that correctly, that could be a huge win for privacy
- quotemstr 14y ago
  How so? A unique identifier is a unique identifier.
  - angelbob 14y ago
    If you don't store a leakable cookie on the client and only use a connection ID (which is autoassigned by the server and cannot be easily forged) then a lot of cross-domain evil hackery goes away.
    - danellis 14y ago
      I don't know what you mean by 'leakable' exactly, but you can already mark cookies as HttpOnly (not accessible from Javascript) and Secure (only transmitted over HTTPS).
      - angelbob 14y ago
        "Leakable" in the sense that you can hand it to somebody else and they can actually use it. A cookie is leakable because the client chooses to send it, so copying it to somebody else is really bad. A server-assigned per-connection ID is not leakable unless you can spoof the IP address of the one you're sending as.
        jvdongen 14y ago
        In principle you could already achieve that by binding a session (and corresponding cookie) to the client's IP address. No big deal. Problem you may have is that some clients are behind proxy farms and can arrive with different source IP addresses within the scope of a single session. If you do not bind the 'server-assigned-per-connection ID' to an IP address they become just as 'leakable' as a session cookie.
        vidarh 14y ago
        Doing server side IP checks is already easy, but in reality it can lead to massive amounts of user complaints when their sessions keep disappearing, because as it turns out, some user segments have a lot of users coming through proxies where each request is not guaranteed to come from the same IP.
        eli 14y ago
        Or their mobile device roams from WiFi to cell data. I suspect anyone suggesting that an IP be part of the session security has never actually tried it on a large scale.
        angelbob 14y ago
        Sure. It'd be nice to have some kind of actual standard to at least make a good best guess here. Cookies are a really, really bad start.
        ay 14y ago
        Then a translation device at carrier assigns the client a different address. I heard first-hand that this is very annoying.
  - ars 14y ago
    If the browser created the unique identifier instead of the server it could create a new one as often as necessary. I think that would actually work very nicely.
    - rmc 14y ago
      Browsers can do that now with cookies, by just clearing old cookies.
  - pdeuchler 14y ago
    Of course, but the question is what that unique identifier denotes. An IP address tells people who you are, where you are, and what you do. A unique identifier ('+18El1iZRFCIiqRpfw4dJR8mXjJn2UxPrjwoRNpjSWg=' for instance) tells what you do without the who and where [1]. It's even possible advertisers could still target users based upon these identifiers, just without the background knowledge that makes these sorts of things privacy issues. [1] I'll admit, it could be argued that the what can determine the who and where
dudenewcomb 14y ago
Maybe he's trying to bring light to Unhosted?

comex 14y ago

Removing cookies from a protocol which is otherwise fully compatible with HTTP/1, in the sense of being able to be interposed as a proxy or substituted in the web server without breaking apps, is a terrible idea. > Cookies are, as the EU commision correctly noted, fundamentally flawed, because they store potentially sensitive information on whatever computer the user happens to use, and as a result of various abuses and incompetences, EU felt compelled to legislate a "notice and announce" policy for HTTP-cookies. > But it doesn't stop there: The information stored in cookies have potentialiiy very high value for the HTTP server, and because the server has no control over the integrity of the storage, we are now seing cookies being crypto-signed, to prevent forgeries. Anyone with a grain of skill is capable of using cookies as identifiers only; it's hard to see what cookies vs identifiers has to do with "notice and announce" or security. An explicit session mechanism could provide benefits over using cookies for the same purpose, but what exactly would removing cookies achieve other than breaking the world?

dangrossman 14y ago
Why must HTTP 2.x be backwards compatible with 1.x? Should SSH have not been created because you can't talk to it with a telnet client? If a new protocol offers sufficient benefits, it would be worth having to make minor changes to apps to support both. Cookies suck, from a technical and regulatory-compliance standpoint. Plus, I'll finally stop having to clear my cookies every month or so just to log in to my PayPal and American Express accounts. Both sites keep creating unique cookies on every login until there are so many that they pass their own web servers' max header length limits.
- joahua 14y ago
  I dunno - obviously different projects diverge on required backwards compatibility (Python 2/3?), but SSH isn't called Telnet v2. For something as fundamental as HTTP the author argues changes need to be radical to drive adoption, but at the same time there's not necessarily wide-spread impetus to do so if the burden is too high. This is engineering on a 15 year time-scale, which I feel a little young (at 24) to well comprehend! It's not just apps in the web-app sense, but user agents (all the way down to embedded systems) that would need changing to take advantage of the 'sufficient benefits'. That's a pretty massive undertaking.

patrickaljord 14y ago

> See for instance how SSH replaced TELNET, REXEC, RSH, SUPDUP > Or I might add, how HTTP replaced GOPHER[3]. telnet and gopher were used by a few thousands servers only and were not consumer facing technologies (for the most part), it doesn't make sense to compare that to IPv4 and HTTP that are used by millions (billion?) of servers.

sukuriant 14y ago
Why not?
bmelton 14y ago
Sukuriant's point might be terse, but it's not invalid either. In 20 years' time, I'm pretty sure that the next iteration of us will be saying something like "When they phased out HTTP, there were only a few billion servers..." Better will always make its way in, even if there are entrenched systems running the 'old faithful' code already out there. IE6 is being phased out, yes, VERY slowly, but we're already way closer to getting it down to an irrelevant number than we would be if there weren't pushes. Will we ever get rid of it completely? Maybe not. I'm sure there's still a gopher server out there somewhere or another, and it's not that uncommon to get Telnet access to some commodity (crappy) web hosts, but SSH is pervasive and good, and we're all better off for it.
jfoutz 14y ago
Who won't support spdy by the end of the year? The pace of protocol obsolescence is increasing. I can't find stats about how much http traffic facebook is responsible for, but i'd bet it's at least 1% Governments and home grown enterprise apps are my guess about who's late to the party.
- mooism2 14y ago
  El cheapo web hosting will be late to the party. TLS certificates will be a stumbling block.
jasonwatkinspdx 14y ago
The factors that caused slow IP4 to IP6 transition are cited as examples by the author you are criticizing. It's very clear they are aware of the scale of http deployment as well as the challenges that brings. In fact, that key point motivates most of their objections to the current standards proposals.

dhruvbird 14y ago

Excellent judge of the protocol. It's interesting that no one is trying to solve the cookie bloat problem with http.

borlak 14y ago

load balancers aren't meant to just be "HTTP routers". they can definitely be used as such for smaller applications and do a good job at it, but a real load balancer needs to be quite complex, being able to adapt to the underlying applications that make use of it. if your goal is to only route HTTP requests, then you're only solving the first step of an increasingly complicated field of computer science (namely, web applications). Cookies aren't going to go away. if you want to improve the protocol to deal with cookies better, that makes sense, but acting like they are some kind of evil on the internet that should be forgotten isn't going to work. it's a bit self-defeating to argue that some protocols failed because of failure to provide new benefits and then argue against Cookies in HTTP!

bickfordb 14y ago
Although I think he should further consider why many websites are using cookies at all or even cryptographically secure cookies, I think he makes a good point about caching and cookies. Websites with large amounts of traffic often move their static content to separate domains to avoid clients sending cookies headers on every request to static/forever-cacheable assets.
nl 14y ago
load balancers aren't meant to just be "HTTP routers". they can definitely be used as such for smaller applications and do a good job at it, but a real load balancer needs to be quite complex, being able to adapt to the underlying applications that make use of it I think Poul-Henning Kamp is fairly well qualified to discuss what load balancers do. (And yes, load balancer are fundamentally HTTP Routers. Yes, sometimes they do content manipulation etc, but all those features are add-ons to the basic use-case)
- youngtaff 14y ago
  Perhaps Kamp is qualified to discuss what some 'load-balancers' do... The proxy developers have always had their doubts about SPDY (you can see them when @mnot first proposed it as a starting point) Terminating SPDY at the HTTP Router makes a lot of sense architecturally but I know some orgs don't terminate SSL at the load-balancer due to the licensing costs. Ultimately we need load-balancing options and someone to develop the opensource proxies (haproxy, varnish etc.) into more sophisticated offerings. Perhaps they'll be a SPDY module for Traffic Server

ajankovic 14y ago

In my opinion it is always a good idea in these kind of situations to set a goal we should strive to. I wonder what the ideal web protocol would look like. If, for example, we didn't have a burden of billions of servers and Internet reliance on HTTP/1.x protocol. What would be the most ideal solution that would suite emerging use cases for the Web? Are there any research papers on this topic?

squidi 14y ago
I agree with this way of thinking. phkamp has come up with some innovative ideas, but if it doesn't come easy for the rest of us there are lots of techniques to help e.g. asking, "if I was superman and could implement any solution, what would it be" http://personalexcellence.co/blog/25-brainstorming-techniques/ http://personalexcellence.co/blog/25-brainstorming-technique...

pilif 14y ago

While in general I understand where he is coming from, I believe his main argument about adoption is flawed. What do you think is more likely going to be adopted? A protocol that's not backwards compatible at all (heck, it even throws out cookies) or something that works over the existing protocol, negotiating extended support and then switching to that while continuing to work the exact same way for both old clients and the applications running behind it? See SPDY which is a candidate for becoming HTTP 2.0. People are ALREADY running that or at least eager to try it. I don't think for a second that SPDY is having the adoption problems of ipv6, SNI issues aside. Even if native sessions would be a cool feature, how many years do you believe it takes before something like that can be reliably used? We're still wary of supporting stuff that a 11 years old browser didn't support.

nl 14y ago
I don't think for a second that SPDY is having the adoption problems of ipv6, SNI issues aside. Google being able to modify both the client (Chrome) as well as few fairly significant server installations has kind of helped there a little bit...
- obtu 14y ago
  The key difference with IPv6 is that SPDY is opaque to routers and only needs changes at the endpoints. IPv6 has the chicken-and-egg problem that it only becomes usable once everyone (hosters, ISPs, core networks, home networks, web frameworks, client devices) has deployed it, and most of those actors don't bother with things that bring no immediate benefit.
  - phkamp 14y ago
    You do know that when IPv6 roll-out was initially contemplated, it would happen on IPv4-convertible addresses only, until everybody had converted. That would have allowed seamless IPv6/IPv4 interop and made migration a breeze. Lacking any "must have" features, nobody could see a reason to bother. Now we have to do the migration without the benefit of seamless conversion.

NHQ 14y ago

"We have also learned that a protocol which delivers the goods can replace all competition in virtually no time." This is an argument for websockets eating protocol lunch.

rdsubhas 14y ago

(These are just my opinions as web developer. Please feel free to downvote if my expectations are wrong) While I can completely agree with the technical merits of this proposal, there are some very two-faced statements. Author begins by pointing out the painfulness of IPv4 to IPv6, says that the next HTTP upgrade should be humble. But then proceeds to kill cookies and remove all the architectural problems in HTTP. Isn't that the same what IPv6 was? Wouldn't such an approach produce the same amount of pain to the implementors (that is us, the web developers)? Any upgrade will certainly have some backward-incompatible changes. But if it is totally backward incompatible, I don't understand why it still needs to be called HTTP. Couldn't we just call it SPDY v2 instead, or some other fancy name? Cookies are a problem. But the safest way to solve that problem is in isolation. Try to come up with some separate protocol extension, see if it works out, throw it away if it doesn't. But why marry the entire future of HTTP with such a do-or-die change? I blindly agree with the author that SPDY is architecturally flawed. But why is it being advocated in such big numbers? Even Facebook (deeply at war with Google) is embracing it. It's because SPDY doesn't break existing applications. Just install mod_spdy to get started. But removing cookies? What happens to the millions of web apps deployed today, which have $COOKIE and set_cookie statements everywhere in the code? How do I branch them out and serve separate versions of the same application, one for HTTP/1.1 and another for HTTP/2.0? More doubts keep coming... Problem with SPDY compressing HTTP headers? Use SPDY only for communication over the internet. Within the server's data center, or within the client's organization - keep serving normal HTTP. There are no bandwidth problems within there. Just make Varnish and the target server speak via SPDY, that is where the real gains are. I could go on. I'm not trying to say that the author's suggestions are wrong. They are important and technically good. But the way they should be taken up and implemented, without pain to us developers, doesn't have to be HTTP/2.0. Good ideas don't need to be forced down others throats.

phkamp 14y ago
No. Ipv6 is Ipv4 with bigger addresses, it didn't try to solve any of the other problems of IPv4 (or attempts to solve them were killed by ISPs). One example was multihoming (having more than one ISP) serveral smart proposals were floated (anycast, nearcast etc) but they were killed by ISP's who protected a lucrative business. If Ipv6 had made multi-ISP multihoming possible without all the trouble of BGP, business would have killed to get it back in the late 1990ies. Cookies only disappear from the wire, they are trivial to simulate on your server (see my other reply here).

bifrost 14y ago

Frankly many problems addressed by these proposals are not things that should be solved in a protocol like HTTP. Do you want every HTTP request to be slowed down with a crypto token exchange and verification? unlikely. In special cases you definately want it, but all the time? Absolutely not.

obtu 14y ago
It may be counter-intuitive, but with the right cyphers, the crypto costs of SSL and SPDY are negligible.
- phkamp 14y ago
  No, they are not. For one thing you have to terminate all your SSL on your loadbalancer in order to distribute the traffic. That makes SPDY a no-go for web-hotels/web-hosting where each customer has their own certificate. Second, there are perfectly valid legally mandated circumstances which forbid end-to-end privacy, from children in schools to inmates in jail and patients in psych. hospitals, not to mention corporate firewalls and the monster that looks out for classified docs not leaking out of CIA.
  - obtu 14y ago
    I was talking about performance, but: SNI addresses your first point. For the second point, whoever needs to enforce snooping already has to deal with SSL.
  - comex 14y ago
    > No, they are not. For one thing you have to terminate all your SSL on your loadbalancer in order to distribute the traffic. That makes SPDY a no-go for web-hotels/web-hosting where each customer has their own certificate. That's what SNI is for. > Second, there are perfectly valid legally mandated circumstances which forbid end-to-end privacy Then install spyware on the user's computer, or add a trusted SSL key and mitm all the things.
- mcbridematt 14y ago

jorangreef 14y ago

"Overall, I find all three proposals are focused on solving yesteryears problems, rather than on creating a protocol that stands a chance to last us the next 20 years."

regularfry 14y ago

How protocol adoption happens, in three steps: 1. Working code. 2. Publicity. 3. Ubiquity. That's it. Kamp is making a lot of the right noises here, but he's already lost ground to SPDY just because they've shipped code. No amount of sitting round tables bashing out the finer details of a better spec will help as much as getting code written - even if it's just a placeholder for an extensible spec, as long as that placeholder does something useful.

gioele 14y ago
> he's already lost ground to SPDY just because they've shipped code. Is SPDY good enough now and fixable enough in the future to leverage this gained ground? As more and more people deploy SPDY, they will understand its problems and there will almost no chances to change it, differently from what Google did at the beginning (they have gone through 1 big change and many small changes in the protocol). When the authors of the main servers (varnish, apache, nginx) will start feeling its limits, will they have to keep it around just for sake of compatibility? Please note that SPDY on the server is not a requirement, just an opportunity. Nothing will change for your users if you want to remove support for SPDY in your server after you deployed it and used it for some time. There are no "http+spdy://example.org" URL around, so supporting HTTP only will always be sufficient. Maybe not as much performant as SPDY but 100% supported.

mike-cardwell 14y ago

I can think of a few organisations that benefit financially, a great amount from the fact that cookies allow them to track users. Ad companies like Google and Microsoft spring to mind. The same organisations building our major browsers... Conflict of interest? You better believe it.

SeoxyS 14y ago
Imagine if browsers reported a user-resettable uuid sent over on each connection open. Advertisers get the benefit of being able to track users, do it more securely since no data is ever stored on the client, and must eat the cost of actually storing the key-value pairs, instead of passing it down to users and expending massive amounts of bandwidth bouncing the data back and forth. It's not about killing cookies, it's about finding a better solution that solves the problem better. Note: I work for a startup that "benefits financially" from tracking users—a feature without which we would not have a business.
- sirclueless 14y ago
  This already exists in spirit, in the form of "Incognito" or "Private" browsing. This doesn't solve the protocol issues of cookie bandwidth and insecure storage, but it does provide a user-resettable store of cookies.

rickmb 14y ago

After the whole IPv6 story, I'm surprised the author ignores the political dimension of designing a new protocol. As Mitch Kapor said "Architecture is Politics". It's not just about what solution is best from a technological perspective, it's about what we want our future to look like. The internet has become way more important than back when these protocols first became standard, and every time a protocol or standard is up for debate, political and commercial forces try to influence it in their favor. Some of the concepts they tried to shove into IPv6 were downright evil, and would have killed the internet as we know it. Personally, I'm relieved all that is left is a small, un-sexy improvement which albeit slowly, will eventually spread and solve the only really critical problem we have with IPv4. I really dread subjecting HTTP to that process. Although I fully agree with the author's critique of cookies for instance, the idea of replacing them with something "better" frankly scares the crap out of me. Especially when the word "identity" is being used. You just know what kind of suggestions some powerful parties will come up with if you open this up for debate, and fighting that will take up all of the energy that should be put towards improving what we already have. As techies we should learn to accept design flaws and slow adoption and look at the bigger picture of the social and political impact of technology: HTTP may be flawed, but things could be way, way worse.

neya 14y ago
This is a well thought, excellent comment that perfectly makes sense. Thank you. When huge corporations track you as how they are tracking you currently, most of the times, they don't get to know who you really are (except Facebook, because they know 'you'). That is, your identity. That's the only level of control you have over your anonymity. And these guys are proposing a new protocol just to remove that. Ridiculous.
- maggit 14y ago
  I suspect you are reading too much into the word "identity". It is just a matter of identifying the endpoint, so the HTTP router can delegate requests from one user to the same backend instance in any given session. This, in turn, will give a user a consistent image of the state at the server independently of cache propagation in the backend. For example. You don't need the user's real life identity to do this, and you don't need the same identity across different sites or across different sessions. This is currently sometimes done by cookies, which makes life difficult for HTTP routers. He is proposing a mechanism to keep the identifying-part while getting rid of problems in the HTTP router layer. The way I read this, it seemed to be without introducing additional privacy concerns and in fact removing some. (Cookies can carry more than identity) There is a detailed technical discussion to be had about implementing all of this, and in this discussion any privacy concerns would become visible and open for discussion. But I think it is a leap to say that the comments in TFA would necessarily make for a world bereft of privacy ;)

moe 14y ago

I have a lot of respect for phkamp, varnish is an impressive piece of engineering. I disagree with the stab he takes at cookie-sessions here, though. He seems to ignore that sessions are not only about identity but also about state. Servers should be stateless, therefor client-sessions (crypt+signed) are usually preferable over server-sessions. Having a few more bytes of cookie-payload is normally an order of magnitude cheaper (in terms of latency) than performing the respective lookups server-side for every request. Very low bandwidth links might disagree, but that's a corner-case and with cookies we always have the choice. Removing cookies in favor of a "client-id" would effectively remove the session-pattern that has proven optimal for the vast majority of websites.

phkamp 14y ago
No, servers should hold _all the state_ and clients none. If I add something to my shopping basket from my mobile phone, I want to be able to add more from my browser, rather than have two shopping baskets. Servers storing stuff on the clients is just plain wrong, and it is wrong from every single angle you can view it: Its wrong from a privacy point of view, it's wrong from a cost-allocation point of view, it's wrong from an architecture point of view and it's wrong from a protocol point of view. But it was a quick hack to add to HTTP in a hurry back in the dotcom days. It must die now.
- duckharp 14y ago
  But what about preferences for anonymous users? Store that on the server side? Append them to the URL? Both options kinda suck. Also, consider dabblet. The way it allows you to store your stuff using github is very smart IMHO.
  - phkamp 14y ago
    Store it on the server: The user-agent gives you a session-id to use as key. It may be that session-keys should tell if they are anonymous or if they represent (locally) authenticated users, but that's a very complex subject I won't claim to have a clear opinion of yet.

sophacles 14y ago

One huge benefit I think people are missing about the notion of a protocol standard session mechanism, is that we can use HTTP auth much easier, and perhaps get away from this notion of every site having to redo the login process. Browsers can handle the "remember login" settings, and logging out is as simple as tab closing. No "remember this computer" or "don't remember this computer" checkbox confusion. No random sites saying "remember me always" and requiring manual logout on a borrowed computer. It certainly helps with the password wallet concept too. Sure all that stuff has become semi-standard as it currently exists, but it is ugly, hacky, and sometimes doesn't work, and other times opens doors for hilarious malfeasance.