7 ms·
TCP Sucks
- deleted 14y ago[deleted]
- krtcut 14y agoThis discussion just shows how change (even just profiling something) gets stalled. Lots of speculation, and then arguments about speculation. Meanwhile... Bram Cohen is lucky because he has a platform, his own platform, that he can use uTP on and he doesn't need anyone's approval. It's working so far ("mature") and that's that. I just wish he'd have given some other protocols a try, a little benchmarking exercise that we could in turn replicate ourselves, so we could see how they compare. That said, I admire the irreverence. We need more of that. As for these arguments, it's like when VJ "saved the internet", if it works, then we're done. Miller Time. Instead here we have a looped version of "Less Filling, Tastes Great." You want dropped packets? Use UDP. Is that acceptable? Acceptable to who? All depends on what you're trying to do. What you're trying to do, not what someone else is trying to do. Don't wait for approval. It may never come. You just get endless debate. If you got something that works, use it.
- caf 14y agoThe solution is for the end user to intervene, and tell all their applications to not be such pigs, and use uTP instead of TCP. Then they’ll have the same transfer rates they started out with, plus have low latency when browsing the web and teleconferencing, and not screw up their ISP when they’re doing bulk data transfers. That still doesn't address the problem when you have many users behind the same queue, some of whom care only about throughput and not latency. You need a scheme which will work when all of those users are acting selfishly.
- bramcohen 14y agoThey can have their throughput without running a denial of service on their net connection. If you assume that they'll DOS themselves for the fun of it anyway, then I can't help you.
- caf 14y agoFor a user that cares only about throughput and not about latency, it's not a DOS.
- paulsutter 14y agoBacking a queue up to seconds of depth doesn't increase throughput. Most of the bandwidth contention is at the edge, right at your DSL connection, so any battle among network connections is a battle among your own usage at any given moment. Separately, bittorrent is not a latency-sensitive app, it's throughput sensitive, and uTP was designed for bittorent.
- scott_s 14y agoMy thought was more fundamental than that: any solution which involves asking users to request different transport protocols is not going to solve the problem. There are far more users who have no idea what a "transport protocol" is than those who do. With that said, I enjoyed the post. It's an interesting problem, and I do find the base idea attractive: allowing applications to opt to be background traffic.
- nkohari 14y agoIt wouldn't (at least in my understanding) be the user that would choose, it would be the application. WoW for example would optimize for latency, whereas BitTorrent would optimize for throughput.
- scott_s 14y agoCorrect, but Bram's argument (as I understand it) was that the users would put pressure on the application developers to opt to be background traffic.
- nkohari 14y agoTwo nearly-universal truths about users that suggest solutions: they're both rare, and usually wrong. :) It's more likely that users will state the problem -- such as, "I want to be able to run WoW and BitTorrent at the same time." From there, the software developers would determine the solution (optimization for latency vs. throughput).
- shaggyfrog 14y ago> Of course, I’ve always used TCP using exactly the API it provides, and even before I understood how TCP worked under the hood gone through great pains to use the minimum number of TCP connections used to the number which will reliably saturate the net connection and provide good piece diffusion. BitTorrent must not not have any books on copyediting.
- thespin 14y agoI'm surprised he doesn't mention CurveCP. He's taken ideas from that author before (e.g. netstrings, which you'll find in the .torrent file format). TCP does suck. If you try to use it for lots of short lived connections. And that pretty much sums up how it's being used nowadays, most the time. For single, long term connections, TCP is fine.
- dfc 14y agoIs "from that author" some sort of passive aggressive jab at djb?
- thespin 14y agoNope. Quite the contrary. I was trying to avoid bringing the strange sentiment to which you are referring to the fore. Mere mention of those initials causes an adverse reaction in a lot of developers and admins. I voluntarily use his software every day. Not perfect (nothing is) but the best there is available. I find his code to be wonderfully lucid if I read it slowly and carefully. My opinion.
- dfc 14y agoI also have a lot of respect for djb. That is why I thought it was strange you did not use his name when you were clearly well acquainted with who the author of netstrings and curvecp is.
- bramcohen 14y agouTP isn't based on CurveCP, and CurveCP is nowhere near as mature as uTP is.
- thespin 14y agoPerhaps I wasn't clear. What I meant was that I would have expected you to have looked at CCP and evaluated it for your purposes, as you did with RED.
- 14y ago
- stephenbez 14y agoIf anyone is curious what uTP is, you can find the protocol defined here: http://bittorrent.org/beps/bep_0029.html http://bittorrent.org/beps/bep_0029.html
- dfc 14y agoIs the shout-out the mere mentioning of bittorrent? Or does nick weaver elude to bram some other way in the full article?
- bramcohen 14y agoWho do you think made the development he's referring to happen?
- dfc 14y agoI obviously know who is behind bittorrent that is why I mentioned it as a possible alternative. But you yourself state that Stanislav Shulanov is the man behind utp. More importantly shout-outs traditionally explicitly reference the person/thing in question. I was not sure if there was a longer/muddier/more-controversial back story that would not be appropriate to discuss in an acm article.
- Alind 14y agoUB alumni . Hi Bram
- jandrewrogers 14y agoDesigning networking protocols that are robust in mathematical sense is unbelievably difficult. In fact, we humans have only found optimal solutions in a few cases if you dig through the mathematics literature. Many real-world networking protocol design scenarios do not have a known non-pathological implementation. Furthermore, there is a large number of decentralized protocol designs that we can prove to have many poor qualities. To bludgeon the equine, people that can significantly advance our understanding of such things tend to win Nobel prizes and similar. It is that difficult. That said, TCP is not the best we can design given everything we know about designing network protocols. It was good enough for the people that designed it at the time, and possibly (my chronology is fuzzy) was approximately as good as the mathematics would have reasonably allowed when it was developed. We can make it work well enough in many cases -- the economics of inertia. Other narrow use cases are better solved differently but are not general solutions. It is one of those problems that sounds like it should be easy to solve on the surface but turns into a bloody epic challenge once you start to dig into it. I am not offering a solution, just noting that very few people can.
- bramcohen 14y agoDid you even read the article?
- scott_s 14y agoI don't think his comment in any way contradicts your post. I thought it added to it; his point is that not only are good protocols hard in practice (which you covered), they are theoretically hard (which you did not cover, hence adding to your point).
- paulsutter 14y agoOne thing you have to realize about Hacker News is that people can accumulate pretty good karma points for postings that have a definitive/authoritative tone blended with some truisms. Not to bludgeon the equine or anything, but I think people find them comforting.
- paulsutter 14y ago
- endymi0n 14y agoHaving uTP running against UTP as an alternative network connection means is a rather unfortunate naming - I imagine a lot of people getting confused to the max, especially as it's pronounced the same. So pretty pretty please: Give the protocol a GOOD name first, then we're talking business! ;-)
- nuje 14y agoIt seems to me that on the IP level the net has been in a technological paralysis for some time. We can't get RED or IPv6 deployed, and and the IETF doesn't seem to get anything useful happen these days. edit: anyone else remember when layer 3 had a bright future ahead of it, IPv6 and end-to-end IPSec (with keys in the DNS) were just around the corner...
- jsprinkles 14y agoToo big to overhaul?
- bramcohen 14y agouTP carries the bulk of all BitTorrent transfers at this point. This would seem to imply a certain level of success.
- hristov 14y agoShameless plug: Extremetcp.com is the solution to the congestion problems of TCP. The best part of ExtremeTCP is that it is not a new protocol. It is TCP. It just uses clever algorithms at the sender side to send data while avoiding congestion. (Since TCP does not actually specify which algorithms one should use as long as one avoids congestion, ExtremeTCP is a perfectly legal version of TCP). Yes, I am involved with this. If interested in testing, please send an email to the contact address in the website.
- paulsutter 14y agoThere are many implementations of improved congestion control for TCP, several of which are implemented in the Linux kernel. It's trivially easy to modify congestion control to get arbitrarily fast performance in high bandwidth-delay environments. I can tell you from experience that implementing fast performance in extremely lossy environments is harder. And hardest of all is to come up with a solution that works on a network that is shared with the common congestion control implementations, and that works with billions of end nodes. I spent years working on this. Feel free to reach out to me on my email address, I'd love to share with you my experience of commercializing such a product.
- geoffschmidt 14y agoIt's hard to evaluate your claim that ExtremeTCP is "the" solution, given that your website (1) does not compare your solution to the significant amount of literature and prior art in this space (eg, TCP Vegas, which is already implemented in the Linux kernel), (2) doesn't make any claims about friendliness to TCP Reno, which is the hardest part of retrofitting a new congestion control strategy onto the public Internet.
- hristov 14y agoGood points. We do have comparisons with most existing TCP solutions out there (at least the ones we were able to get a hold of). I did not put them up because I did not want to drown people in information. But I should put up a white paper that has more detailed test results. By the way we do beat the the Linux TCP stack (which now uses Cubic TCP, afaik). The data shown on the site is Compound TCP, which is the current windows implementation. Let me know if you would like to see the detailed test results.
- viraptor 14y agoOk, maybe I'm missing something, but reading the article I see some weird ideas: RED is hard to deploy, so let's change the base protocol instead. - how does that make sense? Everyone would have to start using new libraries and for backward compatibility we'd have to preserve the tcp layer too. That means standards like http would have to get extensions to use SRV records or suffer delays while utp availability is probed. There's also a complaint that RED will drop packets once the queue is full. I don't get that at all - it will always happen... In addition I get an impression there is some tension/implied superiority between us (people doing uTP) and them (ones doing RED). Why does it look so ugly? There's a known problem, there's an interesting solution for new software (uTP) and some plan to migrate old protocols transparently (RED). When did that turn into some bizarre conflict and why?
- moultano 14y ago> When did that turn into some bizarre conflict and why? Man you weren't kidding. I tried looking up uTP on wikipedia hoping to come away with some technical understanding. It's full of passive aggressive statements that cite forum posts as their support, with no information on how it actually works. Maybe some of the folks in this thread could go fix that.
- bramcohen 14y agoThere's some detailed writeups of the algorithms in the LEDBAT documentation. As for commentary, that's distinctly lacking, and my post is filling in some of the gap.
- bramcohen 14y agoBitTorrent is using uTP just fine, which is only, you know, most of the upload from consumer internet connections, and we're working on getting the same things crammed into TCP with LEDBAT, but that's a slow process. I wasn't complaining about RED dropping packets, just describing how it works. As for the tension, my point is that my solution works and the other one doesn't. If you want to know why the person I quoted was being such a dismissive jerk, you'll have to ask him.
- volatile 14y agoThe author seems to claim that is is implausible for a router vendor to sell a router that drops more packets. The marketing plan is that the because router vendors are unwilling to say ‘has less memory!’ as a marketing tactic, maybe they’d be willing to say ‘drops more packets!’ instead. That seems implausible. Yet he concludes by suggesting the router should drop all the packets. The best way to solve that is for a router to notice when the queue has too much data in it for too long, and respond by summarily dropping all data in the queue. /snip/ Of course, I’ve never seen that proposed anywhere… Based on his earlier reasoning, that would also be implausible.
- bramcohen 14y agoThat's what you would do IF you were going to be serious about making the router drop packets in a way which actually helps. I don't expect it to happen any time soon.
- josefonseca 14y agoI think the catchy title was meant to grab attention to an important present day issue. But TCP actually does not suck, it's been there for longer than I have and served us pretty darned well up until now. Never forget that when the TCP protocol was designed, the biggest concern we had was that a nuke would land on top of our heads at any minute and the network should keep working. Also, the "Internet" was thought to be a small niche network of networks among the military and academics. I guess this is all well known, it's just my reaction to the editorialized title.
- msbhvn 14y agoJust to be clear, "TCP Sucks", despite successfully running the majority of the global Internet traffic. "TCP Sucks" so bad we're basically going to copy a lot of it: window based congestion control, SACK, timestamps, ability to add new options, etc. "TCP Sucks" because it is not perfect and has an issue, an issue that requires router / switch upgrades. We're going to fix that by breaking backward compatibility with _tons_ of applications and requiring an OS update on _every_ client and/or application. All this assuming our relatively new and unproven thing is as good as TCP in all other ways and fixes this issue of TCP perfectly. Hmmm. Me thinks that TCP does not suck so much.
- bramcohen 14y agoDid you read the article or just the headline?
- msbhvn 14y agoI did read it, guess I just got too caught up in the title to brush it off (looks like some others here also wondered about that). Anyway, uTP looks cool, LEDBAT sounds very interesting and BitTorrent is of course, completely awesome. I just don't think TCP sucks. I'm constantly surprised at how well it works for how simple most of it is and how complicated and intractable the rest is.
- romaniv 14y agoSounds like "it kind of worked so far, so let's use it forever, with multiple layers of band-aids if necessary". Besides, unles I'm missing something, the two protocols can be used side-by-side, I.e. you can slowly phase one out by the other where necessary.