6 ms·

Bufferbloat: Dark Buffers in the Internet

55 points by kpi 15y ago

SMrF 15y ago
I admit, though I found the topic fascinating I only had time to skim. But I did notice something which may be of interest to others: CeroWrt is a branch of OpenWrt geared towards fixing the buffer problem on your home network. So I'm guessing if you get this configured correctly you might see significant improvements in latency. I'm intrigued. I posted it as a separate topic here: http://news.ycombinator.com/item?id=3461790 http://news.ycombinator.com/item?id=3461790
pors 15y ago
Wow, just scrolling to the bottom if this article took ages. #sent2kindle
cypherpunks01 15y ago
Overall, does anyone have a feel for how close we might be to a congestion collapse on today's internet? I've been reading about this for a few years now and have obvious worries about real solutions not being implemented until after the storm hits. Or is this being overstated? Are there mechanisms in place that would mitigate or prevent large-scale congestion collapses of this kind?
- EricBurnett 15y ago
  The worldwide internet infrastructure is remarkably fragile. I can think of a few example off the top of my head where it's been abused or broken: 1. China 'borrowing' traffic: http://arstechnica.com/security/news/2010/11/how-china-swallowed-15-of-net-traffic-for-18-minutes.ars http://arstechnica.com/security/news/2010/11/how-china-swall... 2. Myriad undersea cable cuts slowing / segmenting the internet 3. Juniper routers going down: http://www.networkworld.com/news/2011/110711-internet-outage-252851.html http://www.networkworld.com/news/2011/110711-internet-outage... 4. There was an incident (2 years ago?) where an European ISP broadcast a low-cost route incorrectly and blackholed a large portion of the internet, although I can't find a link right now. With respect to buffers specifically, AT&T's network issues when the iPhone launched were largely self-inflicted by having overly large buffers: http://blogs.broughturner.com/2009/10/is-att-wireless-data-congestion-selfinflicted.html http://blogs.broughturner.com/2009/10/is-att-wireless-data-c... So yes, widespread issues are certainly possible. The internet is not nearly as resilient as people would like to believe.
jamesgeck0 15y ago
I've observed behavior similar to this from my cable ISP and chalked it up to malice. It's interesting that the problem is so widespread, but apparently not very well understood by so many providers.
- pja 15y ago
  Mobile data links are especially prone to this kind of madness in my experience. I quite often experience data links which sit at 4-5s ping times for extended periods.
jvehent 15y ago
Feel free to try this at home: http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:networking:traffic_control#a_word_about_bufferbloat http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:n...
Merrack 15y ago
Must be nice to have Vint Cerf on hand to consult on your networking issues!
jimrandomh 15y ago
Yep, I definitely see this from time to time, especially over mobile networks. Interestingly, most of my problems went away when I started using OpenVPN. It connects with UDP to my Amazon EC2 VM, which is close enough to the internet's core that everything after that is low-latency. I'm not sure why this works, but it does suggest a practical testing ground for bufferbloat-workaround strategies: you can't make the rest of the internet adopt your weird new protocol, but you can install it on a VM, and route all your traffic that way. As an added bonus, this gets you security on untrusted networks and a static IP. And you could potentially use this sort of trick to get a cool multi-homed connection, although there's no software for that currently. It'd be interesting to see what possibilities there are when combining a fast but bufferbloated or high-latency connection, with a slow but low-latency connection.

55 points by kpi 15y ago

SMrF 15y ago

I admit, though I found the topic fascinating I only had time to skim. But I did notice something which may be of interest to others: CeroWrt is a branch of OpenWrt geared towards fixing the buffer problem on your home network. So I'm guessing if you get this configured correctly you might see significant improvements in latency. I'm intrigued. I posted it as a separate topic here: http://news.ycombinator.com/item?id=3461790 http://news.ycombinator.com/item?id=3461790

pors 15y ago

Wow, just scrolling to the bottom if this article took ages. #sent2kindle

cypherpunks01 15y ago

Overall, does anyone have a feel for how close we might be to a congestion collapse on today's internet? I've been reading about this for a few years now and have obvious worries about real solutions not being implemented until after the storm hits. Or is this being overstated? Are there mechanisms in place that would mitigate or prevent large-scale congestion collapses of this kind?

EricBurnett 15y ago
The worldwide internet infrastructure is remarkably fragile. I can think of a few example off the top of my head where it's been abused or broken: 1. China 'borrowing' traffic: http://arstechnica.com/security/news/2010/11/how-china-swallowed-15-of-net-traffic-for-18-minutes.ars http://arstechnica.com/security/news/2010/11/how-china-swall... 2. Myriad undersea cable cuts slowing / segmenting the internet 3. Juniper routers going down: http://www.networkworld.com/news/2011/110711-internet-outage-252851.html http://www.networkworld.com/news/2011/110711-internet-outage... 4. There was an incident (2 years ago?) where an European ISP broadcast a low-cost route incorrectly and blackholed a large portion of the internet, although I can't find a link right now. With respect to buffers specifically, AT&T's network issues when the iPhone launched were largely self-inflicted by having overly large buffers: http://blogs.broughturner.com/2009/10/is-att-wireless-data-congestion-selfinflicted.html http://blogs.broughturner.com/2009/10/is-att-wireless-data-c... So yes, widespread issues are certainly possible. The internet is not nearly as resilient as people would like to believe.

jamesgeck0 15y ago

I've observed behavior similar to this from my cable ISP and chalked it up to malice. It's interesting that the problem is so widespread, but apparently not very well understood by so many providers.

pja 15y ago
Mobile data links are especially prone to this kind of madness in my experience. I quite often experience data links which sit at 4-5s ping times for extended periods.

jvehent 15y ago

Feel free to try this at home: http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:networking:traffic_control#a_word_about_bufferbloat http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:n...

Merrack 15y ago

Must be nice to have Vint Cerf on hand to consult on your networking issues!

jimrandomh 15y ago

Yep, I definitely see this from time to time, especially over mobile networks. Interestingly, most of my problems went away when I started using OpenVPN. It connects with UDP to my Amazon EC2 VM, which is close enough to the internet's core that everything after that is low-latency. I'm not sure why this works, but it does suggest a practical testing ground for bufferbloat-workaround strategies: you can't make the rest of the internet adopt your weird new protocol, but you can install it on a VM, and route all your traffic that way. As an added bonus, this gets you security on untrusted networks and a static IP. And you could potentially use this sort of trick to get a cool multi-homed connection, although there's no software for that currently. It'd be interesting to see what possibilities there are when combining a fast but bufferbloated or high-latency connection, with a slow but low-latency connection.