7 ms·

Isn't part of the idea with HTTP2 that the server can see everything you're asking for and more intelligently prioritize vs a bunch of separate independent conn

by TwoBit 6y ago

Isn't part of the idea with HTTP2 that the server can see everything you're asking for and more intelligently prioritize vs a bunch of separate independent connections?

CyberRabbi 6y ago
I’m not aware of any open source servers that implement such an optimization and if proprietary software exists that does, it’s trivial to implement custom logic to do that across multiple connections as well.
- jlokier 6y ago
  It's actually not trivial to optimize dynamic prioritisation across multiple connections. To keep a TCP efficient, the sending process needs to fill up the send buffer to a reasonable amount, generally by writing data to the kernel with write()/send(). If the sending process doesn't do this, the sending TCP will not send full frames as fast as the network or receiver allows, and do so at a steady cadence. But to prioritise the data flow of multiple responses across multiple connections, the sender needs to pause existing TCP flows in progress as soon as it has higher priority response data ready to send if the higher priority TCP will send at full rate. Yet if the higher priority TCP is sending slower than full rate temporarily due to slow start, congestion control or receive window full, etc., the lower priority TCP needs to be resumed just enough to fill the gaps. It's not even possible to do this with the ordinary sockets API, so it's certainly not trivial. With HTTP/2 and TCP in kernel, it's still not optimal because prioritised response data will be queued behind data already in the TCP send buffer, but at least there's only one send buffer's worth to compete with rather than lots. With HTTP/3 and QUIC in userspace, adaptive prioritisation can be optimised further because the decision is made on every packet just before it leaves the machine. Per-packet prioritisation would be possible in theory with HTTP/1 and HTTP/2 by implementing TCP in userspace, or by implementing HTTP in kernelspace, in either case tightly coupling TCP and HTTP. I don't know of anyone doing it, because it would be a lot of work for a marginal gain.
  - CyberRabbi 6y ago
    The optimization proposed was not about packet prioritization but prioritizing assets to send.
    - jlokier 6y ago
      The optimization proposed was whatever HTTP/2 does: > "part of the idea with HTTP2 that the server can see everything you're asking for and more intelligently prioritize" HTTP/2 prioritisation does in fact optimize continuously by starting lower priority streams when higher priority data isn't yet available, then pausing streams in mid-send as other higher priority streams' data becomes available. The size of the send buffer makes a difference to how fast this can react, as I described. Details at Cloudflare: https://blog.cloudflare.com/http-2-prioritization-with-nginx/ https://blog.cloudflare.com/http-2-prioritization-with-nginx... > "if the most important response takes longer to generate than lower priority responses, the server may end up starting to send data for a lower priority response and then interrupt its stream when the higher priority response becomes available" > "the problem with large send buffers is that it limits the nimbleness of the server to adjust the data it is sending on a connection as high priority responses become available. Once the response data has been written into the TCP send buffer it is beyond the server’s control and has been committed to be delivered in the order it is written" You proposed to replace this with HTTP/1.1 and multiple TCPs, which does not provide an equivalent optimization. Browsers do in fact optimize HTTP/1.1 per asset by keeping a list of requests in priority order and running a limited number of TCP connections in parallel, but if they can use HTTP/2 that usually works out faster. On the internet for reasons described in the Cloudflare article, and also because giving as many requests to the server as possible up front allows the HTTP server to start fetching or generating lower priority assets sooner, hiding some backend latency - especially significant with load balancers, other reverse proxies, or HDD cold storage.

by TwoBit 6y ago

Isn't part of the idea with HTTP2 that the server can see everything you're asking for and more intelligently prioritize vs a bunch of separate independent connections?

CyberRabbi 6y ago

I’m not aware of any open source servers that implement such an optimization and if proprietary software exists that does, it’s trivial to implement custom logic to do that across multiple connections as well.

jlokier 6y ago
It's actually not trivial to optimize dynamic prioritisation across multiple connections. To keep a TCP efficient, the sending process needs to fill up the send buffer to a reasonable amount, generally by writing data to the kernel with write()/send(). If the sending process doesn't do this, the sending TCP will not send full frames as fast as the network or receiver allows, and do so at a steady cadence. But to prioritise the data flow of multiple responses across multiple connections, the sender needs to pause existing TCP flows in progress as soon as it has higher priority response data ready to send if the higher priority TCP will send at full rate. Yet if the higher priority TCP is sending slower than full rate temporarily due to slow start, congestion control or receive window full, etc., the lower priority TCP needs to be resumed just enough to fill the gaps. It's not even possible to do this with the ordinary sockets API, so it's certainly not trivial. With HTTP/2 and TCP in kernel, it's still not optimal because prioritised response data will be queued behind data already in the TCP send buffer, but at least there's only one send buffer's worth to compete with rather than lots. With HTTP/3 and QUIC in userspace, adaptive prioritisation can be optimised further because the decision is made on every packet just before it leaves the machine. Per-packet prioritisation would be possible in theory with HTTP/1 and HTTP/2 by implementing TCP in userspace, or by implementing HTTP in kernelspace, in either case tightly coupling TCP and HTTP. I don't know of anyone doing it, because it would be a lot of work for a marginal gain.
- CyberRabbi 6y ago
  The optimization proposed was not about packet prioritization but prioritizing assets to send.
  - jlokier 6y ago
    The optimization proposed was whatever HTTP/2 does: > "part of the idea with HTTP2 that the server can see everything you're asking for and more intelligently prioritize" HTTP/2 prioritisation does in fact optimize continuously by starting lower priority streams when higher priority data isn't yet available, then pausing streams in mid-send as other higher priority streams' data becomes available. The size of the send buffer makes a difference to how fast this can react, as I described. Details at Cloudflare: https://blog.cloudflare.com/http-2-prioritization-with-nginx/ https://blog.cloudflare.com/http-2-prioritization-with-nginx... > "if the most important response takes longer to generate than lower priority responses, the server may end up starting to send data for a lower priority response and then interrupt its stream when the higher priority response becomes available" > "the problem with large send buffers is that it limits the nimbleness of the server to adjust the data it is sending on a connection as high priority responses become available. Once the response data has been written into the TCP send buffer it is beyond the server’s control and has been committed to be delivered in the order it is written" You proposed to replace this with HTTP/1.1 and multiple TCPs, which does not provide an equivalent optimization. Browsers do in fact optimize HTTP/1.1 per asset by keeping a list of requests in priority order and running a limited number of TCP connections in parallel, but if they can use HTTP/2 that usually works out faster. On the internet for reasons described in the Cloudflare article, and also because giving as many requests to the server as possible up front allows the HTTP server to start fetching or generating lower priority assets sooner, hiding some backend latency - especially significant with load balancers, other reverse proxies, or HDD cold storage.