8 ms·

The whole point of this article is that performant Wireguard-over-TCP support in Wireguard simply does not work. You're not fighting the prevalence of an idea,

The whole point of this article is that performant Wireguard-over-TCP support in Wireguard simply does not work. You're not fighting the prevalence of an idea, you're fighting an inherent behavior of the system as currently constituted. In more detail, let's imagine we make a Wireguard-over-TCP tunnel. The "outer" TCP connection carrying the Wireguard tunnel is, well, a TCP connection. So Wireguard can't stop the connection from retransmitting. Likewise, any "inner" TCP connections routed through the Wireguard tunnel are plain-vanilla TCP connections; Wireguard cannot stop them from retransmitting, either. The retransmit-in-retransmit behavior is precisely the issue. So, what could we possibly do about this? Well, Wireguard certainly cannot modify the inner TCP connections (because then it wouldn't be providing a tunnel). Could it work with a modified outer TCP connection? Maybe---perhaps Wireguard could implement a user-space "TCP" stack that sends syntactically valid TCP segments but never retransmits, then run that on both ends of the connection. In essence, UDP masquerading as TCP. But there's no guarantee that this faux-TCP connection wouldn't break in weird ways because the network (especially, as you've discovered, any cloud provider's network!) isn't just a dumb pipe: middleboxes, for example, expect TCP to behave like TCP. Good news (and oops), it looks like I've just accidentally described phantun (and maybe other solutions): https://github.com/dndx/phantun https://github.com/dndx/phantun I'd be curious if this manages to sidestep the issues you're seeing with AWS and OVH.

deleted 2y ago
[deleted]
aarmenaa 2y ago
> The retransmit-in-retransmit behavior is precisely the issue. But you're concerned about an issue I do not have. In practice retransmits are rare between my endpoints, and if they did occur poor performance is acceptable for some period of time. I just need it to me fast most of the time. To reiterate: I do not care about what happens in non-optimal conditions. > it looks like I've just accidentally described phantun (and maybe other solutions): https://github.com/dndx/phantun https://github.com/dndx/phantun I'll definitely look into that. They specifically mention being more performant than udp2raw, so that's nice.
- lxgr 2y ago
  > In practice retransmits are rare between my endpoints You seem to be mistaken about how (most) TCP implementations work. They regularly trigger packet loss and retransmissions as part of their mechanism to determine the optimal transmission rate over an entire path (made up of potentially multiple point-to-point connections with dynamically varying capacity). That mechanism breaks down horribly when using TCP-over-TCP.
  - mlyle 2y ago
    But, my connection speed is usually greater and my loss is much less to my VPN endpoint than to whatever services I am accessing though that endpoint. As a result it doesn't affect things much. Further, accessing it with UDP is not always possible.
    - lxgr 2y ago
      > [...] my loss is much less [...] Unless it's actually zero, any loss on the "outer" TCP stream will cause a retransmission, visible to the inner one as a sharp jump in latency of all data following the loss. Most TCP stacks don't handle that very well either.
      - mlyle 2y ago
        Sure, even when outer loss is pretty close to zero, it's conceptually not great. On the other hand, I get 400mbps over TCP-over-TCP connections, and can't connect in any other reasonable way. 400mbps > 0. Even tunneling in UDP is not great due to MTU effects.
  - screcth 2y ago
    Can't the tunneling software detect when the upper TCP is retransmitting segments and drop them? That would give the lower TCP enough time to transmit the original segment.
    - lxgr 2y ago

deleted 2y ago

[deleted]

aarmenaa 2y ago

> The retransmit-in-retransmit behavior is precisely the issue. But you're concerned about an issue I do not have. In practice retransmits are rare between my endpoints, and if they did occur poor performance is acceptable for some period of time. I just need it to me fast most of the time. To reiterate: I do not care about what happens in non-optimal conditions. > it looks like I've just accidentally described phantun (and maybe other solutions): https://github.com/dndx/phantun https://github.com/dndx/phantun I'll definitely look into that. They specifically mention being more performant than udp2raw, so that's nice.

lxgr 2y ago
> In practice retransmits are rare between my endpoints You seem to be mistaken about how (most) TCP implementations work. They regularly trigger packet loss and retransmissions as part of their mechanism to determine the optimal transmission rate over an entire path (made up of potentially multiple point-to-point connections with dynamically varying capacity). That mechanism breaks down horribly when using TCP-over-TCP.
- mlyle 2y ago
  But, my connection speed is usually greater and my loss is much less to my VPN endpoint than to whatever services I am accessing though that endpoint. As a result it doesn't affect things much. Further, accessing it with UDP is not always possible.
  - lxgr 2y ago
    > [...] my loss is much less [...] Unless it's actually zero, any loss on the "outer" TCP stream will cause a retransmission, visible to the inner one as a sharp jump in latency of all data following the loss. Most TCP stacks don't handle that very well either.
    - mlyle 2y ago
      Sure, even when outer loss is pretty close to zero, it's conceptually not great. On the other hand, I get 400mbps over TCP-over-TCP connections, and can't connect in any other reasonable way. 400mbps > 0. Even tunneling in UDP is not great due to MTU effects.
- screcth 2y ago
  Can't the tunneling software detect when the upper TCP is retransmitting segments and drop them? That would give the lower TCP enough time to transmit the original segment.
  - lxgr 2y ago