greaber

6 ms·

1.
by greaber 5d ago
As for speed, see https://greaber.github.io/syq-bench/rclone.html . As for other stuff, it depends what you use rclone for.
2.
by greaber 5d ago
I did some benchmarks against rclone today. https://greaber.github.io/syq-bench/rclone.html Syq was sometimes much faster and never more than a few percent slower. I didn't do any tuning of syq since it is meant t
3.
by greaber 6d ago
Syq has compression turned on by default. Also, if you are updating a file that already exists on the other end, syq will only send blocks of the file that changed.
4.
by greaber 6d ago
Also, in syq, a connection is the unit of independent transfer work. Probably it would be possible to get higher bandwidth per TCP connection by some kind of multiplexing, but that architecture would be more complex and probably not have ma
5.
by greaber 6d ago
They are benchmarks on real hardware, and I tried to provide enough details so that anyone can reproduce them (but some of the benchmarks are on servers I have that you can't order on public cloud -- these ones are clearly marked). &qu
6.
by greaber 6d ago
Nice example. Rsync might be better here. Although syq will still avoid copying most of the image if changing the config file just modifies a small number of locations and doesn't globally move data around (like prepending a single byt
7.
by greaber 6d ago
Yeah, I kind of regret writing "better than rsync" in the tagline. Maybe I should have written "better than rsync in some respects".
8.
by greaber 6d ago
Yeah, I get that. I really didn't mean to sound conceited, and I don't think that syq is better than rsync for every use case. I was just trying to convey in a very short space that it is something that rsync users might be intere
9.
by greaber 6d ago
Thanks! Yeah, I was a little surprised (maybe I shouldn't have been) at how negative some of the comments were. I will think about what additional explanations I can provide that will help people. Regarding rclone, I have also spent a
10.
by greaber 6d ago
I entirely agree that multiple connections shouldn't actually be necessary for speed, but they help in multiple situations (long-distance transfers, same DC between servers, NFS), and it's not just about amortizing per-file setup
11.
by greaber 6d ago
Yeah, ControlMaster is cool and was an inspiration, but since the syq client is always talking to the same receiver on the server, we can save a few round trips that ControlMaster still needs, so syq persist feels faster. This architecture
12.
by greaber 6d ago
Currently, there is just a fixed range of ports that it will use if they are open and you don't pass `--no-tcp`. It does its own encryption over the TCP connections. Also worth knowing is that for some connections (e.g. if there are an
13.
by greaber 6d ago
It just installs in $HOME/.local/bin by default (and some other parts go in other locations inside your home directory). You don't need superuser access for anything. The first time you connect to a server, it installs its ma
14.
by greaber 6d ago
I'm seeing 5x speedups on my real workloads, and you can see some synthetic benchmarks here or run your own https://greaber.github.io/syq-bench/
15.
by greaber 6d ago
Thanks for the suggestion! I'm not very familiar with btrfs send and btrfs receive. Browsing the documentation, it looks cool, but maybe you could say more about the specific things that make it so much better for your use case? Could
16.
by greaber 6d ago
- No need to be snarky. It's a new project, while rsync is time tested, and that is certainly an advantage for rsync and all old software. - You can install by piping curl to bash, using brew, or by compiling yourself. One thing about
17.
by greaber 6d ago
There are a bunch of reasons, some of which are explained in the description and in the docs. For me, the most important benefit is speed of copying. Note that syq doesn't currently implement rsync's delta merge algorithm, which a
18.
Show HN: Syq – copy files between machines fast (better than rsync)(greaber.github.io)
41 points by greaber 6d ago|51 comments
19.
by greaber 1y ago
Doesn't this formula also punish circular trade patterns in a strange way, like if the US imports $X of good from country B and country B imports $X of goods from country C and country C imports $X of goods from the US, and there are n
20.
by greaber 6y ago
How does CapRover work with databases? Does it replace something like RDS?
21.
Piggly Wiggly (invention of self-service shopping)(en.wikipedia.org)
3 points by greaber 6y ago|0 comments