5 ms·

What prevents a more general solution like using @flagfiles instead of literally specifying them as arguments?

jeffbee 4y ago
I don't know. Judging from source blame, flagfile has been in gflags since 2007 at the latest, and 2007 is also when the command line length limit was lifted: https://github.com/torvalds/linux/commit/b6a2fea39318e43fee84fa7b0b90d68bed92d2ba https://github.com/torvalds/linux/commit/b6a2fea39318e43fee8...
- joshuamorton 4y ago
  My only guess is that it plays badly with borg run somehow but having the borg cli autogenerate a flagfile seems uncomplicated. Edit: Although there's arguably security implications to letting me push a file from my workstation, but that wouldn't have been an issue in 2006.
- the-rc 4y ago
  I remember 128KB being an issue past mid 2007, which is when that patch was merged. I think that the compression trick actually happened a bit after that. Looking at the patch, it appears that yes, it's one of the temporary hacks I was referring to. It did add some breathing room, because it no longer constrains the aggregate of argv plus the environment, but it keeps 128KB as the max size for each flag, which was still problematic for good old GFE and its service list. I can't remember if they managed to implement external flag files first or if they had to resort to --flag, --flag2, as web folks do nowadays to work around e.g. cookie limits (I'm looking at you, oauth2-proxy, embedding massive claims).
the-rc 4y ago
As far as I recall, until the plumbing was heavily redone a decade later or so, you could only push (propagate) an updated command line to replicas, not a separate file as well. On top of that, I vaguely remember something about bootstrapping and ensuring that the remaining flags and the flag file always be updated atomically, in lockstep. For regular Unix folks, imagine issuing a remote ssh command. You can't easily do that AND bundle an auxiliary data file at the same time.

I don't know. Judging from source blame, flagfile has been in gflags since 2007 at the latest, and 2007 is also when the command line length limit was lifted: https://github.com/torvalds/linux/commit/b6a2fea39318e43fee84fa7b0b90d68bed92d2ba https://github.com/torvalds/linux/commit/b6a2fea39318e43fee8...

joshuamorton 4y ago
My only guess is that it plays badly with borg run somehow but having the borg cli autogenerate a flagfile seems uncomplicated. Edit: Although there's arguably security implications to letting me push a file from my workstation, but that wouldn't have been an issue in 2006.
the-rc 4y ago
I remember 128KB being an issue past mid 2007, which is when that patch was merged. I think that the compression trick actually happened a bit after that. Looking at the patch, it appears that yes, it's one of the temporary hacks I was referring to. It did add some breathing room, because it no longer constrains the aggregate of argv plus the environment, but it keeps 128KB as the max size for each flag, which was still problematic for good old GFE and its service list. I can't remember if they managed to implement external flag files first or if they had to resort to --flag, --flag2, as web folks do nowadays to work around e.g. cookie limits (I'm looking at you, oauth2-proxy, embedding massive claims).

the-rc 4y ago

As far as I recall, until the plumbing was heavily redone a decade later or so, you could only push (propagate) an updated command line to replicas, not a separate file as well. On top of that, I vaguely remember something about bootstrapping and ensuring that the remaining flags and the flag file always be updated atomically, in lockstep. For regular Unix folks, imagine issuing a remote ssh command. You can't easily do that AND bundle an auxiliary data file at the same time.