6 ms·

Syd the perhaps most sophisticated sandbox for Linux

55 points by hayali 2y ago

qbane 2y ago
I found this document hard to follow. Perhaps the repo is more informative: https://gitlab.exherbo.org/sydbox/sydbox https://gitlab.exherbo.org/sydbox/sydbox
- nolta 2y ago
  Also this tutorial: https://man.exherbolinux.org/sydtutorial.7.html https://man.exherbolinux.org/sydtutorial.7.html
maxmcd 2y ago
Huh. Does anyone know how it works without additional privileges? I ran syd as a normal user on my system and it blocked network calls. Does it fall back to ptrace? I don't think my user has the ability to create network namespaces. edit: Ah, ok. There's a little more here: https://gitlab.exherbo.org/sydbox/sydbox/-/blob/main/doc/toctou-or-gtfo.md?ref_type=heads https://gitlab.exherbo.org/sydbox/sydbox/-/blob/main/doc/toc.... And the readme does mention ptrace. Would be great to understand how to introspect what this is using and when. ie: in some cases I might not be able to tolerate the performance penalty of ptrace. edit2: ah ok wayyyy more info here: https://man.exherbolinux.org/syd.1.html https://man.exherbolinux.org/syd.1.html and benchmarks: https://man.exherbolinux.org/syd.1.html#BENCHMARKS https://man.exherbolinux.org/syd.1.html#BENCHMARKS So it is slower, but much faster than I would have expected. Wild.
quotemstr 2y ago
This program seems fragile and too eager to break longstanding system ABI --- for example, silently turning O_RDWR into O_WRONLY, prohibiting PROT_EXEC on memfds, and, well, > Syd also blocks executable+anonymous memory > In the operation of Syd, certain system calls are not fully emulated due to seccomp(2) limitations, resulting in the sandbox process continuing these calls directly. These include execve(2), execveat(2) for execution, chdir(2), fchdir(2) for directory changes, and open(2) operations with O_PATH flag. Consequently, this behavior exposes vulnerabilities to time-of-check to time-of-use attacks, allowing for the circumvention of Exec Sandboxing and Force Sandboxing to execute denylisted paths, the bypass of Stat Sandboxing for unauthorised directory access without disclosing directory contents (owing to getdents(2) call emulation), and the detection of hidden files without revealing file metadata, as stat(2) calls are emulated > As of version 3.19.0, Syd turns the "O_PATH" flag in open(2) system call arguments to the "O_RDONLY" flag and emulates the system call as usual which avoids the TOCTOU vector When you see hacks piled on top of hacks this way, there's usually something wrong with the fundamental architecture of the system. I think I'd rather just use bwrap, a LSM, or a VM. That said, there are a few nuggets of good ideas in here, e.g. disabling TIOCSTI and rate-limiting launches of programs that die with SIGSEGV. These enhancements should be in the core system, not a launcher utility.
- qznc 2y ago
  So it really might be „the most sophisticated“ one. Unfortunately, for security we rather need simplicity than sophistication. There should be „obviously no errors“ instead of „no obvious errors“ to quote Hoare.
  - hayali 2y ago
    This one almost made me laugh. Syd is a _unikernel_ and as such much simpler in design than GVisor which is a full-blown user-space kernel.
    - deleted 2y ago
      [deleted]
- hayali 2y ago
  Imho, you're judging syd too harshly without really understanding it: > for example, silently turning O_RDWR into O_WRONLY This is only done for Crypt sandboxing and admittedly it's mostly aimed for encrypting small files that are rarely rewritten, such as config. > prohibiting PROT_EXEC on memfds Syd is secure by default and almost always gives you options to relax specific restrictions. For this one, you want: https://man.exherbolinux.org/syd.2.html#trace/allow_unsafe_memfd https://man.exherbolinux.org/syd.2.html#trace/allow_unsafe_m... > Syd also blocks executable+anonymous memory Same, can be disabled if not needed: https://man.exherbolinux.org/syd.2.html#trace/allow_unsafe_memory https://man.exherbolinux.org/syd.2.html#trace/allow_unsafe_m... > > As of version 3.19.0, Syd turns the "O_PATH" flag in open(2) system call arguments to the "O_RDONLY" flag and emulates the system call as usual which avoids the TOCTOU vector This is a kernel limitation and there's an open kernel bug to implement O_PATH similar to O_CLOEXEC: https://bugzilla.kernel.org/show_bug.cgi?id=218501 https://bugzilla.kernel.org/show_bug.cgi?id=218501
- hayali 2y ago
  Also note, Syd has been used as Exherbo's default sandbox for 16 years now and Exherbo is a source-based distribution which enables package testing by default (we call them "build_options: recommended_tests"), and every package build/test failure that happens under sandbox but not without sandbox is considered a sydbox bug. We have fixed a lot over the years. Moreover, we have a CI task to run a random selection of a hundred gnulib tests (under 64-bit and 32-bit respectively, x86-64, x86, armv{8..7}) on each push. We can therefore proudly say that Syd is reliable when it comes to functional correctness.
wicket 2y ago
> Syd is a user space wrapper This sentence does not give me confidence. A general purpose sandbox should be implemented in the kernel. The kernel syscall interface will always remain available regardless of any sandbox that has been implemented in user space. EDIT (to answer a few of the replies here): The description lacks details so it's not entirely clear how Syd is intended to be used or who its target audience is. If this is intended for a security conscious user to wrap their own executables, similar to Firejail, I guess this serves a purpose but I would certainly hesitate to suggest that it is the "most sophisticated sandbox for Linux". We already have kernel-based SELinux, AppArmor, other LSMs and grsecurity which can ensure that every single executable will run in a sandbox regardless of the experience of the end-user, something a user space wrapper cannot achieve. It's not about whether or not it uses kernel facilities but about ensuring that absolutely everything will be sandboxed.
- sargun 2y ago
  With seccomp, you have syscall filtering. The bits that are left exposed are largely around mm, and depending on how the sandbox works, non-syscall APIs like io_uring. It essentially actuates kernel APIs to sandbox, and then redirects a number of APIs to use userspace reimplementations.
- kevin_nisbet 2y ago
  I'm not sure this is correct, I suspect this works on a similar principal as something like gvisor where as I understand it syscalls are redirected to another userspace program. In gvisors case the kernel basically get's re-implemented in user-space to provide the secure container like implementation. Also, as I recall some of the kernel based syscall based sandboxing have had a number of issues with dealing with some of the syscalls.
- simcop2387 2y ago
  It's a bit non-obvious from that description but Syd does in fact use kernel facilities to do the sandboxing. A sibling comment links to some better documentation (the syd man page) that explains what it uses https://man.exherbolinux.org/syd.1.html https://man.exherbolinux.org/syd.1.html 1) Seccomp, a BPF based kernel filter for syscalls 2) Bind mounts inside a filesystem namespace to control what files are visible 3) Landlock - more path restriction type stuff and permission changes to paths that are local to the application being wrapped 4) seccomp-notify (used with ptrace to inspect types of arguments to syscalls that bpf isn't allowed to access for security reasons, i.e. pointers)
a-french-anon 2y ago
That page looks like a madman's ramblings. Anyone has a comparison with bubblewrap, which is what I'm current using?
mfro 2y ago
Would love to see recording of the lectures I assume this is supporting material for.
kapilvt 2y ago
sort of reminds me of https://github.com/google/gvisor https://github.com/google/gvisor, re syscall interception and checking. gvisor had some significant performance impacts for io/syscall heavy workloads, but potentially seccomp/bpf could do better albeit that's mostly filtering/transform on param re more minimal touchpoint.
deleted 2y ago
[deleted]
Raistanis 2y ago
Yes
nklhlk 2y ago
[flagged]
- hayali 2y ago
  Thank you for your kind words. As the author of syd and an Exherbo developer, I am working on a sibling distro called "Hardened Exherbo": https://hexsys.org https://hexsys.org. The idea is to contain all service daemons with Syd. I don't have much to show yet but I have successfully contained some daemons: 1. rsyslog: https://gitlab.exherbo.org/exherbo/arbor/-/blob/hex/packages/sys-apps/rsyslog/files/rsyslog.syd-3 https://gitlab.exherbo.org/exherbo/arbor/-/blob/hex/packages... 2. openntpd: https://gitlab.exherbo.org/exherbo/arbor/-/blob/hex/packages/net-misc/openntpd/files/openntpd.syd-3 https://gitlab.exherbo.org/exherbo/arbor/-/blob/hex/packages... 3. nginx: https://gitlab.exherbo.org/exherbo/net/-/blob/hex/packages/www-servers/nginx/files/nginx.syd-3 https://gitlab.exherbo.org/exherbo/net/-/blob/hex/packages/w... rsyslog and openntpd profiles may be slightly outdated. I am particularly proud about the nginx profile, it demonstrates many things above all SafeSetID and Binary verification. Note, nginx profile is only configured for static file serving, if you have app servers you're gonna have to allow them as well. Syd has a trace mode when the access violations are only logged and allowed. The utility Pandora uses this mode to provide a learning mode. You can read more about pandora here: https://crates.io/crates/pandora_box https://crates.io/crates/pandora_box Pandora is really nice, it'll trim too long paths turning them into globs and calculate checksums for all the binaries and libraries used and invoking it is as easy as e.g. "pandora profile firefox".
  - hayali 2y ago
    slight correction, it's "pandora profile -mtrace/allow_unsafe_memory:1 firefox", as firefox by default uses JIT which needs WX memory.

55 points by hayali 2y ago

I found this document hard to follow. Perhaps the repo is more informative: https://gitlab.exherbo.org/sydbox/sydbox https://gitlab.exherbo.org/sydbox/sydbox

nolta 2y ago
Also this tutorial: https://man.exherbolinux.org/sydtutorial.7.html https://man.exherbolinux.org/sydtutorial.7.html

maxmcd 2y ago

Huh. Does anyone know how it works without additional privileges? I ran syd as a normal user on my system and it blocked network calls. Does it fall back to ptrace? I don't think my user has the ability to create network namespaces. edit: Ah, ok. There's a little more here: https://gitlab.exherbo.org/sydbox/sydbox/-/blob/main/doc/toctou-or-gtfo.md?ref_type=heads https://gitlab.exherbo.org/sydbox/sydbox/-/blob/main/doc/toc.... And the readme does mention ptrace. Would be great to understand how to introspect what this is using and when. ie: in some cases I might not be able to tolerate the performance penalty of ptrace. edit2: ah ok wayyyy more info here: https://man.exherbolinux.org/syd.1.html https://man.exherbolinux.org/syd.1.html and benchmarks: https://man.exherbolinux.org/syd.1.html#BENCHMARKS https://man.exherbolinux.org/syd.1.html#BENCHMARKS So it is slower, but much faster than I would have expected. Wild.

quotemstr 2y ago

This program seems fragile and too eager to break longstanding system ABI --- for example, silently turning O_RDWR into O_WRONLY, prohibiting PROT_EXEC on memfds, and, well, > Syd also blocks executable+anonymous memory > In the operation of Syd, certain system calls are not fully emulated due to seccomp(2) limitations, resulting in the sandbox process continuing these calls directly. These include execve(2), execveat(2) for execution, chdir(2), fchdir(2) for directory changes, and open(2) operations with O_PATH flag. Consequently, this behavior exposes vulnerabilities to time-of-check to time-of-use attacks, allowing for the circumvention of Exec Sandboxing and Force Sandboxing to execute denylisted paths, the bypass of Stat Sandboxing for unauthorised directory access without disclosing directory contents (owing to getdents(2) call emulation), and the detection of hidden files without revealing file metadata, as stat(2) calls are emulated > As of version 3.19.0, Syd turns the "O_PATH" flag in open(2) system call arguments to the "O_RDONLY" flag and emulates the system call as usual which avoids the TOCTOU vector When you see hacks piled on top of hacks this way, there's usually something wrong with the fundamental architecture of the system. I think I'd rather just use bwrap, a LSM, or a VM. That said, there are a few nuggets of good ideas in here, e.g. disabling TIOCSTI and rate-limiting launches of programs that die with SIGSEGV. These enhancements should be in the core system, not a launcher utility.

qznc 2y ago
So it really might be „the most sophisticated“ one. Unfortunately, for security we rather need simplicity than sophistication. There should be „obviously no errors“ instead of „no obvious errors“ to quote Hoare.
- hayali 2y ago
  This one almost made me laugh. Syd is a _unikernel_ and as such much simpler in design than GVisor which is a full-blown user-space kernel.
  - deleted 2y ago
    [deleted]
hayali 2y ago
Imho, you're judging syd too harshly without really understanding it: > for example, silently turning O_RDWR into O_WRONLY This is only done for Crypt sandboxing and admittedly it's mostly aimed for encrypting small files that are rarely rewritten, such as config. > prohibiting PROT_EXEC on memfds Syd is secure by default and almost always gives you options to relax specific restrictions. For this one, you want: https://man.exherbolinux.org/syd.2.html#trace/allow_unsafe_memfd https://man.exherbolinux.org/syd.2.html#trace/allow_unsafe_m... > Syd also blocks executable+anonymous memory Same, can be disabled if not needed: https://man.exherbolinux.org/syd.2.html#trace/allow_unsafe_memory https://man.exherbolinux.org/syd.2.html#trace/allow_unsafe_m... > > As of version 3.19.0, Syd turns the "O_PATH" flag in open(2) system call arguments to the "O_RDONLY" flag and emulates the system call as usual which avoids the TOCTOU vector This is a kernel limitation and there's an open kernel bug to implement O_PATH similar to O_CLOEXEC: https://bugzilla.kernel.org/show_bug.cgi?id=218501 https://bugzilla.kernel.org/show_bug.cgi?id=218501
hayali 2y ago
Also note, Syd has been used as Exherbo's default sandbox for 16 years now and Exherbo is a source-based distribution which enables package testing by default (we call them "build_options: recommended_tests"), and every package build/test failure that happens under sandbox but not without sandbox is considered a sydbox bug. We have fixed a lot over the years. Moreover, we have a CI task to run a random selection of a hundred gnulib tests (under 64-bit and 32-bit respectively, x86-64, x86, armv{8..7}) on each push. We can therefore proudly say that Syd is reliable when it comes to functional correctness.

wicket 2y ago

> Syd is a user space wrapper This sentence does not give me confidence. A general purpose sandbox should be implemented in the kernel. The kernel syscall interface will always remain available regardless of any sandbox that has been implemented in user space. EDIT (to answer a few of the replies here): The description lacks details so it's not entirely clear how Syd is intended to be used or who its target audience is. If this is intended for a security conscious user to wrap their own executables, similar to Firejail, I guess this serves a purpose but I would certainly hesitate to suggest that it is the "most sophisticated sandbox for Linux". We already have kernel-based SELinux, AppArmor, other LSMs and grsecurity which can ensure that every single executable will run in a sandbox regardless of the experience of the end-user, something a user space wrapper cannot achieve. It's not about whether or not it uses kernel facilities but about ensuring that absolutely everything will be sandboxed.

sargun 2y ago
With seccomp, you have syscall filtering. The bits that are left exposed are largely around mm, and depending on how the sandbox works, non-syscall APIs like io_uring. It essentially actuates kernel APIs to sandbox, and then redirects a number of APIs to use userspace reimplementations.
kevin_nisbet 2y ago
I'm not sure this is correct, I suspect this works on a similar principal as something like gvisor where as I understand it syscalls are redirected to another userspace program. In gvisors case the kernel basically get's re-implemented in user-space to provide the secure container like implementation. Also, as I recall some of the kernel based syscall based sandboxing have had a number of issues with dealing with some of the syscalls.
simcop2387 2y ago
It's a bit non-obvious from that description but Syd does in fact use kernel facilities to do the sandboxing. A sibling comment links to some better documentation (the syd man page) that explains what it uses https://man.exherbolinux.org/syd.1.html https://man.exherbolinux.org/syd.1.html 1) Seccomp, a BPF based kernel filter for syscalls 2) Bind mounts inside a filesystem namespace to control what files are visible 3) Landlock - more path restriction type stuff and permission changes to paths that are local to the application being wrapped 4) seccomp-notify (used with ptrace to inspect types of arguments to syscalls that bpf isn't allowed to access for security reasons, i.e. pointers)

a-french-anon 2y ago

That page looks like a madman's ramblings. Anyone has a comparison with bubblewrap, which is what I'm current using?

mfro 2y ago

Would love to see recording of the lectures I assume this is supporting material for.

kapilvt 2y ago

sort of reminds me of https://github.com/google/gvisor https://github.com/google/gvisor, re syscall interception and checking. gvisor had some significant performance impacts for io/syscall heavy workloads, but potentially seccomp/bpf could do better albeit that's mostly filtering/transform on param re more minimal touchpoint.

deleted 2y ago

[deleted]

Raistanis 2y ago

Yes

nklhlk 2y ago

[flagged]

hayali 2y ago
Thank you for your kind words. As the author of syd and an Exherbo developer, I am working on a sibling distro called "Hardened Exherbo": https://hexsys.org https://hexsys.org. The idea is to contain all service daemons with Syd. I don't have much to show yet but I have successfully contained some daemons: 1. rsyslog: https://gitlab.exherbo.org/exherbo/arbor/-/blob/hex/packages/sys-apps/rsyslog/files/rsyslog.syd-3 https://gitlab.exherbo.org/exherbo/arbor/-/blob/hex/packages... 2. openntpd: https://gitlab.exherbo.org/exherbo/arbor/-/blob/hex/packages/net-misc/openntpd/files/openntpd.syd-3 https://gitlab.exherbo.org/exherbo/arbor/-/blob/hex/packages... 3. nginx: https://gitlab.exherbo.org/exherbo/net/-/blob/hex/packages/www-servers/nginx/files/nginx.syd-3 https://gitlab.exherbo.org/exherbo/net/-/blob/hex/packages/w... rsyslog and openntpd profiles may be slightly outdated. I am particularly proud about the nginx profile, it demonstrates many things above all SafeSetID and Binary verification. Note, nginx profile is only configured for static file serving, if you have app servers you're gonna have to allow them as well. Syd has a trace mode when the access violations are only logged and allowed. The utility Pandora uses this mode to provide a learning mode. You can read more about pandora here: https://crates.io/crates/pandora_box https://crates.io/crates/pandora_box Pandora is really nice, it'll trim too long paths turning them into globs and calculate checksums for all the binaries and libraries used and invoking it is as easy as e.g. "pandora profile firefox".
- hayali 2y ago
  slight correction, it's "pandora profile -mtrace/allow_unsafe_memory:1 firefox", as firefox by default uses JIT which needs WX memory.