6 ms·

I do like this about go. And on Linux it arguably makes sense (I say arguable because DNS without CGO is still a common cause of issues and incompatibility). B

I do like this about go. And on Linux it arguably makes sense (I say arguable because DNS without CGO is still a common cause of issues and incompatibility). But Linux has, to the best of my understanding said "yes, we are ok with users using syscalls". Linux doesn't think that glibc is the only project allowed to interface with the kernel. But for other platforms like OpenBSD and windows they are quite simply relying on implementation details that the vendors consider to be a private and unsupported interface. This whole thing is also separate from "is making libc the only caller of the syscalls instruction" a good and meaningful security improvement.

p_l 3y ago
Windows also provides language-independent ways of calling low-level (and pretty high-level) OS interfaces. The library you have to link to access system services is not going to pollute your language environment with bad runtime.
akira2501 3y ago
> I say arguable because DNS without CGO is still a common cause of issues and incompatibility DNS without CGO works perfectly. The vendor specific ad hoc mechanisms for extending DNS in a site local context are not well supported. If they were implemented more sensibly, then Go, or any other language, would have no problem taking advantage of them even without the "C Library Resolver." Speaking of which, that "C Library Resolver," in my opinion, has one of the worst library interfaces in all of unix. It's not at all a hill worth new projects dying on.
- anonacct37 3y ago
  > DNS without CGO works perfectly It does not. I know this because it impacts my daily work and the work of others. Honestly if you could make my day and go figure out exactly what's going wrong with the pure go DNS implementation it would make my life alot simpler and I wouldn't have to maintain shell scripts that update etc/hosts to hard code in ipv4 addresses for the APIs I access with terraform. https://github.com/hashicorp/terraform-provider-google/issues/6782 https://github.com/hashicorp/terraform-provider-google/issue...
  - akira2501 3y ago
    It seems like the explanation might be right there in that issue. The server is occasionally not successfully sending the A record and you're getting bad fallback behavior due to the way the dial call is imprecisely constructed and this is masking the underlying problem. The code, as written, is doing exactly what you would expect in this scenario. Should it be your DNS resolver library that takes stock of your OS environment and only make calls for A records and not AAAA records when it "detects" some configuration? Shouldn't your application itself have an environment variable or command line option that allows you to specify that your dials should only be done using tcp4? Wouldn't this be immensely useful to have outside of "auto detection" in some library somewhere?
    - j16sdiz 3y ago
      Oh.. right. So we need another configuration for each and every application for something almost always system wide and can change dynamically (like wifi reconnecting) ?
    - deleted 3y ago
      [deleted]
    - tsimionescu 3y ago
      Why should every application be aware of whether it is running on a v4 or v6 network? If the application merely wants to connect to an external service, that is firmly in the OS's job to decide. As a user, if ipv6 is flaky today, I want one central place to configure for ipv4 only, I don't want to go and change every application'settings only to revert that tomorrow.
- ninkendo 3y ago
  > The vendor specific ad hoc mechanisms for extending DNS in a site local context are not well supported DNS is one of those things that OS vendors think should be extendable and configurable. It allows VPN apps to redirect DNS only for certain subdomains, for example, which enables proper split-horizon DNS. I think this is totally reasonable behavior, and it’s undeniably useful. If a particular programming language reimplements DNS on its own, you lose guarantees that the OS is striving to provide to the user. You can make the case that OS’s shouldn’t make these guarantees, and we’re free to disagree on that, but from a practical standpoint it is a very useful feature and it sucks that pure Go apps don’t work with it.
  - wbl 3y ago
    The right way to do this is specify a resolver to use on localhost
  - treffer 3y ago
    Or you make the case that the OS should ship a DNS server. It can handle DNS forwarding and iterative queries anyway (iterative queries as you often need them for dnssec) and the logic for serving DNS responses is ok'ish (DNS name dedup and length restrictions being the biggest complexity issue). I am not super happy with systemd-resolved but it solves this particular issues. No requirement to use libc, but same (os configurable) behavior for all users.
    - tptacek 3y ago
      The difficulty and overhead of running those queries from everybody's phones is one of the reasons (a small reason, but one of them) DNSSEC isn't deployed.
toast0 3y ago
Most unixes allow for static binaries. And most unixes allow for you to run a static binary compiled for an older OS on a newer OS. In order for that to work, the syscall interface needs to be at least mostly stable. Yes, if you're writing netstat or lsof or ps or something, you need tight coupling with the binary and the kernel, and you can argue Linux does that better, but most people aren't writing netstat or lsof or ps.
- IAmLiterallyAB 3y ago
  Even those get most of their info from /proc and /sys rather than special syscalls
  - epcoa 3y ago
    On the BSDs they sure don't, /proc (for its role referred to) and /sys are peculiar to Linux, /proc is not even mounted by default on FreeBSD, is deprecated and doesn't exist on OpenBSD. On OpenBSD/FreeBSD most of the info comes out with sysctl(3) or reading kernel memory directly.
    - saagarjha 3y ago
      > reading kernel memory directly God I hope not
      - adrian_b 3y ago
        I do not know what the previous poster was talking about, but, while unrestricted access to kernel memory is of course unacceptable, a perfectly secure and useful means for the kernel to provide information to a user process is to map a page as read-only in the user address space, from where the user process could get the information without the overhead of a system call. For information of general interest such a special kernel page could be mapped as read-only in the address space of all user processes. Much of the information that is provided in the special file systems /proc and /sys could have been provided in some appropriate data structures in such read-only shared memory, for a faster access, by avoiding the overhead of file system calls and of file text parsing.
Varriount 3y ago
Does Go use direct syscalls on Windows? Last time I looked, I could have sworn Go's source used the standard Windows system DLLs.
- pjmlp 3y ago
  On Windows it never did, and on other platforms, other than Linux, it has been forced to accept that isn't the way OS vendors play the game.

p_l 3y ago

Windows also provides language-independent ways of calling low-level (and pretty high-level) OS interfaces. The library you have to link to access system services is not going to pollute your language environment with bad runtime.

akira2501 3y ago

> I say arguable because DNS without CGO is still a common cause of issues and incompatibility DNS without CGO works perfectly. The vendor specific ad hoc mechanisms for extending DNS in a site local context are not well supported. If they were implemented more sensibly, then Go, or any other language, would have no problem taking advantage of them even without the "C Library Resolver." Speaking of which, that "C Library Resolver," in my opinion, has one of the worst library interfaces in all of unix. It's not at all a hill worth new projects dying on.

anonacct37 3y ago
> DNS without CGO works perfectly It does not. I know this because it impacts my daily work and the work of others. Honestly if you could make my day and go figure out exactly what's going wrong with the pure go DNS implementation it would make my life alot simpler and I wouldn't have to maintain shell scripts that update etc/hosts to hard code in ipv4 addresses for the APIs I access with terraform. https://github.com/hashicorp/terraform-provider-google/issues/6782 https://github.com/hashicorp/terraform-provider-google/issue...
- akira2501 3y ago
  It seems like the explanation might be right there in that issue. The server is occasionally not successfully sending the A record and you're getting bad fallback behavior due to the way the dial call is imprecisely constructed and this is masking the underlying problem. The code, as written, is doing exactly what you would expect in this scenario. Should it be your DNS resolver library that takes stock of your OS environment and only make calls for A records and not AAAA records when it "detects" some configuration? Shouldn't your application itself have an environment variable or command line option that allows you to specify that your dials should only be done using tcp4? Wouldn't this be immensely useful to have outside of "auto detection" in some library somewhere?
  - j16sdiz 3y ago
    Oh.. right. So we need another configuration for each and every application for something almost always system wide and can change dynamically (like wifi reconnecting) ?
  - deleted 3y ago
    [deleted]
  - tsimionescu 3y ago
    Why should every application be aware of whether it is running on a v4 or v6 network? If the application merely wants to connect to an external service, that is firmly in the OS's job to decide. As a user, if ipv6 is flaky today, I want one central place to configure for ipv4 only, I don't want to go and change every application'settings only to revert that tomorrow.
ninkendo 3y ago
> The vendor specific ad hoc mechanisms for extending DNS in a site local context are not well supported DNS is one of those things that OS vendors think should be extendable and configurable. It allows VPN apps to redirect DNS only for certain subdomains, for example, which enables proper split-horizon DNS. I think this is totally reasonable behavior, and it’s undeniably useful. If a particular programming language reimplements DNS on its own, you lose guarantees that the OS is striving to provide to the user. You can make the case that OS’s shouldn’t make these guarantees, and we’re free to disagree on that, but from a practical standpoint it is a very useful feature and it sucks that pure Go apps don’t work with it.
- wbl 3y ago
  The right way to do this is specify a resolver to use on localhost
- treffer 3y ago
  Or you make the case that the OS should ship a DNS server. It can handle DNS forwarding and iterative queries anyway (iterative queries as you often need them for dnssec) and the logic for serving DNS responses is ok'ish (DNS name dedup and length restrictions being the biggest complexity issue). I am not super happy with systemd-resolved but it solves this particular issues. No requirement to use libc, but same (os configurable) behavior for all users.
  - tptacek 3y ago
    The difficulty and overhead of running those queries from everybody's phones is one of the reasons (a small reason, but one of them) DNSSEC isn't deployed.

toast0 3y ago

Most unixes allow for static binaries. And most unixes allow for you to run a static binary compiled for an older OS on a newer OS. In order for that to work, the syscall interface needs to be at least mostly stable. Yes, if you're writing netstat or lsof or ps or something, you need tight coupling with the binary and the kernel, and you can argue Linux does that better, but most people aren't writing netstat or lsof or ps.

IAmLiterallyAB 3y ago
Even those get most of their info from /proc and /sys rather than special syscalls
- epcoa 3y ago
  On the BSDs they sure don't, /proc (for its role referred to) and /sys are peculiar to Linux, /proc is not even mounted by default on FreeBSD, is deprecated and doesn't exist on OpenBSD. On OpenBSD/FreeBSD most of the info comes out with sysctl(3) or reading kernel memory directly.
  - saagarjha 3y ago
    > reading kernel memory directly God I hope not
    - adrian_b 3y ago
      I do not know what the previous poster was talking about, but, while unrestricted access to kernel memory is of course unacceptable, a perfectly secure and useful means for the kernel to provide information to a user process is to map a page as read-only in the user address space, from where the user process could get the information without the overhead of a system call. For information of general interest such a special kernel page could be mapped as read-only in the address space of all user processes. Much of the information that is provided in the special file systems /proc and /sys could have been provided in some appropriate data structures in such read-only shared memory, for a faster access, by avoiding the overhead of file system calls and of file text parsing.

Varriount 3y ago

Does Go use direct syscalls on Windows? Last time I looked, I could have sworn Go's source used the standard Windows system DLLs.

pjmlp 3y ago
On Windows it never did, and on other platforms, other than Linux, it has been forced to accept that isn't the way OS vendors play the game.