6 ms·

I feel like there's a lot of potential between Go and Cosmopolitan libc. Go itself does not use libc much, as shown in the blog, but some great libraries like S

by sbstp 2y ago

I feel like there's a lot of potential between Go and Cosmopolitan libc. Go itself does not use libc much, as shown in the blog, but some great libraries like SQLite3 need it (unless you use https://pkg.go.dev/modernc.org/sqlite https://pkg.go.dev/modernc.org/sqlite). The ability to build a single static binary that works on Linux, Mac and Windows using Go would be life changing for the internal tools I develop at work.

latchkey 2y ago
> The ability to build a single static binary that works on Linux, Mac and Windows using Go would be life changing for the internal tools I develop at work. Just curious, life changing in what way? Obviously, 1 is better than 3, but I'm wondering if there is some other interesting reason.
oguz-ismail 2y ago
I recall reading about new Macs with ARM chips not supporting static binaries. Is it not true?
- deleted 2y ago
  [deleted]
- zamadatix 2y ago
  That's been Apple's stance on full static linking MacOS in general, many years prior to the move to ARM e.g. https://developer.apple.com/library/archive/qa/qa1118/_index.html https://developer.apple.com/library/archive/qa/qa1118/_index... You're welcome to ignore it of course, it's just unofficial and a large pain.
  - oguz-ismail 2y ago
    >You're welcome to ignore it of course How do you mean? Like, is it possible to run such binaries on M1? If so I'd really like to know how
    - telotortium 2y ago
      You can always disassemble libc and look for the system call numbers used by the syscall assembly instructions. It’s just that these numbers (and associated arguments and return values) are not stable and can and do change upon kernel updates (in which case libc will be updated to keep the libc interface stable). I believe Linux is the only major OS these days to guarantee binary compatibility of the syscall interface.
      - oguz-ismail 2y ago
        I know this works on Macs with Intel chips. But the ones with ARM chips just won't execute fully static binaries, and I'm wondering if there's a workaround.
        saagarjha 2y ago
        Nope.
        oguz-ismail 2y ago
        Thanks. That sucks
        saagarjha 2y ago
        If you can convince Apple to change this code let me know: https://github.com/apple-oss-distributions/xnu/blob/94d3b452840153a99b38a3a9659680b2a006908e/bsd/kern/mach_loader.c#L877 https://github.com/apple-oss-distributions/xnu/blob/94d3b452...
        deleted 2y ago
        [deleted]
        telotortium 2y ago
        I’m guessing not. According to man ld on macOS, the -static flag, to produce a fully static executable, is only used to build the kernel. I don’t believe fully-static executables were ever officially supported on macOS, although they would work.
- cyberax 2y ago
  All Macs don't support static binaries. That's because the syscall interface on macOS is not stable, only libc is guaranteed to be stable.
fsmv 2y ago
The same exact binary working isn't going to happen without runtime performance penalties because the syscall numbers are different on different platforms. Also I believe on windows it's not possible to avoid linking some system libraries to use windows.h stuff, there is no stable ABI.
- pjmlp 2y ago
  Linux is the exception among modern OSes to have a stable syscall ABI, everyone else offers only the proper OS API as entry point into OS services. Once upon at time, static linking was the only thing OSes were capable of, all of them moved away from that, and there is no turning back outside embedded bare metal deployments, just become some people are obsessed with static linking.
- actionfromafar 2y ago
  Though msvcrt.dll has a stable subset of functions available on all Windows versions.
spease 2y ago
Hmmm…what about compiling to wasm, and/or then converting the wasm to C? https://github.com/wasm3/wasm3/releases/tag/v0.4.8 https://github.com/wasm3/wasm3/releases/tag/v0.4.8 https://github.com/WebAssembly/wabt/tree/main/wasm2c https://github.com/WebAssembly/wabt/tree/main/wasm2c
- daenney 2y ago
  Has been done and works pretty great: https://github.com/ncruces/go-sqlite3 https://github.com/ncruces/go-sqlite3. Though doesn’t convert the WASM to C, it runs the WASM in Wazero instead.
  - spease 2y ago
    How does that solve what the person I replied to was asking for?
    - ncruces 2y ago
      If I understood you, it doesn't help much, no, but neither does what you suggested. You're suggesting compiling Go to Wasm (presumably using the wasip1 target?), then converting that to C using wabt, then using Cosmopolitan to create an APE… is that it? Well, that's not going to work. First of all, Go's wasip1 target doesn't even support cgo, so if you want SQLite, you're dead right there. Then, even if you used say TinyGo (which might support cgo, not sure), WASI just isn't a great target to compile SQLite into. WASI is a pretty limited syscall layer. You'd end up with no file locking, no shared memory. Also no threads. Then, on top of that, you'd layer Cosmopolitan issues. Having written a portable SQLite VFS from scratch, I am not impressed with how they just paper over file locking semantics incompatibilities between OSes, and then confidently ship a forking webserver with SQLite bundled in. It takes a certain temerity, and not running many SQLite torture tests. Wasm as an intermediate target is great for (single threaded) CPU stuff. WASI is great if you can fit it, but otherwise, it's not, not really.

by sbstp 2y ago

latchkey 2y ago

> The ability to build a single static binary that works on Linux, Mac and Windows using Go would be life changing for the internal tools I develop at work. Just curious, life changing in what way? Obviously, 1 is better than 3, but I'm wondering if there is some other interesting reason.

oguz-ismail 2y ago

I recall reading about new Macs with ARM chips not supporting static binaries. Is it not true?

deleted 2y ago
[deleted]
zamadatix 2y ago
That's been Apple's stance on full static linking MacOS in general, many years prior to the move to ARM e.g. https://developer.apple.com/library/archive/qa/qa1118/_index.html https://developer.apple.com/library/archive/qa/qa1118/_index... You're welcome to ignore it of course, it's just unofficial and a large pain.
- oguz-ismail 2y ago
  >You're welcome to ignore it of course How do you mean? Like, is it possible to run such binaries on M1? If so I'd really like to know how
  - telotortium 2y ago
    You can always disassemble libc and look for the system call numbers used by the syscall assembly instructions. It’s just that these numbers (and associated arguments and return values) are not stable and can and do change upon kernel updates (in which case libc will be updated to keep the libc interface stable). I believe Linux is the only major OS these days to guarantee binary compatibility of the syscall interface.
    - oguz-ismail 2y ago
      I know this works on Macs with Intel chips. But the ones with ARM chips just won't execute fully static binaries, and I'm wondering if there's a workaround.
      - saagarjha 2y ago
        Nope.
        oguz-ismail 2y ago
        Thanks. That sucks
        saagarjha 2y ago
        If you can convince Apple to change this code let me know: https://github.com/apple-oss-distributions/xnu/blob/94d3b452840153a99b38a3a9659680b2a006908e/bsd/kern/mach_loader.c#L877 https://github.com/apple-oss-distributions/xnu/blob/94d3b452...
        deleted 2y ago
        [deleted]
      - telotortium 2y ago
        I’m guessing not. According to man ld on macOS, the -static flag, to produce a fully static executable, is only used to build the kernel. I don’t believe fully-static executables were ever officially supported on macOS, although they would work.
cyberax 2y ago
All Macs don't support static binaries. That's because the syscall interface on macOS is not stable, only libc is guaranteed to be stable.

fsmv 2y ago

The same exact binary working isn't going to happen without runtime performance penalties because the syscall numbers are different on different platforms. Also I believe on windows it's not possible to avoid linking some system libraries to use windows.h stuff, there is no stable ABI.

pjmlp 2y ago
Linux is the exception among modern OSes to have a stable syscall ABI, everyone else offers only the proper OS API as entry point into OS services. Once upon at time, static linking was the only thing OSes were capable of, all of them moved away from that, and there is no turning back outside embedded bare metal deployments, just become some people are obsessed with static linking.
actionfromafar 2y ago
Though msvcrt.dll has a stable subset of functions available on all Windows versions.

spease 2y ago

Hmmm…what about compiling to wasm, and/or then converting the wasm to C? https://github.com/wasm3/wasm3/releases/tag/v0.4.8 https://github.com/wasm3/wasm3/releases/tag/v0.4.8 https://github.com/WebAssembly/wabt/tree/main/wasm2c https://github.com/WebAssembly/wabt/tree/main/wasm2c

daenney 2y ago
Has been done and works pretty great: https://github.com/ncruces/go-sqlite3 https://github.com/ncruces/go-sqlite3. Though doesn’t convert the WASM to C, it runs the WASM in Wazero instead.
- spease 2y ago
  How does that solve what the person I replied to was asking for?
  - ncruces 2y ago
    If I understood you, it doesn't help much, no, but neither does what you suggested. You're suggesting compiling Go to Wasm (presumably using the wasip1 target?), then converting that to C using wabt, then using Cosmopolitan to create an APE… is that it? Well, that's not going to work. First of all, Go's wasip1 target doesn't even support cgo, so if you want SQLite, you're dead right there. Then, even if you used say TinyGo (which might support cgo, not sure), WASI just isn't a great target to compile SQLite into. WASI is a pretty limited syscall layer. You'd end up with no file locking, no shared memory. Also no threads. Then, on top of that, you'd layer Cosmopolitan issues. Having written a portable SQLite VFS from scratch, I am not impressed with how they just paper over file locking semantics incompatibilities between OSes, and then confidently ship a forking webserver with SQLite bundled in. It takes a certain temerity, and not running many SQLite torture tests. Wasm as an intermediate target is great for (single threaded) CPU stuff. WASI is great if you can fit it, but otherwise, it's not, not really.