6 ms·

I wrote this post: the title should be "Patching GCC to build Actually Portable Executables", because it refers to Cosmopolitan Libc and jart's Actually Portabl

I wrote this post: the title should be "Patching GCC to build Actually Portable Executables", because it refers to Cosmopolitan Libc and jart's Actually Portable Executable format. With my gcc patch, you can now build software like vim, emacs, ninja, bash, git, gcc etc with Cosmopolitan Libc, via their usual autotools/cmake-style build system. The built executables should run on Linux, FreeBSD, MacOS, OpenBSD, NetBSD, and also Windows (although I haven't tested Windows yet.) Here's a list of software I got to build with this technique: https://github.com/ahgamut/superconfigure https://github.com/ahgamut/superconfigure The superconfigure script is just a wrapper around the usual configure script used to build your software, supplying flags like --enable-static. If you want to build gcc using Cosmopolitan Libc -- try out this repo: https://github.com/ahgamut/musl-cross-make/tree/gccbuild https://github.com/ahgamut/musl-cross-make/tree/gccbuild

foresto 3y ago
Nice! Do you foresee any problems with using this behind a transpiled language like Nim?
- ahgamut 3y ago
  I haven't used Nim much, but I remember a repo on Github had setup a build script for compiling Nim with Cosmopolitan Libc. https://github.com/Yardanico/cosmonim https://github.com/Yardanico/cosmonim This gcc patch makes such build scripts simpler, because you will need to change less of your code -- let me know how it works!
ChickeNES 3y ago
Have you attempted to build busybox?
- ahgamut 3y ago
  Yes, I have gotten busybox to build successfully, but at present it's not as nice as bash or vim. I expect someone will write a better build script soon enough. Maybe it will be you! The default build parameters in busybox make a lot of OS-related assumptions, so I'd recommend you use make menuconfig instead of make depconfig. in the TUI, you can disable all console and networking utilities, linux modprobe utilities, some linux system utilities, free, uptime, etc. Start with a minimal config and then slowly add things. In terms of source code changes: modify `u_signal_names.c` to use a switch statement instead of the amazing "use SIGHUP as an array index" method it follows, use int32_t instead of the "smallint" type that busybox seems to prefer, and IIRC there's an enum in there somewhere that should be rewritten as a #define. That should get you pretty close to a successful build.
endgame 3y ago
Will this patch of yours make it into GCC upstream? APE executables and cosmopolitan libc are incredibly cool tech, and it'd be great if they were easier to use.
- ahgamut 3y ago
  Reading the GCC source code was fun and educational, and I'd love to find out if this could patch make it into GCC. Haven't discussed it with any of the GCC developers yet. In the meantime, the Cosmopolitan Libc monorepo contains binaries of gcc-11 and binutils that have been built with my patch, you can use those for now :)
cratermoon 3y ago
Could this be done for Rust? rustup target add cosmo maybe?
- ahgamut 3y ago
  I did that last year :) https://github.com/ahgamut/rust-ape-example https://github.com/ahgamut/rust-ape-example Also got ripgrep to work: https://github.com/ahgamut/ripgrep/tree/cosmopolitan https://github.com/ahgamut/ripgrep/tree/cosmopolitan Might need to update the build scripts a little to handle the latest updates in Rust and Cosmo (we'll need gcc-11 now), but I expect to get Rust working again soon enough.
mgaunard 3y ago
Building is not such a big achievement, what's more interesting is to know whether it actually works? As in, was there extensive testing to prove there are no regressions compared to the glibc version?
- ahgamut 3y ago
  Right now I use the gcc, git, vim, and ninja APE binaries on my local system for daily work, and so far I have seen no visible regressions. I agree with you though -- an extensive testing setup will reveal more things for improvement, be it in my patch or in the libc itself. At the moment I only have direct access to Debian 11/Fedora 35/FreeBSD 13, but I expect we will soon find a nice way to run tests across the different operating systems automatically. One of my ideas was to setup a separate drive or folder with all the executables, accessible from different operating systems/VMs, and then run the tests from each. What do you think?
  - gdprrrr 3y ago
    I think github actions has windows images
  - mgaunard 3y ago
    I don't know enough about APE or cosmo to know how risky the changes are. Here apparently the changes are mostly related to error handling, and I guess that's not something that's very well tested with normal usage.
blueflow 3y ago
I recently saw that you got banned from lobste.rs for being a sockpuppet from jart and just now i realized that you are an actual, different person. [unconstructive complaint about lobste.rs moderation skipped] I hope HN treats you better.
- supriyo-biswas 3y ago
  I took a peek at the moderation log and the banning of jart and their descendants was a real shitshow[1]. I’m sure a lot of legit folks lost their accounts, and most of the descendants weren’t even posting anything related to APE or jart’s other projects. [1] https://lobste.rs/moderations https://lobste.rs/moderations
  - distcs 3y ago
    Sockpuppets or voting rings may not be posting related posts/comments at the moment but they could in future. And they could be silently +1-ing each other's posts/comments which you can't see but mods can. I've done moderating work when phpbb forums had their heydey. The clever sockpuppet accounts or voting rings don't start posting about the projects of their main account right from the first day. The clever ones remain dormant, mostly idle, participating in other discussions. They look just like normal accounts. But much much later when the main account posts a "show project" post, the sockpuppets or voting rings add +1 or post a few words of praise and then move on to commenting on +1-ing other posts. And voting rings become more difficult to detect when they don't post anything related, post unrelated stuff but quietly upvote the main account's posts/comments.
    - supriyo-biswas 3y ago
      Without said things actually occurring, there is no reason to say that the accounts are acting as sock puppets.
      - distcs 3y ago
        Only 2 of all the banned accounts have been flagged as sock-puppeteers. You can't see the sockpuppetery anymore because the mod deletes their posts/comments. But the evidence is still there as negative points. Those profiles have negative points. So they must have posted something sometime, got downvoted, got flagged by users and got detected as sockpuppets. Sock puppet is only one possibility. Other possibility is legit accounts forming a voting ring. You and me cant see voting actions. Mods can. The mods have access to server access logs. Mods can see who +1-ed whose posts/comments. So if you need evidence for sock-puppetry or voting-ringery, try asking the mods about it. We can only speculate here. I'm only describing what I've found in the wild from my days of moderating (small) forums. I mean to say that only because you can't see the other accounts posting related posts/comments does not make them any more likely or any less likely to be sockpuppet or voting ring. They may be legit. But they could be a ring. Or they may be legit. Only the mods with the access logs can tell.

foresto 3y ago

Nice! Do you foresee any problems with using this behind a transpiled language like Nim?

ahgamut 3y ago
I haven't used Nim much, but I remember a repo on Github had setup a build script for compiling Nim with Cosmopolitan Libc. https://github.com/Yardanico/cosmonim https://github.com/Yardanico/cosmonim This gcc patch makes such build scripts simpler, because you will need to change less of your code -- let me know how it works!

ChickeNES 3y ago

Have you attempted to build busybox?

ahgamut 3y ago
Yes, I have gotten busybox to build successfully, but at present it's not as nice as bash or vim. I expect someone will write a better build script soon enough. Maybe it will be you! The default build parameters in busybox make a lot of OS-related assumptions, so I'd recommend you use make menuconfig instead of make depconfig. in the TUI, you can disable all console and networking utilities, linux modprobe utilities, some linux system utilities, free, uptime, etc. Start with a minimal config and then slowly add things. In terms of source code changes: modify `u_signal_names.c` to use a switch statement instead of the amazing "use SIGHUP as an array index" method it follows, use int32_t instead of the "smallint" type that busybox seems to prefer, and IIRC there's an enum in there somewhere that should be rewritten as a #define. That should get you pretty close to a successful build.

endgame 3y ago

Will this patch of yours make it into GCC upstream? APE executables and cosmopolitan libc are incredibly cool tech, and it'd be great if they were easier to use.

ahgamut 3y ago
Reading the GCC source code was fun and educational, and I'd love to find out if this could patch make it into GCC. Haven't discussed it with any of the GCC developers yet. In the meantime, the Cosmopolitan Libc monorepo contains binaries of gcc-11 and binutils that have been built with my patch, you can use those for now :)

cratermoon 3y ago

Could this be done for Rust? rustup target add cosmo maybe?

ahgamut 3y ago
I did that last year :) https://github.com/ahgamut/rust-ape-example https://github.com/ahgamut/rust-ape-example Also got ripgrep to work: https://github.com/ahgamut/ripgrep/tree/cosmopolitan https://github.com/ahgamut/ripgrep/tree/cosmopolitan Might need to update the build scripts a little to handle the latest updates in Rust and Cosmo (we'll need gcc-11 now), but I expect to get Rust working again soon enough.

mgaunard 3y ago

Building is not such a big achievement, what's more interesting is to know whether it actually works? As in, was there extensive testing to prove there are no regressions compared to the glibc version?

ahgamut 3y ago
Right now I use the gcc, git, vim, and ninja APE binaries on my local system for daily work, and so far I have seen no visible regressions. I agree with you though -- an extensive testing setup will reveal more things for improvement, be it in my patch or in the libc itself. At the moment I only have direct access to Debian 11/Fedora 35/FreeBSD 13, but I expect we will soon find a nice way to run tests across the different operating systems automatically. One of my ideas was to setup a separate drive or folder with all the executables, accessible from different operating systems/VMs, and then run the tests from each. What do you think?
- gdprrrr 3y ago
  I think github actions has windows images
- mgaunard 3y ago
  I don't know enough about APE or cosmo to know how risky the changes are. Here apparently the changes are mostly related to error handling, and I guess that's not something that's very well tested with normal usage.

blueflow 3y ago

I recently saw that you got banned from lobste.rs for being a sockpuppet from jart and just now i realized that you are an actual, different person. [unconstructive complaint about lobste.rs moderation skipped] I hope HN treats you better.

supriyo-biswas 3y ago
I took a peek at the moderation log and the banning of jart and their descendants was a real shitshow[1]. I’m sure a lot of legit folks lost their accounts, and most of the descendants weren’t even posting anything related to APE or jart’s other projects. [1] https://lobste.rs/moderations https://lobste.rs/moderations
- distcs 3y ago
  Sockpuppets or voting rings may not be posting related posts/comments at the moment but they could in future. And they could be silently +1-ing each other's posts/comments which you can't see but mods can. I've done moderating work when phpbb forums had their heydey. The clever sockpuppet accounts or voting rings don't start posting about the projects of their main account right from the first day. The clever ones remain dormant, mostly idle, participating in other discussions. They look just like normal accounts. But much much later when the main account posts a "show project" post, the sockpuppets or voting rings add +1 or post a few words of praise and then move on to commenting on +1-ing other posts. And voting rings become more difficult to detect when they don't post anything related, post unrelated stuff but quietly upvote the main account's posts/comments.
  - supriyo-biswas 3y ago
    Without said things actually occurring, there is no reason to say that the accounts are acting as sock puppets.
    - distcs 3y ago
      Only 2 of all the banned accounts have been flagged as sock-puppeteers. You can't see the sockpuppetery anymore because the mod deletes their posts/comments. But the evidence is still there as negative points. Those profiles have negative points. So they must have posted something sometime, got downvoted, got flagged by users and got detected as sockpuppets. Sock puppet is only one possibility. Other possibility is legit accounts forming a voting ring. You and me cant see voting actions. Mods can. The mods have access to server access logs. Mods can see who +1-ed whose posts/comments. So if you need evidence for sock-puppetry or voting-ringery, try asking the mods about it. We can only speculate here. I'm only describing what I've found in the wild from my days of moderating (small) forums. I mean to say that only because you can't see the other accounts posting related posts/comments does not make them any more likely or any less likely to be sockpuppet or voting ring. They may be legit. But they could be a ring. Or they may be legit. Only the mods with the access logs can tell.