8 ms·

Hey, so I built this thing, most of it at so far at least. And yeah, right now it isn't doing many things better than Homebrew. Setting of relative paths for b

Hey, so I built this thing, most of it at so far at least. And yeah, right now it isn't doing many things better than Homebrew. Setting of relative paths for bottle installs is still not perfect, well it works for every bottle I have tested except rust. Getting bottles working 100% is very doable though imo. Build from source formulae is still pretty f*ed + I do not know if it is really feasible given that the json API lacks information there and a full on Ruby -> Rust transpiler is way out of scope. Will probably settle for automatic build system detection based on archive structure there. + Maybe do my own version of the .rb scripts but in a more general machine readable format, not .rs lol Casks seem to work but I have only tested some .dmg -> .app ones and .pkg installers so far though. As with bottles 100% doable. Given that almost all formulae are available as bottles for modern ARM mac this could become a fully featured package manager. Actually didn't think so many people would look at it, started building it for myself because Homebrew just isn't cutting it for what I want. Started working on a declarative package + system manager for mac because I feel ansible is overkill for one machine and not really made for that and nix-darwin worms itself into the system so deep. Wrapping Brew commands was abysmally slow though so I started working on this and by now I am deep enough in I won't stop xD Anyway I am grateful for every bug report, Issue and well meaning pull request.

AlphaSite 1y ago
Can you just shell out to homebrew for unsupported cases? I don’t imagine the overhead of ruby will be that high compared to compiling the code.
jrochkind1 1y ago
What makes you interested in a rust implementation of brew? I'm guessing it's that you hoping that it is eventually more performant -- are there specific areas of current brew you have identified as performance bottlenecks likely to eventually benefit from a rust implementation? Or any more info to share about assumptions/hopes that motivated this or any other motivations?
- dijit 1y ago
  I say this from ignorance, but coming from a lineage of linux package managers; brew must be doing something wrong - and upon immediate introspection I doubt that its language specific. The performance of apt/dnf in comparison is surreal; but dnf (or at least yum, its predecessor) is written in Python; which has even worse performance characteristics than Ruby. Clearly something is wrong, I wonder how different they are architecturally.
  - woodruffw 1y ago
    Have you tried Homebrew in the last year or so? I think a lot of people have an impression of Homebrew's performance from the "bad old days," i.e. back when Homebrew had to evaluate every single local formula file to perform any operations at all. (There's still low handing fruit, but it's not like it was a few years ago where `brew list` took seconds to run. It now runs nearly instantaneously for me locally, like most of the other happy path commands.)
    - dijit 1y ago
      Yes, I use it on my daily driver, and it's at least two-orders of magnitude slower than apt. However, the speed increase coincided with my upgrade to an M-Series laptop, so it's possible I just presumed there was a significant hardware speedup in the time we're talking about.
    - WD-42 1y ago
      Another performance characteristic that used to piss me off when I used brew was that at least for one program, it had to invoke brew to launch. Something about finding a path. So that's an entire ruby interpreter starting and stopping just to (in this case) run a bash script. Horribly slow.
      - woodruffw 1y ago
        That’s been fixed for at least a year! But yes, that used to be very slow, particularly when loading brew also meant loading every formula.
nicoburns 1y ago
Perhaps you could embed something like https://github.com/artichoke/artichoke https://github.com/artichoke/artichoke to run the Ruby scripts for compatibility.
- adamnemecek 1y ago
  Please don't.
- alexykn 1y ago
  Looked at that. I do not really think implementing something like artichoke or rutie would be a good idea. I do not want my project to become overly bloated and to achieve my real goal of a declarative system management thing I think sticking to bottles (that cover almost all formulae thanks to the amazing homebrew community) and casks, getting those to work 100% is the better approach. Thank you for the suggestion though!
yincrash 1y ago
One thing that Homebrew does not do easily is to easily allow for creation of universal libraries and binaries - https://github.com/orgs/Homebrew/discussions/4647 https://github.com/orgs/Homebrew/discussions/4647 Maybe that could be a place where sapphire differentiates?
- deleted 1y ago
  [deleted]
Scramblejams 1y ago
Cool project, good luck with it! If I may surface one use case: Several years ago I had to manage a bunch of Macs for CI jobs. The build process (Unreal's UAT) didn't support running more than one build process at a time, and Docker was really slow, so I'd hoped to use different user accounts to bypass that and get some parallelization gains. Homebrew made that very difficult with its penchant for system-wide installs. So a feature request: I'd love to see a competitive package manager that limits itself to operating somewhere (overridable) in the user's home directory.
- alexykn 1y ago
  Initial idea for this really came from my dayjob too, we have macs but no way to centrally manage them. The client / server part for the declarative system manager I want to build on top of this is quite far out yet though. At least several months
- amarshall 1y ago
  Nix effectively has per-user packages, but it’s hard to read into your full use case from your comment.
- watermelon0 1y ago
  IIRC the main reason here is that brew path is hardcoded during the build process of packages, which means that you wouldn't be able to use bottles. I didn't check, but there is a chance that path is also hardcoded in (some) formulae, so even building from the source might not help here.
  - scribu 1y ago
    You could run the build process with chroot or inside Docker, so that the hardcoded paths actually resolve to a designated subdirectory.
    - mananaysiempre 1y ago
      Incidentally, that’s what is usually done in Nixpkgs in similar situations when there’s no better alternative, see buildFHSEnv et al.
3np 1y ago
> probably settle for automatic build system detection based on archive structure there Please add knobs for the end user to manually configure this per package and global default before adding autodetection. As a user to is very frustrating to have to patch the package manager to override some well-intentioned automagic which didn't consider my setup or dig through sources to uncover some undocumented assumption. yarn is a cautionary example.
- alexykn 1y ago
  I'll add manual override flags and also let users not only build from source from formulae but any dir on their machine they want, only makes sense
samhclark 1y ago
You mentioned a declarative package manager for Mac. I've really liked using Homebrew Bundle [1] over the last couple years. It's about the level of declarative that I've wanted and has made it really easy to bootstrap new laptop or VM (since it also works on Linux). The format for a Brewfile was pretty easy to figure out. The way I ended up using it was that `brew install` would temporarily install something, without adding it to my Brewfile. And a little `brew add` wrapper would add the package to my Brewfile to keep it on the system permanently. That part with the wrapper could have used some love and would be a nice fit for a new brew-compatible frontend IMO. Maybe you could expand on that for Sapphire, if that also scratches your declarative itch? [1] https://docs.brew.sh/Brew-Bundle-and-Brewfile https://docs.brew.sh/Brew-Bundle-and-Brewfile
- JimDabell 1y ago
  That would be fantastic. My bootstrap scripts already use bundle but it’s easy to fall into config drift.
NewJazz 1y ago
Just FYI you are describing a ports system, of which there are many. From macports, to alpine packages, to nixpkgs... Tons of research and automation in this area already. Would encourage looking at Nix and distri in particular. Many Nix packages can be built on/for macOS! https://nixos.org/manual/nixpkgs/stable/#preface https://nixos.org/manual/nixpkgs/stable/#preface https://michael.stapelberg.ch/posts/tags/distri/ https://michael.stapelberg.ch/posts/tags/distri/
- alexykn 1y ago
  I know nix, did use nix-darwin for a while and actually integrated my own scripts called from the flake into setting up my system xD
internet2000 1y ago
Good luck. Homebrew hasn't cut it since they started disallowing custom compile flags, and making the user jump through hoops if they want anything slightly different.
stevage 1y ago
Does the "casks" and "bottles" language imply that it's intended be compatible with Homebrew? That isn't made explicit in the description.
- MatthiasPortzel 1y ago
  Yes, this is only a replacement for the Homebrew CLI. It doesn’t have its own package repository and moreover it doesn’t have the ability to build packages (yet)—it’s just downloading and installing the binaries built by Homebrew.
godelski 1y ago
This is awesome! Is there uv support?[0] One of my biggest gripes about brew is how they manage dependencies. The devs have a conflicting philosophy that creates bloat. Package maintainers must define settings, settings should always use the latest Python version. It makes no sense. Maintainers won't update unless things break so you got a bunch of Python versions running around. And it won't use system Python! Uv seems to provide an elegant solution for this. You can build a venv for each package and rust version will only have the specified deps. Since uv finds all your Python instances (and packages) and soft links them you have way less bloat and venvs become really useful. You can also use run and other tools to handle executables Plus is also rust so good synergy ;) [0] https://astral.sh/blog/uv https://astral.sh/blog/uv
- Kwpolska 1y ago
  You should only be using Homebrew for installing software that happens to be written in Python, not dependencies for your own Python projects. If you do that, the Python version does not matter, it's just whatever version is required to make the package work.
  - godelski 1y ago
    > not dependencies for your own Python projects I'm not doing that. Honestly I'm not sure how to do that and it sounds like a real pain. > the Python version does not matter This is incorrect. Go check what versions of Python brew has installed for you. It's definitely not your system version... It's not "what works" it is "what the maintainer specified". And according to the brew devs this is supposed to be /the latest version that works/. Which was my point. People don't update just on a Python change. That's not going to happen without automation. (I even suggested we be allowed to specify the minimum version and I was told it's maintainer's responsibilities). You can trivially find packages that can be used with newer versions of Python than their brew formulas specify.
    - pm215 1y ago
      My experience was actually the opposite -- a package I used broke because somebody did an automated "bump the python version this package uses" change without noticing that upstream for the package hadn't yet made a release with the necessary changes to make it work with that python version, so it fell over on startup. (They were quick with reverting it when I reported the problem.)
miki123211 1y ago
> Build from source formulae is still pretty f*ed + I do not know if it is really feasible given that the json API lacks information there and a full on Ruby -> Rust transpiler is way out of scope. Will probably settle for automatic build system detection based on archive structure there. + Maybe do my own version of the .rb scripts but in a more general machine readable format, not .rs lol Last I checked (which was about a year ago), Homebrew had ~7000 formulas (not including casks). I think it would be feasible to transcribe most of them to your format of choice with AI, run the build in a loop and ask the LLM to fix errors, and reserve manual intervention for the few cases that the LLM can't fix.
- alexykn 1y ago
  Yeah, this would 100% be feasible. Great idea. It's during the week now so I don't have much time besides work. I'll probably open a Poll on git within the next couple days about what to do about a real from source packaging system with it's own dsl, maybe just yaml/toml or something a bit more powerful like lua with mlua. No matter the choice of packaging apporach I would like to keep the generated packages "bottle compliant" -> at least with the json api spec, since this also installs from there. There seems to be at least some in my project so even though I am very opinionated on a lot of things I don't think I should decide on that completely alone, especially given that this is the first time I play with packaging.
  - alexykn 1y ago
    on github xD

AlphaSite 1y ago

Can you just shell out to homebrew for unsupported cases? I don’t imagine the overhead of ruby will be that high compared to compiling the code.

jrochkind1 1y ago

What makes you interested in a rust implementation of brew? I'm guessing it's that you hoping that it is eventually more performant -- are there specific areas of current brew you have identified as performance bottlenecks likely to eventually benefit from a rust implementation? Or any more info to share about assumptions/hopes that motivated this or any other motivations?

dijit 1y ago
I say this from ignorance, but coming from a lineage of linux package managers; brew must be doing something wrong - and upon immediate introspection I doubt that its language specific. The performance of apt/dnf in comparison is surreal; but dnf (or at least yum, its predecessor) is written in Python; which has even worse performance characteristics than Ruby. Clearly something is wrong, I wonder how different they are architecturally.
- woodruffw 1y ago
  Have you tried Homebrew in the last year or so? I think a lot of people have an impression of Homebrew's performance from the "bad old days," i.e. back when Homebrew had to evaluate every single local formula file to perform any operations at all. (There's still low handing fruit, but it's not like it was a few years ago where `brew list` took seconds to run. It now runs nearly instantaneously for me locally, like most of the other happy path commands.)
  - dijit 1y ago
    Yes, I use it on my daily driver, and it's at least two-orders of magnitude slower than apt. However, the speed increase coincided with my upgrade to an M-Series laptop, so it's possible I just presumed there was a significant hardware speedup in the time we're talking about.
  - WD-42 1y ago
    Another performance characteristic that used to piss me off when I used brew was that at least for one program, it had to invoke brew to launch. Something about finding a path. So that's an entire ruby interpreter starting and stopping just to (in this case) run a bash script. Horribly slow.
    - woodruffw 1y ago
      That’s been fixed for at least a year! But yes, that used to be very slow, particularly when loading brew also meant loading every formula.

nicoburns 1y ago

Perhaps you could embed something like https://github.com/artichoke/artichoke https://github.com/artichoke/artichoke to run the Ruby scripts for compatibility.

adamnemecek 1y ago
Please don't.
alexykn 1y ago
Looked at that. I do not really think implementing something like artichoke or rutie would be a good idea. I do not want my project to become overly bloated and to achieve my real goal of a declarative system management thing I think sticking to bottles (that cover almost all formulae thanks to the amazing homebrew community) and casks, getting those to work 100% is the better approach. Thank you for the suggestion though!

yincrash 1y ago

One thing that Homebrew does not do easily is to easily allow for creation of universal libraries and binaries - https://github.com/orgs/Homebrew/discussions/4647 https://github.com/orgs/Homebrew/discussions/4647 Maybe that could be a place where sapphire differentiates?

deleted 1y ago
[deleted]

Scramblejams 1y ago

Cool project, good luck with it! If I may surface one use case: Several years ago I had to manage a bunch of Macs for CI jobs. The build process (Unreal's UAT) didn't support running more than one build process at a time, and Docker was really slow, so I'd hoped to use different user accounts to bypass that and get some parallelization gains. Homebrew made that very difficult with its penchant for system-wide installs. So a feature request: I'd love to see a competitive package manager that limits itself to operating somewhere (overridable) in the user's home directory.

alexykn 1y ago
Initial idea for this really came from my dayjob too, we have macs but no way to centrally manage them. The client / server part for the declarative system manager I want to build on top of this is quite far out yet though. At least several months
amarshall 1y ago
Nix effectively has per-user packages, but it’s hard to read into your full use case from your comment.
watermelon0 1y ago
IIRC the main reason here is that brew path is hardcoded during the build process of packages, which means that you wouldn't be able to use bottles. I didn't check, but there is a chance that path is also hardcoded in (some) formulae, so even building from the source might not help here.
- scribu 1y ago
  You could run the build process with chroot or inside Docker, so that the hardcoded paths actually resolve to a designated subdirectory.
  - mananaysiempre 1y ago
    Incidentally, that’s what is usually done in Nixpkgs in similar situations when there’s no better alternative, see buildFHSEnv et al.

3np 1y ago

> probably settle for automatic build system detection based on archive structure there Please add knobs for the end user to manually configure this per package and global default before adding autodetection. As a user to is very frustrating to have to patch the package manager to override some well-intentioned automagic which didn't consider my setup or dig through sources to uncover some undocumented assumption. yarn is a cautionary example.

alexykn 1y ago
I'll add manual override flags and also let users not only build from source from formulae but any dir on their machine they want, only makes sense

samhclark 1y ago

You mentioned a declarative package manager for Mac. I've really liked using Homebrew Bundle [1] over the last couple years. It's about the level of declarative that I've wanted and has made it really easy to bootstrap new laptop or VM (since it also works on Linux). The format for a Brewfile was pretty easy to figure out. The way I ended up using it was that `brew install` would temporarily install something, without adding it to my Brewfile. And a little `brew add` wrapper would add the package to my Brewfile to keep it on the system permanently. That part with the wrapper could have used some love and would be a nice fit for a new brew-compatible frontend IMO. Maybe you could expand on that for Sapphire, if that also scratches your declarative itch? [1] https://docs.brew.sh/Brew-Bundle-and-Brewfile https://docs.brew.sh/Brew-Bundle-and-Brewfile

JimDabell 1y ago
That would be fantastic. My bootstrap scripts already use bundle but it’s easy to fall into config drift.

NewJazz 1y ago

Just FYI you are describing a ports system, of which there are many. From macports, to alpine packages, to nixpkgs... Tons of research and automation in this area already. Would encourage looking at Nix and distri in particular. Many Nix packages can be built on/for macOS! https://nixos.org/manual/nixpkgs/stable/#preface https://nixos.org/manual/nixpkgs/stable/#preface https://michael.stapelberg.ch/posts/tags/distri/ https://michael.stapelberg.ch/posts/tags/distri/

alexykn 1y ago
I know nix, did use nix-darwin for a while and actually integrated my own scripts called from the flake into setting up my system xD

internet2000 1y ago

Good luck. Homebrew hasn't cut it since they started disallowing custom compile flags, and making the user jump through hoops if they want anything slightly different.

stevage 1y ago

Does the "casks" and "bottles" language imply that it's intended be compatible with Homebrew? That isn't made explicit in the description.

MatthiasPortzel 1y ago
Yes, this is only a replacement for the Homebrew CLI. It doesn’t have its own package repository and moreover it doesn’t have the ability to build packages (yet)—it’s just downloading and installing the binaries built by Homebrew.

godelski 1y ago

This is awesome! Is there uv support?[0] One of my biggest gripes about brew is how they manage dependencies. The devs have a conflicting philosophy that creates bloat. Package maintainers must define settings, settings should always use the latest Python version. It makes no sense. Maintainers won't update unless things break so you got a bunch of Python versions running around. And it won't use system Python! Uv seems to provide an elegant solution for this. You can build a venv for each package and rust version will only have the specified deps. Since uv finds all your Python instances (and packages) and soft links them you have way less bloat and venvs become really useful. You can also use run and other tools to handle executables Plus is also rust so good synergy ;) [0] https://astral.sh/blog/uv https://astral.sh/blog/uv

Kwpolska 1y ago
You should only be using Homebrew for installing software that happens to be written in Python, not dependencies for your own Python projects. If you do that, the Python version does not matter, it's just whatever version is required to make the package work.
- godelski 1y ago
  > not dependencies for your own Python projects I'm not doing that. Honestly I'm not sure how to do that and it sounds like a real pain. > the Python version does not matter This is incorrect. Go check what versions of Python brew has installed for you. It's definitely not your system version... It's not "what works" it is "what the maintainer specified". And according to the brew devs this is supposed to be /the latest version that works/. Which was my point. People don't update just on a Python change. That's not going to happen without automation. (I even suggested we be allowed to specify the minimum version and I was told it's maintainer's responsibilities). You can trivially find packages that can be used with newer versions of Python than their brew formulas specify.
  - pm215 1y ago
    My experience was actually the opposite -- a package I used broke because somebody did an automated "bump the python version this package uses" change without noticing that upstream for the package hadn't yet made a release with the necessary changes to make it work with that python version, so it fell over on startup. (They were quick with reverting it when I reported the problem.)

miki123211 1y ago

> Build from source formulae is still pretty f*ed + I do not know if it is really feasible given that the json API lacks information there and a full on Ruby -> Rust transpiler is way out of scope. Will probably settle for automatic build system detection based on archive structure there. + Maybe do my own version of the .rb scripts but in a more general machine readable format, not .rs lol Last I checked (which was about a year ago), Homebrew had ~7000 formulas (not including casks). I think it would be feasible to transcribe most of them to your format of choice with AI, run the build in a loop and ask the LLM to fix errors, and reserve manual intervention for the few cases that the LLM can't fix.

alexykn 1y ago
Yeah, this would 100% be feasible. Great idea. It's during the week now so I don't have much time besides work. I'll probably open a Poll on git within the next couple days about what to do about a real from source packaging system with it's own dsl, maybe just yaml/toml or something a bit more powerful like lua with mlua. No matter the choice of packaging apporach I would like to keep the generated packages "bottle compliant" -> at least with the json api spec, since this also installs from there. There seems to be at least some in my project so even though I am very opinionated on a lot of things I don't think I should decide on that completely alone, especially given that this is the first time I play with packaging.
- alexykn 1y ago
  on github xD