6 ms·

Any comment on the costs estimated in the article? especially the buildkite costs?

• Jarred 2mo ago
  CI has always been expensive for Bun including before the acquisition. We build for [macOS, Linux, FreeBSD, Android, Windows] x [ARM64, x64] and then run tests on multiple Linux distros with multiple shards, multiple macOS versions and Windows for each architecture. We recently started cross-compiling all the builds on Linux arm64 and that made it a little faster (I wrote a CLI tool to download the correct macOS headers for cross-compilation). We also have a daily cron job that asks claude to make the slowest tests faster while adding more assertions.
  • JustSkyfall 2mo ago
    How come Bun uses Buildkite instead of self-hosting the CI infra?
    • simonw 2mo ago
      Presumably because they "build for [macOS, Linux, FreeBSD, Android, Windows] x [ARM64, x64]" and self hosting all of that would be time-consuming and expensive.
    • CamouflagedKiwi 2mo ago
      Probably because they don't want to self-host Windows or MacOS servers when they can pay someone else to do that for them (or Linux ones, I assume that is within their wheelhouse for production but CI is a bit of a different beast to model inference).
    • Jarred 2mo ago
      It spawns ephemeral EC2/Azure instances, which is a lot cheaper than the GitHub actions runners we used before that. We shard to a lot of machines for tests and I’d be worried about running out if we used dedicated servers. BuildKite is fine but I wouldn’t be that surprised if we move off of BuildKite to a custom thing at some point. Months ago, we switched from CMake to a handrolled typescript build system and it made our builds faster and simpler.
      • JustSkyfall 2mo ago
        Thanks for answering! I suppose my second question would be how much Bun spends on CI per month atm - I'd imagine that robobun would have ramped up the bill a fair bit
      • latchkey 2mo ago
        you don't need buildkite or blacksmith anymore. codex trivially built me a self-hosted gh actions runner workflow for ephemeral vm's (just a big bash script that manages the vms with qemu). i even sped up the builds with my own custom base image with everything installed in it that i need too. it works flawlessly. fanning that out to starting and stopping instances wouldn't be too much of a stretch.
        • m00dy 2mo ago
          Does it work with macOS ?
          • latchkey 2mo ago
            weird question, i don't see why not. osx has a fantastic virtualization story. https://developer.apple.com/documentation/virtualization/running-macos-in-a-virtual-machine-on-apple-silicon https://developer.apple.com/documentation/virtualization/run...
            • m00dy 2mo ago
              bro, I was a committer for libvirt
  • sandeepkd 2mo ago
    I think the question for CI costs is still out there. While I do not think it should be in tunes of thousands a day but thats the skepticism presented in the article. True costs are really important to make a good decision in situations like this. One has to consider the fact that lots of people are going to use these numbers to justify the rewrite in future.
  • jojobas 2mo ago
    So are the "$800k + CI" estimates accurate? And your team works weekends just to make tests faster?