8 ms·

"We are committed to fixing these problems, as long as it doesn't involve buying things other than AI computers, hiring humans, or using non-Microsoft products.

"We are committed to fixing these problems, as long as it doesn't involve buying things other than AI computers, hiring humans, or using non-Microsoft products." Calling Azure the solution to this problem when it is in fact the source of most of these problems is just fantastic doublespeak. Github is ripe for disruption and I hope it is disrupted soon.

mort96 28d ago
If you're a big company, you can afford having one engineer spend one or two days per year to maintain your self-hosted GitLab or Forgejo. On top of better reliability than GitHub, you'll get the additional bonus that your source code won't accidentally leak through being in Copilot's training set. If you're a hobbyist, Codeberg is great, has a nice community and automatically shields you from slop contributions.
- ivraatiems 28d ago
  The issue with these systems is that they lack Github's sophistication for issue tracking, knowledge transfer, and automation. I think Gitlab is a mature product in its own space and unlikey to change, for instance, at this point. Codeberg also has the issue of having a political stance which means they will not accept just anyone's use of the platform. That is absolutely their right and I have no issue with it, but it's unattractive to me - as someone who agrees with most of their current politics - because the day they decide they don't like me, I'm screwed.
  - mort96 28d ago
    I never found GitHub's systems for issue tracking to be all that great. Cross-repository issues and development plans are hard to track within a git host. I've always used an external panning and issue tracking tool, mostly Linear, and it works really well. GitLab's Linear integration is excellent, FWIW. I've actually worked with a couple of companies who do use GitHub for their code, and they all use Linear in addition to GitHub. I understand the concern you're talking about wrt. Codeberg, but I wouldn't view it as a significantly bigger risk than anything else. Any platform can suddenly decide that your project is against ToS (GitHub will absolutely not accept just anyone's use of their platform either) and Codeberg introducing some rules recently doesn't, in my mind, drastically increase the risk of a dramatic ToS change in the future. But we all have to make our own risk evaluations and I won't judge yours. Luckily, moving between Git hosts isn't that difficult; setting up CI again and losing merge request history does suck but it's not the end of the world, unlike something like, say, losing your AWS/GCP/whatever account.
  - Shish2k 28d ago
    "sophistication" seems like a strange way to describe GitHub to me - I've found in every individual aspect (code browsing, issue tracking, code review, package management, etc), it's the worst out of all the systems I use regularly... But it's good _enough_ for most people, and it has all those features in one place, which is more convenient than wrangling 10-15 high quality but disconnected systems
  - rcleveng 28d ago
    Gitlab has the benefit of having very little traffic, both free and paid. Their limits are still way above the current usage so less likely to be an issue
    - mh- 28d ago
      Worth mentioning GitLab's paid enterprise offering are more expensive than GitHub's, on a per-seat basis. Lots of companies moved because it was cheap, but it's not anymore. Ironic that companies might choose to migrate to them now for stability, rather than price.
    - otterley 27d ago
      GitLab has its own problems. It might be somewhat more available than GitHub lately, but it too is not immune to incidents, particularly involving workflow dispatch and managed runner availability. Plus the UX isn’t nearly as good. Sometimes it feels downright clunky in comparison.
  - shimman 28d ago
    Dude no one uses github tracking for anything serious, come on. The only thing github has over gitea/forgejo is discussions. That's the only real social "innovation" github has contributed to open source development and it's just a shitty tacked on forum. Also for accuracy, Codeberg has a pro-human and anti-corporation stance. Both of which are definitely en vogue at the moment. Much better than GitHub's pro slop sentiment, which is doing nothing but destroy their reputation.
- unrented7977 28d ago
  Speaking from experience, it cost mW about a week or two per year to maintain GitLab for the startup I worked at. My personal GitLab on the other hand really does take only a day or two per year. That said, a week or two per year is just what it costs to maintain any one thing period. I spent about that much time maintaining PCs in the office, or my personal proxmox setup. It's not onerous at all. GitLab is super bloated and a little sucky to admin, but it's not too bad all things considered. I'm admin in my new job's GitHub org and it sucks a whole lot more to maintain.
dcrazy 28d ago
> We installed as much hardware as available power allowed in our existing data centers while accelerating our migration to Azure. And from the RCA [1]: > The immediate cause of the failure was network saturation on load balancers in Central US due to a new peak in traffic. [1]: https://www.githubstatus.com/incidents/zkxwbgr0cnmx https://www.githubstatus.com/incidents/zkxwbgr0cnmx
- ivraatiems 28d ago
  "While accelerating our migration to Azure," meaning, they will only solve problems if it helps them also use Azure more. It is unbelivable that aload of 2.8b commits was totally fine, and a load of 2.9b was a sitewide outage, unless they have no reporting or their tooling is completely incompetent. If things can fall apart so easily, throwing more capacity at the problem won't fix it.
  - dcrazy 28d ago
    You’re torturing your own logic to make Azure the villain here. And it also sounds like you lack experience with capacity exhaustion. Things fail slowly, then suddenly.
    - ivraatiems 28d ago
      [flagged]
      - dcrazy 28d ago
        Baseless accusation made from a position of zero information.
        ivraatiems 28d ago
        Opinion based on stated facts. Please share the information you have which contradicts the conclusions I have drawn from Github's statement. (And we know they're liars. They report very few of the actual incidents they have; see for example https://mrshu.github.io/github-statuses/ https://mrshu.github.io/github-statuses/)
awesome_dude 28d ago
> Github is ripe for disruption and I hope it is disrupted soon. It's an expensive, low revenue generating site. There are, and have always been, competitors, including "host it all yourself" solutions, but nothing has really stuck. How is it "ripe" for disruption?
- ivraatiems 28d ago
  They had $1b revenue in 2023 and now probably more than $2b in revenue... do you have cost figures showing what their expenses are?
  - bluedino 28d ago
    > We have since added more than 3 million CPU cores, 120 petabytes of high-speed storage, and significant network capacity. We installed as much hardware as available power allowed in our existing data centers while accelerating our migration to Azure. That can't be cheap.
    - cyberax 28d ago
      A server box now has around 256 CPU cores. So that's about 12000 servers. If each one is $10k that's $120 million. Not a lot compared to Github's income.
      - bluedino 28d ago
        Does that $10k server not need RAM?
        mh- 28d ago
        If parent commenter is off by an order of magnitude in their costs, their point still stands.
        Goronmon 28d ago
bpavuk 28d ago
I'm betting on Tangled and Codeberg. Tangled has a better press and in general is a dark horse, Codeberg has the "brand" and some network effects from projects that moved to there. (famously, Zig.) I heard that Sourcehut is having a moment as well, and I love the idea of email-based workflow and not having to have an account to contribute to someone's project hosted there, but I'm not maintaining anything worthwhile paying the $4/mo sub.
- DANmode 28d ago
  What if it was $2?
  - bpavuk 28d ago
    that is more manageable but c'mon I can't even keep Google One 100GB up on a consistent basis, that's how poor I am. self-hosting would be a far better option because apparently I find enough people to provide free Hetzner VPSes and stuff as long as I can sell this as mutually beneficial. for context, I would GLADLY move there my Neovim plugin. all it does is brings the current jj message into your editor and lets you integrate it with a status bar (or anything in nvim, really). that would be a decent measure against drive-by slop contributions, and I'd accept contribs over private github mirror from those who I know but can't bother setting up git mail EDIT: TIL that one can host SourceHut themselves. discoverability may still be a problem (sr.ht just ranks higher in search engines) but 1) fixable with github mirror that points to sourcehut instance as a canonical development platform, 2) it's moderately easy to sync contributions between tangled and sourcehut, so tangled is also an option EDIT 2: the email part would be PITA, so $4/mo is attractive on that background
    - fragmede 27d ago
      Oracle has a free tier that you can self host a git server on, if that's what you're looking for.
      - bpavuk 26d ago
        I spent a month trying to set this thing up, to no avail
kjellsbells 28d ago
Ok, but there's no universe where a major Microsoft-owned property is not being forced to run on Azure. Just like AWS pushing to get off Oracle back in the day. It would be career-destroying to suggest otherwise regardless of technical merit (and tbf, no infrastructure is bulletproof, unless you want to port GitHub to z/OS on mainframe)
- semiquaver 27d ago
  LinkedIn gave up after four years of trying: https://www.cnbc.com/amp/2023/12/14/linkedin-shelved-plan-to-migrate-to-microsoft-azure-cloud.html https://www.cnbc.com/amp/2023/12/14/linkedin-shelved-plan-to... Azure just has very poor performance and reliability characteristics. It’s a particularly bad migration target for a colo-based company that mainly runs on owned hardware (such as GitHub or LinkedIn). Requires much larger architecture changes than (say) a company coming from AWS.
joezydeco 27d ago
The Azure horror stories are on this blog post and HN discussion. https://news.ycombinator.com/item?id=47616242 https://news.ycombinator.com/item?id=47616242

mort96 28d ago

If you're a big company, you can afford having one engineer spend one or two days per year to maintain your self-hosted GitLab or Forgejo. On top of better reliability than GitHub, you'll get the additional bonus that your source code won't accidentally leak through being in Copilot's training set. If you're a hobbyist, Codeberg is great, has a nice community and automatically shields you from slop contributions.

ivraatiems 28d ago
The issue with these systems is that they lack Github's sophistication for issue tracking, knowledge transfer, and automation. I think Gitlab is a mature product in its own space and unlikey to change, for instance, at this point. Codeberg also has the issue of having a political stance which means they will not accept just anyone's use of the platform. That is absolutely their right and I have no issue with it, but it's unattractive to me - as someone who agrees with most of their current politics - because the day they decide they don't like me, I'm screwed.
- mort96 28d ago
  I never found GitHub's systems for issue tracking to be all that great. Cross-repository issues and development plans are hard to track within a git host. I've always used an external panning and issue tracking tool, mostly Linear, and it works really well. GitLab's Linear integration is excellent, FWIW. I've actually worked with a couple of companies who do use GitHub for their code, and they all use Linear in addition to GitHub. I understand the concern you're talking about wrt. Codeberg, but I wouldn't view it as a significantly bigger risk than anything else. Any platform can suddenly decide that your project is against ToS (GitHub will absolutely not accept just anyone's use of their platform either) and Codeberg introducing some rules recently doesn't, in my mind, drastically increase the risk of a dramatic ToS change in the future. But we all have to make our own risk evaluations and I won't judge yours. Luckily, moving between Git hosts isn't that difficult; setting up CI again and losing merge request history does suck but it's not the end of the world, unlike something like, say, losing your AWS/GCP/whatever account.
- Shish2k 28d ago
  "sophistication" seems like a strange way to describe GitHub to me - I've found in every individual aspect (code browsing, issue tracking, code review, package management, etc), it's the worst out of all the systems I use regularly... But it's good _enough_ for most people, and it has all those features in one place, which is more convenient than wrangling 10-15 high quality but disconnected systems
- rcleveng 28d ago
  Gitlab has the benefit of having very little traffic, both free and paid. Their limits are still way above the current usage so less likely to be an issue
  - mh- 28d ago
    Worth mentioning GitLab's paid enterprise offering are more expensive than GitHub's, on a per-seat basis. Lots of companies moved because it was cheap, but it's not anymore. Ironic that companies might choose to migrate to them now for stability, rather than price.
  - otterley 27d ago
    GitLab has its own problems. It might be somewhat more available than GitHub lately, but it too is not immune to incidents, particularly involving workflow dispatch and managed runner availability. Plus the UX isn’t nearly as good. Sometimes it feels downright clunky in comparison.
- shimman 28d ago
  Dude no one uses github tracking for anything serious, come on. The only thing github has over gitea/forgejo is discussions. That's the only real social "innovation" github has contributed to open source development and it's just a shitty tacked on forum. Also for accuracy, Codeberg has a pro-human and anti-corporation stance. Both of which are definitely en vogue at the moment. Much better than GitHub's pro slop sentiment, which is doing nothing but destroy their reputation.
unrented7977 28d ago
Speaking from experience, it cost mW about a week or two per year to maintain GitLab for the startup I worked at. My personal GitLab on the other hand really does take only a day or two per year. That said, a week or two per year is just what it costs to maintain any one thing period. I spent about that much time maintaining PCs in the office, or my personal proxmox setup. It's not onerous at all. GitLab is super bloated and a little sucky to admin, but it's not too bad all things considered. I'm admin in my new job's GitHub org and it sucks a whole lot more to maintain.

dcrazy 28d ago

> We installed as much hardware as available power allowed in our existing data centers while accelerating our migration to Azure. And from the RCA [1]: > The immediate cause of the failure was network saturation on load balancers in Central US due to a new peak in traffic. [1]: https://www.githubstatus.com/incidents/zkxwbgr0cnmx https://www.githubstatus.com/incidents/zkxwbgr0cnmx

ivraatiems 28d ago
"While accelerating our migration to Azure," meaning, they will only solve problems if it helps them also use Azure more. It is unbelivable that aload of 2.8b commits was totally fine, and a load of 2.9b was a sitewide outage, unless they have no reporting or their tooling is completely incompetent. If things can fall apart so easily, throwing more capacity at the problem won't fix it.
- dcrazy 28d ago
  You’re torturing your own logic to make Azure the villain here. And it also sounds like you lack experience with capacity exhaustion. Things fail slowly, then suddenly.
  - ivraatiems 28d ago
    [flagged]
    - dcrazy 28d ago
      Baseless accusation made from a position of zero information.
      - ivraatiems 28d ago
        Opinion based on stated facts. Please share the information you have which contradicts the conclusions I have drawn from Github's statement. (And we know they're liars. They report very few of the actual incidents they have; see for example https://mrshu.github.io/github-statuses/ https://mrshu.github.io/github-statuses/)

awesome_dude 28d ago

> Github is ripe for disruption and I hope it is disrupted soon. It's an expensive, low revenue generating site. There are, and have always been, competitors, including "host it all yourself" solutions, but nothing has really stuck. How is it "ripe" for disruption?

ivraatiems 28d ago
They had $1b revenue in 2023 and now probably more than $2b in revenue... do you have cost figures showing what their expenses are?
- bluedino 28d ago
  > We have since added more than 3 million CPU cores, 120 petabytes of high-speed storage, and significant network capacity. We installed as much hardware as available power allowed in our existing data centers while accelerating our migration to Azure. That can't be cheap.
  - cyberax 28d ago
    A server box now has around 256 CPU cores. So that's about 12000 servers. If each one is $10k that's $120 million. Not a lot compared to Github's income.
    - bluedino 28d ago
      Does that $10k server not need RAM?
      - mh- 28d ago
        If parent commenter is off by an order of magnitude in their costs, their point still stands.
        Goronmon 28d ago

bpavuk 28d ago

I'm betting on Tangled and Codeberg. Tangled has a better press and in general is a dark horse, Codeberg has the "brand" and some network effects from projects that moved to there. (famously, Zig.) I heard that Sourcehut is having a moment as well, and I love the idea of email-based workflow and not having to have an account to contribute to someone's project hosted there, but I'm not maintaining anything worthwhile paying the $4/mo sub.

DANmode 28d ago
What if it was $2?
- bpavuk 28d ago
  that is more manageable but c'mon I can't even keep Google One 100GB up on a consistent basis, that's how poor I am. self-hosting would be a far better option because apparently I find enough people to provide free Hetzner VPSes and stuff as long as I can sell this as mutually beneficial. for context, I would GLADLY move there my Neovim plugin. all it does is brings the current jj message into your editor and lets you integrate it with a status bar (or anything in nvim, really). that would be a decent measure against drive-by slop contributions, and I'd accept contribs over private github mirror from those who I know but can't bother setting up git mail EDIT: TIL that one can host SourceHut themselves. discoverability may still be a problem (sr.ht just ranks higher in search engines) but 1) fixable with github mirror that points to sourcehut instance as a canonical development platform, 2) it's moderately easy to sync contributions between tangled and sourcehut, so tangled is also an option EDIT 2: the email part would be PITA, so $4/mo is attractive on that background
  - fragmede 27d ago
    Oracle has a free tier that you can self host a git server on, if that's what you're looking for.
    - bpavuk 26d ago
      I spent a month trying to set this thing up, to no avail

kjellsbells 28d ago

Ok, but there's no universe where a major Microsoft-owned property is not being forced to run on Azure. Just like AWS pushing to get off Oracle back in the day. It would be career-destroying to suggest otherwise regardless of technical merit (and tbf, no infrastructure is bulletproof, unless you want to port GitHub to z/OS on mainframe)

semiquaver 27d ago
LinkedIn gave up after four years of trying: https://www.cnbc.com/amp/2023/12/14/linkedin-shelved-plan-to-migrate-to-microsoft-azure-cloud.html https://www.cnbc.com/amp/2023/12/14/linkedin-shelved-plan-to... Azure just has very poor performance and reliability characteristics. It’s a particularly bad migration target for a colo-based company that mainly runs on owned hardware (such as GitHub or LinkedIn). Requires much larger architecture changes than (say) a company coming from AWS.

joezydeco 27d ago

The Azure horror stories are on this blog post and HN discussion. https://news.ycombinator.com/item?id=47616242 https://news.ycombinator.com/item?id=47616242