5 ms·

I don't really understand why this is happening at this scale, it's not like they just became broke and can't afford a proper server... can someone explain?

plufz 4mo ago
See previous days articles. Agentic coding. Going from 1b annual commits to estimated 14b or more from one year to another.
embedding-shape 4mo ago
The faster you move, the more you screw up, almost no company producing software have figured out how to move fast and not screw up. It's so hard, that companies even used to boast about how much they didn't care about screwing up, as long as they moved fast. Add in new "productivity" tools that help you move even faster, with even less regards for how much you screw up (even though the tool could be used for you to move at the same speed, but with less screw ups), and an engineering culture which boils down to "Why not?", and you get platforms run by Microsoft that are unable to achieve two nines of reliability.
dicksent 4mo ago
ai
fareesh 4mo ago
Agents are shipping code faster all over the world and in some cases 24 hours a day. Additionally, some significant number of non-developers are now developers i.e. they are also shipping to github regularly. This is not limited to just pushing code but all the bells and whistles that github added as features under the assumption of some predictable growth are now exceeding the original plans. I suspect a lot of their existing systems have to be re-architected for unanticipated scale, and it won't happen overnight for sure.
- prepend 4mo ago
  They were sucking 5 years ago before agents existed. I don’t think this has anything to do with recent changes. https://damrnelson.github.io/github-historical-uptime/ https://damrnelson.github.io/github-historical-uptime/
  - p-e-w 4mo ago
    Whoa, if that is even remotely accurate then the talk about agents is a complete red herring.
    - theolivenbaum 4mo ago
      If I remember correctly the status page was not precise before the acquisition - so take with a big grain of salt the 100% pre-acquisition values
      - prepend 4mo ago
        I remember the status page being quite accurate before the acquisition. I don’t like this whole casting of doubt upon sources without providing superior or even alternate sources. It makes it hard to discuss when one person presents a source and another says “I’m not sure that is accurate.” In a vague way. What am I supposed to do with that? Research more sources that may or may not align with how you feel?
  - Octoth0rpe 4mo ago
    Pretty damning. Would also be interesting to see the number of commits overlayed. The graph tells a great story about the correlation with MS's takeover, but I wonder if at the same time that uptime went to shit, MS was shifting over large numbers of enterprise contracts to github. That would be a more complete story IMO. None of which excuses this. Can you imagine someone's reaction in 2017 if you told them that github would be below 90% uptime in 2026? It would be unimaginable.
  - sarchertech 4mo ago
    That’s nonsense. GitHub didn’t have 100% uptime before 2020. I remember downtime back then. And Microsoft didn’t make changes that fast. The only thing that changed is the accuracy of their status page. Also go back and look at the unofficial status page from 3 years ago. It’s regularly above 99% and has been dropping steadily since then. Then in the last 3 months has dropped to below 85%.
- potatoman22 4mo ago
  [flagged]
baq 4mo ago
They’re on track to 30x volume yoy by their own words
prepend 4mo ago
I suspect it’s caused because Microsoft is using buggy Microsoft tech instead of the original stack. They’re making political decisions based on what they sell vs what’s actually useful for their use case. It’s kind of impossible to find out if this is true though.
- u_fucking_dork 4mo ago
  That doesn’t track because GitHub Enterprise Cloud has great uptime. This is all load based, vibe coded ai slop code shipped at record numbers from users who will never convert to paid. The real question is what are they doing about that?
  - prepend 4mo ago
    How is github enterprise cloud uptime tracked?
philipwhiuk 4mo ago
Most of the outages are actually the unavailability of single AI models, not the core service.

plufz 4mo ago

See previous days articles. Agentic coding. Going from 1b annual commits to estimated 14b or more from one year to another.

embedding-shape 4mo ago

The faster you move, the more you screw up, almost no company producing software have figured out how to move fast and not screw up. It's so hard, that companies even used to boast about how much they didn't care about screwing up, as long as they moved fast. Add in new "productivity" tools that help you move even faster, with even less regards for how much you screw up (even though the tool could be used for you to move at the same speed, but with less screw ups), and an engineering culture which boils down to "Why not?", and you get platforms run by Microsoft that are unable to achieve two nines of reliability.

dicksent 4mo ago

fareesh 4mo ago

Agents are shipping code faster all over the world and in some cases 24 hours a day. Additionally, some significant number of non-developers are now developers i.e. they are also shipping to github regularly. This is not limited to just pushing code but all the bells and whistles that github added as features under the assumption of some predictable growth are now exceeding the original plans. I suspect a lot of their existing systems have to be re-architected for unanticipated scale, and it won't happen overnight for sure.

prepend 4mo ago
They were sucking 5 years ago before agents existed. I don’t think this has anything to do with recent changes. https://damrnelson.github.io/github-historical-uptime/ https://damrnelson.github.io/github-historical-uptime/
- p-e-w 4mo ago
  Whoa, if that is even remotely accurate then the talk about agents is a complete red herring.
  - theolivenbaum 4mo ago
    If I remember correctly the status page was not precise before the acquisition - so take with a big grain of salt the 100% pre-acquisition values
    - prepend 4mo ago
      I remember the status page being quite accurate before the acquisition. I don’t like this whole casting of doubt upon sources without providing superior or even alternate sources. It makes it hard to discuss when one person presents a source and another says “I’m not sure that is accurate.” In a vague way. What am I supposed to do with that? Research more sources that may or may not align with how you feel?
- Octoth0rpe 4mo ago
  Pretty damning. Would also be interesting to see the number of commits overlayed. The graph tells a great story about the correlation with MS's takeover, but I wonder if at the same time that uptime went to shit, MS was shifting over large numbers of enterprise contracts to github. That would be a more complete story IMO. None of which excuses this. Can you imagine someone's reaction in 2017 if you told them that github would be below 90% uptime in 2026? It would be unimaginable.
- sarchertech 4mo ago
  That’s nonsense. GitHub didn’t have 100% uptime before 2020. I remember downtime back then. And Microsoft didn’t make changes that fast. The only thing that changed is the accuracy of their status page. Also go back and look at the unofficial status page from 3 years ago. It’s regularly above 99% and has been dropping steadily since then. Then in the last 3 months has dropped to below 85%.
potatoman22 4mo ago
[flagged]

baq 4mo ago

They’re on track to 30x volume yoy by their own words

prepend 4mo ago

I suspect it’s caused because Microsoft is using buggy Microsoft tech instead of the original stack. They’re making political decisions based on what they sell vs what’s actually useful for their use case. It’s kind of impossible to find out if this is true though.

u_fucking_dork 4mo ago
That doesn’t track because GitHub Enterprise Cloud has great uptime. This is all load based, vibe coded ai slop code shipped at record numbers from users who will never convert to paid. The real question is what are they doing about that?
- prepend 4mo ago
  How is github enterprise cloud uptime tracked?

philipwhiuk 4mo ago

Most of the outages are actually the unavailability of single AI models, not the core service.