7 ms·
Red Squares – GitHub outages as contributions
- pards 4mo agoThis design is perfect irony. I love it.
- keyle 4mo agoThis is one of the most creative idea I've seen this year. Tasteful and clever. Bravo!
- cyanydeez 4mo agodouble entendre: Is it load based or github-employee based that weekends are sparser. or just a multifactor of both.
- Shoetp 4mo agoYes
- globular-toast 4mo agoDidn't they blame "AI" for the increased load? I'm not sure why AI usage would be more during the week than the weekend, but it could be. It does look like Friday outages were a bit rarer, which could be due to having a "no deployments on Friday" rule.
- mirekrusin 4mo agoFrom the chart it seems they should have policy to deploy on weekends only.
- lnenad 4mo agoThe memes are really painful now. I feel for the team that's is trying to survive underwater.
- renegade-otter 4mo agoWith management screaming down their necks: YOU NEED TO USE MOAR AI!
- nojonestownpls 4mo ago$250k can do a good job of easing meme-induced pain. "survive underwater" what a joke. Yes, there will be good engineers there who will be sad to see it go this way, but they choose to be there, get paid better than 99% of humanity for that.
- lnenad 4mo agoYou are removing the humanity from these people. Yes some don't care and just take their salaries. But I'm sure some people if not majority is stressed due to the situation.
- bharxhav 4mo agoWould be interesting to see if this correlated with their release cycles.
- hosteur 4mo agoWell, outages seem to be distributed across all days except weekends. So this seems like people fucking around with stuff being a major factor.
- samlinnfer 4mo agoSurely it just means more people working, resulting in more load, resulting in more outages?
- pwagland 4mo agoOr even both. In any kind of continuous deployment, you'd expect outages at the point of deployment, or shortly thereafter as the unintended consequences ripple. Then the load during the working days makes those ripples larger and into outages.
- embedding-shape 4mo agoMost outages are caused by changes by humans ("actors"?), very rarely are things "People just dig our stuff so much we can't keep up" but more often "We didn't think about this performance drawback when we built thing X, now it's hurting us", and of course, more outages when you try to fix those issues without fully considering the scope and impact.
- rufo 4mo agoAt least as of when I left the company, GitHub was being deployed to fairly close to once every 60-90 minutes (the frequency of a deploy train/merge queue batch going out) 24 hours a day, at least during weekdays… there are a fair number of international engineers and deploy trains get crowded during main US business hours, so while there are fewer PRs going out at odd hours US time, there were typically still some. There aren’t dedicated releases as such for GitHub-hosted instances - everything you release needs to be gated behind a feature flag or other mechanism if it’s not going live immediately, and your code either needs to handle the database in both its pre- and post-migrated state, or you need to run the migration in advance of your code shipping out. Fun fact: it used to be the case that GitHub was actually _less_ reliable if nobody deployed to it… there used to be various resource leaks that we didn’t see when people were deploying all day, since then the app wasn’t getting restarted constantly. After GitHub went down during a holiday break we had volunteers to deploy GitHub once a day during holiday breaks, until the underlying issues were eventually fixed.
- figmert 4mo agoFar fewer outages during the weekends. Perfect, wasn't gonna do any work then anyway.
- sd9 4mo agoWeekends are the untapped frontier. Still room to scale.
- skor 4mo agochange is the biggest cause then?
- kshahkshah 4mo agoWait until they go 996!
- ahstilde 4mo agoyup! When i did an analysis last month, GitHub is up 89.3% on weekdays and 96.5% on weekends. Incidents touch 62% of weekdays and 11% of weekends. Claude shows the same pattern: 92.5% weekday, 97.8% weekend. Tuesday through Thursday is the danger zone. Sunday is practically a different service. https://www.aakash.io/tech-chase/github-and-claude-are-down-three-out-of-four-days https://www.aakash.io/tech-chase/github-and-claude-are-down-...
- jrumbut 4mo agoI had an occasion recently where I was working a lot of late nights/early mornings with AI use. And I'd be getting these instant, beautiful responses, and then, as soon as the sun started coming in the windows, it would take longer and fail more, and by the time the clock struck 9 AM, every LLM had turned back into a pumpkin.
- danfritz 4mo agoI wonder how well this corolates with azure incidents. Especially for the US regions.
- ngruhn 4mo agoI live in Europe. I've not noticed these constant outages. But I only use GitHub after work.
- progbits 4mo agoInteresting. I'm in EU and see these constantly but usually in the afternoon so it bothers me less as I'm already wrapping up, but my US coworkers are getting hit much worse.
- whirlwin 4mo agoI've started experiencing them first time during Q1 this year up until now. Used to be pretty stable before that.
- p2detar 4mo agoI also bet my money on Azure. Someone who allegedly worked there recently posted an article here on the numerous problems with Azure. Sadly I didn’t bookmark it.
- hosteur 4mo agoThe article you are thinking of was likely written by Axel Rietschin who worked on Azure core compute team. https://isolveproblems.substack.com/p/how-microsoft-vaporized-a-trillion https://isolveproblems.substack.com/p/how-microsoft-vaporize... HN thread: https://news.ycombinator.com/item?id=47616242 https://news.ycombinator.com/item?id=47616242
- chrisweekly 4mo agoWow. Yikes. I never liked Azure, but this level of dysfunction is just astonishing.
- philprx 4mo ago"Good job, Microsoft, amazing uptime."
- aykutseker 4mo ago[dead]
- Fokamul 4mo agoClearly their team needs more LLM usage.
- ramon156 4mo agoPlease tell me this makes sense This website has no overused ai-generated animations and... I quite enjoy it. The original website[1] has a fade-in animation, big round cards, shadows, all the jazz you can think of, it's there. This site is very readable, very honest and sober. I don't need to sift through buzzwords to figure out tiny details. Thank you, OP! 1: https://mrshu.github.io/github-statuses/ https://mrshu.github.io/github-statuses/
- airstrike 4mo agocan you correlate this to data on # of commits, actions, etc?
- korrectional 4mo agoI don't really understand why this is happening at this scale, it's not like they just became broke and can't afford a proper server... can someone explain?
- plufz 4mo agoSee previous days articles. Agentic coding. Going from 1b annual commits to estimated 14b or more from one year to another.
- embedding-shape 4mo agoThe faster you move, the more you screw up, almost no company producing software have figured out how to move fast and not screw up. It's so hard, that companies even used to boast about how much they didn't care about screwing up, as long as they moved fast. Add in new "productivity" tools that help you move even faster, with even less regards for how much you screw up (even though the tool could be used for you to move at the same speed, but with less screw ups), and an engineering culture which boils down to "Why not?", and you get platforms run by Microsoft that are unable to achieve two nines of reliability.
- dicksent 4mo agoai
- fareesh 4mo agoAgents are shipping code faster all over the world and in some cases 24 hours a day. Additionally, some significant number of non-developers are now developers i.e. they are also shipping to github regularly. This is not limited to just pushing code but all the bells and whistles that github added as features under the assumption of some predictable growth are now exceeding the original plans. I suspect a lot of their existing systems have to be re-architected for unanticipated scale, and it won't happen overnight for sure.
- prepend 4mo agoThey were sucking 5 years ago before agents existed. I don’t think this has anything to do with recent changes. https://damrnelson.github.io/github-historical-uptime/ https://damrnelson.github.io/github-historical-uptime/
- elAhmo 4mo agoFunny to see this closely match contribution graphs with effectively no downtime on weekends.
- jve 4mo agoA graph I have to question is even accurate. > Across 170 days with at least one incident · worst day Thu, Nov 20, 2025 (1.1 days) 1.1 days total how is that possible? Scrolling over that day doesn't indicate the math behind the scenes - 1.3 hours single bullet point. Also Nov 19 has a bullet point 1.3 day outage but total is 8.1 hours
- hxtk 4mo agoThe missing status page [1] treats it as downtime any time any component of the system is down, and calculates the overall uptime based on the time that doesn't overlap with any individual category outages, and the overall downtime as any time overlapping with at least one individual category outage to avoid double-counting They show 24h of minor outage on that date. I'm guessing that this site is taking the downtime in a given day across all services and adding it up, which would mean the worst possible day has 10 days of downtime (a day of downtime for each major category). 1: https://mrshu.github.io/github-statuses/ https://mrshu.github.io/github-statuses/
- thenewnewguy 4mo agoI see a bullet point for "1.0 days of 1.3 days", and when I mouse over the previous day (Wedensday 2025-11-19), I see "7.8 hours of 1.3 days". I haven't actually checked any sources to confirm there really was downtime on those days, but if we assume those numbers are true 7.8 hours + 1 day is about 1.3 days.
- rvz 4mo agoAnother reminder that a self hosted git repository would have more uptime than GitHub and centralizing everything to GitHub was a very bad idea. [0] [0] https://news.ycombinator.com/item?id=22867803 https://news.ycombinator.com/item?id=22867803
- efreak 4mo agoThat depends entirely on where you host it. My PC has more downtime than GitHub does. It's only up enough to push to it ~75% of the time (I do use it as a ssh remote fwiw, but most projects get pushed to a VM that only starts on login, and my PC itself isn't always running). If you're working on an oss project (ie distributing source anyways), probably better a good idea to mirror to GitHub as a backup anyways, let Microsoft pay for backup+bandwidth for you.
- faangguyindia 4mo agoAll these companies brag about being hyperscalers and cannot scale github. Similarly, i see google releasing advancement after advancement in LLM yet i see antigravity sub where people are crying all time.
- jpb0104 4mo agoSetup my self-hosted Forgejo last night. Very pleased so far.
- hosteur 4mo agoYeah me too. I moved all my public projects to codeberg and my internal repos to self-hosted forgejo. Hosting forgejo is really easy as well. It being a single binary makes it really easy to handle with almost zero maintenance.
- revolution88 4mo agoFor 30th of April, 2026 it shows it was down 1.0 days of 2.6 days (minor incident) :)
- debarshri 4mo agoWould be funny if you host it on github pages.
- Gigacore 4mo agoIt is funny how weekends are almost always up!
- WesSouza 4mo agoWell done.
- FaithMB 4mo agoI like this more than I expected. The intensity gradient is a nice touch too.
- dvh 4mo agoI didn't know azure was this bad, completely changed my opinion on their cloud offerings
- dijit 4mo agoReally? A 10 minute interaction with the platform was enough to inform me that no serious engineer is in charge, and no serious engineer chooses this platform. It is a platform for CFOs to avoid having another vendor relationship.
- chrisweekly 4mo agoIt's even worse than I'd imagined. See this peer comment w/ link to scathing analysis from an insider: https://news.ycombinator.com/item?id=48035171 https://news.ycombinator.com/item?id=48035171
- vaylian 4mo agoI know that there was a plan to move GitHub to Azure, but I don't know what the status is. It could very well be, that GitHub is not running on Azure yet.
- __natty__ 4mo agoContrast between official [0] and third party status pages [1] is huge. How their terms of service for SLA are legal if they are so different from real world usage of their product? I really like GitHub and their services but every time when it’s broken and their status page is green something screams inside me. [0] https://www.githubstatus.com/ https://www.githubstatus.com/ [1] https://mrshu.github.io/github-statuses/ https://mrshu.github.io/github-statuses/
- philipwhiuk 4mo ago> How their terms of service for SLA are legal if they are so different from real world usage of their product? Because the SLA likely doesn't consider some features of GitHub under the SLA, whereas an outage/issues for a single model is seen as problem on the Third Party page.
- xyzzy_plugh 4mo agoTheir terms of service are legal because their terms of service require YOU, the CUSTOMER, to track their availability against the agreed upon SLA and to pursue credits when they break their SLA. At a recent gig we experienced many, many GitHub outages that were not tracked on their status page, and we kept a log (i.e. just search in slack). After our business people argued with our account executives at GitHub we got hundreds of dollars of credits. Then the business peopled complained because hundreds of dollars of credits is not worth their time. And so GitHub continues to have terrible uptime and nothing is done about it.
- everfrustrated 4mo agoThis. We talked to our account reps and engineering folks at GitHub - they had no monitoring to track if they had kept their end of the contracted SLA. They expected us to log any faults and as you say the process wasn't worth it - even with massive outages - just for a few beans in credits. GitHub has low availability simply because it doesn't cost them and they wear no legal or contractual damage from it. If a competitor came to me and said, we will _pay_ you damages for the time your developers are offline not able to use our product to do their jobs, we would sign up immediately.
- nautilus12 4mo agoGuess where AI Coding entered the picture
- tealpod 4mo agoIf anyone wants a aggregated status page for github, cloud & AI services. https://status-page.org/ https://status-page.org/
- u_fucking_dork 4mo agoEvery time one of these vibe coded meme sites gets posted there’re endless comments about how it’s not actually because of load, the GitHub team is shit, their tech stack is shit, Microsoft is shit, Azure is shit, etc. Just compare the GitHub status page for public GitHub vs the enterprise cloud pages. Enterprise has much better numbers and I’ve personally can’t remember the last time there was an outage that prevented me from doing work. If the problems didn’t revolve around load, I’d expect to see the same uptime problems reflected on the enterprise offering.
- Anoian 4mo agoWhy cannot both sides be right? Meaning that the load is crazy high but also that the tech stack and everything else is bad?
- semiquaver 4mo agoWhat do you mean by enterprise cloud? The default GitHub enterprise cloud plan is hosted on the same infra as “public github.” Do you have a link to what you mean?
- taspeotis 4mo agoBottom of the status page links to the Enterprise regions? > Check GitHub Enterprise Cloud status by region: > - Australia: au.githubstatus.com > - EU: eu.githubstatus.com > - Japan: jp.githubstatus.com > - US: us.githubstatus.com
- lenerdenator 4mo agoInterested in this difference as well. It'd make sense if there was a "you get what you pay for" attitude at MS re: public GitHub. It's not a good position for the free users, but, what else are you gonna do? Stand up your own? Retrain yourself on your SDLC in a new platform? They need a competitor.
- chrisweekly 4mo agoGitLab
- collinmanderson 4mo ago> Disruption with Gemini 2.5 Pro model > Disruption with Grok Code Fast 1 in Copilot > Incident with Copilot Grok Code Fast 1 > Claude Opus 4 is experiencing degraded performance It doesn't seem fair to blame Github for this? There's nothing they can do about it?
- yreg 4mo agoTo me this makes it uninteresting. Degraded preformance of hyperscalers seems off topic to bundle with e.g. github.com availability. I think the author just wanted the chart to be as red as possible.
- lenerdenator 4mo agoDepends on who's hosting the models.
- YetAnotherNick 4mo agoI think it is fair to blame Github if they repackage other services. We run a much smaller service than Github and have all sort of fallbacks to different providers and different models.
- collinmanderson 4mo agoGithub Copilot also lets you use other models when one has an outage.
- rnotaro 4mo agoBut Github also allows you do fallback to other models...
- Aurornis 4mo agoThe pattern recently is to collect every individual service degradation and present them as all equally significant. Erase the severity and then present them all as “GitHub outages” or reduce it to an uptime graph. I’m not happy with GitHub’s recent major outages either, but there is an ugly side of the pile-on where we’re getting these vibecoded attention seeking websites and social media posts to collect upvotes, likes, karma, and attention that blur the lines between small service degradations and total site outages to be more dramatic.
- anant-singhal 4mo agoInteresting, no outages on weekends
- gen220 4mo agoThis idea has been around! I made this one in January to help slice and dice uptime by incident category. https://isgithubcooked.com https://isgithubcooked.com
- predkambrij 4mo agoThe answer is: yes
- culi 4mo ago"Billing" is all the way in the green and "Pull Requests" all the way in the red
- Raed667 4mo agoThis is a much better project
- bucktrack 4mo ago[flagged]
- letmetweakit 4mo agoIf you think that Github sucks, make something else and try to do it better, or don't use it.
- predkambrij 4mo agoFeature request: best streak
- Robdel12 4mo agoThe last time one of these were posted it had pre-MS acquisition pegged at 100.0% and everyone ate it up. This one is including external LLM services as apart of GitHub being “down”.
- giancarlostoro 4mo agoIf we had this for Anthropic (and I an a happy Claude Max subscriber mind you) I wonder how bad this would look. Probably worse.
- asklfjnasfas 4mo ago[dead]
- whirlwin 4mo agoWe just invested a lot migrating 300+ pipelines from Azure DevOps to GitHub Actions. What a bummer timing-wise. Anyone got an alternative to GitHub Actions?
- scotty79 4mo agoFogejo is self-hosted Github with actions.
- scotty79 4mo ago*Forgejo
- vqtska 4mo agoForgejo has open source reimplementation of github actions
- traverseda 4mo agoYou can run your own github actions compatible-ish server. https://github.com/nektos/act https://github.com/nektos/act Personally my favorite is probably drone-ci. I'd suggest not buying in too hard on any one of these CI systems and just writing shell scripts. Shell scripts are portable, and you can use whatever to trigger them.
- ktm5j 4mo agoWhat does 'none' mean? Wouldn't all of the blank boxes (no recorded outage) qualify as 'none'? I'm confused.
- qoez 4mo agoFunny how it's less during weekends, as if by them working on it it gets worse and when they leave it alone it's solid.
- otagekki 4mo agoKind of normal I think, at the companies I've worked for, most outages were caused by a change in production, for which the impact was not properly assessed.
- NiloCK 4mo agoAlso just load. I'm sure many github services are closer to capacity / logjam at 11Am on Tuesday than at 3AM Saturday.
- adampunk 4mo agoAlmost as though it's load and not code causing the issue.
- Romario77 4mo agoit's because people work (and use GitHub) a lot more on weekdays. A lot of outages are load related.
- SlightlyLeftPad 4mo agoWow, that’s a great visualization. How many 7s of uptime is that?
- shevy-java 4mo agoWill Microslop ever fix GitHub again?
- mring33621 4mo agolooks like Klingon OMG it's a secret message!
- dbgrman 4mo agoLOL! Perfect. one change request: brighter should be more intense. Right now the more intense days are dark red and to a color blind person like me, that doesn't pop out.
- bnyhil31-afk 4mo agoI agree. I am not color blind, but some more contrast would be helpful. Thanks for sharing!
- NimraNoor 4mo agoAzure was always bad but this is next level.
- remify 4mo agoThey should deploy on sundays !
- 1970-01-01 4mo agoI've been pushing to GitHub all morning with 0.0000 issues. So you can complain all you want, but I have no reason to do so. YMMV.
- traderj0e 4mo agoWonder if there's the same thing but in green on someone's commit history
- devy 4mo agoWe need one for Anthropic's Claude: https://status.claude.com/ https://status.claude.com/ IMO, Claude is not fairing any better than Github.
- mawax 4mo agoExcept Claude is not catching flak for it.
- culi 4mo agoI wonder how GitLab, BitBucket, Sourceforge, Codeberg, etc compare
- Sir_Gooner 4mo agoDon't forget muh SourceHut!!! I already removed the hentai when you hit an error page lmaooo
- londons_explore 4mo agoI mostly only use GitHub for hobby weekend stuff... And that's when it's down least, yay!
- dtran24 4mo agoIf there was a GitHub replacement, what would people want from it? Just reliability? Anything else?
- bzreaper 4mo agosame for me, at my company I can't do pr reviews rn
- shantnutiwari 4mo agoNice. Can the Github team arrange the outages so it spells "Github"; or even better, "vibe coding"? Thanks (do i need a /s?)