HACKER Q&A
📣 mikebonnell

Is GitHub down?


Not loading for me at all, but status page shows green across the board.


  👤 edgyquant Accepted Answer ✓
Everytime github goes down, and my push/pull is rejected, I immediately assume they’ve discovered I’m incompetent and fired me. And I’m the head of engineering at my company.

👤 klysm
Seems to be a fairly catastrophic failure. https://github.com/ fails to load. https://www.githubstatus.com/ shows all green as of this writing. Nothing on the twitter yet https://twitter.com/githubstatus

edit: The outage is now acknowledged on the status page https://www.githubstatus.com/

edit: EU folks appear to have things working so it looks like a regional network fault


👤 hn8305823
140.82.113.0/24 is visible in the global routing table:

  route-views>sh bgp 140.82.113.0
  BGP routing table entry for 140.82.113.0/24, version 62582026
  Paths: (19 available, best #4, table default)
The route is verified by RPKI so it's not a route hijack.

Edit: deleted traceroute


👤 mbreese
https://www.githubstatus.com/

Just flipped to red.

See here: https://www.githubstatus.com/incidents/gqx5l06jjxhp

>Investigating - We are currently experiencing an outage of GitHub products and are investigating.

>Jun 29, 2023 - 17:52 UTC


👤 lucb1e
https://codeberg.org open source GitHub without Microsoft (it's a German non-profit). You can also host your own lightweight https://forgejo.org instance

In case anyone questions whether centralizing literally everything onto GitHub is a good idea, at least as a mirror for things you depend on


👤 ChrisArchitect

👤 stefanos82
So down right now...I wonder why they still use https://www.githubstatus.com/ that reports everything is alright when it's not!

👤 lucb1e
To all the "same" and "not for me" posters: the very least you could add is a location

👤 hejcloud
I feel sympathy for all the engs at companies I've implemented CI/CD based on Gh Actions in recent years. It's not like I didn't tell them that Github showed to be somewhat unreliable in the recent years and in contrast to their claim "it's just the build pipeline, not the product" I think it is a horrible incident if you're not able to deploy to production and have barely any ad-hoc backup.

I'm always evangelizing Argo or Flux and some self-hosted Gitlab or gitea, but seems like they all prefer to throw their money at Github as of now.


👤 ActualHacker
Since nobody can work, I'll just leave this here: "I must have put a decimal point in the wrong place or something. Shit! I always do that. I always mess up some mundane detail."

👤 charlieyu1
Loads for me

Unless it is cached

Edit: I could even login


👤 whitewingjek
Yes, github status says all is operational but hacker news is faster. Go figure.

👤 gaoshan
My team ran some code that crushed our Github actions very shortly before this outage. Nervous laughter around the office that it was us.

👤 sdsd
I am laughing so hard right now. My best friend mocked me for using git.lain.faith to host my code, saying it wasn't reliable. Well, well, well. In the last year GitLain hasn't gone down once.

I know he was still right in a way, who knows when git.lain.faith will just disappear. But still. I'm going to send some texts to bother him right now, hahaha.


👤 onionisafruit
I heard from somebody at GitHub that this one will make a good incident report. No other details or estimates for recovery time.

👤 Macuyiko
Update

We have identified the root cause of the outage and are working toward mitigation Posted 4 minutes ago. Jun 29, 2023 - 18:02 UTC


👤 oblongatacur
I noticed Github's OIDC token changed about a half an hour ago. Security incident?


👤 jesterman

👤 renonce
Not down for me accessing from Hong Kong. I suspect this is a regional outage.

👤 rvz
Unsurprising. There is at least one outage with GitHub every single month. [0]

[0] https://news.ycombinator.com/item?id=35967921


👤 turtleyacht
Won't load for me.

https://githubstatus.com shows all green, but it's not the case...


👤 kashnote
Are these status pages updated manually? At the very least it should be able to detect that the home page doesn't even load and turn itself red.

👤 gwbas1c
Yaay! I just pushed and my new commit showed up in CI!

👤 gwbas1c
I can't push from my desktop, and https://github.com/ spins in the browser

👤 computershit
Yup. Totally down, happened right as I opened a PR.

👤 acim
I just noticed that artifacts download didn't work although the web site was up. There was some varnish proxy error.

👤 numbsafari
If they used AI to rebuild their system and migrated it Azure, I bet they would stop having all the problems.

👤 joshstrange
Yep, looks like it's down. Can't pull/push and can't even get the web to load at all.

👤 aftaylor2
Unable to load Github website or push using git-remote-https as of the last several minutes.

👤 MorningInfidel
Same. Spins for a while then dreaded `ERR_CONNECTION_TIMED_OUT` chrome error page.

👤 saintblasphemer
Looks like we are back online.

👤 distortionfield
Down for me as well, Operation Timed Out errors on all attempted SSH connections

👤 Gordonjcp
No it isn't, it's working absolutely fine and has been all afternoon.

👤 ny711
This should be a weekly ASK HN; seems to happy pretty frequently at this point

👤 circuit10
It works for me at the moment

👤 JohnMakin
First noticed when trying to pull a helm chart - get a 503 backend error page.

👤 squalo
Not a single day passes without a MAJOR outage in a Microsoft owned service.

👤 escape-big-tech
often people point out how unreliable self-hosted services are, well, hosted services are just as unreliable if not more.

this, ladies and gentlemen, is why you should always self host critical infrastructure


👤 coder9874
Yep it's down. Why do they even bother with the status page

👤 connorgutman
Up in Africa (Morocco).

👤 datalus
And just when I was about to get into flow state...

👤 duckkg5
Status page updated showing all red https://www.githubstatus.com/

👤 GrumpyNl
Same here, it cant send a verification sms.

👤 jetpackjoe
Its down for me

👤 ukrainiancrab
Cannot use oauth2 in algoexpert :/

👤 wcarss
I can't even load the status page.

👤 Xeamek
looking on the bright side, at least we'll get an interesting post-mortem to read in a day or two.

👤 ushakov
Actions won't start for me

👤 mydriasis
Yes. Can't even pull ;(

👤 ughitsaaron
Good time to grab a beer!

👤 tednotfred
Down for me right now.

👤 sparc24
Azure Strikes Again!

👤 themusicgod1
It shouldn't matter. Nobody should be using github post-2018.

👤 facu17y
seems to be a network issue, not a service issue

👤 almost_usual
Works for me?

👤 etimberg
Same for me

👤 abe-101
Same here

👤 ActualHacker
Yep

👤 saintblasphemer
Same here.

👤 rk32
its all down currently

👤 adpirz
Same