HACKER Q&A
📣 crgt

Is Apple down?


https://developer.apple.com doesn't work App Store doesn't work iMessage doesn't work. Not just me - coworkers also struggling.

Any idea what's going on?

  👤 Animats Accepted Answer ✓ 

    nslookup
    > server a.ns.apple.com
    Default server: a.ns.apple.com
    Address: 2620:149:ae0::53#53
    Default server: a.ns.apple.com
    Address: 17.253.200.1#53
    > developer.apple.com
    Server:  a.ns.apple.com
    Address: 2620:149:ae0::53#53

    developer.apple.com canonical name = developer-cdn.apple.com.akadns.net.
    ** server can't find developer-cdn.apple.com.akadns.net: REFUSED
Ah. So Apple's own DNS servers are redirecting developer.apple.com to something on "akadns.net", which is operated by Akamai. But Apple's own DNS servers refuse to resolve that, probably because it's not in the apple.com zone.

More:

    nslookup
    > developer-cdn.apple.com.akadns.net
    Server:  127.0.0.53
    Address: 127.0.0.53#53

    Non-authoritative answer:
    developer-cdn.apple.com.akadns.net canonical name = world-gen.g.aaplimg.com.
    world-gen.g.aaplimg.com canonical name = apple-c.g.aaplimg.com.
    apple-c.g.aaplimg.com canonical name = apple-cf.g.aaplimg.com.
    apple-cf.g.aaplimg.com canonical name = apple-lr.g.aaplimg.com.
    > server a.ns.apple.com
    Default server: a.ns.apple.com
    Address: 2620:149:ae0::53#53
    Default server: a.ns.apple.com
    Address: 17.253.200.1#53
    > developer-cdn.apple.com.akadns.net
    Server:  a.ns.apple.com
    Address: 2620:149:ae0::53#53

    ** server can't find developer-cdn.apple.com.akadns.net: REFUSED
It's clearly a botched DNS configuration. Not clear what the intent was. Did they really want to point "developer.apple.com", a web site, to "developer-cdn.apple.com.akadns.net", which is a DNS server? Or am I misreading that?

It's generally considered bad form to have all the DNS servers for "example.com" under "example.com", by the way. If you mess up "example.com", or it goes down, getting to it to fix it can be difficult.

Anyway, this looks like an attempt to outsource something to Akamai that went badly wrong.

👤 tshaddox
Yep.

Wife: My Apple Maps isn't working.

Me: Hmm, it's not working for me either. They must be having server problems. You should use Google Maps for now.

Wife: I can't download Google Maps either, the App Store doesn't seem to be working.

👤 donohoe
You wouldn't think it if you went by this:

https://www.apple.com/support/systemstatus/

👤 perihelions
Here's a lot of crowd-sourced anecdata points:

- "Multiple Apple services are down such as: (Will be updating this list)"

https://old.reddit.com/r/apple/comments/tjg8tz/megathread_ap... ("[Megathread] Apple Outages")

👤 samwillis
I chose the perfect time to restore a repaired iPhone, don’t seem to be able to fully login to iCloud, it’s hanging on the login screen…

Edit: It’s also refusing to download any apps, doesn’t even show the progress circle. Just a download icon next to the app name on the Home Screen and errors out when you click it.

Edit: Login and app downloads now working as of 6.00GMT

👤 spansoa
It's times like this that force us to remind ourselves how reliant we are on critical services like these. On one hand, we can celebrate (Internet snow-day!) but on the other we are forced to shop around for alternatives too.

I often wondered how medieval the world would become if there was a huge sun flare ejection that breached the magnetic field and destroyed a bunch of data-centers. Think of the mess we'd be in!

👤 jcims
I’m sure it’s happens more than I’m aware but i have to say that i can’t recall an App Store outage since i got back in the platform 3-4 years ago. Not bad!
👤 rvieira
I picked a terrific time to lose my temper and do a `rm -rf /Library/Developer/CommandLineTools ; xcode-select --install` /facepalm
👤 lima
Looks like their DNS servers are responsive, but refuse to serve records:

    $ dig -t NS developer.apple.com
    [...]
    apple-lr.g.aaplimg.com.     14400   IN      NS      b.gslb.aaplimg.com.
    apple-lr.g.aaplimg.com.     14400   IN      NS      a.gslb.aaplimg.com.

    $ dig @a.gslb.aaplimg.com developer.apple.com
    [...]
    ;; ->>HEADER<<- opcode: QUERY, status: REFUSED
    ;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
    ;; WARNING: recursion requested but not available
    ;; WARNING: recursion requested but not available
Most likely a configuration mistake that'll be undone as soon as they figured out how to re-deploy their DNS servers while DNS is down.

Unlikely to be BGP shenanigans as some people on Twitter claim. My network has direct peerings to Apple's AS714.

👤 chewmieser
Definitely. Downdetector shows a bunch of reports too (e.g. https://downdetector.com/status/apple-music/). I noticed issues with Music and News, seems like a ton of their services are down
👤 synaesthesisx
Yes. Even developer.apple.com won't load at all for me. Who wants to take bets on DNS as the culprit?
👤 gjsman-1000
MacRumors says Apple is down.

https://www.macrumors.com/2022/03/21/icloud-and-apple-servic...

Big outage... is it some stupid DNS issue again?

👤 chuinard
My app update was rejected because my Upgrade screen was unable to fetch prices from their servers and instead showed an infinite spinner.
👤 donatj
My Apple Music stopped working mid song and is being weird now. Everything seems to be working fine for my wife. Weirdly spotty.
👤 oxplot
iCloud Private Relay is shown as affected as well. This is an interesting case when it comes to failure behavior. From security perspective, you want your connection to stop working instead of falling back to insecure. Is this the case? Can anyone confirm?
👤 ChrisMarshallNY
They seem to have been having a bit of a lie-down, today. I can't submit TestFlight builds, but now, it is taking longer, before the server throws a nutty, so I guess the fix is on its way.
👤 Maursault
MobleMe is still down.

https://www.mac.com/

https://www.me.com/

👤 SlimyHog
Yeah, I'm seeing anecdotal reports of a bunch of services out
👤 rateofclimb
App Store Connect was down for me but appears to be up again now.
👤 selimthegrim
I haven’t been able to cancel subscriptions lately. I filed a refund request and complaint to Apple, maybe it didn’t get through because of this?
👤 mariojv
It's a partial outage for me. I was just able to send an iMessage, but directions on Maps are not working. I live in central Texas.
👤 leviathan
I've been struggling with a DNS downtime at Mediatemple all day. Is there a possible more global DNS issue?
👤 SalimoS
Yes, got a notification that Apple private relay is unavailable

And another notification that it’s back online 40min later

👤 teeray
I had abnormal trouble pulling video I uploaded to iCloud yesterday. Something is up.
👤 wanderer_
I noticed a blip in iMessage earlier, but it sorted itself out before too long.
👤 brown9-2
the domain name developer.apple.com resolves through a series of CNAMEs to Apple's CDN (applimg.com), which if it was down would explain other things like iMessage also being unavailable.
👤 sys_64738
AAPL is down too, today.
👤 hit8run
For me in Germany: iMessage up App Store up Developer site down
👤 ComputerGuru
It’s coinciding with an AWS outage. Probably not unrelated.
👤 camhart
https://www.the-sun.com/tech/4944089/apple-maps-down-icloud-...
👤 gunzor
Can't upload an ipa to App Store Connect for an hour.
👤 jhgb
> Any idea what's going on?

Must be gravity. (Sorry, I had to.)

👤 alopes
Had a few issues with the App Store with OS 12.3
👤 traceroute66
Must be regionalised. Nothing wrong here.
👤 spike021
iMessage texts are working fine for me but an image I sent to a friend is stuck. Music is also down for me.
👤 novateg
Down for me via CloudFlare WARP
👤 fargle
It's always DNS.
👤 vaxman
gotECC?

https://earthsky.org/sun/sun-activity-solar-flares-cme-week-...

LPDDR5 in the SoC doesn’t.

👤 Melatonic
It is always DNS ;-D
👤 ragnot
Down for me.
👤 windex
fedora is down.
👤 barkingcat
maybe their dns expired?