HACKER Q&A
📣 beezle

Yahoo disregarding RFC 5321 retries


Had a firewall issue that temporarily affected port 25 but was not identified for just over 24 hours. In the process of resolving, a test message was sent from a yahoo account to the domain in question.

Yahoo sent an 'unable to deliver message after multiple retries, giving up' message after exactly three hours.

Three hours is exceedingly short and my understanding has always been that mail is re-tried for a number of days with a backoff algorithm controlling how often.

Is this now common? I am left to wonder how much mail was lost for good during this outage because others have also taken the Yahoo approach.

RFC 5321 suggests:

Retries continue until the message is transmitted or the sender gives up; the give-up time generally needs to be at least 4-5 days. It MAY be appropriate to set a shorter maximum number of retries for non-delivery notifications and equivalent error messages than for standard messages. The parameters to the retry algorithm MUST be configurable.


  👤 jedberg Accepted Answer ✓
Back in the day when the RFC was written it was common for people to run their own mail servers and have them be offline for a day or two, so it was a better experience for the sender to have retries for days.

Nowadays it's assumed that the mail server is set up for high availability on good networks so it's a better experience for the sender to know right away that it wasn't deliverable so that they can use another communication channel for semi-urgent messages.

The ideal situation would be for the sender to get a retry warning while still retrying (which is they way it used to be done), but I can see why Yahoo might not want to tie up their resources that way.


👤 denton-scratch
Not much sympathy, really.

If you rely on a free mail server, you can expect to get everything you paid for. Pending outbound mail has to hang around in a queue, which costs money.

In the days when the original specs were written, mail was often multi-hop, and delivery was unreliable. Most email nowadays gets transferred in seconds, so queueing something for 4 days seems unnecessary.

But people do still run their own mailservers, on home equipment - which might fail to accept delivery, because it blew up, or suffered a power-cut, or needed to be rebooted, or whatever.

It's not reasonable to expect a freemail provider to maintain a sensible outbound queue. And it's not reasonable for a freemail provider to scrub the queue after 3 hours. Solution: get a real mail provider.


👤 tedunangst
1. They probably figure "the network is reliable" and therefore any failure that isn't short lived is permfail. I would disagree, but I imagine their own user base complains that they don't get bounces until days later. "That was a very important serious business email. How dare you not tell me it wasn't delivered promptly!" (And I imagine the old standard of a warning email that message hasn't been delivered but retries are ongoing isn't handled well by many people.)

2. I'd double check you didn't bring the server back online but misconfigured such that it rejected the message.


👤 throwawayboise
> I am left to wonder how much mail was lost for good

You'll never know this, because SMTP does not have delivery guarantees. It's totally possible for any message to just disappear. Not likely these days, but possible.


👤 throwaway47292
4-5 days in queue is infeasible with modern amount of spam, especially generative spam (e.g. someone gets verisign's .com zone and sends one email to every john@domain)

My gut feeling is that almost nobody waits more than few hours, probably even less.

But at least whoever tried to send you email got a 'well we tried to send but failed' so they know to reach you in some other way.


👤 beezle
Thanks for all the replies. As a few have mentioned, directly or indirectly, this may well be an industry shift away from the long standing method of handling temporarily undeliverable email, at least for the largest providers.

This used to be the typical way of handling things:

This is the mail system at host abc.xyz #################################################################### # THIS IS A WARNING ONLY. YOU DO NOT NEED TO RESEND YOUR MESSAGE. # #################################################################### Your message could not be delivered for more than 4 hour(s). It will be retried until it is 5 day(s) old. For further assistance, please send mail to postmaster. If you do so, please include this problem report. You can delete your own text from the attached returned message.

Bottom line seems to be, if you host your own email expect inbound mail to begin permanently failing after three hours should your server (or routing to it) be down longer.


👤 dangus
(Take this comment as as least rude and combative as possible, I'm not trying to attack you. Also, I'm no expert on email)

It sounds like your mail server's uptime is below modern industry standards.

A 24 hour outage is somewhat long in the email world, isn't it?

That's a genuine question, I'm not sure, but it sounds like a long time.

Google's SLA is three nines, which is just under 9 hours per year.

If I were to guess, Yahoo doesn't really expect any mail server to be 100% down for anything close to 4 hours at a time.


👤 nickdothutton
Email delivery behaviour is governed by money these days, not the standards.

👤 paxys
There's a big difference between desktop mail clients that have an outbox and web-based experiences of services like Yahoo and Gmail. I doubt any of the latter kind will retry sending an email over several days.