The middle ground for email self-hosting?

I have been selfhosting my email stack on a cheap VPS (~ $10) for many years now, probably since 2009 or so. I used to set up everything manually, but that was quite painful and there were some rough edges.

Nowadays, I just use mailcow (https://github.com/mailcow/mailcow-dockerized) for the setup part and have a much more polished experience. Email deliverability is not a problem. Generally, you just have to make sure to correctly setup DKIM/SPF (and DMARC) and check if your IP is on some blacklist. You can get it removed easily. (Edit: Also required is forward-confirmed reverse DNS, see below).

There was one provider that denied incoming mails from me, even though I got the IP removed from every blacklist I could find. I wrote a short mail to the admin contact and got told I had to host a web page with contact information on the same IP. Since being whitelisted there, everything works like a charm, couldn't be happier.

Many VPS providers have an SMTP gateway that you can use to ensure that your email does not get marked as spam. I use transip.eu and never have problems. Prior to setting up my mail server to use their gateway, I ONLY had problems with Microsoft-based receivers such as @outlook.com. So please, don't use Microsoft for anything email related, as they are currently one of the worst offenders in making it hard for people to run their own mail server.

I do outbound email for my domain from a generic Gmail account, where my domain address is added as an alternate "Send mail as". This gets sent through Mailgun's free SMTP.

Incoming email to my domain is forwarded through Cloudflare's free service to the generic Gmail account.

This seems to pass all quality checks to avoid being sent to spam.

A major theme in self-hosted email discussions is deliverability issues (particularly to larger email service providers), and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine, and possibly the chatter about failed deliveries is caused by misconfigured servers and/or misunderstandings.

Seems like it shouldn't be hard to check and collect reference statistics with a survey, though I'm failing to find surveys of that kind, and getting accounts on public services would be the tricky part for me personally (since I don't like to provide my phone number), so not doing that myself either. Only occasionally tried to check it with others, and messages were delivered fine in those cases -- but that's just a few samples.

That definitely outsources the most painful part of the problem. Though I'd probably use a forwarding-specific service - e.g. a lot of people swear by SES, I've used smtp2go just for Hotmail deliveries, I'm sure there are others.

You might have a bit of SPF fiddling to do, just because you might be fighting the default self-hosting assumption that incoming and outgoing servers are the same.

The middle ground is to keep copies of all emails on your computer through a email client (via IMAP or POP), so that when you are locked out by your email provider, you can resume your business and communication with your contacts with your new email address. With IMAP, you can even upload your old emails to your next provider.

Incoming email is simple, MTAs have no problem at all delivering to residential IPs if that's what your MX says. So ideally you should run your own postfix + dovecot at your premises and point your MX to that. You have to take additional steps for sending email.

> would my custom domain be free from being marked by spam?

The receipient's mail service gets to choose if it thinks your email is spam, this will happen whatever your sending arrangements, outlook is not immune from sending spam and is no magic guarantee others will give it a free pass somehow.

Recipients score your email on a variety of characteristics, many of which are under your control. A major consideration is the sending netblock, eg, residential ADSL blocks are likely to be rejected or scored to hell. Garbage netblocks like linode with a terrible reputation likewise. A clean (no history of spamming) IP in a clean (reputable) netblock will be scored higher. You can look up sender reputations here, which is the service the big email providers use.

https://senderscore.org/

So to send your own mail, you should rent a dedicated server on your own IP, you can do this for $30/mo or so. All you need to run there is postfix + SASL auth to forward your (and only your) emails.

Then you must configure DKIM etc correctly and check your emails are validly signed, DKIM requires being able to add TXT fields to your DNS.

It's very possible to do this yourself securely after a bit of a learning curve and have it require minimal ongoing maintenance.

Do the same but use a http://postmarkapp.com or https://www.smtp2go.com account where you likely pay nothing unless you're a heavy sender.

Had an issue with my self hosted email going to spam and these services solved it.

I'm going to write this assuming you're a non-technical professional, lawyer perhaps, looking for a private email solution that doesn't rely on third parties.

Bottom line: There's no "middle ground", any middle ground you cede is allowing a third party some kind of access. Hosting your own email has become expensive and time-consuming (although IMHO it's still extremely worthwhile, and I do it in spite of what a pain in the ass it is). Be prepared to spend at least $50/mo and at least 6 hours in setup and 1-2 hours a month debugging if you do it personally. Or you can find someone to help (see below). You need your own IP address. You need a dedicated box, not a VPS. And check the IP address in advance to make sure it's clean, and not blacklisted. Tell the datacenter you're going to be doing email and ask them if they're okay with that for a clean IP. Use https://mxtoolbox.com/blacklists.aspx to test the IP address they're offering you, or IPs in their range. Unlike some people are saying, you should never do this off a VPS if you have an interest in keeping the email secure and functioning for a long time.

My personal go-to would be dedicated hosting in the Netherlands, Switzerland, Isle of Man or Norway. Clean IPs, your own box, start with a clean server. But then you're talking $250/mo or so.

If you don't know how to set it up, there are people who can do it for you. You will need to essentially trust that person with access to all your correspondence, but if they do it properly, no one at the server farm[0] or elsewhere will have access to your correspondence... which puts you in the 0.01% of people on earth whose email isn't read by big tech companies.

[0] -who doesn't physically access the server: Look for ones in cages and ask who has physical access and why.

I just don't understand the attraction of self hosting email. The pain seems extreme, even for those who understand the considerable number of nuances.

To me the happy middle ground is email on your own domain but using an existing provider such as G / MS or whoever. That way you've got control but don't need to worry about the pain.

It does require paying for but really on balance not much. If you're spending more than an hour a year maintaining your self hosted email (which you will, big time!) then your Google Workspace / O365 is paid for.

The situation I've found frustrating is about family email on same domain. I've gone in a huge loop that has ended up back with GWorkspace which is quite costly for 3-4 family users. But still - not even close to the horror of self hosting...

I used to do something similar: I hosted my own IMAP, while using Gandi for receiving mail and sending mail. That meant I didn't lose mail on the receiving side if my mail server had an issue, and I didn't fail to deliver mail to others because I hadn't jumped through enough hoops. (Literally everyone I personally know who has run their own mail server has had one or both of those problems at least once.)

This worked well for me because it gave me the feeling of having more control and privacy and security over my email.

I switched away from that solution when I realized that in practice I have less ability to effectively provide security than the whole security and product teams of a major email provider.

You can also try to use your domain registrar SMTP as relay, in case they offer you a mail service, and setting their SPF records to the domain. I have set it up with Gandi and works pretty good.

FWIW, a $4/m Exchange Online Plan 1 is probably enough. Or even the $1/m Exchange Online Protection if you don't have any need for a hosted mailbox. Anything that'll get you ongoing access to the Exchange Admin Center.

I use a single M365 Business Basic account, as a conventional mailbox, for one of my domains. From within the Exchange Admin Center there's extensive control over mail flow -- domains to accept mail for, inbound and outbound connectors for routing mail between on-prem mail servers. Best as I can tell, literally ongoing any subscription that gets you an account with access to EAC ought be enough to route any or all of your email through EO in either direction.

https://www.microsoft.com/en-us/microsoft-365/exchange/compa...

https://www.microsoft.com/en-us/microsoft-365/exchange/excha...

https://docs.microsoft.com/en-us/exchange/standalone-eop/sta...

https://docs.microsoft.com/en-us/exchange/mail-flow-best-pra...

I've done this with Amazon SES, which is PAYG and costs me pennies.

https://www.pxeger.com/2020-07-02-hybrid-cloud-email-with-am...

It is a bit overcomplicated, because I also set up SES to receive email, but I could run that instead with an ordinary Postfix server. It would be much simpler for outgoing only, I think

I know this is slightly off topic from OP's question, but I'm chiming in with one piece of first-hand advice:

You can self host mail alongside gmail/outlook on your own domain. More than one email service can run concurrently, without any problems.

That often overlooked fact allows you to quickly set up something like gmail on your domain, then use the trial period to see if you can self-host with any success. If you can, then you can shut down the trial, or move on to trial another paid service like 365 while you're still "trialing" your own host.

It really helped me make the transition.

I have good luck using https://forwardemail.net for having email address on my own custom domain. This will catch inbound email (configured from DNS MX record) to my gmail, and for outbound I set the custom domain as gmail alias.

Doesn't solve privacy, data ownership, nor google lock-in issue (but at least if I lost my gmail, I can move to a real email selfhost solution and keep my address). As my need is just to have custom domain address for the cool factor of it, this simple setup works flawlessly.

You should be aware that Microsoft's SMTP servers parse the mime structure of the mails and restructure it in a non-standard way. I have no idea why, but for example this breaks PGP signatures.

I have experience self hosting for incoming mails; outgoing mails are sent via mailchimp.

We have had this setup for several years. It is not difficult to setup, emails are delivered reliably, and email delivery cost is negligible.

Some of our users use Outlook / Thunderbird / Apple Mail as a client, some use GMail as a client (check external mail / send as user) and some use Rainloop which I set up on the mail server.

I use a free Gmail account as my mail client and direct all my custom domains from Fastmail to Gmail. Messages are delivered very quickly and I don't have spam issues.

Truth is I like Gmail but I think Google have dropped the ball with, "Let me point all my custom domains to a Gmail account. I would even pay you but I don't want Workspace".

Don't know about the effect on the spam ranking, but on the flip side you'd lose some privacy.

For example, email notifications sent by Stripe are delivered over TLS'd connections. My bank does this too. If you are to proxy these, the relay will obviously be in the loop on all emails that aren't local to your mail server.

I think your best bet is Amazon ses. In my recent testing, all mail always go through - unless the recipient marks it thus.

It's damn cheap too, like almost free for low volumes.

I think Amazon uses this for their workmail also and has become pretty strict at policing abuse.

I am only speaking for gmail though, so ymmv for hotmail et al which I haven't checked.

I’d go AuthSMTP or similar rather than 365 for pure outbound as it will be much more cost effective.

Another alternative is https://thehelm.com:

Helm is a personal, private email server that won't share your data.

The Verge

I'd say just rent hosted dovecot from a reputable company and you're then free to build your own filters, tools, pipelines on top of that.

My favorite is mailbox.org

I did different setups over time, but currently settled on forwardemail.net for incoming catch-all custom domain and iCloud+ SMTP servers for sending.

I've done similar with AWS's SES before with no trouble - I imagine you'd have similar results with MSFT.

This is how I do it, I can highly recommend it. My residential ISP provides a free relay service so I use that.

I'm still wondering why there's no go-to choice to spin up a docker container that has everything preconfigured.