Hosting email server for our users?

We've been building self-hosted apps (for photos, music) and considering building an email client. Most of the email clients we've seen simply try to connect to Gmail or POP3, but we're curious what it would take to host an email service. What would be a simple stack to add users, give them an address, handle their inbox, have a simple API?

I'm aware we'd have to make sure the IP address is seen by Google and others, that we'd have to configure DKIM and such, and generally worry about spam and deliverability. That's understood. My question is focused on what would be a good server app or stack to make it easy to maintain and work with a web client.

Would love some pointers. Thanks!