9 ms·

It remains a constant frustration to me that Postfix, Exim, Cyrus, Dovecot and the like still feel as if they belong very much in the "here be dragons" territor

It remains a constant frustration to me that Postfix, Exim, Cyrus, Dovecot and the like still feel as if they belong very much in the "here be dragons" territories of the Unix world. Configuring these systems is an exercise in constant frustration and bafflement. They're such a pain to use that since becoming the sysadmin in charge of our work email servers, I gave up running personal ones as well and just pay FastMail to deal with it for me - ain't nobody got time for that. I often dream of taking a sabbatical and writing modern, user-friendly SMPT and IMAP[0] servers. If only so I could use them at work on my return and save myself a lot of time and stress. [0] Possibly even with Exchange ActiveSync support - Microsoft licenses the protocol, but I've no idea what it costs http://www.microsoft.com/en-us/legal/intellectualproperty/IPLicensing/Programs/exchangeactivesyncprotocol.aspx http://www.microsoft.com/en-us/legal/intellectualproperty/IP...

clarry 12y ago
Have you tried OpenSMTPD[1]? I've been running it for a few years now, admittedly with a very basic setup. My config is about ten lines long, written in five minutes or so. I haven't really needed to touch it since. [1] https://www.opensmtpd.org/ https://www.opensmtpd.org/
- thisishugo 12y ago
  It's on my radar, but to date I've not had (or more accurately made) the time to seriously evaluate it. I'm looking at an architecture overhaul soon, so will make the effort to take a proper look before then.
- Torgo 12y ago
  I love OpenSMTPD. The only issue I had with it was, no filter API (just fixed, as of a month ago or so) and to get things like spam and antivirus filtering, or domainkeys, you have to proxy chain it. Also, the API changed very rapidly, and I had to run nightlies for a while to get around a few issues (I'm on stable now though with no problems.) These things add complexity to the configuration, but I would still rather do this than maintain any other mail server software.
  - stevekemp 12y ago
    I use qpsmtpd as a SMTP proxy, which handles all my anti-spam checks, and does virtual domain lookups. The setup is documented here, and pretty nice https://github.com/skx/ms-lite/ https://github.com/skx/ms-lite/
robomartin 12y ago
At some level I've always viewed this as an example of how FOSS can fail to deliver good products at times. It should be embarrassing to the FOSS community that something as fundamental as email services requires super human effort to setup, administer and maintain. I don't know how it is we got here. It's a real head shaker.
- mpnordland 12y ago
  Well, I'm no super admin, but my mail server's been humming along fine for several months now, without needing intervention except for heartbleed.
jauer 12y ago
It is very much "here be dragons" but a lot of that is because of legacy compatibility issues from when mail was delivered directly to local users. If you don't care about local access (IMAP & POP is good enough, no mutt to the mail spool for you) Dovecot is a huge improvement on everything else. You configure it to listen directly for LMTP and use MySQL for user information and Maildir for data it's almost as easy as running a random php+mysql webapp. Even serverside filtering with sieve "just works". The inbound SMTP/Spam filtering stack is still a PITA but that's because of security issues (spam). DBmail (dbmail.org) is one rework that seemed sane but stores all messages in a RDBMS and I didn't want to deal with scaling it at work (little ISP, ~10k mail users, ~1.5 million messages per day including spam) but it was fine for personal use. I've been playing with homegrown POP & SMTP servers that use a s3 compatible datastore as the backend but that's a side project. People go crazy over email so I really want a simple to operate, sane, zero point of failure mail system...
- mike-cardwell 12y ago
  Re "zero point of failure mail system": I also use Dovecot, and I learnt about DSync recently, although haven't played with it yet. It provides two way synchronisation of mailboxes between two Dovecot servers, so you can store the same mail on two boxes, in completely different locations. The clever thing is, it can recover from a split brain scenario completely, safely, and without any losses. If your two servers can't see each other for a few hours and you make conflicting changes on both of them, then it apparently is able to recover completely and entirely automatically when the connection comes back up. http://wiki2.dovecot.org/Tools/Dsync http://wiki2.dovecot.org/Tools/Dsync You could have one remote server in a DC, and the other locally in your office. Point your mail clients at the one in the DC, but then configure your office router to intercept connections to the DC server and re-route them to the local office server. When you're in the office, you hit the local office server. When you're outside of the office, you hit the DC server. [edit] Any chance your can elaborate on your S3 test setup? I've considered something similar. I would be interested if there was any mail server software that already does this. The alternative would be to use an S3 based filesystem as your store, but this doesn't seem very efficient. You'd definitely want local caching of messages in this setup.
ushi 12y ago
I experienced something completely different. I set up a postfix/dovecot/ldap/nginx/roundcube/spam/... for a medium sized office with some special needs last month and was amazed how easy it is. Especially postfix and dovecot are a breeze to work with. The main problem was, that you can do everything in X different ways and have to think careful which way you want to go. In addition i am running a very simple private setup for years now without any problems.
- kkmickos 12y ago
  That is exactly why I love postfix/dovecot, there are X ways to do it and I can get it to fit my specific needs perfectly.
crucialfelix 12y ago
Setting up mail servers is the one things I truly dread. Its truly barbaric. I cuss and froth at the mouth whenever I have to go down there into that postfix dungeon. Its the reason I got into Puppet and Ansible - vainly hoping all those barbaric problems could be encapsulated.
corford 12y ago
There's probably something wrong with me but I quite enjoy the masochistic rituals of getting postfix/dovecot/opendkim/spamassassin/zarafa humming. Postfix is actually a beautiful piece of software once you spend some time with it and get to know it. Like most things, once you've done it a few times (and written everything down!) it isn't so bad :) LDAP on the other hand... that's something that _always_ defeats me.
- CraigJPerry 12y ago
  >> Postfix is actually a beautiful piece of software Couldn't agree more. I've also learned a ton from the source code. If anyone has even a passing interest in security, i'd highly recommend browsing the postfix source code.
  - moe 12y ago
    No. Just no. The architecture is, at best, an anachronism. Designs of much more elegance have been explored (qmail). But as a matter of fact, neither postfix, nor qmail, nor exim, nor (god forbid) sendmail belong into our day and age anymore (I have run 3 of them at scale). A modern MTA is way overdue. Please keep writing MTAs kids! In modern languages. Until you get one right. Thanks!
    - e12e 12y ago
      I started out with Exim (3.x or 4 -- can't remember now) and stuck with it mostly for the same reasons that Debian didn't change away -- I wasn't convinced postfix was a significantly cleaner/simpler design -- that it would be worth it to learn how the "other" system worked. I did have some exposure to qmail before that, and it did feel like it was simpler in some good ways -- but then it became a sort of abandonware, sadly. Any thoughts on Lamson ? (http://lamsonproject.org/ http://lamsonproject.org/) [edit: I did recently switch a few vps' over to nullmailer rather than use exim4 configured for smarthost delivery -- once I figured out how to get tls working thanks to this blogpost http://metz.gehn.net/2012/11/nullmailer-with-starttls/) http://metz.gehn.net/2012/11/nullmailer-with-starttls/)]
    - 12y ago
deleted 12y ago
[deleted]
kev009 12y ago
Well, you only need two from your list. And I don't really think Postfix+Dovecot is much harder than Nginx+$dynamiclanguage. It's just something fewer people felt the need to do historically.
uuid_to_string 12y ago
Part of the problem is that many years ago certain people decided it would be a good idea to tightly couple email to domain names (DNS). Previously email needed only IP addresses to work. The result is that now when you are configuring SMTP you have to also configure DNS. That means more things that can go wrong, and more things to check as you are setting things up. It also means you may need to pay a fee for a domain name. This is because we all submit to the notion of an ICANN root and commercial registrars selling (renting) names that cost nothing to create. Thus email is not solely under your control. You generally have to play the ICANN DNS game, only because your email recipients are playing. Nothing stops anyone from running their own root though. And this is what is done with private DNS inside organizations. And then, as if that DNS complication was not already enough to take control of email away from you, you have various schemes trying to prevent spam that discriminate for or against mail you send based on IP address and domain name. Can you operate email without DNS? Technically yes. There was a time before DNS, and email worked just fine. Practically speaking, today you need DNS, whether it's under ICANN's root or your own. All this hassle steers you to just accept third party email hosting. Profiting from this arrangement has become a career for many a man. And with "the cloud" many are hoping to cash in yet again, as organizations who once ran controlled own email feel pressured to let a cloud computing vendor control it for them. The fact that all this third party control makes warrantless search and surveillance so easy is but one side effect. Centralising hundreds and thousands of accounts in third parties make the spammer's job easier, too. If you think about it, there are many unwanted side effects of centralizing email. When every sender and recipient are connected directly to each other via a network, why would you want to prevent them from sending messages to each other directly? With the constant connectivity and bandwidth we have today in many places, the centralisation and outsourcing of email is baffling to me... it is nonsensical... until you remember how much of a PITA it is setting up email.:) It's no wonder we let third parties handle it. Is this PITA by design? Who cares? Let's just fix it. More of these projects should exist. Or made public (I imagine many of these are personal setups now being released for public use). I have my own that uses qmail.
- nodata 12y ago
  What? It was so much better when we used ip addresses in e-mail addresses?
vacri 12y ago
I also find them baffling and frustrating. It seems that senior sysadmins are fond of "But email is so easy", and I'm sure it is... once you already have your battle-scars (I'm finding Buildbot lies in the same category). Another sysadmin friend of mine says that setting up your own proper mailserver should be a rite of passage before you can call yourself a sysadmin. Edit: it's not so much that it's about configuring Application X, but that there are so many moving parts, from local aliases to firewall fun to DNS entries. Then there's the fun of 'deliverability', greylisting and so forth.
selvan 12y ago
Do anyone have thoughts/feedback on Haraka - a mail server used by craigslist http://haraka.github.io/ http://haraka.github.io/ ( Runs with Node.js + Javascript based plugin system )?
- stevekemp 12y ago
  I've got an extensive qpsmtpd setup - the Perl-based precursor to haraka - and I've been meeting to port it over for over a year now. My setup, with some documentation, is here https://github.com/skx/ms-lite/ https://github.com/skx/ms-lite/ and evolved from a commercial service http://book.mail-scanning.com/ http://book.mail-scanning.com/ I've used Haraka standalone for a couple of custom-jobs, but I've never used it at any significant volume. That said qpsmtpd was a pleasure to use, deploy, and develop against, so I'd expect to have a similar feeling with it.
- baudehlo 12y ago
  I think it's great :) On a serious note - you'll find the Haraka developers more friendly and active than any other MTA out there. Just join the #haraka IRC channel on freenode and we're happy to help anyone.
unstoppable 12y ago
I dread having to set up mail servers. Setting up a typical dovecot + postfix install with a webmail frontend that looks as good as gmail should be as easy and simple as setting up wordpress, but it isn't even close.

clarry 12y ago

Have you tried OpenSMTPD[1]? I've been running it for a few years now, admittedly with a very basic setup. My config is about ten lines long, written in five minutes or so. I haven't really needed to touch it since. [1] https://www.opensmtpd.org/ https://www.opensmtpd.org/

thisishugo 12y ago
It's on my radar, but to date I've not had (or more accurately made) the time to seriously evaluate it. I'm looking at an architecture overhaul soon, so will make the effort to take a proper look before then.
Torgo 12y ago
I love OpenSMTPD. The only issue I had with it was, no filter API (just fixed, as of a month ago or so) and to get things like spam and antivirus filtering, or domainkeys, you have to proxy chain it. Also, the API changed very rapidly, and I had to run nightlies for a while to get around a few issues (I'm on stable now though with no problems.) These things add complexity to the configuration, but I would still rather do this than maintain any other mail server software.
- stevekemp 12y ago
  I use qpsmtpd as a SMTP proxy, which handles all my anti-spam checks, and does virtual domain lookups. The setup is documented here, and pretty nice https://github.com/skx/ms-lite/ https://github.com/skx/ms-lite/

robomartin 12y ago

At some level I've always viewed this as an example of how FOSS can fail to deliver good products at times. It should be embarrassing to the FOSS community that something as fundamental as email services requires super human effort to setup, administer and maintain. I don't know how it is we got here. It's a real head shaker.

mpnordland 12y ago
Well, I'm no super admin, but my mail server's been humming along fine for several months now, without needing intervention except for heartbleed.

jauer 12y ago

It is very much "here be dragons" but a lot of that is because of legacy compatibility issues from when mail was delivered directly to local users. If you don't care about local access (IMAP & POP is good enough, no mutt to the mail spool for you) Dovecot is a huge improvement on everything else. You configure it to listen directly for LMTP and use MySQL for user information and Maildir for data it's almost as easy as running a random php+mysql webapp. Even serverside filtering with sieve "just works". The inbound SMTP/Spam filtering stack is still a PITA but that's because of security issues (spam). DBmail (dbmail.org) is one rework that seemed sane but stores all messages in a RDBMS and I didn't want to deal with scaling it at work (little ISP, ~10k mail users, ~1.5 million messages per day including spam) but it was fine for personal use. I've been playing with homegrown POP & SMTP servers that use a s3 compatible datastore as the backend but that's a side project. People go crazy over email so I really want a simple to operate, sane, zero point of failure mail system...

mike-cardwell 12y ago
Re "zero point of failure mail system": I also use Dovecot, and I learnt about DSync recently, although haven't played with it yet. It provides two way synchronisation of mailboxes between two Dovecot servers, so you can store the same mail on two boxes, in completely different locations. The clever thing is, it can recover from a split brain scenario completely, safely, and without any losses. If your two servers can't see each other for a few hours and you make conflicting changes on both of them, then it apparently is able to recover completely and entirely automatically when the connection comes back up. http://wiki2.dovecot.org/Tools/Dsync http://wiki2.dovecot.org/Tools/Dsync You could have one remote server in a DC, and the other locally in your office. Point your mail clients at the one in the DC, but then configure your office router to intercept connections to the DC server and re-route them to the local office server. When you're in the office, you hit the local office server. When you're outside of the office, you hit the DC server. [edit] Any chance your can elaborate on your S3 test setup? I've considered something similar. I would be interested if there was any mail server software that already does this. The alternative would be to use an S3 based filesystem as your store, but this doesn't seem very efficient. You'd definitely want local caching of messages in this setup.

ushi 12y ago

I experienced something completely different. I set up a postfix/dovecot/ldap/nginx/roundcube/spam/... for a medium sized office with some special needs last month and was amazed how easy it is. Especially postfix and dovecot are a breeze to work with. The main problem was, that you can do everything in X different ways and have to think careful which way you want to go. In addition i am running a very simple private setup for years now without any problems.

kkmickos 12y ago
That is exactly why I love postfix/dovecot, there are X ways to do it and I can get it to fit my specific needs perfectly.

crucialfelix 12y ago

Setting up mail servers is the one things I truly dread. Its truly barbaric. I cuss and froth at the mouth whenever I have to go down there into that postfix dungeon. Its the reason I got into Puppet and Ansible - vainly hoping all those barbaric problems could be encapsulated.

corford 12y ago

There's probably something wrong with me but I quite enjoy the masochistic rituals of getting postfix/dovecot/opendkim/spamassassin/zarafa humming. Postfix is actually a beautiful piece of software once you spend some time with it and get to know it. Like most things, once you've done it a few times (and written everything down!) it isn't so bad :) LDAP on the other hand... that's something that _always_ defeats me.

CraigJPerry 12y ago
>> Postfix is actually a beautiful piece of software Couldn't agree more. I've also learned a ton from the source code. If anyone has even a passing interest in security, i'd highly recommend browsing the postfix source code.
- moe 12y ago
  No. Just no. The architecture is, at best, an anachronism. Designs of much more elegance have been explored (qmail). But as a matter of fact, neither postfix, nor qmail, nor exim, nor (god forbid) sendmail belong into our day and age anymore (I have run 3 of them at scale). A modern MTA is way overdue. Please keep writing MTAs kids! In modern languages. Until you get one right. Thanks!
  - e12e 12y ago
    I started out with Exim (3.x or 4 -- can't remember now) and stuck with it mostly for the same reasons that Debian didn't change away -- I wasn't convinced postfix was a significantly cleaner/simpler design -- that it would be worth it to learn how the "other" system worked. I did have some exposure to qmail before that, and it did feel like it was simpler in some good ways -- but then it became a sort of abandonware, sadly. Any thoughts on Lamson ? (http://lamsonproject.org/ http://lamsonproject.org/) [edit: I did recently switch a few vps' over to nullmailer rather than use exim4 configured for smarthost delivery -- once I figured out how to get tls working thanks to this blogpost http://metz.gehn.net/2012/11/nullmailer-with-starttls/) http://metz.gehn.net/2012/11/nullmailer-with-starttls/)]
  - 12y ago

deleted 12y ago

[deleted]

kev009 12y ago

Well, you only need two from your list. And I don't really think Postfix+Dovecot is much harder than Nginx+$dynamiclanguage. It's just something fewer people felt the need to do historically.

uuid_to_string 12y ago

Part of the problem is that many years ago certain people decided it would be a good idea to tightly couple email to domain names (DNS). Previously email needed only IP addresses to work. The result is that now when you are configuring SMTP you have to also configure DNS. That means more things that can go wrong, and more things to check as you are setting things up. It also means you may need to pay a fee for a domain name. This is because we all submit to the notion of an ICANN root and commercial registrars selling (renting) names that cost nothing to create. Thus email is not solely under your control. You generally have to play the ICANN DNS game, only because your email recipients are playing. Nothing stops anyone from running their own root though. And this is what is done with private DNS inside organizations. And then, as if that DNS complication was not already enough to take control of email away from you, you have various schemes trying to prevent spam that discriminate for or against mail you send based on IP address and domain name. Can you operate email without DNS? Technically yes. There was a time before DNS, and email worked just fine. Practically speaking, today you need DNS, whether it's under ICANN's root or your own. All this hassle steers you to just accept third party email hosting. Profiting from this arrangement has become a career for many a man. And with "the cloud" many are hoping to cash in yet again, as organizations who once ran controlled own email feel pressured to let a cloud computing vendor control it for them. The fact that all this third party control makes warrantless search and surveillance so easy is but one side effect. Centralising hundreds and thousands of accounts in third parties make the spammer's job easier, too. If you think about it, there are many unwanted side effects of centralizing email. When every sender and recipient are connected directly to each other via a network, why would you want to prevent them from sending messages to each other directly? With the constant connectivity and bandwidth we have today in many places, the centralisation and outsourcing of email is baffling to me... it is nonsensical... until you remember how much of a PITA it is setting up email.:) It's no wonder we let third parties handle it. Is this PITA by design? Who cares? Let's just fix it. More of these projects should exist. Or made public (I imagine many of these are personal setups now being released for public use). I have my own that uses qmail.

nodata 12y ago
What? It was so much better when we used ip addresses in e-mail addresses?

vacri 12y ago

I also find them baffling and frustrating. It seems that senior sysadmins are fond of "But email is so easy", and I'm sure it is... once you already have your battle-scars (I'm finding Buildbot lies in the same category). Another sysadmin friend of mine says that setting up your own proper mailserver should be a rite of passage before you can call yourself a sysadmin. Edit: it's not so much that it's about configuring Application X, but that there are so many moving parts, from local aliases to firewall fun to DNS entries. Then there's the fun of 'deliverability', greylisting and so forth.

selvan 12y ago

Do anyone have thoughts/feedback on Haraka - a mail server used by craigslist http://haraka.github.io/ http://haraka.github.io/ ( Runs with Node.js + Javascript based plugin system )?

stevekemp 12y ago
I've got an extensive qpsmtpd setup - the Perl-based precursor to haraka - and I've been meeting to port it over for over a year now. My setup, with some documentation, is here https://github.com/skx/ms-lite/ https://github.com/skx/ms-lite/ and evolved from a commercial service http://book.mail-scanning.com/ http://book.mail-scanning.com/ I've used Haraka standalone for a couple of custom-jobs, but I've never used it at any significant volume. That said qpsmtpd was a pleasure to use, deploy, and develop against, so I'd expect to have a similar feeling with it.
baudehlo 12y ago
I think it's great :) On a serious note - you'll find the Haraka developers more friendly and active than any other MTA out there. Just join the #haraka IRC channel on freenode and we're happy to help anyone.

unstoppable 12y ago

I dread having to set up mail servers. Setting up a typical dovecot + postfix install with a webmail frontend that looks as good as gmail should be as easy and simple as setting up wordpress, but it isn't even close.