5 ms·

What good Web Developers should know about sending E-mail

119 points by DougRathbone 15y ago

edanm 15y ago
When faced with sending email from my app, I chose to outsource everything, in order to not learn this stuff.. I set up an account at PostMark (postmarkapp.com), which made it very simple to start sending emails. I did have to change a few DNS settings, but they explained exactly how, and after that everything was a breeze. I really recommend them to anyone who doesn't want to learn this stuff. It's pretty cheap too (I'm on the free tier so far, but it's a few bucks for a 1000 emails).
- tomjen3 15y ago
  I looked into PostMark, but sendgrid is less than 1/10 of the price, so I am not quite sure why anybody would stay with postmark.
  - edanm 15y ago
    Good point. I looked into SendGrid, but went with PostMark afterwards. It was easier to set up. The reason, if I recall, is that SendGrid made emails come out with a "sent via SendGrid" header, which I didn't like. To remove it, I'd have had to sign up for a paid deal with SendGrid. PostMark let me get things working first, then decide whether to pay or not.
    - ajtaylor 15y ago
      When implementing Sendgrid, I think I spent about 30 min reading over the DNS setup docs and 30 min actually creating the records. I'm no sysadmin, but I thought it was quite easy. Also, they have a free account which allows you to send 200 emails per month. That would allow you to try it out and then decide that Yes Virginia, it is worth the money. :)
  - malyk 15y ago
    Sendgrid would not reliably send our emails fast enough. We were getting queued for any gmail address we were sending too and sendgrid told us that there was nothing they could do about it. Since we have a requirement to get into peoples inbox ASAP we bailed on them as they couldn't help. Then we went to critsend, which was great until we ran into a problem and couldn't get in touch with customer service for a week (and they never got back to us). Then we tried Dyn since they called us up after reading a press hit and offered us a free trial period. Things were good there until we found out we couldn't deliver to comcast.com email addresses. Again, they were no help. So we switched to Postmark and things have been awesome. Yes, you pay more than some other places, but your email gets to you recipients and it does so fast. And yes, we had our DNS configured configured correctly for each of those services.
- dennyferra 15y ago
  Is Amazon Simple Email Services an option?
  - mnutt 15y ago
    It is; we use it and it is very reasonably priced. However their APIs are very painful to work with and there is no web interface.
  - riobard 15y ago
    It sure is. We are currently using Postmark to send notifications to our users. Postmark does not allow bulk emails like newsletters, and our user count is way beyond the ceiling of Newsberry (Postmark's bulk sending service). Yes, we could negotiate a deal with them, but it's just not as transparent as we want it to be. Frankly, having to worry about the number of subscribers is kinda painful for a fast-growing startup---we cannot predict how many more users we will get next month! Not to mention the price of Postmark is 15x that of Amazon SES. It's not a big deal for a small amount of emails, but it accumulates pretty quickly when we need to send frequent updates, weekly newsletters, and occasionally campaign messages. We have our own analytic systems and template designers, which make Postmark's value added services useless for us (keep in mind that's not to say it's useless to you). What we need is a reliable and affordable mailing service with a flat, per use fee. Amazon SES fits the bill nicely. We rolled out a new delivery system using SES and just finished sending a testing batch of newsletters. Simple put, SES is amazing: bounce rate is well below 0.2%, almost no complaints, and zero rejects for a 100k batch! We will test a few more times later, and if everything goes well like this, we will completely switch over to SES.
acabal 15y ago
I managed my own mail server for scribophile.com for a year or two. Figuring out how to reliably send email is a complete nightmare and as much black magic as it is engineering. When I finally had to move to new hosting, the thought of having to migrate my email server settings and basically starting over scared the crap out of me. That's when I discovered Sendgrid, and I've been using them since then. Whatever their monthly cost is, it's worth it for not having to deal with the email configuration nightmare.
- ajtaylor 15y ago
  Hear, hear! At $work we started using Sendgrid last month, and it's worth 10x the money we pay every month (Silver level is $80/mo for 300k emails). Our usage will be ramping up over the coming months, and I feel so much more secure knowing that the chances of getting on a blacklist are greatly reduced because of Sendgrid.
  - ams6110 15y ago
    Related, anyone know of anyone providing hosted mailing lists, e.g. ezmlm or mailman? I've been looking for someone providing this and not finding much of anything.
    - bmelton 15y ago
      Have you looked at Mailchimp? It's not as much for transactional emails as it is for scheduled, repeated broadcast emails, so I suspect it's probably more along the lines of what you're looking for. I know they also provide the web interface (and APIs) for allowing users to subscribe, unsubscribe and all that jazz.
      - diminish 15y ago
        I am using mailchimp for multiple clients, and AWS simple email sending for some transactional clients; they are all quite helpful. Anyone recommending sendgrid, why is it a better choice?
Volpe 15y ago
Is setting up a (free!) google apps account on your domain, and then using google (gmail) a viable option? (as long as you are within gmails limits on sending emails)
- joelhaasnoot 15y ago
  I do this for a variety of reasons (including that not all my dev machines have working email), and it works well for us. Not a massive amount of email being sent, but seems to work well. Any horror stories?
  - jaxn 15y ago
    I used Google AppEngine for a mobile game that sent email to notify players. While there was a link to turn off emails in every email and a setting in the app, enough people clicked "mark as spam" that we got shut off. It happened a couple of times and each time Google turned us back on. My general experience is that Google is more heavy handed than a true ESP like Sendgrid or Postmark. ESPs take spam seriously, but they also know spam reports are a fact of life.
- bradleyland 15y ago
  Not a very good option in my experience. Google has some pretty complicated abuse detection algorithms in place. For example, if an account sends email through SMTP, but never receives any email, logs in via POP, or through the web interface, the account may get shut down. Also, Google's tolerance for "Mark as Spam" on the email you send is very, very low. It boils down to this: Google Apps isn't the same as SendGrid, Postmark, etc. Google Apps is intended for use by people, not web apps. Having said all that, these are my experiences. I've tried it on three separate occasions in the past, and every time, I ended up fighting deliverability problems until I finally moved to a different solution. The most common is that the Google Apps user through which you send email gets disabled for "abuse", and you have to log in through the web interface in order to re-enable it.
dramaticus3 15y ago
Email, still not dead. My work has gradually moved from Web -> Email. I learned about email while working as a sys admin for a website sending 100k mailouts and now I've become a bit of a specialist writing email apps. Postfix is my weapon of choice, you can extend it externally via its unix pipe & socket routing. Currently I'm building an outgoing SMTP service for people network hopping - Wifi, 3g etc. It has been interesting to see the rise in inter-SMTP server communications going SSL. Google & Hotmail, for instance, both use TLS to send their email, even when there's no requirement to do so, TLS is chosen first. So people often trumptet BBM as end-to-end secure but if you use Google Mail with your mobile email and set it to TLS only and only send to other Google & Hotmail users then the same is true. I've also had a report of a German ISP that deep packets SMTP traffic and changes STARTTLS to XXX "for security reasons". I suspect that they have a tap in front of their closed source MTA to log the traffic to satisfy the European Data Retention Directive.
- icebraining 15y ago
  >So people often trumptet BBM as end-to-end secure but if you use Google Mail with your mobile email and set it to TLS only and only send to other Google & Hotmail users then the same is true. I actually thought they had end-to-end encryption with different keys for every device (PGP-like). That would make them much more secure than that, since the servers wouldn't be able to access the contents. But I've been reading about it before replying and apparently they use a single key per server, not to mention that if you're not on a private BES, you're using a global key (they call it 'scrambled', not encrypted). What a joke.
  - DougRathbone 15y ago
    interesting you've found that about BES - link?
    - icebraining 15y ago
      EDIT: It's mostly about BIS; BES servers can actually implement end-to-end, if they're IT department enables the S/MIME module and create, distribute and teach users how to use PKI certificates. But if you're not doing that, you're not really protected. This[1] document from the Communications Security Establishment of Canada explains it well. Citing: PIN-to-PIN transmission security: PIN-to-PIN is not suitable for exchanging sensitive messages. Although PIN-to-PIN messages are encrypted using Triple-DES, the key used is a global cryptographic “key” that is common to every BlackBerry device all over the world. This means any BlackBerry device can potentially decrypt all PIN-to-PIN messages sent by any other BlackBerry device, if the messages can be intercepted and the destination PIN spoofed. Further, unfriendly third parties who know the key could potentially use it to decrypt messages captured over the air. Note that the “BlackBerry Solution Security Technical Overview” document published by RIM specifically advises users to “consider PIN messages as scrambled, not encrypted”. [1]: http://www.cse-cst.gc.ca/its-sti/publications/itsb-bsti/itsb57b-eng.html http://www.cse-cst.gc.ca/its-sti/publications/itsb-bsti/itsb...
Scone 15y ago
V helpful, thanks
- DougRathbone 15y ago
  Glad you liked it!
citricsquid 15y ago
What good web developers should know about sending e-mail: fuck it, use postmark.
- DougRathbone 15y ago
  Lol - agreed, although send grid is more my favourite especially for small sites where the free account that allows for 200 emails usually covers you
rlpb 15y ago
That site hurts my eyes.
- DougRathbone 15y ago
  lol - thats what happens when you get a developer to design a site. its on the never ending list of todo's thats for sure
Jem 15y ago
Good web developers should have their logo / sitename linked to the home page - usability 101. :p
atc 15y ago
Black on grey, really?!
- Nickste 15y ago
  I was wondering whether I was the only person who firebugged the background to #FFFFFF before reading?
JonLim 15y ago
I'm the Product Manager for PostageApp (www.postageapp.com) and we built our email delivery engine from the ground up (rather than using existing solutions like Port25) and we learned a huge deal about getting emails into inboxes. This article is a fantastic start and will definitely help you out with delivering email, but there are some minor nuances among different SMTP servers and email providers that are just so quirky that you really can't account for 100% of everything. Organic growth of your email volume to users who genuinely want your emails is a great way to make sure you stay in their good books, as well as observing some SPAMAssassin rules and auditing your subject line and content to not seem spammy. It's a huge and complicated matter (getting emails into inboxes) and it's not documented, thankfully we are getting a better and better idea of how to do it.
dminor 15y ago
Other things that may help: - Promptly remove addresses that bounce back (mail to dead addresses is a red flag). - Set up feedback loops with the major ISPs so you can remove people who report you as spam. - Track image opens and clickthroughs and unsubscribe people who haven't read your emails in a year. - Make your unsubscribe link prominent.
joshfraser 15y ago
This is why SendGrid is so valuable. I learned all those tricks for getting emails into inboxes, but it took me a while to learn and it was a constant battle keeping up with everything.

119 points by DougRathbone 15y ago

edanm 15y ago

When faced with sending email from my app, I chose to outsource everything, in order to not learn this stuff.. I set up an account at PostMark (postmarkapp.com), which made it very simple to start sending emails. I did have to change a few DNS settings, but they explained exactly how, and after that everything was a breeze. I really recommend them to anyone who doesn't want to learn this stuff. It's pretty cheap too (I'm on the free tier so far, but it's a few bucks for a 1000 emails).

tomjen3 15y ago
I looked into PostMark, but sendgrid is less than 1/10 of the price, so I am not quite sure why anybody would stay with postmark.
- edanm 15y ago
  Good point. I looked into SendGrid, but went with PostMark afterwards. It was easier to set up. The reason, if I recall, is that SendGrid made emails come out with a "sent via SendGrid" header, which I didn't like. To remove it, I'd have had to sign up for a paid deal with SendGrid. PostMark let me get things working first, then decide whether to pay or not.
  - ajtaylor 15y ago
    When implementing Sendgrid, I think I spent about 30 min reading over the DNS setup docs and 30 min actually creating the records. I'm no sysadmin, but I thought it was quite easy. Also, they have a free account which allows you to send 200 emails per month. That would allow you to try it out and then decide that Yes Virginia, it is worth the money. :)
- malyk 15y ago
  Sendgrid would not reliably send our emails fast enough. We were getting queued for any gmail address we were sending too and sendgrid told us that there was nothing they could do about it. Since we have a requirement to get into peoples inbox ASAP we bailed on them as they couldn't help. Then we went to critsend, which was great until we ran into a problem and couldn't get in touch with customer service for a week (and they never got back to us). Then we tried Dyn since they called us up after reading a press hit and offered us a free trial period. Things were good there until we found out we couldn't deliver to comcast.com email addresses. Again, they were no help. So we switched to Postmark and things have been awesome. Yes, you pay more than some other places, but your email gets to you recipients and it does so fast. And yes, we had our DNS configured configured correctly for each of those services.
dennyferra 15y ago
Is Amazon Simple Email Services an option?
- mnutt 15y ago
  It is; we use it and it is very reasonably priced. However their APIs are very painful to work with and there is no web interface.
- riobard 15y ago
  It sure is. We are currently using Postmark to send notifications to our users. Postmark does not allow bulk emails like newsletters, and our user count is way beyond the ceiling of Newsberry (Postmark's bulk sending service). Yes, we could negotiate a deal with them, but it's just not as transparent as we want it to be. Frankly, having to worry about the number of subscribers is kinda painful for a fast-growing startup---we cannot predict how many more users we will get next month! Not to mention the price of Postmark is 15x that of Amazon SES. It's not a big deal for a small amount of emails, but it accumulates pretty quickly when we need to send frequent updates, weekly newsletters, and occasionally campaign messages. We have our own analytic systems and template designers, which make Postmark's value added services useless for us (keep in mind that's not to say it's useless to you). What we need is a reliable and affordable mailing service with a flat, per use fee. Amazon SES fits the bill nicely. We rolled out a new delivery system using SES and just finished sending a testing batch of newsletters. Simple put, SES is amazing: bounce rate is well below 0.2%, almost no complaints, and zero rejects for a 100k batch! We will test a few more times later, and if everything goes well like this, we will completely switch over to SES.

acabal 15y ago

I managed my own mail server for scribophile.com for a year or two. Figuring out how to reliably send email is a complete nightmare and as much black magic as it is engineering. When I finally had to move to new hosting, the thought of having to migrate my email server settings and basically starting over scared the crap out of me. That's when I discovered Sendgrid, and I've been using them since then. Whatever their monthly cost is, it's worth it for not having to deal with the email configuration nightmare.

ajtaylor 15y ago
Hear, hear! At $work we started using Sendgrid last month, and it's worth 10x the money we pay every month (Silver level is $80/mo for 300k emails). Our usage will be ramping up over the coming months, and I feel so much more secure knowing that the chances of getting on a blacklist are greatly reduced because of Sendgrid.
- ams6110 15y ago
  Related, anyone know of anyone providing hosted mailing lists, e.g. ezmlm or mailman? I've been looking for someone providing this and not finding much of anything.
  - bmelton 15y ago
    Have you looked at Mailchimp? It's not as much for transactional emails as it is for scheduled, repeated broadcast emails, so I suspect it's probably more along the lines of what you're looking for. I know they also provide the web interface (and APIs) for allowing users to subscribe, unsubscribe and all that jazz.
    - diminish 15y ago
      I am using mailchimp for multiple clients, and AWS simple email sending for some transactional clients; they are all quite helpful. Anyone recommending sendgrid, why is it a better choice?

Volpe 15y ago

Is setting up a (free!) google apps account on your domain, and then using google (gmail) a viable option? (as long as you are within gmails limits on sending emails)

joelhaasnoot 15y ago
I do this for a variety of reasons (including that not all my dev machines have working email), and it works well for us. Not a massive amount of email being sent, but seems to work well. Any horror stories?
- jaxn 15y ago
  I used Google AppEngine for a mobile game that sent email to notify players. While there was a link to turn off emails in every email and a setting in the app, enough people clicked "mark as spam" that we got shut off. It happened a couple of times and each time Google turned us back on. My general experience is that Google is more heavy handed than a true ESP like Sendgrid or Postmark. ESPs take spam seriously, but they also know spam reports are a fact of life.
bradleyland 15y ago
Not a very good option in my experience. Google has some pretty complicated abuse detection algorithms in place. For example, if an account sends email through SMTP, but never receives any email, logs in via POP, or through the web interface, the account may get shut down. Also, Google's tolerance for "Mark as Spam" on the email you send is very, very low. It boils down to this: Google Apps isn't the same as SendGrid, Postmark, etc. Google Apps is intended for use by people, not web apps. Having said all that, these are my experiences. I've tried it on three separate occasions in the past, and every time, I ended up fighting deliverability problems until I finally moved to a different solution. The most common is that the Google Apps user through which you send email gets disabled for "abuse", and you have to log in through the web interface in order to re-enable it.

dramaticus3 15y ago

Email, still not dead. My work has gradually moved from Web -> Email. I learned about email while working as a sys admin for a website sending 100k mailouts and now I've become a bit of a specialist writing email apps. Postfix is my weapon of choice, you can extend it externally via its unix pipe & socket routing. Currently I'm building an outgoing SMTP service for people network hopping - Wifi, 3g etc. It has been interesting to see the rise in inter-SMTP server communications going SSL. Google & Hotmail, for instance, both use TLS to send their email, even when there's no requirement to do so, TLS is chosen first. So people often trumptet BBM as end-to-end secure but if you use Google Mail with your mobile email and set it to TLS only and only send to other Google & Hotmail users then the same is true. I've also had a report of a German ISP that deep packets SMTP traffic and changes STARTTLS to XXX "for security reasons". I suspect that they have a tap in front of their closed source MTA to log the traffic to satisfy the European Data Retention Directive.

icebraining 15y ago
>So people often trumptet BBM as end-to-end secure but if you use Google Mail with your mobile email and set it to TLS only and only send to other Google & Hotmail users then the same is true. I actually thought they had end-to-end encryption with different keys for every device (PGP-like). That would make them much more secure than that, since the servers wouldn't be able to access the contents. But I've been reading about it before replying and apparently they use a single key per server, not to mention that if you're not on a private BES, you're using a global key (they call it 'scrambled', not encrypted). What a joke.
- DougRathbone 15y ago
  interesting you've found that about BES - link?
  - icebraining 15y ago
    EDIT: It's mostly about BIS; BES servers can actually implement end-to-end, if they're IT department enables the S/MIME module and create, distribute and teach users how to use PKI certificates. But if you're not doing that, you're not really protected. This[1] document from the Communications Security Establishment of Canada explains it well. Citing: PIN-to-PIN transmission security: PIN-to-PIN is not suitable for exchanging sensitive messages. Although PIN-to-PIN messages are encrypted using Triple-DES, the key used is a global cryptographic “key” that is common to every BlackBerry device all over the world. This means any BlackBerry device can potentially decrypt all PIN-to-PIN messages sent by any other BlackBerry device, if the messages can be intercepted and the destination PIN spoofed. Further, unfriendly third parties who know the key could potentially use it to decrypt messages captured over the air. Note that the “BlackBerry Solution Security Technical Overview” document published by RIM specifically advises users to “consider PIN messages as scrambled, not encrypted”. [1]: http://www.cse-cst.gc.ca/its-sti/publications/itsb-bsti/itsb57b-eng.html http://www.cse-cst.gc.ca/its-sti/publications/itsb-bsti/itsb...

Scone 15y ago

V helpful, thanks

DougRathbone 15y ago
Glad you liked it!

citricsquid 15y ago

What good web developers should know about sending e-mail: fuck it, use postmark.

DougRathbone 15y ago
Lol - agreed, although send grid is more my favourite especially for small sites where the free account that allows for 200 emails usually covers you

rlpb 15y ago

That site hurts my eyes.

DougRathbone 15y ago
lol - thats what happens when you get a developer to design a site. its on the never ending list of todo's thats for sure

Jem 15y ago

Good web developers should have their logo / sitename linked to the home page - usability 101. :p

atc 15y ago

Black on grey, really?!

Nickste 15y ago
I was wondering whether I was the only person who firebugged the background to #FFFFFF before reading?

JonLim 15y ago

I'm the Product Manager for PostageApp (www.postageapp.com) and we built our email delivery engine from the ground up (rather than using existing solutions like Port25) and we learned a huge deal about getting emails into inboxes. This article is a fantastic start and will definitely help you out with delivering email, but there are some minor nuances among different SMTP servers and email providers that are just so quirky that you really can't account for 100% of everything. Organic growth of your email volume to users who genuinely want your emails is a great way to make sure you stay in their good books, as well as observing some SPAMAssassin rules and auditing your subject line and content to not seem spammy. It's a huge and complicated matter (getting emails into inboxes) and it's not documented, thankfully we are getting a better and better idea of how to do it.

dminor 15y ago

Other things that may help: - Promptly remove addresses that bounce back (mail to dead addresses is a red flag). - Set up feedback loops with the major ISPs so you can remove people who report you as spam. - Track image opens and clickthroughs and unsubscribe people who haven't read your emails in a year. - Make your unsubscribe link prominent.

joshfraser 15y ago

This is why SendGrid is so valuable. I learned all those tricks for getting emails into inboxes, but it took me a while to learn and it was a constant battle keeping up with everything.