9 ms·

Show HN: Mailready – meet the new email deliverability standards

82 points by kehers 3y ago

Brajeshwar 3y ago
Some of these popped up on Hacker New recently; - Learn and Test DMARC[1] does a visual breakdown of how email servers communicate, giving you a better understanding of SPF, DKIM, and DMARC and how they work together. - Mail-Tester[2] - test the spammyness of your emails. - MECSA[3] is an online tool developed by the Joint Research Centre (JRC)[4] to assess the security of email communication between providers. 1. https://www.learndmarc.com https://www.learndmarc.com 2. https://www.mail-tester.com https://www.mail-tester.com 3. https://mecsa.jrc.ec.europa.eu/en/ https://mecsa.jrc.ec.europa.eu/en/ 4. https://joint-research-centre.ec.europa.eu/index_en https://joint-research-centre.ec.europa.eu/index_en
rkagerer 3y ago
If you have SPF but not DKIM is that sufficient for deliverability to these providers?
- LeonM 3y ago
  For DMARC alignment you need either SPF alignment or DKIM alignment, either one will do. (note that 'alignment' is not the same as an SPF pass). Which means that you _could_ get away with just SPF alignment, but you wouldn't want to trust on that since SPF is horribly broken and most third party senders don't even bother with SPF alignment anymore. Always focus on DKIM alignment instead. But if you are now just thinking about this, you're in trouble anyway. If you are sending bulk amounts of email (that is, 5k a day per Google's rules) and you are not yet signing with DKIM, then you are probably not ready for adopting a strong DMARC policy ('quarantine' or 'reject') before Feb 1st. Email hardening takes time, the larger/more complex your domain is, the more time you probably need to ensure you are DKIM aligned for all your delegated senders. Don't be tempted to just add a DMARC record with p=reject policy, that would be irresponsible and asking for problems (read: undeliverable email).
- justusthane 3y ago
  For mail that you're sending yourself, from your own infrastructure (e.g. the envelope-from address matches the header-from address), DMARC + SPF should be sufficient and is easy to implement. For third parties that are sending on your behalf, you'll likely need DKIM - but that will be implemented on their side, and all you'll have to do is add the DNS record they give you.
KingOfCoders 3y ago
"Send reconfirmation emails to people that have not interacted with your email (no opens or clicks)" How can I check opens? I'm not aware of any reliable way to check this. Mail clients not loading pixels means the software is unaware of opens?
- toomim 3y ago
  Maybe because they click a link in the confirmation email?
- matt_heimer 3y ago
  Same way as always, image loading. If image URLs are unique and external then you can track their opening. Not all users will allow it so you also track clicking on in email links.
  - TylerE 3y ago
    Doesn't just about every mail service proxy, and often preemptively fetch those?
    - jeroenhd 3y ago
      Not every service, buy many of them do.
      - TylerE 3y ago
        I'm quite sure GMail does... which is basically the entire internet as far as mail delivery.
- jeroenhd 3y ago
  I personally tend to enable delivery/read receipts for new email domains. Big providers will often send you a notification for delivery at least, and prompt for the read receipt. You can also track your DMARC statistics and figure out what mail domains tend to not deliver your email.
mike-cardwell 3y ago
Basic XSS - https://mailready.info/authentication?domain=grepular.com https://mailready.info/authentication?domain=grepular.com
- kehers 3y ago
  Thanks for this. Will fix.
  - mike-cardwell 3y ago
    Your fix is wrong. It states the SPF record is invalid. It is not. https://datatracker.ietf.org/doc/html/rfc7208 https://datatracker.ietf.org/doc/html/rfc7208 > Unrecognized modifiers MUST be ignored no matter where, or how often, they appear in a record. This allows implementations conforming to this document to gracefully handle records with modifiers that are defined in other specifications. A correct SPF validator will ignore the xss modifier, not treat the SPF record as invalid.
a_subsystem 3y ago
Our domain checks out because we use O365. However, we have an old Exchange server sending out via SMTP. We're not sure what the best path forward is for us. Do we change our apps to route through O365? Will probably take days for that. We have in house custom apps that use it heavily. Anyone have good resources on what we should do?
- dashgreen 3y ago
  With the appropriate configuration, your SMTP server could relay to O365, just acting as a forwarder, you don't necessarily have to remove the server. This is very common in use-cases for old devices that barely support SMTP authentication, never mind TLS!
- technion 3y ago
  There's an oss dikim signing transport plugin for on prem exchange that has become somewhat a standard in this space. https://github.com/Pro/dkim-exchange/releases https://github.com/Pro/dkim-exchange/releases
CodeWriter23 3y ago
> What happens if I dont comply? >You get marked as spam? And that's probably the best case scenario. Your email may not get delivered. My best friend works at a large ISP specifically on their email transport system. They discard 97% of the emails they receive. That's straight into the bit bucket, not to your Junk Mail folder.
- KevinMS 3y ago
  hotmail or comcast? My users complain about email just disappearing, no bounce, no spam folders, just gone.
  - CodeWriter23 3y ago
    I can’t disclose his place of employment. He did say this was kinda the reality the big email domains contend with.
- jeroenhd 3y ago
  I have seen something similar at a smaller ISP, but the silently discarded email usually came from domains with bad SPF setups. Really infuriating, because customers would not believe that this wasn't a problem on our end, it was the other side telling us to discard their email!
  - Avamander 3y ago
    SPF alone is not a reason for discarding a letter.
    - jeroenhd 3y ago
      It shouldn't be, but the standard is not clear. The spec is very dodgy in that it acknowledges that every server needs to pick its own policies: > Disposition of SPF fail messages is a matter of local policy. On silently dropping email, the following is listed as a "consideration": > Other dispositions such as "dropping" or deleting email after acceptance are inappropriate because they leave uncertainty and reduce the overall reliability and utility of email across the Internet. There is no MUST (NOT) in the spec when it comes to silently dropping email. The intent of the author of the spec is to always provide feedback, but it doesn't actually say that in cleae terms.
em-bee 3y ago
this doesn't seem to help for private email. i want to send email to my friends at google. yet google blocks delivery. this is not any kind of business or commercial messages. but from my private account to my friends account. SPF and DMARC check out and surely private emails should not need unsubscribe headers. so your site says everything is fine. then why does google still reject my emails?
- riedel 3y ago
  This really getting crazy. My daughter nearly did not get into the swimming course because google just black holes the registration confirmation because my wife's used her Gmail. I really hope that this kind of stuff gets illegal: just taking an email and virtually burning it.
  - em-bee 3y ago
    this really sums up my feelings. i have sent emails to people past that i have no other way to reach and never got a reply, and i have no idea if they even got my email. and the same the other way around. which is one reason why i run my own server. i always believed that spam filtering must be done at the end user, and noone else has the right to block email from reaching me. in particular the most obvious thing, every address that i send to, should automatically be whitelisted as a valid sender, unless i explicitly mark it as spam. the exceptions should be obvious DMARC/DKIM/SPF violations. at one point i was even working on my own email server to implement this kind of whitelisting/filtering myself.
- jeroenhd 3y ago
  > then why does google still reject my emails? Multiple options. For example, your IP address may not have a good reputation. This can happen when a previous tennant used your IP address to send spam, but it also happens when you send very little email to Google/Microsoft servers, not giving you the opportunity to build a good reputation. I briefly considered sending my mail server logs to Gmail so I could get regular whitelisted email delivered, but I changed my mind when I realised Google would probably mark my domain as a bot. This seems particularly bad on IPv6 for some reason. I'm not sure why, maybe it's because their spam filters are treating every address as a /128 rather than a /64 network? The worst server in my experience is Microsoft Exchange. I caught the stupid platform taking my email, _rewriting the email address because it didn't like it (despite being compliant!)_, and _then_ checking the DKIM signature, which obviously failed. It doesn't have IPv6 deliverability issues, though, because like many Microsoft cloud products, it doesn't even support IPv6. Microsoft Outlook also sometimes fails the SPF check... because of DNS issues _on Microsoft's side_. None of this is standards compliant, of course. The best you can do is DKIM+SPF+reverse PTR+strict DMARC+DNSSEC+DANE+using some expensive data center so there aren't many spammers in the nearby IPv4 blocks. Most of these can be generated automatically through online tools or ready-out-of-the-box email servers such as Mailinabox or Mailcow. Also, _check your configuration regularly_, set up alerts or something; sometimes something may break and your domain/email address will start losing reputation. It's infuriating to get email delivered, even if you do everything right. I've given up on that stuff, though, and tell everyone I email to check their spam folder and move it to their inbox to train their spam filter.

82 points by kehers 3y ago

Some of these popped up on Hacker New recently; - Learn and Test DMARC[1] does a visual breakdown of how email servers communicate, giving you a better understanding of SPF, DKIM, and DMARC and how they work together. - Mail-Tester[2] - test the spammyness of your emails. - MECSA[3] is an online tool developed by the Joint Research Centre (JRC)[4] to assess the security of email communication between providers. 1. https://www.learndmarc.com https://www.learndmarc.com 2. https://www.mail-tester.com https://www.mail-tester.com 3. https://mecsa.jrc.ec.europa.eu/en/ https://mecsa.jrc.ec.europa.eu/en/ 4. https://joint-research-centre.ec.europa.eu/index_en https://joint-research-centre.ec.europa.eu/index_en

rkagerer 3y ago

If you have SPF but not DKIM is that sufficient for deliverability to these providers?

LeonM 3y ago
For DMARC alignment you need either SPF alignment or DKIM alignment, either one will do. (note that 'alignment' is not the same as an SPF pass). Which means that you _could_ get away with just SPF alignment, but you wouldn't want to trust on that since SPF is horribly broken and most third party senders don't even bother with SPF alignment anymore. Always focus on DKIM alignment instead. But if you are now just thinking about this, you're in trouble anyway. If you are sending bulk amounts of email (that is, 5k a day per Google's rules) and you are not yet signing with DKIM, then you are probably not ready for adopting a strong DMARC policy ('quarantine' or 'reject') before Feb 1st. Email hardening takes time, the larger/more complex your domain is, the more time you probably need to ensure you are DKIM aligned for all your delegated senders. Don't be tempted to just add a DMARC record with p=reject policy, that would be irresponsible and asking for problems (read: undeliverable email).
justusthane 3y ago
For mail that you're sending yourself, from your own infrastructure (e.g. the envelope-from address matches the header-from address), DMARC + SPF should be sufficient and is easy to implement. For third parties that are sending on your behalf, you'll likely need DKIM - but that will be implemented on their side, and all you'll have to do is add the DNS record they give you.

KingOfCoders 3y ago

"Send reconfirmation emails to people that have not interacted with your email (no opens or clicks)" How can I check opens? I'm not aware of any reliable way to check this. Mail clients not loading pixels means the software is unaware of opens?

toomim 3y ago
Maybe because they click a link in the confirmation email?
matt_heimer 3y ago
Same way as always, image loading. If image URLs are unique and external then you can track their opening. Not all users will allow it so you also track clicking on in email links.
- TylerE 3y ago
  Doesn't just about every mail service proxy, and often preemptively fetch those?
  - jeroenhd 3y ago
    Not every service, buy many of them do.
    - TylerE 3y ago
      I'm quite sure GMail does... which is basically the entire internet as far as mail delivery.
jeroenhd 3y ago
I personally tend to enable delivery/read receipts for new email domains. Big providers will often send you a notification for delivery at least, and prompt for the read receipt. You can also track your DMARC statistics and figure out what mail domains tend to not deliver your email.

mike-cardwell 3y ago

Basic XSS - https://mailready.info/authentication?domain=grepular.com https://mailready.info/authentication?domain=grepular.com

kehers 3y ago
Thanks for this. Will fix.
- mike-cardwell 3y ago
  Your fix is wrong. It states the SPF record is invalid. It is not. https://datatracker.ietf.org/doc/html/rfc7208 https://datatracker.ietf.org/doc/html/rfc7208 > Unrecognized modifiers MUST be ignored no matter where, or how often, they appear in a record. This allows implementations conforming to this document to gracefully handle records with modifiers that are defined in other specifications. A correct SPF validator will ignore the xss modifier, not treat the SPF record as invalid.

a_subsystem 3y ago

Our domain checks out because we use O365. However, we have an old Exchange server sending out via SMTP. We're not sure what the best path forward is for us. Do we change our apps to route through O365? Will probably take days for that. We have in house custom apps that use it heavily. Anyone have good resources on what we should do?

dashgreen 3y ago
With the appropriate configuration, your SMTP server could relay to O365, just acting as a forwarder, you don't necessarily have to remove the server. This is very common in use-cases for old devices that barely support SMTP authentication, never mind TLS!
technion 3y ago
There's an oss dikim signing transport plugin for on prem exchange that has become somewhat a standard in this space. https://github.com/Pro/dkim-exchange/releases https://github.com/Pro/dkim-exchange/releases

CodeWriter23 3y ago

> What happens if I dont comply? >You get marked as spam? And that's probably the best case scenario. Your email may not get delivered. My best friend works at a large ISP specifically on their email transport system. They discard 97% of the emails they receive. That's straight into the bit bucket, not to your Junk Mail folder.

KevinMS 3y ago
hotmail or comcast? My users complain about email just disappearing, no bounce, no spam folders, just gone.
- CodeWriter23 3y ago
  I can’t disclose his place of employment. He did say this was kinda the reality the big email domains contend with.
jeroenhd 3y ago
I have seen something similar at a smaller ISP, but the silently discarded email usually came from domains with bad SPF setups. Really infuriating, because customers would not believe that this wasn't a problem on our end, it was the other side telling us to discard their email!
- Avamander 3y ago
  SPF alone is not a reason for discarding a letter.
  - jeroenhd 3y ago
    It shouldn't be, but the standard is not clear. The spec is very dodgy in that it acknowledges that every server needs to pick its own policies: > Disposition of SPF fail messages is a matter of local policy. On silently dropping email, the following is listed as a "consideration": > Other dispositions such as "dropping" or deleting email after acceptance are inappropriate because they leave uncertainty and reduce the overall reliability and utility of email across the Internet. There is no MUST (NOT) in the spec when it comes to silently dropping email. The intent of the author of the spec is to always provide feedback, but it doesn't actually say that in cleae terms.

em-bee 3y ago

this doesn't seem to help for private email. i want to send email to my friends at google. yet google blocks delivery. this is not any kind of business or commercial messages. but from my private account to my friends account. SPF and DMARC check out and surely private emails should not need unsubscribe headers. so your site says everything is fine. then why does google still reject my emails?

riedel 3y ago
This really getting crazy. My daughter nearly did not get into the swimming course because google just black holes the registration confirmation because my wife's used her Gmail. I really hope that this kind of stuff gets illegal: just taking an email and virtually burning it.
- em-bee 3y ago
  this really sums up my feelings. i have sent emails to people past that i have no other way to reach and never got a reply, and i have no idea if they even got my email. and the same the other way around. which is one reason why i run my own server. i always believed that spam filtering must be done at the end user, and noone else has the right to block email from reaching me. in particular the most obvious thing, every address that i send to, should automatically be whitelisted as a valid sender, unless i explicitly mark it as spam. the exceptions should be obvious DMARC/DKIM/SPF violations. at one point i was even working on my own email server to implement this kind of whitelisting/filtering myself.
jeroenhd 3y ago
> then why does google still reject my emails? Multiple options. For example, your IP address may not have a good reputation. This can happen when a previous tennant used your IP address to send spam, but it also happens when you send very little email to Google/Microsoft servers, not giving you the opportunity to build a good reputation. I briefly considered sending my mail server logs to Gmail so I could get regular whitelisted email delivered, but I changed my mind when I realised Google would probably mark my domain as a bot. This seems particularly bad on IPv6 for some reason. I'm not sure why, maybe it's because their spam filters are treating every address as a /128 rather than a /64 network? The worst server in my experience is Microsoft Exchange. I caught the stupid platform taking my email, _rewriting the email address because it didn't like it (despite being compliant!)_, and _then_ checking the DKIM signature, which obviously failed. It doesn't have IPv6 deliverability issues, though, because like many Microsoft cloud products, it doesn't even support IPv6. Microsoft Outlook also sometimes fails the SPF check... because of DNS issues _on Microsoft's side_. None of this is standards compliant, of course. The best you can do is DKIM+SPF+reverse PTR+strict DMARC+DNSSEC+DANE+using some expensive data center so there aren't many spammers in the nearby IPv4 blocks. Most of these can be generated automatically through online tools or ready-out-of-the-box email servers such as Mailinabox or Mailcow. Also, _check your configuration regularly_, set up alerts or something; sometimes something may break and your domain/email address will start losing reputation. It's infuriating to get email delivered, even if you do everything right. I've given up on that stuff, though, and tell everyone I email to check their spam folder and move it to their inbox to train their spam filter.