Managing Email Delivery – Common Grievances and Solutions

emailwhitelist

The question I have may be more of principle than anything else, but here's my dilemma.

I manage an email system for a small company (about 20 email users). We own a plain-letter .com domain name through Network Solutions. Our email service is hosted by Google Apps.

Recently (Feb. 2011) we've been having customers report that they aren't getting our emails. Upon further investigation it seems that the failed emails are all to a common (well known) domain. We have not received any bounce messages for the emails. We've also contacted a few of the intended recipients, who have reported that the messages are not in their spam box; they simply did not receive anything. In these cases we re-sent the same email to an alternate address on another domain, which was successful received.

One customer contacted their email provider about the issue. The provider recommended that we submit a form to be white-listed by their domain.

Here's where my problem begins. I feel like this is heading down a slippery slope. Doesn't this undermine the very principle of email? If this is the appropriate action to take in these situations where will it end? In theory (following this model) it could be argued that eventually one will first need to "whitelist" (or more appropriately termed "authenticate") themselves with an email host before actually sending any messages. More to this point, what keeps the "bad" spammers from doing the same thing…? We've just gone full circle.

I know avoiding anti-spam measures is a big cat-and-mouse game, but I think this is the wrong way of "patching" the problem. Email standards say that messages should not just disappear silently. I have a problem supporting a model that says "you must do < this > to make sure your emails aren't ignored".

I have a notion to call the provider and voice my complaint, although I have a feeling it will probably fall on deaf ears. Am I missing something here? Is this an acceptable approach to email spam problems? What should I do?

Best Answer

I find far too many legitimate sites are shooting themselves in the foot. They (unwittingly I hope) configure their servers in such a manner that they appear to forging their identity. Common problems include:

Missing or incorrect PTR record.
A record for the PTR does not point back to the originating IP address.
The name used in the HELO command has an invalid top level domain like local, lan, or localdomain.
The name used in the HELO command does not have an A record.
The A record for the name in the HELO command does not return the IP address being used.
No SPF record for the domain being used in the HELO command.
No SPF configured for the domain of the envelope sender.
SPF for the domain of the envelope sender prohibits use of the IP address being used to send the email.
Domain is not listed with DNSWL.org.
Configuring the address of the mail server as a second level domain (example.com) rather than a third or forth level domain (mail.example.com).
Not accepting mail to postmaster.

Avoid all of the above, and you will have a much better chance of getting your mail delivered.

EDIT: I have found Port25 Solutions Inc. has a very good automated verification service listed on their E-mail Authentication page. Many thanks to them for their fine service. It is designed to verify DKIM signatures, but gives excellent feedback for most of the items listed above. Check in the Port25 resources section, and use the appropriate email address to get the results mailed to your desired e-mail address. Remember if you need to do DNS changes it can take a day or so to be reflected in all the caches. Worst case should be two times your Time to Live setting.

Related Solutions

email – How to Send Emails and Avoid Spam Classification

Be sure that your emails don’t look like typical spam emails: don’t insert only a large image; check that the character-set is set correctly; don’t insert “IP-address only” links. Write your communication as you would write a normal email. Make it really easy to unsubscribe or opt-out. Otherwise, your users will unsubscribe by pressing the “spam” button, and that will affect your reputation.

On the technical side: if you can choose your SMTP server, be sure it is a “clean” SMTP server. IP addresses of spamming SMTP servers are often blacklisted by other providers. If you don’t know your SMTP servers in advance, it’s a good practice to provide configuration options in your application for controlling batch sizes and delay between batches. Some mail servers don’t accept large sending batches or continuous activity.

Use email authentication methods, such as SPF, and DKIM to prove that your emails and your domain name belong together. The nice side-effect is you help in preventing that your email domain is spoofed. Also check your reverse DNS to make sure the IP address of your mail server points to the domain name that you use for sending mail.

Make sure that the reply-to address of your emails are a valid, existing addresses. Use the full, real name of the addressee in the To field, not just the email-address (e.g. "John Doe" <john.doe@example.com> ) and monitor your abuse accounts, such as abuse@example.com and postmaster@example.com.

Receiving spam from the own email address. postfix

As we can see, this message uses your address as the envelope sender:

postfix/qmgr[19733]: 750991E018: from=<me@example.com>, size=3207, nrcpt=1 (queue active)

This means you have methods for rejecting such messages right after MAIL FROM (or RCPT TO, as I do). Regarding the headers like From: and Date:, they can be spoofed and contain anything. Additional spam filters like Spamassassin can perform tests against these, but that's another story.

Your email client is showing the time and date provided by the Date: header rather than the time the server has actually received the mail. You can look at the Received headers to see the dates added by the servers the message has gone through, but the email client trusts the Date: header.

METHOD 1: Blacklisting the domain from external sources

The methods aren't in order: the first one is easy to add, but the second one is better in every way.

If this server is the only legitimate source for email from your domain example.com, you could simply block all messages using from the domain, unless from own networks or an authenticated user, using check_sender_access. I personally put everything in smtpd_recipient_restrictions to get more details in the logs before rejecting the connection. For main.cf:

smtpd_recipient_restrictions =
    permit_mynetworks,
    permit_sasl_authenticated,
    . . .
    check_sender_access hash:/etc/postfix/access/sender_access,
    . . .

The /etc/postfix/access/sender_access is a lookup table (remember to postmap) of white- and blacklisted MAIL FROM addresses, domains etc. For blacklisting mail from this domain, e.g.

example.com   550 YOU ARE NOT ME.

METHOD 2: Implementing SPF for your domain and testing sender SPF in Postfix

If you have other sources for mail, you can't use the previous method. Also, SPF is something you should really implement to prevent your domain to be used for sending spam. First you add a TXT record for your domain listing all the authorized senders. See SPF Introduction and Record Syntax.

After that, configure your Postfix to check for SPF (see How To Implement SPF In Postfix). E.g.

Install Perl with Mail::SPF and NetAddr::IP modules.
Install postfix-policyd-spf-perl

main.cf:

smtpd_recipient_restrictions =
    . . .
    reject_unauth_destination,
    check_policy_service unix:private/policy-spf,
    . . .

master.cf:

policy-spf  unix  -       n       n       -       -       spawn
    user=nobody argv=/usr/bin/policyd-spf

Best Answer

Related Solutions

email – How to Send Emails and Avoid Spam Classification

Receiving spam from the own email address. postfix

Related Topic