I'm setting up SPF records for my domain, and am not getting the results that I expect. It's quite possible I'm making some sort of mistake, but first I'd like to ask: does it take time for the changes I make to SPF records to propagate?
domain-name-system – Do Changes in SPF Records Take Time to Propagate?
domain-name-systemlatencyspf
Related Solutions
"DNS propagation" isn't a real phenomenon, per se. Rather, it is the manifest effect of the caching functionality specified in the DNS protocol. Saying that changes "propagate" between DNS servers is a convenient falsehood that's, arguably, easier to explain to non-technical users than describing all of the details of the DNS protocol. It's not really how the protocol works, though.
Recursive DNS servers make queries on behalf of clients. Recursive DNS servers, typically run by ISPs or IT departments, are used by client computers to resolve names of Internet resources. Recursive DNS servers cache the results of queries they make to improve efficiency. Queries for already-cached information can be answered without making any additional queries. The duration, in seconds, that a result is cached is supposed to be based on a configurable value called the Time To Live (TTL). This value is specified by the authoritative DNS server for the record queried.
There is no one answer to all the questions being asked because DNS is a distributed protocol. The behavior of DNS depends on the configuration of the authoritative DNS server for a given record, the configuration of recursive DNS servers making queries on behalf of client computers, and DNS caching functionality built-in to the client computers' operating systems.
It's good practice to specify a TTL value short enough to accommodate neecssary day-to-day changes to DNS records, but long enough so to create a "win" in caching (i.e. not so short as to age-out of cache too quickly to provide any efficiency improvement). Employing a balanced strategy with TTL values results in a "win" for everyone. It reduces both the load and bandwidth utilization for the authoritative DNS servers for a given domain, the root servers, and the TLD servers. It reduces the upstream bandwidth utilization for the operator of the recursive DNS server. It results in quicker query responses for client computers.
As a DNS record's TTL is set lower load and bandwidth utilization on the authoritative DNS servers will increase because recursive DNS servers will not be able to cache the result for a long duration. As a record's TTL is higher changes to records will not appear to "take effect" quickly because client computers will continue to receive cached results stored on their recursive DNS servers. Setting the optimal TTL comes down to a balancing act between utilization and ability to change records quickly and see those changes reflected on clients.
It is worth noting that some ISPs are abusive and ignore the TTL values specified by the authoritative DNS servers (substituting their own administrative override, which is a violation of RFC). There's nothing to be done about this, from a technical perspective. If the operators of abusive DNS servers can be located complaints to their systems administrators might result in their implementing best practices (arguably what amounts to common sense for any network engineer familiar with DNS). This particular type of abuse isn't a technical problem.
If everybody "plays by the rules" changes to DNS records can "take effect" very quickly. In the case of changing the IP address assigned to an "A" record, for example, an exponential backoff of the TTL value would be performed, leading up to the time the change will be made. The TTL might start at 1 day, for example, and be decreased to 12 hours for a 24 hour period, then 6 hours for a 12 hour period, 3 hours for a 6 hour period, etc, down to some suitably small interval. Once the TTL has been backed-off the record can be changed and the TTL brought back up to the desired value for day-to-day operations. (It is not necessary to use an exponential backoff, however this strategy minimizes the time the record will have a low TTL and decreases load on the authoritative DNS server.)
After making a DNS record change logs should be monitored for access attempts being made as a result of the old DNS record. In the example of changing an "A" record to refer to a new IP address a server should remain present at the old IP address to handle access attempts resulting from client computers still using the old "A" record. Once access attempts based on the old record have reached an acceptably low level the old IP address can be disused. If the requests related to an old record are not abating quickly it is possible that (as described above) a recursive DNS server is ignoring the authoritative TTL. Knowing the source IP address of an access attempt, however, does not provide direct information as to the recursive DNS server responsible for supplying an old record. If the IP addresses of errant access attempts are all related to a single ISP it may be possible to locate the offending DNS server and contact its operator.
Personally, I've seen changes "take effect" immediately, in a few hours, and in some cases with a particular brain-damaged ISP, after several days. Doing a backoff of your TTL and being mindful of how the process works will increase your changes for success, but you can't ever be sure what some well-meaning idiot might be doing with their recursive DNS servers.
SPF records detail which servers are allowed to send mail for your domain.
Questions 1-3 really summarise the whole point of SPF: You're supposed to be listing the addresses of all the servers that are authorised to send mail coming from your domain.
If you don't have an exhaustive list at this time, it's generally not a good idea to set up an SPF record. Also a domain can only have one SPF record, so you'll need to combine all the information into a single record.
The individual questions really just help break the list down for you.
- asks you for other domains whose mail servers may relay mail from you; if you have eg a secondary MX server at mail-relay.example.org, and that is the main mail server (MX record) for the domain
example.org
, then you should entermx:example.org
. Your SPF record should include your own domain's MX record under nearly all circumstances (mx
). - asks you for your ip netblocks. If you have colocated servers at 1.2.3.0/28, and your office address space is 6.7.8.0/22, enter
ip4:1.2.3.0/28 ip4:6.7.8.0/22
. IPv6 space should be added as egip6:2a01:9900:0:4::/64
. - if (eg) you also have a machine off in someone else's office that has to be allowed to send mail from your domain, enter that as well, with eg
a:mail.remote.example.com
.
Your mobile phone users are problematic. If they send email by connecting to your mail server using eg SMTP AUTH, and sending through that server, then you've dealt with them by listing the mail server's address in (2). If they send email by just connecting to whatever mail server the 3G/HSDPA provider's offering, then you can't do SPF meaningfully until you have rearchitected your email infrastructure so that you do control every point from which email purporting to be from you hits the internet.
Question 4 is a bit different, and asks what recipients should do with email that claims to be from your domain that doesn't come from one of the systems listed above. There are several legal responses, but the only interesting ones are ~all
(soft fail) and -all
(hard fail). ?all
(no answer) is as useless as ~all
(qv), and +all
is an abomination.
~all
is the simple choice; it tells people that you've listed a bunch of systems who are authorized to send mail from you, but that you're not committing to that list being exhaustive, so mail from your domain coming from other systems might still be legal. I urge you not to do that. Not only does it make SPF completely pointless, but some mail admins on SF deliberately configure their SPF receivers to treat ~all
as the badge of a spammer. If you're not going to do -all
, don't bother with SPF at all.
-all
is the useful choice; it tells people that you've listed the systems that are allowed to send email from you, and that no other system is authorized to do so, so they are OK to reject emails from systems not listed in your SPF record. This is the point of SPF, but you have to be sure that you have listed all the hosts that are authorized to originate or relay mail from you before you activate it.
Google is known to advise that
Publishing an SPF record that uses -all instead of ~all may result in delivery problems.
well, yes, it may; that is the whole point of SPF. We cannot know for sure why google gives this advice, but I strongly suspect that it's to prevent sysadmins who don't know exactly whence their email originates from causing themselves delivery problems. If you don't know where all your email comes from, don't use SPF. If you're going use SPF, list all the places it comes from, and tell the world you're confident in that list, with -all
.
Note that none of this is binding on a recipient's server; the fact that you advertise an SPF record in no way obliges anyone else to honour it. It is up to the admins of any given mail server what email they choose to accept or reject. What I think SPF does do is allow you to disclaim any further responsibility for email that claimed to be from your domain, but wasn't. Any mail admin coming to you complaining that your domain is sending them spam when they haven't bothered to check the SPF record you advertise that would have told them that the email should be rejected can fairly be sent away with a flea in their ear.
Since this answer has been canonicalised, I'd better say a few words about include
and redirect
. The latter is simpler; if your SPF record, say for example.com
, says redirect=example.org
, then example.org
's SPF record replaces your own. example.org
is also substituted for your domain in those look-ups (eg, if example.org
's record includes the mx
mechanism, the MX
lookup should be done on example.org
, not on your own domain).
include
is widely misunderstood, and as the standard's authors note "the name 'include' was poorly chosen". If your SPF record include
s example.org
's record, then example.org
's record should be examined by a recipient to see if it gives any reason (including +all
) to accept your email. If it does, your mail should pass. If it doesn't, the recipient should continue to process your SPF record until landing on your all
mechanism. Thus, -all
, or indeed any other use of all
except +all
, in an include
d record, has no effect on the result of processing.
For more information on SPF records http://www.openspf.org is an excellent resource.
Please don't take this the wrong way, but if you get an SPF record wrong, you can stop a significant fraction of the internet from receiving email from you until you fix it. Your questions suggest you might not be completely au fait with what you're doing, and if that's the case, then you might want to consider getting professional assistance before you do something that stops you sending email to an awful lot of people.
Edit: thank you for your kind words, they're much appreciated.
SPF is primarily a technique to prevent joe-jobbing, but some people seem to have started to use it to try to detect spam. Some of those may indeed attach a negative value to your having no SPF record at all, or an overbroad record (eg a:3.4.5.6/2 a:77.5.6.7/2 a:133.56.67.78/2 a:203.54.32.1/2
, which rather sneakily equates to +all
), but that's up to them and there's not much you can do about it.
I personally think SPF is a good thing, and you should advertise a record if your current mail structure permits it, but it's very difficult to give an authoritative answer, valid for the entire internet, about how people are using a DNS record designed for a specific purpose, when they decide to use it for a different purpose. All I can say with certainty is that if you do advertise an SPF record with a policy of -all
, and you get it wrong, a lot of people will never see your mail.
Edit 2: deleted pursuant to comments, and to keep the answer up-to-date.
Best Answer
Yes, there might be caching or other delays depending on how the zone is being edited (
nsupdate
results in fairly immediate changes, less so if some web front-end talks to a database that maybe eventually does something to update a zone), how zone transfers are done (the master DNS server might push changes, or the slaves could instead be configured to periodically poll that server for updates), and whether you are querying an authoritative DNS server or something else that might have cached the previousTXT
record due to a previous query from your client, and thus is unaware of the changes the master server(s) might already know about.Use
nslookup
ordig
to query different servers (and also check theSOA
serial number, it should have bumped on a change, if not, you're looking at old data).The
TTL
of theTXT
record might be an important thing to know; the fulldig
output should include that.