How can I turn off the outgoing mails filtering only? My postfix server is very slow. Sending 1 mail/sec, and I think the problem is with the outgoing mail filter (amavis). Or is it possible the problem is something else? The DNS is good.
Postfix very slow sending mail
postfix
Related Solutions
What you want to do requires to treat mails from users using your server as their MSA (i.e. their outgoing relay) with a different policy than those received from 3rd parties (i.e. when your mailserver is acting in it's MX role). Fortunately, amavis has just the right tool for you: Policy Banks.
Let's see how you could define a policy for your users:
$policy_bank{'PREQ-SUB'} = {
originating => 1, # indicates client is ours, allows signing
final_spam_destiny => D_DISCARD, # discard spam
final_virus_destiny => D_DISCARD, # discard spam
warnspamsender => 1, # send a warning
forward_method => 'smtp:127.0.0.1:10025', # you probably need to adjust this
smtpd_discard_ehlo_keywords => ['8BITMIME'], # force mail conversion to Q/P
smtpd_greeting_banner => '${helo-name} ${protocol} ${product} SUBMISSION service ready',
spam_admin_maps => ["postmaster\@example.net"], # warn of spam from us
virus_admin_maps => ["postmaster\@example.net"], # warn of viruses from us
};
From the naming of this policy bank, you can already guess that I'm running this as a pre-queue filter that get's triggered if mail is delivered via the submission TCP port 587. To make this configuration work, I told my Postfix MTA to deliver mails that the submission service received to localhost on port 10028 (whereas, when acting as public MX, the server forwards mail to port 10024). To activate two ports in amavis and bind the PREQ-SUB policy to port 10028, I'm using these settings:
# policy bank definition
$inet_socket_port = [10024, 10028]; # listen on listed inet tcp ports
$interface_policy{'10028'} = 'PREQ-SUB'; # mail submitted using TLS on submission/smtps port
The corresponding master.cf entry for Postfix is:
submission inet n - - - - smtpd -o smtpd_tls_security_level=encrypt
-o tls_preempt_cipherlist=$submission_tls_preempt_cipherlist
-o smtpd_tls_protocols=$submission_smtpd_tls_protocols
-o smtpd_tls_ciphers=$submission_smtpd_tls_ciphers
-o smtpd_tls_exclude_ciphers=$submission_smtpd_tls_exclude_ciphers
-o smtpd_sasl_auth_enable=yes
-o smtpd_recipient_restrictions=$submission_smtpd_recipient_restrictions
-o milter_macro_daemon_name=ORIGINATING
-o smtpd_proxy_filter=127.0.0.1:10028
-o syslog_name=postfix-submission/smtpd
-o receive_override_options=no_header_body_checks
Note that this does actually do a bit more than simply send mails to amavis, e.g. set cipher lists and so on (you'll notice the main.cf variable references).
So, what can you do if your users do NOT submit their mail on port 587, or not all of them do? Well, you'll have to leave the land of 100% certainty then. amavis can analyze the contents of a mail and act on the presence of headers. One such headers could be the authenticated user's name which Postfix adds if you set smtpd_sasl_authenticated_header = yes. You could then tell amavis to act on this header:
package Amavis::Custom;
use strict;
BEGIN {
import Amavis::Conf qw(:platform :confvars c cr ca $myhostname);
import Amavis::Util qw(do_log untaint safe_encode safe_decode);
import Amavis::rfc2821_2822_Tools;
import Amavis::Notify qw(build_mime_entity);
}
sub new {
my($class,$conn,$msginfo) = @_;
my($self) = bless {}, $class;
my $auth_sender = 0;
foreach my $line (@{$msginfo->{'orig_header'}}) {
$line =~ s/\n / /g;
# WARNING: you need to improve this to AT LEAST also match
# for your OWN mail servers name!
$auth_sender = 1 if $line =~ m/^Authenticated sender/i;
}
if ($auth_sender) {
do_log(2, sprintf("Load pre-queue submission policy bank"));
Amavis::load_policy_bank('PREQ-SUBMISSION')
}
return $self;
}
1; # insure a defined return
Please don't ignore the warning within this code: Headers are easily fakeable, and other mail servers could insert an "Authenticated sender" header, too, so better match for something like "your-mailserver.example.net.*Authenticated sender".
On a final note, regarding one of your comments: Running a mail server DOES take a lot of time, and requires you to constantly monitor it for abuse. There is not "get out of jail free" card when it comes to taking part in the global email system!
Disclaimer: this answer doesn't contain the solution about how to make postfix bcc email twice. I'm just decipher the content of your maillog line.
Postfix part pre-content_filter
Feb 3 13:30:25 email1mail postfix/submission/smtpd[3319]: connect from somewhere[xxx.xxx.xxx.xxx]
Feb 3 13:30:25 email1mail postfix/submission/smtpd[3319]: 47F943003E: client=somewhere[xxx.xxx.xxx.xxx], sasl_method=PLAIN, sasl_username=alice@mail.example.com
Feb 3 13:30:25 email1mail postfix/cleanup[3323]: 47F943003E: message-id=<54D0BF62.6030008@mail.example.com>
Postfix receive email from alice@mail.example.com and assign a queue-ID 47F943003E
Feb 3 13:30:25 email1mail postfix/qmgr[3313]: 47F943003E: from=<alice@mail.example.com>, size=611, nrcpt=2 (queue active)
At this stage, postfix already make BCC of your email. That's why value of nrcpt is 2 instead 1.
Feb 3 13:30:46 email1mail postfix/smtp[3324]: 47F943003E: to=<archiv@archiv.example.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=21, delays=0.07/0.01/0/21, dsn=2.6.0, status=sent (250 2.6.0 from MTA(smtp:[192.168.1.102]:10026): 250 2.6.0 Message received)
Feb 3 13:30:46 email1mail postfix/smtp[3324]: 47F943003E: to=<carol@mail.example.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=21, delays=0.07/0.01/0/21, dsn=2.6.0, status=sent (250 2.6.0 from MTA(smtp:[192.168.1.102]:10026): 250 2.6.0 Message received)
Two lines above was another proof that postfix already BCC your email then sent it to content filter port.
Amavis part pre-chipermail
Feb 3 13:30:25 email1mail amavis[3061]: (03061-05) ESMTP:[127.0.0.1]:10024 /var/lib/amavis/tmp/amavis-20150203T130744-03061-ZTBAQ9iE: <alice@mail.example.com> -> <archiv@archiv.example.com>,<carol@mail.example.com> SIZE=611 Received: from email1mail.localdomain ([127.0.0.1]) by localhost (mail.example.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP; Tue, 3 Feb 2015 13:30:25 +0100 (CET)
This is the evidence that amavis receive email with two recipients.
Feb 3 13:30:46 email1mail amavis[3061]: (03061-05) Passed CLEAN {RelayedInbound,RelayedOpenRelay}, BCM [xxx.xxx.xxx.xxx]:59491 [xxx.xxx.xxx.xxx] <alice@mail.example.com> -> <archiv@archiv.example.com>,<carol@mail.example.com>, Queue-ID: 47F943003E, Message-ID: <54D0BF62.6030008@mail.example.com>, mail_id: XP5xmOE51_t2, Hits: -1, size: 611, queued_as: 250 2.6.0 Message received, 21204 ms
This is log when amavis forward email (with two recipients) to chipermail
Amavis part post-chipermail
The strange part is when chipermail forward back to amavis. Chipermail splits the the original email into two email with one recipient per email.
Feb 3 13:30:47 email1mail amavis[3060]: (03060-06) ESMTP:[192.168.2.10]:10028 /var/lib/amavis/tmp/amavis-20150203T130722-03060-gusE2h0r: <alice@mail.example.com> -> <archiv@archiv.example.com> Received: from ciphermail ([192.168.1.102]) by localhost (mail.example.com [192.168.2.10]) (amavisd-new, port 10028) with ESMTP for <archiv@archiv.example.com>; Tue, 3 Feb 2015 13:30:47 +0100 (CET)
Feb 3 13:30:47 email1mail amavis[3061]: (03061-06) ESMTP:[192.168.2.10]:10028 /var/lib/amavis/tmp/amavis-20150203T130744-03061-ZTBAQ9iE: <alice@mail.example.com> -> <carol@mail.example.com> Received: from ciphermail ([192.168.1.102]) by localhost (mail.example.com [192.168.2.10]) (amavisd-new, port 10028) with ESMTP for <carol@mail.example.com>; Tue, 3 Feb 2015 13:30:47 +0100 (CET)
Here the evidence of email splitting. Two emails was received by amavis from chipermail.
Feb 3 13:31:08 email1mail amavis[3061]: (03061-06) Passed UNCHECKED {RelayedTaggedInbound}, ACM [xxx.xxx.xxx.xxx] [xxx.xxx.xxx.xxx] <alice@mail.example.com> -> <carol@mail.example.com>, Message-ID: <54D0BF62.6030008@mail.example.com>, mail_id: cD-pyG9zKnpK, Hits: -1, size: 1860, queued_as: 96AC730057, 21510 ms
Amavis forward email (with recipient carol) to postfix with message ID 96AC730057
Feb 3 13:31:09 email1mail amavis[3060]: (03060-06) Passed CLEAN {RelayedOpenRelay}, ACM [xxx.xxx.xxx.xxx] [xxx.xxx.xxx.xxx] <alice@mail.example.com> -> <archiv@archiv.example.com>, Message-ID: <54D0BF62.6030008@mail.example.com>, mail_id: l6papfJwmT6R, Hits: -1, size: 871, queued_as: A4AEC3005E, 22636 ms
Amavis forward email (with recipient archiv) to postfix with message ID A4AEC3005E
Postfix part post-content_filter
Feb 3 13:31:08 email1mail postfix/smtpd[3340]: connect from localhost[127.0.0.1]
Feb 3 13:31:08 email1mail postfix/smtpd[3340]: 96AC730057: client=localhost[127.0.0.1]
Feb 3 13:31:08 email1mail postfix/cleanup[3323]: 96AC730057: message-id=<54D0BF62.6030008@mail.example.com>
Postfix receive email from amavis with message ID 96AC730057
Feb 3 13:31:08 email1mail postfix/qmgr[3313]: 96AC730057: from=<alice@mail.example.com>, size=2357, nrcpt=2 (queue active)
Postfix make BCC of your email again. You can spot that the value of nrcpt is 2.
Feb 3 13:31:08 email1mail postfix/lmtp[3342]: 96AC730057: to=<carol@mail.example.com>, relay=email1mail.localdomain[private/dovecot-lmtp], delay=0.05, delays=0.01/0.02/0.01/0.01, dsn=2.0.0, status=sent (250 2.0.0 <carol@mail.example.com> RXUSJ4y/0FQQDQAAXzmN0w Saved)
Feb 3 13:31:08 email1mail postfix/smtp[3343]: 96AC730057: to=<archiv@archiv.example.com>, relay=archiv.example.com[192.168.1.103]:25, delay=0.11, delays=0.01/0.02/0.06/0.02, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 2138426503)
Postfix doesn't send the email to content filter again. It send carol's email to dovecot and send archiv's email to archiv server
In another part
Feb 3 13:31:09 email1mail postfix/smtpd[3345]: connect from localhost[127.0.0.1]
Feb 3 13:31:09 email1mail postfix/smtpd[3345]: A4AEC3005E: client=localhost[127.0.0.1]
Feb 3 13:31:09 email1mail postfix/cleanup[3323]: A4AEC3005E: message-id=<54D0BF62.6030008@mail.example.com>
Postfix receive email from amavis with message ID A4AEC3005E
Feb 3 13:31:09 email1mail postfix/qmgr[3313]: A4AEC3005E: from=<alice@mail.example.com>, size=1358, nrcpt=2 (queue active)
Postfix make BCC of your email again to itself. You can spot that the value of nrcpt is 2.
Feb 3 13:31:09 email1mail postfix/smtp[3343]: A4AEC3005E: to=<archiv@archiv.example.com>, relay=archiv.example.com[192.168.1.103]:25, delay=0.08, delays=0.01/0/0.06/0.01, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 26BF726509)
Postfix send email with two duplicate recipients to archiv server
As you can see, postfix send three copy of email after pass it to chipermail. Your log in archiv server confirm this behavior.
So, how can we bcc the email before pass to chipermail?
Right now, I don't have an idea. The problem is, postfix always pass the bcc-ed email to content_filter (chipermail and amavis). This architecture prevents to send email to bcc address before pass it to content_filter.
Fine, how can we bcc the only one email after pass to chipermail?
You already have the solution, put receive_override_options = no_address_mappings on the main.cf.
Best Answer
Check whether amavis is actually being run. I suspect your issue is on the remote end, and you can't turn off filtering at that end.
Draining a queue at 1 mail/sec per thread is probably reasonable. Some of the things which may happen on the other end which will slow down your transfer rate include:
All of these are likely to add up to about a second. Verifying your DNS configuration is complete will help. Configuring both SPF and TXT records for SPF may speed up DNS checks in some cases.
If you can configure Postfix to do immediate delivery, then each message should use its own thread. I don't know if Postfix supports multiple delivery threads. Multiple threads can create synchronization issues which may slow down your server.
High volume delivery may require special tuning of the server software. Exim allows the spool database to be split to allow multiple threads to run with less contention. If the email is not personalized, then putting the addresses in a BCC field and grouping recipients by domain may help.