taRgrey - S25R + tarpitting + greylisting (tarpit + greylist policy server)

K2-net home

Japanese

November 14, 2006 (lastupdate April 18, 2012)

taRgrey is a patch that makes postgrey into a tarpitting policy server.
Tarpitting means response delay for blocking spam.
taRgrey is designed to decrease false positives.

taRgrey-patch for postgrey-1.34
http://k2net.hakuba.jp/pub/targrey-0.31-postgrey-1.34.patch (Thank you Kazuhiko san)
taRgrey-patch for postgrey-1.33
http://k2net.hakuba.jp/pub/targrey-0.31-postgrey-1.33.patch
taRgrey-patch for postgrey-1.32
http://k2net.hakuba.jp/pub/targrey-0.31-postgrey-1.32.patch
taRgrey-patch for postgrey-1.31
http://k2net.hakuba.jp/pub/targrey-0.31-postgrey-1.31.patch
taRgrey-patch for postgrey-1.30
http://k2net.hakuba.jp/pub/targrey-0.31-postgrey-1.30.patch
taRgrey-patch for postgrey-1.28/29
http://k2net.hakuba.jp/pub/targrey-0.31-postgrey-1.28.patch
taRgrey-patch for postgrey-1.27
http://k2net.hakuba.jp/pub/targrey-0.30-postgrey-1.27.patch

I had used greylisting combined with the S25R anti-spam system on my mail server.I named this system Rgrey. S25R defines reverse FQDN patterns to presume clients to have a dynamic IP address.

The Selective SMTP Rejection (S25R) System
http://www.gabacho-net.jp/en/anti-spam/anti-spam-system.html
Rgrey greylists clients only when their reverse FQDN matches the S25R rules.
Rgrey - S25R + greylisting
http://k2net.hakuba.jp/rgrey/ (This page is written in Japanese only.)
Therefore almost all legitimate mail servers are not greylisted, then, false positives get fewer. Rgrey on my mail server could block 94% of spam.

After that, I had used tarpitting combined with S25R. I named this system Starpit.

Starpit - S25R+tarpitting
http://d.hatena.ne.jp/stealthinu/20060706/p5 (This page is written in Japanese only.)
Tarpitting doesn't cause a long delay as in greylisting. It can accept mail servers even though they don't retry or retry with a different IP address each time. Starpit on my mail server could block 93% of spam when the delay time was set to 65 seconds.(July 2006 in Japan)

note: (July 2007 in Japan)



I recommend the 120 sec tarpitting for taRgrey mode as this result.

However, Starpit still caused false positives. Most of them were caused in case of email magazine and email news. taRgrey solves this problem. It accepts mail by greylisting when it is resent after blocked by tarpitting.

false positive
ham but reject by spam filter
false negative
spam but pass through spam filter

We set parameters for taRgrey so that anti-greylisting spam cannot pass through. We can safely configure greylisting tight because there are few legitimate mail blocked by Starpit.

Moreover, this patch enables us to specify the number of times of retry for greylisting. Most of anti-greylisting spam retries just once.

Additional functions with this patch are as follows:

Sample of configuration file
http://k2net.hakuba.jp/spam/postfix.conf.2.tar.gz

/etc/postfix/main.cf

smtpd_recipient_restrictions =
    permit_mynetworks
    permit_sasl_authenticated
    reject_unauth_destination
    ...(whitelist and other filter)
    check_client_access regexp:$config_directory/permit_client_nots25r
    check_policy_service   inet:60000
    ...
    permit

smtpd_data_restrictions =
    permit_mynetworks
    permit_sasl_authenticated
    ...(whitelist)
    check_client_access regexp:$config_directory/permit_client_nots25r
    check_policy_service   inet:60000
    permit
/etc/postfix/permit_client_nots25r
/\.dip\.t-dialin\.net$/       WARN
/\.dyn\.optonline\.net$/      WARN
...(other dynamic IP FQDN pattern(not match S25R pattern))
!/(^unknown$)|(^[^.]*[0-9][^0-9.]+[0-9].*\.)|
  (^[^.]*[0-9]{5})|(^([^.]+\.)?[0-9][^.]*\.[^.]+\..+\.[a-z])|
  (^[^.]*[0-9]\.[^.]*[0-9]-[0-9])|
  (^[^.]*[0-9]\.[^.]*[0-9]\.[^.]+\..+\.)|
  (^(dhcp|dialup|ppp|[achrsvx]?dsl)[^.]*[0-9])/ OK  ...(This regexp is one line)
/./                           WARN
# postgrey --tarpit=125 --targrey --retry-count=2 --delay=1800

The policy server is called twice. First, it is called by the smtpd_recipient_restrictions parameter, then it filters the client by tarpitting. At this time, it lists the client address on the tarpitting black list. Next, it is called by the smtpd_data_restrictions parameter, then it finds that the client has been able to pass through tarpitting. At this time, it deletes the client address from the tarpitting black list. If it finds that a client matches the tarpitting black list when it is called at the first time, then it greylists the client without tarpitting.

I designed taRgrey to decrease false positives. However, I heard the demand to use tarpitting for to decrease false negative. I prepared a tarpit&greylist mode with the following option.
"--tarpit=35 --retry-count=2"
It work tarpit&greylist mode. It delay 35 sec and return 450(greylist).

Mr.Nakahara reported a rate of false negative about each method.

It is the result that all these filtered blacklist of HELO and NS in sample configuration only for from connection to match dynamic IP of S25R, and was filtered.
taRgrey has false negative of 2.3 times of tarpit&greylist.

I also made a patch for postfix. It makes non blocking sleep.
The SLEEP action of postfix just calls the sleep UNIX system call. Therefore a sleeping process wastefully remains after a client disconnects during tarpitting. This patch enables postfix to detect disconnection by a client and kill the sleeping process.
postfix-2.3.x / 2.4.x / 2.5.x / 2.6.x / 2.7.x / 2.8.x / 2.9.x

postfix sleep patch
2.3 - 2.7 http://k2net.hakuba.jp/pub/postfix-2.3.3-sleep.patch
2.8 http://k2net.hakuba.jp/pub/postfix-2.8-sleep.patch
2.9 http://k2net.hakuba.jp/pub/postfix-2.9-sleep.patch

for qmail

Mr.chichi wrote taRgrey patch for Qgrey (qmail greylisting)
s25rtarpitgreylist

address

ƣ (SATOH Kiyoshi)
http://d.hatena.ne.jp/stealthinu/