Fwd: [technical-issues] Banning .xyz email from my company's servers
FYI -------- Forwarded Message -------- Subject: [technical-issues] Banning .xyz email from my company's servers Date: Wed, 16 Mar 2016 09:48:29 +0100 From: Olivier MJ Crepin-Leblond <ocl@gih.com> To: Technical issues <technical-issues@atlarge-lists.icann.org> Hello all, this is just to let you know that after being flooded with 980 spam emails from a myriad of .XYZ email addresses, I have taken the unprecedented step of blocking the top level domain .XYZ from our email servers. The flood has been sustained for the past couple of months and for some reason, the emails manage to go through our regular filters like SpamAssassin, as well as various DNS block lists. Since my company's main servers run Postfix, I did the following: / ///etc/postfix/main.cf// // //smtpd_sender_restrictions = check_sender_access regexp:/etc/postfix/blocked_tlds// // // ///etc/postfix/blocked_tlds// // //### block sender access from the following spammy top-level domains// ///\.xyz$/ REJECT Top Level Domain .xyz banned due to spam/ We have not received any more significant amount of spam from the domains listed on https://www.spamhaus.org/statistics/tlds/ therefore we have not added them to our blocking list yet, but if any significant amount of spam starts coming in from them, we'll add them to the blocking list too. Being a strong supporter of Universal Acceptance of TLDs, I am sad to have had to reach this point but also being the bill payer in my company, I could not allow the bandwidth taken nor the time taken by readers of emails to delete this daily flood of spam from their Inbox. It is sad that the unwillingness of the domain name industry to subject themselves to standards of behaviour leads to domains that are completely untrustable and that need to be treated as a pest rather than an asset to the Internet. Kindest regards, Olivier -- Olivier MJ Crépin-Leblond, PhD http://www.gih.com/ocl.html
[is this OT, how did this start?] I use spamassassin system-wide to increase the spam score of a message from certain TLDs to near the threshold where it's just rejected. So for example in local.cf I add a rule like: header DOTTOP_RULE From =~ /.*\.top/i describe DOTTOP_RULE BZS 20160226 score DOTTOP_RULE 2.5 which means just having a .TOP TLD in the From gives it a base score of 2.5, so it wouldn't take much more, tripping some other spamassassin rules, to just get it blocked entirely. But it means in theory a very non-spammy msg from that TLD might still get through. -- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*
Dear Barry, thanks for your follow-up on this. That's a good idea too. I've checked the past messages in the queue and spamassassin scores range from 3.4 to 4.9 (with a trigger score required of 5.2). However, can you get a rejection message sent to the originator of the message? When blocking at Postfix level, the message is not accepted in the system & a bounce is issued. A genuine email originator would get a bounce explaining the bounce and try another method to get in contact. With spamassassin the message would just get dropped, wouldn't it? (apologies, my spamassassin coding is a bit crusty) Kindest regards, Olivier On 17/03/2016 05:32, bzs@theworld.com wrote:
[is this OT, how did this start?]
I use spamassassin system-wide to increase the spam score of a message from certain TLDs to near the threshold where it's just rejected.
So for example in local.cf I add a rule like:
header DOTTOP_RULE From =~ /.*\.top/i describe DOTTOP_RULE BZS 20160226 score DOTTOP_RULE 2.5
which means just having a .TOP TLD in the From gives it a base score of 2.5, so it wouldn't take much more, tripping some other spamassassin rules, to just get it blocked entirely.
But it means in theory a very non-spammy msg from that TLD might still get through.
-- Olivier MJ Crépin-Leblond, PhD http://www.gih.com/ocl.html
Since we've waded into this and you ask questions... Spamassassin itself will not drop emails, it only marks them as spam or not based on the required_score value. You can use other tools such as procmail (with a cooperating MTA such as postfix/sendmail/exim/etc) to drop, return, send to an alternate folder, or put in your INBOX based on the result of spamassassin. Or your own MUA might have facilities to not show it to you or whatever. I wouldn't worry much about true spammers readjusting based on the result, a typical professional spammer sends out about one billion messages per day. About the only automated hint some suspect is they might note whether an address returned unknown user or not since anything but unknown user indicates the email address is valid even if the message was rejected. So some choose to set up procmail (typically) scripts which appear to send back unknown user errors hoping that will drop them from the spammer's database entirely. I tend to think that's optimistic (THEY DON'T REALLY CARE!) but why not try? If one were trying to block a more targeted source and not a true "blind" spammer, perhaps just an annoying person who pitches slightly randomized versions of your domain for sale several times per day, then perhaps those methods would be effective and get one dropped from their database. The real tragedy of spam is the human time wasted over it. On March 17, 2016 at 10:31 ocl@gih.com (Olivier MJ Crepin-Leblond) wrote:
Dear Barry,
thanks for your follow-up on this. That's a good idea too. I've checked the past messages in the queue and spamassassin scores range from 3.4 to 4.9 (with a trigger score required of 5.2). However, can you get a rejection message sent to the originator of the message? When blocking at Postfix level, the message is not accepted in the system & a bounce is issued. A genuine email originator would get a bounce explaining the bounce and try another method to get in contact. With spamassassin the message would just get dropped, wouldn't it? (apologies, my spamassassin coding is a bit crusty) Kindest regards,
Olivier
On 17/03/2016 05:32, bzs@theworld.com wrote:
[is this OT, how did this start?]
I use spamassassin system-wide to increase the spam score of a message from certain TLDs to near the threshold where it's just rejected.
So for example in local.cf I add a rule like:
header DOTTOP_RULE From =~ /.*\.top/i describe DOTTOP_RULE BZS 20160226 score DOTTOP_RULE 2.5
which means just having a .TOP TLD in the From gives it a base score of 2.5, so it wouldn't take much more, tripping some other spamassassin rules, to just get it blocked entirely.
But it means in theory a very non-spammy msg from that TLD might still get through.
-- Olivier MJ Crépin-Leblond, PhD http://www.gih.com/ocl.html
-- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*
Simply had to share on the spam filtering issue: Apparently registrars have the same spam problem.
<legal@namecheap.com>: host smx1.registrar-servers.com[209.188.21.36] said: 550 A URL in this email (lloydspbuk . com) is listed on https://spamrl.com/. Please resolve and retry (in reply to end of DATA command)
In this case a Lloyds bank spoof domain registered via the same party rejecting it. Irony? :) Derek
Olivier If you’re using Postfix I’d recommend setting up Postgrey (see my rather old howto here: http://www.michele.me/blog/archives/2005/10/postgrey-greylisting-in-postfix-... ) Spam Assassin by itself will only score mails. For outright rejection you can plugin a few DNSBLs directly into Postfix and watch a lot of junk simply vanish completely .. Regards Michele -- Mr Michele Neylon Blacknight Solutions Hosting, Colocation & Domains http://www.blacknight.host/ http://blog.blacknight.com/ http://ceo.hosting/ Intl. +353 (0) 59 9183072 Direct Dial: +353 (0)59 9183090 ------------------------------- Blacknight Internet Solutions Ltd, Unit 12A,Barrowside Business Park,Sleaty Road,Graiguecullen,Carlow,Ireland Company No.: 370845 On 17/03/2016, 04:31, "at-large-bounces@atlarge-lists.icann.org on behalf of Olivier MJ Crepin-Leblond" <at-large-bounces@atlarge-lists.icann.org on behalf of ocl@gih.com> wrote:
Dear Barry,
thanks for your follow-up on this. That's a good idea too. I've checked the past messages in the queue and spamassassin scores range from 3.4 to 4.9 (with a trigger score required of 5.2). However, can you get a rejection message sent to the originator of the message? When blocking at Postfix level, the message is not accepted in the system & a bounce is issued. A genuine email originator would get a bounce explaining the bounce and try another method to get in contact. With spamassassin the message would just get dropped, wouldn't it? (apologies, my spamassassin coding is a bit crusty) Kindest regards,
Olivier
On 17/03/2016 05:32, bzs@theworld.com wrote:
[is this OT, how did this start?]
I use spamassassin system-wide to increase the spam score of a message from certain TLDs to near the threshold where it's just rejected.
So for example in local.cf I add a rule like:
header DOTTOP_RULE From =~ /.*\.top/i describe DOTTOP_RULE BZS 20160226 score DOTTOP_RULE 2.5
which means just having a .TOP TLD in the From gives it a base score of 2.5, so it wouldn't take much more, tripping some other spamassassin rules, to just get it blocked entirely.
But it means in theory a very non-spammy msg from that TLD might still get through.
-- Olivier MJ Crépin-Leblond, PhD http://www.gih.com/ocl.html
_______________________________________________ At-Large mailing list At-Large@atlarge-lists.icann.org https://atlarge-lists.icann.org/mailman/listinfo/at-large
At-Large Official Site: http://atlarge.icann.org
It's all in the balance, I guess. On a very high-volume site, the scoring of each incoming mail -- which requires examining content and evaluating it against what could be a complex ruleset -- presents a potentially significant drain on resources. If a reasonable judgment is made that a TLD is a source of no significant non-spam, then it's far more efficient to just block on the TLD. It's certainly not uncommon for people or organizations to say "if you want to communicate with me you need to do so in a way that is acceptable to me". The requirements could mean (in descending level of complexity) a local set of rules, or not being on the spamhaus black list, or not using an undesired TLD. Olivier's issue of bounce messages might be appropriate ... if the recipient of the bounce messages cared at all. I imagine most spamming sites would just drop them. Arguably that "drastic" action -- cutting off access from a whole TLD -- provides a market-based incentive for that TLD to clean up its act. If enough of the world won't accept mail from a TLD, theoretically its sales would drop and there would be a financial incentive to fix that. In the absence of any regulatory enforcement of abuse complaints, this is as effective an agent of change as one can hope for. Universal Acceptance is ICANN's begging the world to live with the products of its TLD expansion, no matter how awful they may be. But given ICANN's lack of any real end-user protections (led by identifiable Board members who believe that end-users are not legitimate stakeholders), this is really the only tool available with which to fight back. - Evan On 17 March 2016 at 05:32, <bzs@theworld.com> wrote:
[is this OT, how did this start?]
I use spamassassin system-wide to increase the spam score of a message from certain TLDs to near the threshold where it's just rejected.
So for example in local.cf I add a rule like:
header DOTTOP_RULE From =~ /.*\.top/i describe DOTTOP_RULE BZS 20160226 score DOTTOP_RULE 2.5
which means just having a .TOP TLD in the From gives it a base score of 2.5, so it wouldn't take much more, tripping some other spamassassin rules, to just get it blocked entirely.
But it means in theory a very non-spammy msg from that TLD might still get through.
-- -Barry Shein
Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* _______________________________________________ At-Large mailing list At-Large@atlarge-lists.icann.org https://atlarge-lists.icann.org/mailman/listinfo/at-large
At-Large Official Site: http://atlarge.icann.org
-- Evan Leibovitch Geneva, CH Em: evan at telly dot org Sk: evanleibovitch Tw: el56
On March 17, 2016 at 13:26 evan@telly.org (Evan Leibovitch) wrote:
It's all in the balance, I guess.
On a very high-volume site, the scoring of each incoming mail -- which requires examining content and evaluating it against what could be a complex ruleset -- presents a potentially significant drain on resources. If a reasonable judgment is made that a TLD is a source of no significant non-spam, then it's far more efficient to just block on the TLD.
That's true. For example we block many nets based on just the IP address and those connections are just dropped nearly instantly. Also pattern matches of sending hosts. For example not accepting anything coming from a host which appears to be an end-user (dhcp-.*, ppp-.*, host names which look like ip addresss like 192.74.137.22.somehost.com that sort of thing) tho generally there's a little more qualification than that, end-user networks which have been sources of spam. So the recommendation was fairly specific to the example given. I'm resistant to dropping an entire TLD and try to use more focused methods such as raising their base spamassassin score or, well, we have a lot of tools like testing regular expressions on Subject: lines, From addresses. Hint: Don't ever open an account with a user name containing the name of any erectile dysfunction medication or variant thereof (e.g., replacing 'i' with 1) and expect me to ever see your mail!
It's certainly not uncommon for people or organizations to say "if you want to communicate with me you need to do so in a way that is acceptable to me". The requirements could mean (in descending level of complexity) a local set of rules, or not being on the spamhaus black list, or not using an undesired TLD.
Olivier's issue of bounce messages might be appropriate ... if the recipient of the bounce messages cared at all. I imagine most spamming sites would just drop them.
Arguably that "drastic" action -- cutting off access from a whole TLD -- provides a market-based incentive for that TLD to clean up its act. If enough of the world won't accept mail from a TLD, theoretically its sales would drop and there would be a financial incentive to fix that.
You're an optimist :-) That assumes a lot of the net would block them which I suspect is not the case. But there have been analogues, some quite troublesome. For example organizations buying returned IP address blocks only to find they're in many, many spam databases. Probably why they were returned.
In the absence of any regulatory enforcement of abuse complaints, this is as effective an agent of change as one can hope for.
Universal Acceptance is ICANN's begging the world to live with the products of its TLD expansion, no matter how awful they may be. But given ICANN's lack of any real end-user protections (led by identifiable Board members who believe that end-users are not legitimate stakeholders), this is really the only tool available with which to fight back.
There are other tools but point taken. Another aspect is that with 90+% of all email being spam and as I said earlier typical "real" spammers sending on the order of a billion messages per day there is the issue of bandwidth and resources in general. It's very nice to have strong gates when the barbarians are at the gate but who paid for those gates and, more importantly, there are barbarians out there! I could show you logs of spammers, for example, sending to generated names such as aaaa@theworld.com, aaab@theworld.com, aaac@theworld.com, etc, millions of them, for days or weeks, until they're just blocked at the IP level. And then a customer asks why it took 20 minutes for an email to get to them or why some path they're using (e.g., interactive web site) is so sluggish. Maybe it's all the spam trying to travel along the same path?!?! People tend to think of this problem only in terms of their own mailbox, what spam they did or didn't see, which is understandable. At a governance level we need to also think about the mind-boggling resource consumption and waste of human resources caused by spam. And the inherent criminality of course, fraud etc. Now if you will all open your psalters to page 334 we will...
- Evan
On 17 March 2016 at 05:32, <bzs@theworld.com> wrote:
[is this OT, how did this start?]
I use spamassassin system-wide to increase the spam score of a message from certain TLDs to near the threshold where it's just rejected.
So for example in local.cf I add a rule like:
header DOTTOP_RULE From =~ /.*\.top/i describe DOTTOP_RULE BZS 20160226 score DOTTOP_RULE 2.5
which means just having a .TOP TLD in the From gives it a base score of 2.5, so it wouldn't take much more, tripping some other spamassassin rules, to just get it blocked entirely.
But it means in theory a very non-spammy msg from that TLD might still get through.
-- -Barry Shein
Software Tool & Die | bzs@TheWorld.com | http:// www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo* _______________________________________________ At-Large mailing list At-Large@atlarge-lists.icann.org https://atlarge-lists.icann.org/mailman/listinfo/at-large
At-Large Official Site: http://atlarge.icann.org
-- Evan Leibovitch Geneva, CH
Em: evan at telly dot org Sk: evanleibovitch Tw: el56
-- -Barry Shein Software Tool & Die | bzs@TheWorld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD The World: Since 1989 | A Public Information Utility | *oo*
Tbh, on some email resources I manage, I've been contemplating combining Postgrey and logs with registrar lookups to decide on an additional filtering mechanisms. Derek On 2016-03-18 09:50 PM, bzs@theworld.com wrote:
Now if you will all open your psalters to page 334 we will...
participants (5)
-
bzs@theworld.com -
Derek Smythe -
Evan Leibovitch -
Michele Neylon - Blacknight -
Olivier MJ Crepin-Leblond