I installed the SARE rule updates channel into a nightly sa-update routine yesterday, and noticed today that the rules didn’t appear to be getting used. I had misread the docs initially it turns out - you must use the updates.spamassassin.org channel if you plan to use any other channels since SA expects to find all rules in one location.

When I tried adding the default channel tonight I was getting the following error tonight while setting up sa-update:

error: GPG validation failed!
The update downloaded successfully, but it was not signed with a trusted GPG
key. Instead, it was signed with the following keys:

24F434CE

Perhaps you need to import the channel's GPG key? For example:

wget http://spamassassin.apache.org/updates/GPG.KEY
gpg --import GPG.KEY

After poking around for a bit I found the last line of the message is misleading - you actually want to run sa-update --import GPG.KEY after downloading it.

It seems I underestimated the spammers. I also misunderstood the comment posting system in Wordpress: I thought that updating the comment_status field to ‘registered_only’ meant that only registered users could leave comments. That’s not what happens. Values of ‘closed’ and ‘open’ appear to work as I understood, so I’m not sure what the other value was meant to do. The result was spammers could directly access wp-comments-posts.php and freely post their advertisements for online poker sites and viagra.

This evening a modified the Wordpress code to enable the ‘registered_only’ setting in the comment_status field to work as I thought it should. It doesn’t appear to break anything, but I haven’t extensively tested yet. The patches to wp-comments.php and wp-comments-posts.php are on the Pleiades patch pages.

It seems the spam wasn’t a single pass, so I’ve had to turn on comments for registered users only. I’m sure there’s spam bot that will register itself out there too, but I don’t want to block comments yet - although why not, I don’t know, given that no one but spammers has commented yet.

Slashdot has a story about the some of the concerns being expressed about the 9-year sentence handed down to a Virginia spammer. The prosecutor explains that the case was not just about spamming, but had a lot to do with the fraudulent nature of the business. The defendant was sending spam to sell products that didn’t work, in essence selling modern day snake-oil.

I wondered how long it would take for comments to get entered, and what they would be. Now I know: about 6 months, and full of SPAM for online poker sites. Weasals.

eWeek reports that MARID dumped Microsoft, and has instead opted for a hybrid solution for sender-authentication using SPF with modifications for checking other header fields (a variation of United SPF). The Microsoft extensions were patent encumbered and although a royalty-free license was available, obtaining the MS license agreement was a concern for the open source community. The Apache Software Foundation published an open letter a couple of weeks ago stating they would not be implementing Sender ID in their products under the MS license agreement.

PhysOrg.com reports that a Revolutionary Spam Firewall has been developed at the University of Queensland. It uses a support vector machine to categorize messages as a whole instead of using statistical probabilities based on individual keywords and other message properties. The developers claim the firewall is more accurate than they are at filtering out spam messages, and can detect the difference between a press release about Viagra and an add selling Viagra.

A startup firm has been created by UniQuest, the University’s commercialization company, to develop the firewall into a commercial product. I’ll be keeping an eye out for this one. I’ve seen claims of incredible accuracy before, but still have hopes it can be attained.

I setup junk mail filters at work last week, and they seem to be doing the trick. So far I’ve received no complaints of missing mail or blocked mail. We’re rejecting 28% of attempted deliveries, and have tagged 35% of the mail that passes the initial tests as spam. All told, 53% of all mail coming into the centre is unwanted, and blocked or marked as such.

I was poking around in my spam folders tonight comparing the results of Spam Assassin against a simple Procmail recipe that tags messages with URLs in them. I run both filters all the time, catching almost all spam. I need to add a recipe that tags base64 encoded messages to clean up the rest.

Interestingly enough, simply throwing out all messages with URLs that don’t come from people on a whitelist catches more spam than Spam Assassin does. Since almost all spam comes with a URL it’s easy to filter for unless you receive plenty of legitimate email with URLs, and can’t easily maintain a whitelist. The whitelist method has its drawbacks, primarily with web transactions that involve email verification. It’s a trivial task to find the message at the end of a spam folder to find the address and add it to the whitelist.

Microsoft has endorsed the Sender Policy Framework (SPF). Having them onside will help push the proposed standard forward, and it is a major step towards minimizing forged mail. It won’t solve the spam problem completely, since spammers are free to setup their own domains, but it helps to increase the cost of spamming. Blocking known spam-sending domains is trivial, so spammers will have to regularly purchase new domains in order to send to MTAs that only accept mail from domains with SPF records.

I’ve setup an SPF record in the DNS records for this domain. Every so six weeks or so, some spammer uses my address as the reply-to on a bulk mailing, and I get a pile of message undeliverable responses from stupid MTAs that are configured to send replies to obviously forged messages. Once we finish moving servers around at the hospital, we’ll be setting up SPF records in the DNS there.

I’ll also be configuring Postfix both here and at the office to drop bounced messages that were forged from us.