Welcome, Guest. Please Login
 
  HomeHelpSearchLogin FAQ Radified Ghost.Classic Ghost.New Bootable CD Blog  
 
Page Index Toggle Pages: 1
Send Topic Print
Comparing POP3 spam filtering software (Read 9231 times)
Dan Goodell
Special Guest
*****
Offline



Posts: 552
N California


Back to top
Comparing POP3 spam filtering software
Mar 29th, 2009 at 5:12am
 
This is a follow-on to another thread in which I described using virtualization to run a comparison of several email filtering programs.  See that thread for details of how the virtual machines were setup and how the test was conducted.  This thread is for any discussion of the results of the comparison itself.


The Objective:  Recently a few people have asked me to recommend spam filtering software to use with Outlook Express, so I thought I'd run a test comparing a few of these programs head to head.  (Keep your eye on the ball here, folks--the subject is spam filtering software, not the wisdom or folly of using Outlook Express.)  It wasn't my aim to determine the best program out there, but just to find something that works well enough to recommend to people.  If I could review 10 candidates, I figured one of them ought to be good enough that I could recommend.


The Test Candidates:  After some google searches and scanning a few newsgroup discussions, I chose 10 mail filtering programs for the test.  (Actually 11, but one failed to install.)  I'm not claiming these are the 10 best programs, they're just 10 that seemed popular or that I already had some experience with.  Guessing people using Outlook Express may be more frugal than average, I limited the selection to freeware programs only.  Besides, if I found a freeware program that worked well, it would be hard to tell someone to buy something else.


Providing Content for the Test:  Since I have control over my own domain mail server, I created 10 temporary mail accounts that would fill with identical collections of junk and non-junk mail.  That way, each program would be tested on the same content.


The Virtual Machines:  Using VirtualPC, I created 10 copies of XP that could coexist simultaneously on my home lan.  A different anti-spam program was installed on each virtual machine, and one of the temporary mail accounts was associated with each copy of OE.

Since I would be testing the effectiveness a typical non-techie might be able to expect, I deliberately chose not to read any manuals or installation instructions.  I would change a few settings if they were obvious, but other than that they'd have to work right out of the box.  If the program had an option to use a public blacklist, I enabled it.


The Conclusion:  Most of these programs need to be "trained"--at the beginning, you have to tell them which messages are junk or non-junk.  It's kind of tedious at first, but as they learn they start predicting more accurately, so there are fewer and fewer question marks that need user intervention.

In the first couple days it wasn't obvious which programs would do well, but as I got into the second week clear differences began to emerge.  At the end of two weeks, 4 were clearly running away from the pack.  Not only were the others less accurate in scoring spam, some of them permanently deleted good messages without the user knowing it!  (I knew, because I was watching 10 identical mailboxes.)


POPFile and K9 get my "Editor's Choice" awards.  Both use a similar style--you run OE to download mail (via the spam filter's proxy), then subsequently launch the filter's UI to make any training corrections.  Within two weeks, both were correctly marking spam nearly all the time.  Mail identified as spam is tagged, so you can setup an OE rule to redirect it into a separate folder.  Nothing is deleted by the spam filter, so you have a chance to scan your custom OE folder for any misidentified good mail.

Mailwasher uses a different concept--you use its UI first to check what mail is waiting on the server.  It premarks what it thinks is spam, but the user has a chance to correct that.  Finally, it deletes the spam directly from the server, then launches OE to download whatever is left.

Mailwasher was somewhat less accurate than the top two, and the conceptual difference means you will always work from both the filter's UI plus OE's interface.  In contrast, as POPFile or K9 became more accurate, there was less need to get into the filter's UI at all.  Nevertheless, it's nice to delete junk before it gets downloaded to your computer in the first place.  Another feature I really like about Mailwasher is that it shows you both the sender's name and email address, which makes it easy to spot spam purporting to be from a friend or colleague's name when the email address was wrong.  On the down side, Mailwasher Free devotes a big portion of its UI to ads nagging you to upgrade to the paid version.  (Fair disclosure: on my personal machine, I have for years used an older paid version of Mailwasher.)

Finally, Spamihilator was as accurate as the top two, but I just didn't think it was as intuitive or easy to use.  That may be a matter of preference, though, and another user may like it better than I did.  Like the top two, you first run OE, then check the filter's UI.  Unlike the other two, it doesn't mark spam, it blocks it so it doesn't get through to OE.  If it erroneously blocks a good message, you won't know until you check the filter's UI, where you can release the message and let it through the next time you run OE.


 
 
IP Logged
 

MrMagoo
Übermensch
*****
Offline


Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)


Back to top
Re: Comparing POP3 spam filtering software
Reply #1 - Mar 29th, 2009 at 8:36am
 
Dan Goodell wrote on Mar 29th, 2009 at 5:10am:
Recently a few people have asked me to recommend spam filtering software to use with Outlook Express, so I thought I'd run a test comparing a few of these programs head to head.

Answer:
Use Gmail.  My Gmail account used to get very little spam.  Then, Google bought Postini, the industry anti-spam leaders.  Now, I don't get any spam.

Ok, so that doesn't answer the question people asked you.  But it does answer the question they meant to ask.  What they want is an easy way to access all their email accounts with no spam.  Gmail does that, and all you do is sign up.  No additional software, no additional sign-ups, no additional research.  Google handles the backups.  They give you access from any computer, or your phone.  It looks nice, and it behaves exactly like you want it to.

I know, a lot of people aren't ready for this type of advice.  They are just now learning that you don't really get emails from the king of Nigeria offering you a million dollars, and here I am talking about moving your email into 'the cloud.'  But in the end, its a better solution and the way the world is heading.

The only disadvantage I can think of is the possible privacy implications of giving Google all your information.  So far, I think they have been good stewards of the information that they collect about all of us every we use any of their products, but it can be scary to think about how much they really do know about people.

Email filtering should be done on the server.  Spam should never make it to the client.  But, for the people whose servers doesn't provide proper protection, this information is invaluable.  

Thanks for the detailed test results.
 
WWW  
IP Logged
 
Spanky
Radmeister
**
Offline


Rad's non-Admin Test profile
in Seamonkey

Posts: 73
Same as Rad


Back to top
Re: Comparing POP3 spam filtering software
Reply #2 - Mar 29th, 2009 at 11:38am
 
i have used, in the past, 3 of your 4 recommendeded filters, except for k9, which i never heard of.

i used to get lots of spam, tho no more. i think the rad vps has a spam filter.

i have many times sent mail to friends, only to be rejected as spam. tried to tweak the message, but once you're labeled a spammer, it seems, yer a goner, and no amount of msg tweaking can help.

some never return your msg.
'what do you mean you never received my email?"

later: "oh, i found it it my spam folder."
 
WWW  
IP Logged
 
Dan Goodell
Special Guest
*****
Offline



Posts: 552
N California


Back to top
Re: Comparing POP3 spam filtering software
Reply #3 - Mar 29th, 2009 at 6:52pm
 
MrMagoo wrote: "Answer: Use Gmail.  [...] Ok, so that doesn't answer the question people asked you.  But it does answer the question they meant to ask.  What they want is an easy way to access all their email accounts with no spam."

Well, I don't think so.  What they want is to keep doing things the same way they always have.  They have a single email address, through their local cable or dsl provider, who set them up with OE long ago, and they don't want to change any of that.  It's what they're comfortable with.  I've suggested gmail, but that's not what they want to hear.  They're longtime users of Norton Internet Security who have gradually gotten fed up with it bogging down their computers, so are removing NIS and just want a substitute for the AV (that's easy) and spam filter.  Nothing more.

Heck, I've got a lot of customers who refuse to get rid of AOL!  They insist on defiling their computers with all that garbage AOL software, even though they can get to their AOL email more easily through an ordinary web browser.

 
 
IP Logged
 
MrMagoo
Übermensch
*****
Offline


Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)


Back to top
Re: Comparing POP3 spam filtering software
Reply #4 - Mar 29th, 2009 at 7:49pm
 
Dan Goodell wrote on Mar 29th, 2009 at 6:52pm:
What they want is to keep doing things the same way they always have.

MrMagoo wrote on Mar 29th, 2009 at 8:36am:
I know, a lot of people aren't ready for this type of advice.They are just now learning that you don't really get emails from the king of Nigeria offering you a million dollars, and here I am talking about moving your email into 'the cloud.'

I know, lots of people don't like change.  They finally feel like they know how to use their computer and they don't want to mess that up.  And sometimes you have to answer the question people ask - not the question that they should be asking - in order to keep them happy.  On the other hand, switching to Gmail isn't any harder than selecting, configuring, and maintaining a spam filter.

My point is that for people who are a little more flexible, like most readers of this forum, it's time to consider that email on the desktop my soon start becoming a thing of the past.  Businesses haven't run their email that way for years - its stored on a server and accessed through Exchange.  Spam filtering and data backups are done at the server level, and individual users don't have to worry about it.  They can also get their email from any computer (almost - you have to have Outlook installed and usually a VPN connection.)  

Gmail is finally providing a similar experience for individual users.  Granted, webmail has been available for years, but none have had the features, polish, and raw storage capacity of Gmail.

Dan Goodell wrote on Mar 29th, 2009 at 6:52pm:
They have a single email address, through their local cable or dsl provider, who set them up with OE long ago, and they don't want to change any of that... Heck, I've got a lot of customers who refuse to get rid of AOL!

Yes, I know the type, and stressing them out by trying to tell them how to use their computer usually turns out to be fruitless and only upsets them, so I completely understand your position.  

Gmail allows you to send and receive mail from any address that is POP3 capable, so you should be able to integrate most ISP email accounts into Gmail.  I'd take good odds that most of these stubborn users would be very happy with the change after the first week, but I know that talking them into the change is sometimes no small task

 
WWW  
IP Logged
 
Page Index Toggle Pages: 1
Send Topic Print