Home Effective Spam Filtering Techniques With Eudora Email

 


OVERVIEW

STRATEGY

REGEXP  INFO

THE FILTERS
 1 Virus Attached?
 2 Duplicate Fm-To
 3 Whitelist (Passlist)  
 4 Friendly Domains
 5 Newsletters
 6 List Subscriptions
 7  Keywords
 8 Personality
 9 Bogus Address
10 Username in
   Subject 

11 Click Here
12 !!!!!!!!!!!!
13 Remote Images  
    or Database Links
  
14 Bcc From
    Unknown
 
15 Bad Word List #1
16 Bad Word List #2
17 Tracking Codes
    in Subject

18 Bad Word List #3
19 Bad Word List #4
20 Bad Word List #5
21 Too Many HTTP's
22 Adult Links

23 Bogus Hotmail,
    AOL and Yahoo


MOST EFFECTIVE
    SEARCH TERMS

LINKS

FILTER VERBS

Other Interesting
Eudora Filters:

Numerical User
   Name

HTML Contents
Asian Characters
Blank Subject
Secret Keyword
   With Auto-Reply

 

OVERVIEW

Tired of getting so much spam in your email?  Get only the email you want, and trash the rest, using Eudora Email's powerful  built-in email filtering capabilities!


It is possible, using a few strategic filters in Eudora email, to allow all the email you wish to keep into your email inbox, and automatically filter out most of the spam to a special quarantine mailbox or even the trash, where you can review it at your convenience and then flush it away. And with a little more effort and using a few additional filters as shown here, it is possible to eliminate up to 99% or more of the spam you receive. These pages shows how I use Eudora Versions 5.1 and later for Windows to filter spam from my email, with less than 1 in 100 spam emails making it through to my Inbox.

FILTERING GOALS: My goal is to filter out as close to 100% of spam as possible, while not filtering out any "friendly" email. "Friendly" email includes email from people I don't know, so these filters will try to minimize as much as possible the chance of false positives for email from senders that are not whitelisted in the "passlist". In the main filter set, aggressively identifying all spam takes precedence over protecting email from unknown senders. The most recent update of this filter set (download available here)  is currently identifying 99.7% of my 2200 most recent test spams. Without whitelisting it also matches 12.5% of 2000 of my most recent "friendly" incoming emails. In actual use with a whitelist filter however I get no false positives, after the occasional new emailer is added to the passlist.

A second filter set has been developed that places the protection of "friendly email" ahead of identifying spam (at the expense of allowing a higher number of spam to reach the inbox). This approach should be more useful to people who get a lot of legitimate email from unknown senders. The current version of this filter set (download available here) is filtering 97% spam, 2.0% false positives on my test email collection without any whitelist filters.

*NOTE from Cecil - Thanks to everyone who has written to me with words of support for these pages recently. While I can't answer every email personally, it is good to know that some of you are finding this tutorial helpful. I have not made any major revisions to these pages or the filter sets since early March (03) as I have turned my attentions to other subjects. With regard to SPAM volume - In the time from January till May this year the number of spams I receive daily has risen from 57 to over 180!! The problem of spam is growing exponentially. At this rate, I predict the problem of spam will reach "critical mass" sometime later this year, and the proverbial "something" will be done about it in the legal and/or technical fields.
 - Cecil, May 11 2003

As of August 2003, I am no longer regularly checking my long-time email address of "cecilw@pullman.com" since the spam it receives has grown to close to 300 pieces per day. I will occasionally download it and add the spam to my collection until the account expires later this year. To contact me by email, please write to me at cecilw at cecilw dot com. Thanks!

BACKGROUND INFO:

I started this website in late Dec. 2002 after realizing that the spam in my email inbox outnumbered legitimate email by a wide and growing margin. This web site is an ongoing work in progress, changing and evolving as I find better ways to filter out spam.

I have three main email accounts at home and at work, and I check for mail on each of them from both home and at multiple locations at work throughout the day using Eudora 5.1 or 5.2 with three "personalities". I don't get a lot of valid email from people I don't know, and I received about 1700 pieces of unsolicited spam email in January 2003,  for an average of about 57 pieces of spam a day. (*August 2003 - this is up to well over 200 spams a day!). The filters I use and show here are successfully catching and diverting close to 100% of incoming unsolicited and unwanted spam email to my trash mailbox on most days, with almost no  "false positives".  I recommend scanning  the trash mailbox quickly for any email that doesn't belong there, and then delete it.  Your mileage may vary, depending on your particular email style and needs and which spam lists your email address has been "opted into" against your will.

I have a file  of over 4000 (and growing) spam email messages that I use for testing filters on. I'm currently (Aug 03) using about 32 combined spam filters, one virus filter and several whitelist filters in Eudora 5.2 for Windows, and this catches all but a small number of these messages. An emerging trend in new spam email is to make it very short, and too vague or innocuous to bother writing filters for - perhaps a tribute to the effectiveness of spam filtering in general. The number of filters I use goes up or down as I continue to experiment with new filtering rules and refine my word and phrase lists.

The number of spam filters I now use is down from a recent high of over 175 filters  (and I've heard of instances of people using, or trying to use, more than a thousand spam filters). It was clear the problem of spam was growing increasingly out of control, so I decided to start investigating how to filter out spam and unwanted email more effectively and efficiently. It is possible to use just a few spam filters  and remove "most" spam email. And for many people that's good enough. But by adding a few more filters as shown here it is possible to get up to 99% or better spam detection and yet minimize the number of false-positives.

NEXT PAGE - STRATEGY

By  Cecil Williams

 

 

Hit Counter