Linogate Internet Technologies
   

The two SPAM filter modes of DEFENDO

DEFENDO's SPAM filter can be called at two different stages while processing an inbound email: Just after it was received (relay SPAM filter) or while delivering it to a local user's mailbox (user specific SPAM filter). Please check the comparison below for the differences between both stages.
Attention!
Before 4.2-3.4 it was possible to enable both SPAM filter stages at a time. This may lead to overload conditions and even system failure due to the high resource consumption of a duplicate analysis.
Relay SPAM filter
  • Applies when delivering to a local mailbox and when forwarding to an internal mail server
  • Single policy which applies to all users
  • Detected SPAM can be redirected to a specific recipient address
  • Configurable by administrators only Expert -> Mail Server")
  • All results and actions will be logged
User specific SPAM filter
  • Applies only to emails delivered to a local DEFENDO mailbox
  • Can be enabled and configured individually per user
  • Each user must deal with detected SPAM himself
  • Can be configured by administrators ("Administration -> Users") and mostly also by authorized users ("Home -> Email options"). The options of the tabs "SPAM scores" and "SPAM modules" in menu "Expert -> Mail Server" apply to all users
  • No logging

Optimizing the SPAM filter

Finding out the best SPAM filter configuration is an individual matter. Please consider the following advices while tuning the DEFENDO SPAM filter to yield the best results.
Use the realtime lists
Many spammers adapt their emails to the static rulesets of common SPAM filters quickly. It is highly recommended to enable the realtime inquiries offered by DEFENDO's SPAM filter, i.e. the RBL and URIBL servers as well as the Razor2 system.
Selecting reasonable thresholds
Using the unmodified SPAM filter ruleset, the majority of SPAM mails will be scored about 3 to 10 points. In real-life some amateurish SPAM will yield up to about 30 points. In contrast clever SPAM might pass undetected with 0 points.
When enabling the SPAM filter you should start with a rather low threshold for tagging emails. A value between 2 and 3 is a good choice. There is also on option to silently discard SPAM mails. Note that there may be legal issues with this feature. Please make sure that all relevant laws and regulations are satisfied. Even if allowed, you shouldn't enable this option until you're done tuning the SPAM filter and the users have confidence in it. Otherwise the SPAM filter might become the scapegoat for supposed lost emails. By then you should be experienced enough to select a reasonable threshold for discarding emails anyway. However we discourage the use of thresholds below 5 points.
If the SPAM filter classifies some emails as SPAM which infact are not, you should consider the analysis details provided by the tagged email. In particular if you encounter repeated false positives from the same sender you might want to ask him to avoid the peculiarity (e.g. unnecessary flashy formatting). Usually the sender will have problems with the SPAM filters of other recipients, too and will be grateful to finally know the reason. In some cases you might even point to a misconfiguration or an other problem (e.g. if the sender's mail server is listed as open relay). Simply adding the sender to the SPAM filter's whitelist is of course the most obvious solution. If however the emails of various senders are misclassified as SPAM an unfitted userdefined SPAM filter rule could be the reason. Please check the analysis details. Otherwise you might have to increase the threshold.
Now what about undetected SPAM? An abbreviated version of the SPAM filter's analysis results is provided in the headers of each analyzed email. In Outlook-Express you can view the mail headers when selecting the messages "Properties" on tab "Details". The header "X-Spam-Status" contains among others the score and the detected attributes (tests). If many SPAM mails narrowly miss the threshold for tagging an email as spammed you should lower the threshold slightly. If you encounter many SPAM mails with no or very low score you should enable the realtime services offered by DEFENDO if you didn't already do that (see above). Make sure a recent DEFENDO version is installed ("Administration -> Update") as the static SPAM filter ruleset is updated along with the regular DEFENDO updates. If it turns out that the analysis of many undetected SPAMs indicate the same attribute but this attribute isn't scored enough to filter the mail, you might want to define a user defined SPAM filter rule which assigns a higher score to this test.
User defined SPAM filter rules
Use this feature with care while fine tuning DEFENDO's SPAM filter. In our experience an intense use of this feature is not necessary if you obey the pieces of advice given above. Quite the contrary is often the case: Unsuitable patterns and extreme scores result in more false positives and then again to even more SPAM filter rules. If more than, say, 20 rules have been defined and the amount of rules is still growing it's ususally time to revise the rules - or better delete all of them.
When selecting the score for a self defined rule you should always keep the thresholds in mind. Only if you are absolutely sure that a rule will by no means match a non-SPAM mail you may choose a score which exceeds the discard threshold. The score for a rule which is just very likely to match SPAM mails only should be rather moderate. Select a value just below the threshold for tagging an email as on real SPAM mails probably an additional characteristic will be detected and so the total score exceeds the threshold.
Please keep in mind that a pattern will also partially match a word. For example many SPAM mails promote the drug "Cialis", however the pattern "cialis" will also match "specialist" which is probably not what you want to do. Whenever possible you should use patterns which match on whole words only (e.g. "_cialis_").

Management of detected SPAM

Currently DEFENDO offers two ways to deal with SPAM: Tag or discard it. Particularely while optimizing the SPAM filter configuration but also later in normal operation it makes sense to postprocess SPAM mail in the client. It will make it easier to find emails which have been categorized as SPAM by mistake. Almost any mail program is able to distribute emails to different folders automatically. Of course this applies to DEFENDO's webmail client, too. The SPAM filter adds the header "X-Spam-Level" to each analyzed email. For each point of score the header contains an "x" character. So an email with a SPAM score of 3.4 will containt the header "X-Spam-Level: xxx". Use the filter rules of your mail client to e.g. deliver email with 10 or more "x" characters in a folder which is e.g. deleted unsighted each day. A lower priority rule will sort out email with a minimum of e.g. 4 times "x". The corresponding folder can be deleted weekly. Finally a third rule will take the mails with 2 or 3 "x". This folder is checked from time to time if it contains any false positives.
If it doesn't violate privacy regulations you can even manage the SPAM mails detected by DEFENDO's relay SPAM filter centrally. Instead of delivering a SPAM mail to its original recipient it can be redirected to a specific address. Manage the emails which arrive at this address as described above.