Department SPAM Filters
Reviewed by Murray Anderegg 02/27/2013
Spam is unsolicited bulk email. Most email sent to Computer Science users now goes directly to Google rather than passing through the department’s SMTP server. Google has its own set of spam filters, which are pretty effective. This article deals with mail that goes through the department’s SMTP server.
During the spring semester 2013 the department expects to close up access to the SMTP server from the outside world. As part of this transition the department no longer expects to run spam filters internally.
Spam represents approximately 98% of what shows up at the @cs.unc.edu email gateway server. Computer Services filters out most of it, but some of it gets through, and we strongly discourage users from opening any content in spam messages. The @cs.unc.edu email gateway server has available several spam filters, whose descriptions are detailed below. Most of the department spam filters attempt to stifle delivery from substandard mail servers, or bots. The @cs.unc.edu mail gateway does not normally drop mail, but instead attempts to return an error message to the connecting machine for any undeliverable mail.
Levels of filtering
Spam filtering is available in several levels:
- Non-optional – Greeting pause, bad MX record, virus/dangerous attachment scan, spam scoring
- Minimal – Non-optional filters plus greylisting
- Medium – Minimal filters plus Fatal DNS Validity checks
- Maximum – (More false positives) Medium filters plus DNS validy check and blacklisting
Default filtering levels
The default filter for a given user is determined by their user category:
- faculty – minimal filtering; can request medium or maximum filtering
- non-faculty – maximum filtering; can request medium or minimal filtering
- alumni – maximum filtering
When an incoming mail connection is made, the mail server pauses before sending a banner to the connecting machine. The standard protocol on mail requires that the connecting machine wait until the banner is received before starting the delivery of mail. When the connecting machine fails to wait for the banner, a fatal delivery error is sent to the connecting machine.
Bad MX record
When a mail server locator record for the return address on a message resolves to an unroutable network address, e.g. a home network address such as 10.0.1.1 or 18.104.22.168, then there is no way for the mail server to return the message if it is undeliverable. Therefore, the mail server refuses the connection. In addition, if the mail server locator record resolves to the @cs.unc.edu mail gateway server and the message is not from an address accepted by the @cs.unc.edu, then the message is refused. Finally, if the mail locator record resolves to the special ‘localhost’ address, 127.0.0.1, which is a special short-circuit to the local machine, then the mail is refused.
Virus scan/dangerous attachment scan
The @cs.unc.edu mail gateway server finally scans any message that gets this far through the filters. If a virus is found, then the message is refused with an error status indicating the virus to the connecting machine. If the message contains a “Dangerous Attachment” that could easily be executed on the recipients machine, then the attachment is removed from the message and replaced with a notice indicating the removal.
This is a score of the “spam”-ness of the message. This runs the message through the SpamAssassin package and attaches a header with name, X-CS-Spam-Score, to the message. This header is only attached when the score is greater than a three. This header can then be used for filtering of the messages.
Greylisting refers to a process of returning a “request for redelivery” anytime a “new” mail delivery is attempted. In this case “new” refers to the first time that a combination of sender’s email address and connecting server is seen by the @cs.unc.edu mail server. The redelivery request is a “temporary” failure code that indicates to the connecting machine that the mail can not be accepted at this time, and that the connecting machine should attempt to redeliver the message the next time through its queue.
Fatal DNS validity check
The DNS Validity check attempts to ensure that the machine delivering mail is who it claims to be. The @cs.unc.edu mail server ensures that the address maps to a name, which maps back to the same address. If the DNS check fails, then the message is refused with a fatal error message indicating that the @cs.unc.edu mail server was unable to resolve the connecting machine’s name and IP address.
Temporary DNS validity codes
If there are inconsistencies in the DNS records for the connecting IP address, then the mail is refused with a “temporary” failure code indicating that the connecting machine should retry the delivery on its next pass through its queue. This allows for cases where there is a brief problem with DNS that may resolve itself in a few minutes.
When a machine has received a large number of refusals to accept mail, then the machine is blacklisted for three days. Some machines refuse to take “no” for an answer, and this allows our server to efficiently refuse connections for a short period of time.
NOTE: When the department migrates to using Google mail as the @cs.unc.edu email gateway server, then the department will use Google’s filters and will only filter mail that does not come through Google.