The Email Problem and Solutions

Today it’s impossible to think business and personal communications without email. Sending and receiving emails costs you and me nothing. It’s free! The zero cost (for users), the efficiency of delivery, and ease of use has made it so popular. But now email has become a victim of it’s own success. Just my quick test with one email server for 4 days showed that 96% of the emails received were abusive.

The email protocol (SMTP) was designed at a time when very few people were using emails and everyone basically knew each other. So, security was not a concern, but today the world has changed and that trust isn’t there anymore, but the SMTP protocol we’re using remains the same.

So, how is today’s technology dealing with this problem?

If it comes to your Inbox it’s your problem

This is the approach taken by most of the ISPs and small companies who cannot afford the cost of extra security or simply think that it’s end users problem. So, you and I receive emails with viruses, spams and phishing traps. We’re on our own to decide what to do with them. This approach has proven to be the most ineffective (since most users aren’t aware of the security issues). It has damaged corporate images, caused loss in productivity, money and business. If the users need to fight email problems at this level, then getting a good email client or third-party softwares is the best option. Some of the end user solutions available in the market are:

Apple Mail – Mac users have one of the best email client, with pretty good built-in spam filter. It can learn for over a period of time and can be quite effective in identifying spams. It has additional options to identify spams, for example, if your ISP uses SpamAssassin, Brightmail or another spam-analysis tool, Mail leverages that analysis.

Thunderbird – It’s a cross platform email client form Mozilla, with a built-in junk mail filter. It also has a learning capability, as you keep marking messages as spam, over time thunderbird’s filtering improves. Thunderbird also has anti-phishing protection that will tell you if it thinks the message might be a scam to steal your passwords, personal information, credit cards, etc…

There are some desktop anti-spam softwares that can be installed as plugin to the email client:
Trend Micro Anti-Spam Pilot – Free plugin to Outlook provided by Trend Micro.
Comodo Anti-Spam Desktop 2005 – Free anti-spam for Windows PC, supports popular email clients.

If you’re using a Windows PC then I think it’s obvious that you need an anti-virus software too. Some popular anti-viruses are Norton, McAfee and Trend-Micro. If you’re looking for a free one then get AVG Anti-Virus Free.

Stop it before it reaches to users Inbox

This is also called a gateway solution, where the incoming/outgoing emails are routed via email filters before delivering to the users Inbox. This has proven to be the most effective solution available today, but it’s by no means a 100% solution. Some new viruses, spams and phising emails do manage to bypass the filter. Here are different types of gateway solutions available:

Spamassassin – Very popular free mail scanner software, which works with most of the widely deployed email servers like Sendmail, Postfix, Qmail and many more. It uses a wide variety of local and network tests to identify spam signatures.

Ironport, Barracuda, Sophos, Mcafee, Symantec and many others… – They all provide a single box gateway solution to fight virus and spam. These email security appliances are relatively easy to implement in the existing email environment. They run special softwares and signatures are updated constantly. All you need to do is perform a simple configuration and point the MX record to these appliances.

Messagelabs and Postini – These are hosted gateway solutions to solve the email problem. The selling point of such service has been “no initial hardware and software investment” from the customers. It works similar to the Security appliance scenario, you have to point your MX record to these providers SMTP servers, where the emails will be filtered and only good emails allowed to reach your mail server.

Most of the popular free webmail providers such as Gmail, Yahoo and Hotmail use gateway solutions that is tightly integrated with the user’s web interface.

Let’s patch the email system

This is indeed a very smart solution, which requires some minor addition/modifications in DNS and SMTP server, but the problem is the scale in which emails are deployed today. It’s proving almost impossible to ask all the email server and domain name owners to make such a small change. Here are some popular extensions proposed to fix the SMTP protocol.

Sender Policy Framework (SPF) – SPF is an extension to the SMTP protocol, which allows to identify and reject emails from forged addresses. This is how it works: ” 1) the domain owner publishes this information in an SPF record in the domain’s DNS zone, and when someone else’s mail server receives a message claiming to come from that domain, then (2) the receiving server can check whether the message complies with the domain’s stated policy. If, e.g., the message comes from an unknown server, it can be considered a fake.”

Sender ID – This is Micorsoft’s protocol, and it was derived from SPF. “The Sender ID Framework is an e-mail authentication technology protocol that helps address the problem of spoofing and phishing by verifying the domain name from which e-mail is sent. Sender ID validates the origin of e-mail by verifying the IP address of the sender against the purported owner of the sending domain.”

DKIM (DomainKeys Identified Mail) – DKIM is a method for email authentication using signatures, this is an enhanced protocol based on Yahoo’s DomainKeys. The sender’s MTA signs and receiver’s MTA verifies. “DKIM uses DNS-based self-certified keys. Because the scope of DKIM is limited, it does not need generalized, powerful and long-term certificates, issued by separate authorities.”

CSA (Client SMTP Authorization) and CSV (Certified Server Validation) are some other solutions proposed to solve the email problem.

Forget about the existing email system and let’s design a new one

This is the approach taken by some experts, who think that email at it’s current state is broken and there’s no point trying to patch it. Even if you come up with a good patch it’s difficult to implement anyway. So, why not design a new email system from scratch, with the security in mind. This sounds very radical but some people did propose such ideas and demonstrated a working system:

Internet Mail 2000 – This is a project launched by D. J. Bernstein, the author of popular Qmail MTA. “IM2000 is a project to design a new Internet mail infrastructure around the following concept: Mail storage is the sender’s responsibility” . The sender’s ISP, rather than the receiver’s ISP, is the always-online post office from which the receiver picks up the message. Meng Wong’s RSS Email is the implementation of IM2000. Here’s the presentation at Google in July 2006 Turning Email Upside Down: RSS/Email and IM2000 (Google Video)

Can e-mail be saved? – An old article from 2004, but is interesting to see how six of the industry’s most provocative thinkers envision the future email.

Some popular techniques used in most of the solutions

blacklists and whitelists – a list of email addresses, domains and ip addresses to either exclusively allow or block emails. There’re some public blacklists that are used by SMTP servers and gateways to block potential spammers. Spamhaus and Sorbs are popular public blacklist providers.

Bayesian spam filtering and trainable systems – based on Bayes’s theorem, “probability that an email is spam, given that it has certain words in it, is equal to the probability of finding those certain words in spam email, times the probability that any email is spam, divided by the probability of finding those words in any email”. These type of systems can be trained on a per-user basis.

Heuristic filtering – Heuristic filtering uses various tests for spam and assigns a numerical score to each test. Each message is scanned for these patterns, and the applicable scores tallied up. If the total is above some fixed value, the message is identified as spam.

Reverse DNS lookup of the connecting IP, Content filtering, Signature-based filtering (such as Vipul’s Razor), Greylisting (temporarily reject messages from unknown sender mail servers) and enforcing RFC Standards are also used to identify spam email.

Conclusion

Latest anti-virus and anti-spam techniques are proving to be quite effective, but it’s not implemented in all the email servers. For the spammers only 1-in-100,000 success rate is enough to pay for their efforts. As these anti-spam techniques become more effective, criminals are finding new ways to attack, and they always seem to be one step ahead of the security professionals. The recent problem with the image spam boom and Messagelabs Intelligence Report shows that. I personally think that we need a new email protocol like IM2000 or RSS/Email but also understand, it’s very unlikely to happen anytime soon. If 96% of spam still cannot kill the current email protocol, I wonder what needs to happen to replace it…

This article originally appeared in SecurityTNT.com