Exchange 2010: Niggling Fears About Storage Requirements

Last Updated (Tuesday, 30 November 1999 10:00) Wednesday, 09 December 2009 09:40

I was just reading this Ferris piece on Exchange mailbox sizes and thought it would be useful to document Cleartext's strategy for our hosted email clients storage. The Ferris article starts;

'Exchange 2010's database strategy is very interesting. The new Database Availability Groups and the benefits they offer for data protection and quick recovery are striking.

Overall, Microsoft is optimistic that mailboxes will be able to grow to 10GB or more. However, it's unclear how large mailboxes will perform in practice. Several concerns spring to mind:'  read more...

We (Cleartext) offer both mailbox hosting (via an Axigen ISP platform) and SaaS email archiving.

Our strategy is to keep hosted mailboxes as low as 200-500Mb and bundle these mailboxes with the SaaS archiving service for permanent email storage. This has several advantages;

1) Users mailboxes are easier to manage and migrate if needed and are less prone to data loss caused by IT systems issues or more often 'user error'.

2) The SaaS archive has full text and attachment indexing meaning it's quicker and easy to find email when needed.

3) The client can implement their retention policy on the SaaS platform to ensure compliance with eDiscovery regulations. Avoiding issues where staff delete email from their inbox.

This is a change to the usual large mailbox offerings from organisation slike Google but we believe is more appropriate for business deployments.

Keeping your MX records tidy

We often notice that despite our advice clients insist on leaving a 'backup' MX record in their DNS, this means that they a) don't understand how spammers operate b) don't understand that we have primary, secondary and tertiary routes for their email.

So I thought it timely to explain how MX routing works and why it's not a good idea to leave an  'extra' MX record in place that DOESN'T point to us. Lets assume your companies domain name is 'your-company.com' and you have such a backup record in place, lets say it's value is 100 and it's named postoffice.your-isp.net.

Mail servers route inbound email for a domain to the MX record with the lowest value, so looking at your MX records;

your-company.com.    3600 IN    MX 10 mx811.clearemail.net.
your-company.com.    3600 IN    MX 30 mx813.clearemail.net.
your-company.com.    3600 IN    MX 100 postoffice.your-isp.net.
your-company.com.    3600 IN    MX 20 mx812.clearemail.net.

Any mail server sending mail to anyone at 'your-company.com' will try to deliver to us at  the MX 10 value above (mx811.clearemail.net), and if that fails then 20, then 30. If all fail then the sending mail server will try to send to MX 100. postoffice.your-isp.net.

Often clients initially setup a backup mail route like the MX 100 you have above because there's a worry that the main routes will all be unavailable, which is very, very remote given these (MX 10, 20, 30) all point to different parts of our infrastructure.

The reason we advise against this practice is that spammers have realised that some organisations do this so they send their spam to the <span style="text-decoration: underline;"><strong>highest</strong></span> route first, that would be to MX 100. This routes the email to your-isp.net and that system will then deliver email to your mail server. This bypasses Cleartext (or any other managed email security platform) thereby causing several things to happen;

1) Our multi-layered spam and virus filtering will not be applied.
2) Inbound email will not be archived and therefore unavailable for e-discovery
3) Any custom email rules, perhaps for HR reasons will not be applied
4) This inbound email will not be recorded anywhere in our logs because it's bypassed us.

Looking at the above, 1) isn't too much of an issue because your ISP may be applying rudimentary filtering therefore catching some of the spam, but they may let through phishing emails, trojans etc, 2) could be an issue because this email won't be archived which means you may not be complying with e-discovery legislation and 3,4) could also be an issue if you need to trace email that someone says they sent to you, or HR needs to for some reason.

Now it's arguable that 2-4 above won't be too much of an issue because legitimate mail servers will send to 10, 20 or 30 first, but even so there's still a chance genuine mail will route this way and do you want that if you end up in court with the other party doing email discovery on your organisation?

So, to summarise, if you use a managed email security service and have such a 'backup' MX record in place you currently have a 'backdoor' into your email system which could let spam or malware in and that routes email without your corporate policy being applied.

So make sure you don't get caught out by having email routed around the very platform that's supposed to be providing your email security and compliance requirements.

Where Is the Real-Time Web Message Bus?

Looking at my Google Alerts this morning I saw this one;

So after quick read at of this article on ReadWriteWeb I posted the following as a comment.

I'll declare a vested interest in XMPP first, our service Cleartext ESM (Enterprise Social Messaging) uses XMPP.

Now I'll also say that it's worth noting that XMPP does enterprise and internet scale and is good at bridging these through an ecosystem of components, I think this is a key benefit.

There's a third issue with Twitter in addition to business model and latency at large scale (which I don't believe you can solve with http polling) and that is shifting technology road map. For anyone to build complicated enterprise and internet level apps on Twitter this needs to be solved and as the platform is evolving rapidly that's a big ask.

For example parts of our platform rely on the fact that re-tweets and hash tags are in the message. Twitters recent news that RT's will become part of the API is good news for short messages but bad news for solutions proxying Twitter if only because we all have to do more R&D, which equates to delays to market and increased costs.

I'm firmly believe that email succeeded because its an open, standards based, federated platform. XMPP delivers the same for IM and mirco-blogging. I think this is the real time bus you're looking for.

Average email sizes 2003-2009

Saturday, 12 September 2009 12:33

I recall some research MessageLabs did around 2003 that said email sizes averaged less than 20k, in fact maybe the number was 16k.

Anyway about a year ago I took a look back over 2008’s data and we were seeing an average size of about 30k, the same view this year and I see that the average size of an email is now around 70k for outbound email and 200k for inbound email.

So that looks like an 85k average email size to me which is fair jump from 2003’s 16k. Note that these numbers are after the spam filters so we don't get the numbers messed up by spam.

I’d be interested to hear from any other organisations on the sizes they are seeing.