Note: This blog was written together with Dave Stork after reading a Zimbra and Exchange product comparison. You can find the article on Dave’s blog here, including a personal note by Dave.

In a blog post by Christopher Wells, alias vSamurai, the author positions VMware Zimbra Collaboration Server (ZCS 7.x) as an enterprise-ready drop-in replacement for Microsoft Exchange Server 2010 environments of all sizes. He also suggests Zimbra is a better multi-tenant solution for ISPs. The author does this by comparing both products in a feature comparison.

These reviews are helpful in order for companies to make an informed decision. After all, there’s nothing wrong with a bit of competition. However, Dave Stork and I wanted to create a response, because some statements are flawed or just plain wrong. In the process, we will be following the structure of the referenced blog:

Backup and Restore
The author starts off by claiming that “the ease with which backup and restore can be performed in Zimbra outweighs the capabilities of Exchange”. While it’s interesting to note the author implicitly admits Exchange is more capable, he misses the point. The product should follow a well-designed backup and recovery strategy, based on customer demands and compliance regulations. Where Exchange has server, database, mailbox and single item recovery options, Zimbra is built on top of MySQL, meaning recovery requires brick level restore or (partially) restoring information from MySQL dumps. Also, in Zimbra the databases only contains meta information; the actual messages and attachments are stored on the file system. While this makes sense for Zimbra, as many SQL people consider storing binary data in databases a bad practice, it increases the complexity of backup and restore, because meta information and file system needs to be in sync. Note that Exchange’s Extensible Storage Engine (ESE) is purpose-built for storing mailbox information, including attachments.

Scalability
Then, the author claims that Zimbra has better scaling capabilities than Exchange. First, let’s start by looking at the definition of scaling. A system is said to scale well if:

• it can handle increased load without (serious) performance penalties, or
• the system is able to accommodate growth by adding resources (scale up) or additional systems (scale out).

Ideally, scaling up should show a linear pattern, meaning two systems equal can handle twice the load. Scaling out most of the time doesn’t, which makes sense when looking at how computers are designed using shared resources like buses for example.

Now, scaling isn’t solely a matter of hardware; a system also requires software built to scale. The role-based model of Exchange, with its specific roles for serving mailboxes and handling replication, routing e-mail and servicing clients, is a good example of a thought-out scalability supporting concept. Of course, you can install all roles on a single server, which is currently the recommended practice by Microsoft, but you’re still able to design fit-for-purpose farms and clusters.

Thus, the ability to scale is determined by the whole set of components playing well together, hardware and software. With this in mind we’d like to include an interesting table which is part of the VMware (acquired Zimbra early 2010) study “Zimbra Collaboration, Server Performance on VMware vSphere 5.0”:

In their analysis, VMware primarily focuses on the CPU utilization figure. That figure implies that Zimbra has more headroom than Exchange using the same configuration. However, Exchange also has several background processes which perform tasks in the background, like optimizing the database to reduce the number of IOPS. Yes this takes up a certain % of CPU cycles, but optimizing storage for sequential access could explain the significant 240% decrease in IOPS for Exchange. Lower IOPS reduces storage requirements – and costs – for Exchange. The over 60% lower latency figure for Exchange is also an indication overall processing of messages is faster in Exchange.

Costs
As often in these Open Source Software (OSS) discussions, the cost card is played. The author claims that on average, Zimbra is 50% cheaper than Exchange. However, this claim is made without any supporting references or figures, making it difficult to verify this statement. However, from our experiences, those claims are often primarily based on retail prices and licensing costs. What is often overlooked (or ignored) in comparisons with OSS, are training costs or hidden costs like support or maintenance.

Functionality is also a potential cost saver, as companies can work more efficiently due to added or enhanced functionality. These savings depend on customer needs, although some are widely used and immediately contribute to lower costs, like for example AutoDiscover (automatic configuration of Outlook 2007 and later clients or ActiveSync devices).

Exchange natively supports Outlook, common browsers and mobile devices; Zimbra requires an Outlook plug-In, Zimbra Connector for Microsoft Outlook, increasing support and maintenance costs. Note that this connector is only available for Zimbra Collaboration Server Network Edition Professional users.

Regarding maintenance, Exchange requires Exchange, Active Directory and (optionally, but a big bonus) PowerShell skills. Zimbra consists of a set of 3^rd party products, requiring knowledge of each product, like Postfix, mbox e-mail storage, MySQL, Apache. OpenLDAP, SpamAssassin, ClamAV and shell scripting. Of course, more components mean more products to configure and maintain, increasing maintenance costs.

Storage Benefits
A full paragraph is dedicated to the benefits of using Zimbra with NetApp storage. However, the NetApp products and technologies mentioned are not Zimbra specific, and therefor in our opinion do not add anything to the discussion.

Feature Comparison
The author then continues with a “direct” feature comparison between Zimbra and Exchange. Let’s have a look:

1. Platform Architecture
First, author claims ESE is over 20 years old, the .EDB file is non-modular and the ESE engine is non-tunable. Yes, ESE exists for over 20 years, but that’s also 20 years of experience in building a fit-for-purpose database engine. With each new Exchange version, ESE was redesigned to meet evolving requirements and expectations in a changing world. When looking at the VMware IOPS comparison in the Scalability section, it’s Zimbra that should worry about storage.

Second, author claims Database Availability Groups (DAGs), based on Fail-over Clustering, isn’t a proven technology for large deployments. Exchange 2010 is on the market since October 2009. Like many Exchange fellows, we have designed or seen large Exchange deployments (i.e. thousands of mailboxes). Also, if millions of Office 365 users aren’t proof of a successful large scale multi-tenant ISP-like deployment based using multiple data center DAGs, what is?

To be honest, is it really that important which exact technology is used and how old it is? In the end functionality and performance are more important, as they are relevant in any business case for Exchange. What would a decision maker most likely ask, “Does it use Microsoft SQL Server?” or “What can we do with it and how much will it cost?”. We think and know out of experience it will probably be the latter.

2. Reliability & Robustness
The author claims Microsoft is considering (moving Exchange storage to) SQL and needs to prove robustness of the new architecture. While Microsoft has considered the SQL storage engine several times, it decided to stick with the optimized ESE engine. This was also true for Exchange 2010 back in 2009, like you can read in this blog. Main reason for deciding to stick with ESE is performance.

When pleading for ZCS, the author states “Linux has better uptime”. While this may have been true in the Windows 98 era, from experience, managed Exchange systems can reach similar uptime figures. On the contrary, I’ve seen Linux systems crashing every few days. The only conclusion you can draw here is that reliability not only depends on hardware and software components and their quality, it also depends a lot on if and how systems are managed. Also, don’t confuse uptime with availability, as planned downtime will reset my uptime statistic, but that’s all it is: a statistic.

3. Tiered Storage (was Platform Scalability)
Tiered storage, or Hierarchical Storage Management, is about classifying data in terms of things like security, performance or pricing. Exchange itself partly supports this concept, using elements like DAGs, databases, mailboxes, personal archives and retention policies. For example, you can home your mailbox on multiple lean and mean servers using fast SAS storage while personal archives, used to automatically store e-mail older than 1 year using retention policies, are served by a fat server using inexpensive SATA disks on JBOD storage.

ZCS utilizes a built-in HSM solution which automatically moves items from the (fast) primary volume to the (cheaper) secondary volume. The database holds information on the actual location where the item resides. Conceptually, this matches the Exchange concept of primary mailbox and personal archive using retention policies. However, retention policies are more powerful and – when permitted – give users control over what to archive and when. When Exchange customers want to use a deeper level of storage tiering, they can opt for 3^rd party solutions like Symantec Enterprise Vault (item-level stubbing) or storage solutions.

Note however, there are some important factors to take into consideration with stubbing:

• Data stored on a different tier, e.g. tape, isn’t always available online;
• Tiered storage adds complexity, introducing the need to compare reduced costs for storage against additional costs due to increased complexity;
• Stubbing may impact future migration or transition options, e.g. vendor support, or recovery options.

4. High Availability
Author claims DAGs do not provide Exchange infrastructure protection and have a learning curve. The first part of that claim is absolutely true: DAGs are designed to increase the availability of Exchange databases served by Exchange servers holding the Mailbox role, while providing a fail‑over mechanism. Covering for the other tasks are the other Exchange roles. Mail flow within an Exchange Environment is automatically redundant when you have multiple Hub Transport servers, as they monitor connectivity and possible routes for delivery. For client access, multiple Client Access servers can be made redundant using load balancing technology. Exchange has these built-in features that work independent of where Exchange is running, i.e. they also work in a non-virtualized system and no additional high priced product is required to make the underlying services highly available.

Regarding the learning curve claim, every new technology has a learning curve. DAG is built on top of fail-over clustering (nothing new) and easier to manage than its predecessors, CCR and SCR. Then again, we’d prefer Exchange admins who know what they’re doing, rather than somebody who learned an SRM trick.

Speaking of which, the whole argument that “ZCS with VMware’s Site Recovery Manager (SRM) is proven, scalable and effective” is apparently nothing more than a plug for VMware’s SRM product in conjunction with VMware licenses (vSphere required), as we see no credible arguments.

5. Platform Extensibility
The author states that Microsoft recommends using its proprietary shell. We assume he means PowerShell, which is here to stay. Other vendors, like Cisco or Quest, are adopting it and offer modules to manage their products using PowerShell. Heck, even Zimbra offers PowerShell scripts to manage Zimbra through encapsulated SOAP requests. For the record, we both don’t know of any Exchange admin complaining about some Linux product requiring bash (Bourne-Again shell) or perl for scripting, turning this in a non-argument.

The author continues by apparently mixing a few things up. The argument given for ZCS is that “SOAP API allows server access using web services framework for client access and Zimlets for integration with 3rd-party services” while Exchange offers “limited SOAP access” and “Outlook add-ins require developer effort”. This is apples versus oranges; Outlook is a fat client and Zimlets are like web parts. If you want to make a nice dashboard, we’d suggest you use something like Sharepoint instead of bloating your e-mail web client.

Finally, SOAP and Exchange Web Services (EWS) are targeted at developers, PowerShell at automation. If you’re curious about the power of EWS, we’d suggest you check out the excellent blog by Glen Scales.

6. Platform Openness
While Exchange is mostly closed source, a lot has changed since the 90’s. Exchange has a developer center nowadays, where SDK and APIs are published on how to interact with certain parts of the Exchange ecosystem, e.g.:

• Exchange Server Web Services SDK
• Transport Agents SDK
• Backup and Restore SDK
• Exchange Server protocol documentation

7. Open Standard Protocols Support
It’s true that the current Outlook version doesn’t support all available standards for exchanging calendaring or contact information. However, for most companies that isn’t an issue. When required, solutions and workarounds are available.
Also see “Mobile Support”.

8. Rebranding
The author claims Outlook Web Access (OWA) has a single theme. That might have been the case with the RTM version, but since SP1 we have over 28 themes to choose from. If that’s not enough, there’s even an Exchange Server 2010 SP1 Outlook Web App Customization SDK to take customization into your own hands. Note that the SDK also documents integrating IM (e.g. Lync).

9. Web Client Support
Regarding Web Client support, the author states “limited browser support for OWA” (Outlook Web App). Since SP1, OWA has full support for IE7+, Firefox 3.01+ (Windows, MacOS, Linux), Chrome 3.0.195.27+ (Windows), Safari 3.1+ (MacOS). In addition, OWA Mini, targeted at simple mobile browsers, reincarnated in Exchange 2010 SP2.

Yes, there are browsers out there that don’t have the full featured Premium OWA (like Opera), but “limited browser support for OWA” is a bit over-simplified, especially if you take into consideration the combined market shares of the fully supported browsers (without Safari, between 81-91% since December 2011).

10. Mac Support
Outlook team and Mac Outlook are produced by two different teams, which might be one of the reasons for the feature disparity between Outlook 2010 and Outlook for Mac 2011. Apart from differences caused by the underlying operating system, we agree features should be as on par as possible for all available platforms.

Note that the mentioned Zimbra desktop client doesn’t support Exchange’s native MAPI protocol, adding the requirement to enable the IMAP or POP protocol on the Exchange server.

11. Linux
The author proceeds by arguing there’s no Outlook client or Exchange Server for Linux. That is a moot point; there’s also no Zimbra server for Windows. Also, when somebody’s trying to convince you using arguments like, “ZCS server components love the Linux platforms”, that’s not very convincing now, and is often seen with discussions when emotions prevail over rational thinking.

12. Mobile Support
More and more (mobile) clients are adopting the Exchange ActiveSync (EAS) protocol for exchanging e-mail, calendar, contact and task information with Exchange. In fact, even Blackberry announced they will adopt EAS in their upcoming Blackberry 10 OS product. This is probably driven by Microsoft releasing EAS protocol as part of their Open Specifications Promise, turning EAS more or less into the de‑facto standard for (corporate) e-mail synchronization for mobile clients.

Zimbra partially supports EAS for e-mail, calendar and contacts, but requires the Zimbra Mobile add-on. It is a bit unclear if tasks are synced, here it seems so for Pro users but here it is advised against while here the screenshots tell yet another story. Confusing.

13. Multi-tenancy
The author doesn’t show how Zimbra is a better multi-tenancy solution for ISPs when compared to Exchange 2010. But since Exchange 2010 Service Pack 2, there is no need for third party hosting software as it is now fully incorporated in Exchange without extra costs.
However; the intent was possibly to prove this implicitly via the costs argument of on-premises deployments. One other way is to look at actual hosted Zimbra and Exchange solutions available commercially.

Let’s compare costs from random Zimbra providers (picked from Zimbra’s Partners list), Exchange hosting providers and Office 365 subscriptions. It is not an extensive comparison, but it should give us an indication. Some (not all) are shown here:

*) discounts possible with more mailboxes
**) Note that prices are per month, but only apply with an annual subscription.

This table shows that the Exchange subscriptions are comparable or provide more functionality for lower costs. We do not see the 50% cost benefit argument at all and in our opinion shows that Exchange 2010 is a very viable multi-tenancy solution for ISPs.

One very important difference we want to point out is the available storage per mailbox. This tended to be a lot (several factors) more with Exchange than with Zimbra, without heavily impacting the price. This fact alone suggests that Exchange can be a very viable groupware solution to ISPs.

Final words
This concludes the authors’ feature comparison, but there are still some important elements missing, like product support, directory integration, IPv6 readiness, traffic management (e.g. ethical walls) or IRM. Also, what about integration or support of Unified Communications technologies, like single inbox – including voicemail – or voice access to mailbox?

Now don’t get the impression we want to condemn Christopher for trying to compare both products, even though by reading just the header and counting the numerous VMware-related logos on the site we were a bit hesitant regarding what the “conclusion” would be (we have a saying here, We from WC Eend recommend WC Eend).

We do appreciate good comparisons, because it can shake up our opinions of what is and what should be with Exchange and start interesting discussions. It‘s also an opportunity to learn about similar products. We believe competition is healthy and comparisons can be educational; It can help companies make a better fit for their needs and budget, or at least provide a starting point.

It is however crucial for a fair comparison that the facts, conclusions and opinions stated are correct and sound. Unfortunately, this is not the case with this article. There are numorous factual errors and most opinions stated are poorly argumented. To add to that, the author uses a feature list which can be found on the internet in several places, like here. This may be an indication authors are copying content, without knowledge or cross-checking facts.

Therefore, with the information provided in Christophers blogpost, one can’t conclude that Zimbra is an adequate replacement for all environments, Enterprise or SMB. Also, we do not see any indication that Zimbra is better suited for multi-tenancy by ISPs. If anything, we think we have shown that Exchange is a more than capable, competitive and well-though product.

You’re invited to comment or share your opinions in the comments below.

Update (April 10th): Apparently, on March 21st Wells posted a follow up on his Zimbra versus Exchange viewpoint. Looking at it, Wells seems to enjoy the attention.  Despite saying discussing viewpoints keeps vendors’ focus sharp, he doesn’t come up with arguments on why our post was – in Wells’ words – flawed. While I believe Zimbra serves a purpose – and it certainly isn’t on my radar as Wells says – I feel Zimbra or other non-Exchange evangelists should be able to take feedback like a pro. When you ignore other viewpoints or remain silent when asked for arguments, it’s more like a monologue rather than the interaction Wells claimed he’s in favour of.

Finally, our post didn’t go unnoticed, as Tony Redmond referred to it an article on Windows IT Pro. In the article, called Dispelling myths and other half truths, Redmond addresses some of Wells’ flawed claims as well.