Unknown's avatar

About Michel de Rooij

Michel de Rooij, with over 25 years of mixed consulting and automation experience with Exchange and related technologies, is a consultant for Rapid Circle. He assists organizations in their journey to and using Microsoft 365, primarily focusing on Exchange and associated technologies and automating processes using PowerShell or Graph. Michel's authorship of several Exchange books and role in the Office 365 for IT Pros author team are a testament to his knowledge. Besides writing for Practical365.com, he maintains a blog on eightwone.com with supporting scripts on GitHub. Michel has been a Microsoft MVP since 2013.

Thoughts on “Five things that annoy me about Exchange 2010”


An article on SearchExchange by Greg Shields, MVP Remote Desktop Services, VMWare vExpert and a well-known writer and speaker, covered Greg’s annoyances with Exchange Server 2010. I doubt these points will be in the top 5 annoyances of the common Exchange 2010 administrator, apart from some arguments being flawed. Here’s why.

Role Bases Access Control Management
First, the article complains about the complexity of Exchange 2010’s Role Based Access Control system, or RBAC for short. It’s clear what RBACs purpose is, managing the security of your Exchange environment using elements like roles, groups, scopes and memberships (for more details, check out one of my earlier posts on RBAC here). RBAC was introduced with Exchange 2010 to provide organizations more granular control when compared to earlier versions of Exchange, where you had manage security using a (limited) set of groups and Active Directory. In smaller organizations the default setup may – with a little modification here and there – suffice.

For other, larger organizations this will enable them to fine-tune the security model to their business demands. And yes, this may get very complex.

Is this annoying? Is it bad that it can’t be managed from the GUI in all its facets? I think not for two reasons. First is that Exchange administrators should familiarize themselves with PowerShell anyway; it is here to stay and the scripting language of choice for recent Microsoft product releases. Second is that, in most cases, setting RBAC up will be a one-time exercise. Thinking it through and setting it up properly is just one of the aspects of configuring the Exchange 2010 environment. Also, I’ve seen pre-Exchange 2010 organizations with delegated permission models that also took a significant amount of time to fully comprehend, beating the authors “20 minute test” easily.

DAGs three server minimum
The article talks about the requirement to put the 3rd copy in a Database Availability Group (DAG) on a “witness server” in a separate site to get the best level of high availability. It looks like the author mixed things up as there’s no such thing as a “witness server” nor a requirement to host anything as such in a separate site. In DAG, there’s a witness share (File Share Witness or FSW for short) which is used to determine majority for DAG configurations with an even number of members. This share is hosted on a member server, preferably as part of the e-mail infrastructure, e.g. a Hub Transport server, located in the same site as the (largest part of) population resides.

Also, there’s no requirement for a 3rd DAG member when high availability is required. The 3rd DAG member is a Microsoft recommendation when using JBOD storage. For disaster recovery a remote 3rd DAG member could make sense, but then you wouldn’t require a “witness server” given the odd number of DAG members.

Note is that there’s a difference between high availability and disaster recovery. Having multiple copies in the same site is to offer high availability; having remote copies is to provide resilience.  More information on DAGs and the role of the File Share Witness in my Datacenter Activation Coordination article here.

Server Virtualization and DAGs
Next, the article continues that Exchange 2010 DAGs don’t support high availability options provided by the virtualization platform, which is spot on. Microsoft and VMWare have been squabbling for some time over DAGs in combination of with VMWare’s HA/DRS options, leading to the mentioned support statement from Microsoft.  VMWare did their part by putting statements like “VMware does not currently support VMware VMotion or VMware DRS for Microsoft Cluster nodes“; what doesn’t help is putting this in the best practices guide as a side note on page 64. More recently, VMWare published a support table for VMWare HA/DRS and Exchange 2010 indicating a “YES” for VMWare HA/DRS in combination with Exchange 2010 DAG. I hope that was a mistake.

In the end, I doubt if DAGs being non-supported in conjunction with VMWare HA/DRS (or similar products from other vendors) will be a potential deal breaker, like the author states. That might be true for organizations already utilizing those options as part of their strategy. In that case it would come down to evaluating running Exchange DAGs without those options (which it happily will). Not only will that offer organizations Exchange’s availability and resilience options with a much greater flexibility and function set than a non-application aware virtualization platform would, it also saves you some bucks in the process as well. For example, where VMWare can recover from data center or server failures, DAGs can also recover from database failures and several forms of corruption.

Exchange 2010 routing
The article then continues with Exchange 2010 following Active Directory sites for routing. While this is true, this isn’t something new. With the arrival Exchange 2007, routing groups and routing group connectors were traded for AD sites to manage routing of messages.

The writers annoyance here is that Exchange must be organized to follow AD site structure. Is that bad? I think not. Of course, with Exchange 2003 organizations could skip defining AD sites so they should (re)think their site structure anyway since more and more products use AD site information. I also think organizations that haven’t designed an appropriate AD site structure following recommendations may have issues bigger than Exchange. In addition, other products like System Center also rely on a prope AD site design.

Also, when required organizations can control message flow in Exchange using hub sites or connector scoping for instance. It is also possible to override site link costs for Exchange. While not all organizations will utilize these settings, they will address most needs for organizations. Also, by being site-aware, Exchange 2010 can offer functionality not found in Exchange 2003, e.g. autodiscover or CAS server/CAS Arrays having site-affinity.

Ultimately, it is possible to set up a separate Exchange forest. By using a separate forest with a different site structure, organizations can isolate directory and Exchange traffic to route it through different channels.

CAS High Availability complicated?
The last annoyance mentioned in the article is about the lack of wizards to configure CAS HA features, e.g. to configure a CAS array with network load balancing like the DAG wizard installs and configures fail-over clustering for you. While true, I don’t see this as an issue. While setting up NLB was not too complex and fit for small businesses, nowadays Microsoft recommends using a hardware load balancer, making NLB of less importance. And while wizards are nice, most steps should be performed as part of a (semi)automated procedure, e.g. reconfiguring after a fail/switch-over. This procedure or script can be tested properly, making it less prone to error.

The article also finds network load balancing and Windows fail-over clustering being mutually exclusive an annoyance. Given that hardware load balancing is recommended and cost effective, supported appliances became available this restriction is becoming a non- issue.

More information on configuring CAS arrays here and details on NLB with clustering here.

Final words
Now don’t get the impression I want to condemn Greg for sharing his annoyances with us. But when reading the article I couldn’t resist responding on some inaccuracies, sharing my views in the process. Most important is that we learn from each other while discussing our perspectives and views on the matter. Having said that, you’re invited to comment or share your opinions in the comments below.

KB979744 re-released to fix issues after installing MS11-028


A quick notice on the Exchange Team re-releasing hotfix KB979744 after identifying the issue which could cause problems on Windows Server 2008 SP2 (not R2). The problem can lead to Exchange Management Shell or Exchange Management Console not starting, MRS crashing or Event Viewer not opening after installing MS11-028 (KB2449742 or KB2449741, depending on your Windows Server 2008 level).

If you didn’t install the MS11-028 hotfix yet:

If you have the MS11-028 hotfix installed and you experience the issue:

Forefront Protection for Exchange Rollup 3


Microsoft released Hotfix Rollup 3 for Forefront Protection for Exchange Server (KB2538719).

Here’s the list of fixes included in this rollup:

  1. Mail queues and sluggish Exchange/Outlook performance
  2. Increased “Available Disk Space” Health Point threshold to 250MB
  3. Error: The DNS Blocklist lookup domain blocklist.messaging.microsoft.com could not be contacted
  4. The Exchange Information store crashes with Forefront Protection for Exchange installed
  5. An attempted upgrade of Forefront Protection for Exchange fails with a “Registration Service Failed” error
  6. You receive Forefront Protection Health Notifications indicating a status of “Green to Green”
  7. Forefront generates a MaxDisabledWait error within 15 minutes after starting
  8. A MaxDisabledWait error occurs and Forefront Protection does not recover
  9. Forefront Protection doesn’t apply keyword filtering within hyperlink strings
  10. Forefront Protection for Exchange crashes while scanning a TAR file
  11. An engine update fails in Forefront Protection for Exchange
  12. Emails that are 90 MB or larger are being sent to the Forefront archive folder
  13. The Microsoft Forefront Server Protection Eventing Service will not start following an upgrade from a beta version of Forefront Protection for Exchange
  14. Forefront Protection for Exchange detects files as “Engine Error” when no engines have been enabled for scanning
  15. Messages quarantined due to engine error can now be delivered as complete email
  16. High CPU conditions in EdgeTransort.exe process result in crash
  17. You receive Forefront generated email notification that the Cloudmark engine or Worm list could not update
  18. Exchange email queues at startup following an abnormal shutdown

For more details on the fixes consult the related knowledge base article. You can request the hotfix rollup through the support center here.

VMWare HA/DRS and Exchange DAG support


Last year an (online) discussion took place between VMWare and Microsoft on the supportability of Exchange 2010 Database Availability Groups in combination with VMWare’s High Availability options. Start of this discussion were the Exchange 2010 on VMWare Best Practices Guide and Availability and Recovery Options documents published by VMWare. In the Options document, VMWare used VMware HA with DAG as an example and contains a small note on the support issue. In the Best Practices Guide, you have to turn to page 64 to read in a side note, “VMware does not currently support VMware VMotion or VMware DRS for Microsoft Cluster nodes; however, a cold migration is possible after the guest OS is shut down properly.” Much confusion rose; was Exchange 2010 DAG supported in combination with those VMWare options or not?

In a reaction, Microsoft clarified their support stance on the situation by this post on the Exchange Team blog. This post reads, “Microsoft does not support combining Exchange high availability (DAGs) with hypervisor-based clustering, high availability, or migration solutions that will move or automatically failover mailbox servers that are members of a DAG between clustered root servers.” This meant you were on your own when you performed fail/switch-overs in an Exchange 2010 DAG in combination with VMWare VMotion or DRS.

You might think VMWare would be more careful when publing these kinds of support statements. Well, to my surprise VMWare published a support article 1037959  this week on “Microsoft Clustering on VMware vSphere: Guidelines for Supported Configurations”. The support table states a “Yes” (i.e. is supported) for Exchange 2010 DAG in combination with VMWare HA and DRS. No word on the restrictions which apply to those combination, despite the reference to the Best Practices Guide. Only a footnote for HA, which refers to the ability to group guests together on a VMWare host.

I wonder how many people just look at that table, skip those guides (or overlook the small notes on the support issue) and think they will run a supported configuration.

Updating Exchange 2010 DAG Members


With all the (re-)releases of rollups, the question might rise on how to perform a proper up or downgrade of all DAG configuration members.

Basically, the procedure is straightforward and should be followed per DAG member:

  1. Appoint (next) DAG member;
  2. Move away all active copies on that DAG member;
  3. Prevent copies from activating on that DAG member;
  4. Perform maintenance, e.g. down or upgrade DAG member;
  5. Enable possible activation on that DAG member again;
  6. Optionally redistribute database copies.

Note that in a DAG configuration with 2 members, you need to be aware that during maintenance you have a temporary situation with no fail-over options. If that’s undesirable, consider implementing a 3rd DAG member.

To make the above procedure  easier and automated regarding moves and activation (un)blocking, additional scripts are available since SP1 for Exchange 2010. These scripts are located in the Scripts folder, below the Exchange installation folder. By default the location of the scripts will be C:\Program Files\Microsoft\Exchange Server\v14\Scripts.

Utilizing them, the procedure is quite easy as you can see below. Note that the example uses a DAG named DAG1 with nodes ex2010a and ex2010b as members. They both host 2 databases, ex2010mdb1 and ex2010mdb2; both host 1 active copy and a passive copy of the other database.

  1. Appoint (next) DAG member, e.g. ex2010a;
  2. Run StartDagServerMaintenance.ps1 targeting that DAG member, e.g.:
    .\StartDagServerMaintenance.ps1 –server ex2010a

    image
  3. Perform maintenance;
  4. Run StopDagServerMaintenance.ps1 targeting that DAG member, e.g.:
    .\StopDagServerMaintenance.ps1 –server ex2010a
  5. Repeat steps 2-3 for the other DAG member(s):image
  6. Optionally run RedistributeActiveDatabases.ps1 for the DAG, e.g.:
    .\RedistributeActiveDatabases.ps1 –DagName DAG1 –BalanceDBsByActivationPreference –Confirm:$false

    image

Be advised that when upgrading on major levels (RTM to SP1 or SP1 to SP2), you can’t move a database to a lower level host. This means that when upgrading a node from SP1 to SP2 and moving a database to that SP2 node in the process, you can’t move that database to any SP1 nodes in the DAG. Keep this in mind when planning your upgrade, because it will impact the availability level by limiting your fallback options, albeit temporarily.