Exchange 2007 SP3 Update Rollup 6

Today the Exchange Team released Rollup 6 for Exchange Server 2007 Service Pack 3 (KB2608656). This update raises Exchange 2007 version number to 8.3.245.2.

Here’s the list of changes included in this rollup:

  • 2289607  The week numbers displayed in OWA do not match the week numbers displayed in Outlook for English users and French users in an Exchange Server 2007 environment
  • 2498852  “0×80041606″ error message when you perform a prefix search by using Outlook in online mode in an Exchange Server 2007 environment
  • 2499841  An arrow icon does not appear after you change the email message subject by using OWA in an Exchange Server 2007 SP3 environment
  • 2523695  A “System.ArgumentOutOfRangeException” exception occurs when you click the “Scheduling Assistant” tab in Exchange Server 2007 OWA
  • 2545080  Users in a source forest cannot view the free/busy information of mailboxes in a target forest when the cross-forest Availability service is configured between two Exchange Server 2007 forests
  • 2571391  Applications or services that depend on the Remote Registry service may stop working in an Exchange Server 2007 environment
  • 2572010  The Microsoft Exchange Information Store service may crash after you run the Test-ExchangeSearch cmdlet in an Exchange Server 2007 environment
  • 2575360  A new feature is available to automatically stop the Microsoft Exchange Information Store service when a time-out is detected in an Exchange Server 2007 SP3 environment
  • 2591655  A journaling report remains in the submission queue when an email message is delivered successfully in an Exchange Server 2007 environment
  • 2598980  The PidLidClipEnd property of a recurring meeting request has an incorrect value in an Exchange Server 2007 environment
  • 2616427   An Outlook Anywhere client loses connection when a GC server restarts in an Exchange Server 2007 environment
  • 2617784  Journal reports are expired or lost when the Microsoft Exchange Transport service is restarted in an Exchange Server 2007 environment
  • 2626217   Certain changes to address lists may not be updated in an Exchange Server 2007 environment
  • 2629790   The Exchange IMAP4 service may stop responding on an Exchange Server 2007 Client Access server when users access mailboxes that are hosted on Exchange Server 2003 servers
  • 2633801   The SCOM 2007 SP1 server cannot alert certain issues in an Exchange Server 2007 organization
  • 914533  The Microsoft Exchange Information Store service may stop responding on an Exchange Server 2007 server
  • 976977  The scroll bar does not work in OWA when there are more than 22 all-day event calendar items in an Exchange Server 2007 user’s calendar
  • 2641312  The update tracking information option does not work in an Exchange Server 2007 environment
  • 2653334  The reseed process is unsuccessful on the SCR passive node when the circular logging feature is enabled in an Exchange Server 2007 environment
  • 2656040  An Exchange Server 2007 Client Access server may respond slowly or stop responding when users try to synchronize the Exchange ActiveSync devices with their mailboxes
  • 2658613  The “PidLidClipEnd” property of a no ending recurring meeting request is set to an incorrect value in an Exchange Server 2007 environment

When running ForeFront Protection for Exchange, make sure you disable ForeFront before installing the rollup and re-enable it afterwards, otherwise the Information Store and Transport services may not start. You can disable ForeFront using fscutility /disable and enable it using the fscutility /enable command.

Note that update rollups are cumulative, i.e. they contain fixes released in earlier update rollups for the same product level (RTM, SP). This means you don’t need to install previous update rollups during a fresh installation but can start with the latest rollup.

You can download Exchange 2007 SP3 Rollup 6 here.

Thoughts on “Automatic E-mail Server Notifications in Exchange 2010″

In an article on MsExchange.org, Markus Klein elaborates on the reasons behind the changed message delivery notification (MDN) behavior in Exchange 2010. Examples of MDNs are read or delivery receipts or out of office messages. Issues may arise with MDNs because Exchange 2010 (and Exchange 2007) will use a blank sender address and not all e-mail systems can cope with that, making Exchange compliant with the related RFC. The article ends with workarounds to mitigate the issue. Here are my thoughts on that article.

The article refers to RFC2298, dated March 1998. However, MDNs are defined by RFC3798 of May 2004, which obsoletes RFC2298. Nevertheless, like Klein indicated, both RFCs dictate the following:

The envelope sender address (i.e., SMTP MAIL FROM) of the MDN MUST be null (<>), specifying that no Delivery Status Notification messages or other messages indicating successful or unsuccessful delivery are to be sent in response to an MDN.

The idea behind using a blank sender address is that e-mail systems will not return DSN messages, e.g. mailbox unavailable or disk quota exceeded, as a reply to an MDN, preventing potential message loops. However, there are some side-effects as not all e-mail systems or messaging hygiene products are RFC compliant. For example, the default setting of ForeFront Protection 2010 for Exchange is to block messages with an empty sender address. These products may simply block those messages, since blank senders could potentially be an indicator for spoofed messages. When you suspect such product to be causing the issue, check and reconfigure when appropriate.

The author continues the article by describing how to configure and troubleshoot routing of MDNs to the internet. The author shows how to enable and inspect the receive connector logs. Instead, I suggest monitoring the send connector logs when troubleshooting MDN delivery. Inspecting the send connector log files, you can get a clue on why MDN delivery fails and will see if Exchange is trying to deliver the MDN at all, and if so, the reason why. To enable send connector logging use the following cmdlet:

Set-SendConnector <ConnectorID> -ProtocolLoggingLevel verbose

The log files are generated in the “V14\TransportRoles\Logs\ProtocolLog\SmtpSend” folder below the location where you installed Exchange.

Finally, the author suggests the following workarounds:

  1. Use Outlook “out of office”
  2. Switch Relay Provider
  3. Implement Exchange Server Edge Roles

The first workaround is a less preferable option, as it’s configured per-user as a rule and rules, stored in the user’s mailbox, can’t easily be managed. When using the OOF option, administrators can, using the Get-MailboxAutoReplyConfiguration and Set-MailboxAutoReplyConfiguration cmdlets. Also, it makes the end user responsible for working around the issue. Meanwhile, despite this instruction, you can still expect lots of users to keep using the OOF function.

The second and third suggestions are non-options, since they don’t eliminate the issue and will only add a product and an extra hop to the e-mail route. Yes, you can switch to using a different SMTP relay or implement an Exchange Edge server which will accept MDN messages with an empty sender address. However, that may not be the final destination of the e-mail message, so the (unpredictable) MDN delivery issue remains. Nobody can guarantee that the e-mail system or message hygiene appliance at the recipient blocks blocks your OOF message with an empty sender address. You can read that between the lines of the PSS statement the author quotes as well:

The Exchange edge server will not reject the OOF message as the edge server will be incorporated into the Exchange organization. The HUB server will transfer the OOF messages in the address of OOF mailbox to the edge server and the edge server will then send the messages with empty return path e.g. blank sender, MAIL FROM: <> “null” to Internet.

Now, when the issue lies outside of your Exchange organization, e.g. the hosted message hygiene service or destination mail system, you might be left with no other option than to violate RFC3798 by adding a sender address. In Exchange this isn’t possible, but other e-mail gateways could help you with that. Note that when using a hosted message hygiene service or appliance for outbound messages, using a non-blank sender might be less of an issue since you’re offloading the delivery, compared to trying to deliver the message to the destination mail system yourself.

However, when opting to resort to these measures, I’d strongly suggest reconsidering sending out of office messages (or MDNs in general) outside of your Exchange organization, regardless of the sender. Spammers love confirmed e-mail addresses, so treasure your business e-mail addresses like you probably treat your own personal address.

Note that this blog isn’t to condemn the author of the discussed article, but to clarify things up since many people moving from Exchange 2003 to Exchange 2007 or Exchange 2010 may run into these behavioral differences. You’re invited to comment or share your opinions in the comments below.

Exchange Management Console & IE9 issue fixed

Finally, today the Exchange team made available a fix to solve the issues when using the Management Console of Exchange 2007 or 2010 in conjunction with Internet Explorer 9.

As you probably know, when using Internet Explorer 9 you can’t close the Exchange Management Console properly as it gives you the error “You must close all dialog boxes before you can close Exchange Management Console” having no dialogs open.

To solve this issue, you had to do the resort to measures like killing the EMC process using Task Manager.

To properly install the hotfix:

  1. Request hotfix ID 2624899 from support here. For a direct download link click here.
  2. Download and install MS11-081 (2586448). You can retrieve this update here.
  3. Install the hotfix ID 2624899.

Microsoft states it expects to incorporate this fix in a future update of Internet Explorer 9.

While releasing a fix for the IE9 issue is great after all these month, I can’t help but wonder why the fix has not been made public.

Loadbalancing, ActiveSync and Affinity

Recently, a client was experiencing load issues on the Exchange 2010 Client Access Servers. The client also had installed a hardware load balancer to balance client traffic.

While investigating the PAL results, the ActiveSync connections chart showed a significantly unbalanced number of ActiveSync connections between the CAS servers.

It turned out the client had load balanced all client traffic using Source IP affinity for all protocols. This means each client gets assigned the same CAS server, based on the client’s IP address. While this may sound reasonable, for ActiveSync this may not be optimal. Reason is that most mobile telephony providers use some form of NAT translation for their clients, resulting in these devices to appear having the same IP address.

When organizations standardize on a NAT utilizing mobile telephony provider, the problem might emerge sooner as all of their mobile clients will be assigned to the same Client Access Server.

In the picture above you’ll see the top two mobile devices are being NAT’ed. When the top device connects to the Exchange environment, it gets assigned the 1st CAS server based on its IP address. When the 2nd mobile device connects, the load balancer sees the same IP address after which it will direct that traffic to 1st CAS server as well.

While affinity is not required for ActiveSync, it is recommended since for each newly appointed CAS server, the notification subscription to the mailbox to be informed of updates would have to be recreated. Of course, this would result in a performance penalty and increased latency. Another option would be Session ID, but some EAS clients unnecessarily create a new SSL session ID.

After switching affinity from Client IP to Authorization HTTP Header the ActiveSync clients spread out more evenly. When using Authorization HTTP Header affinity, the load balancer uses the base64 encoded credentials as part of the http client request, e.g.

POST http://mail.eightwone.com/Microsoft-Server-ActiveSync/default.eas?Cmd=Sync&..
..
Authorization: Basic YW55IGNhcm5hbCBwbGVhc3VyZS4=

After switching affinity for ECP as well (should be Cookie or Session ID), the load issues were gone.

Where in the past mobile clients were insignificant to Outlook clients when compared in numbers, the ongoing consumerization of IT movement results in an increasing mobile client population. The number of ActiveSync users may easily outweigh the number of Outlook clients, as many users use a phone or tablet (or both) in addition to Outlook, if they use Outlook at all.

Exchange 2007 SP3 Update Rollup 5

Today the Exchange Team released Rollup 5 for Exchange Server 2007 Service Pack 3 (KB2602324). This update raises Exchange 2007 version number to 8.3.213.1.

Here’s the list of changes included in this rollup:

  • 981820  New X-headers of a message item do not appear when the message item is retrieved by IMAP4 or by POP3 in an Exchange Server 2007 SP2 environment
  • 2292150  A deleted hyperlink remains in the HTML source of an email message if you create the email message by using OWA in an Exchange Server 2007 environment
  • 2411423  The Msftefd.exe process constantly consumes up to 100 percent of CPU resources when your mailbox language is set to German on an Exchange Server 2007 server
  • 2450078  The sent time in an email message body is incorrect when you reply or forward the email message by using an EWS application in an Exchange Server 2007 environment
  • 2451415  “There was a problem logging onto your mail server” error message when you use a POP3 client to access a mailbox in an Exchange Server 2007 SP3 environment
  • 2536652  EdgeTransport.exe randomly stops responding on a Hub Transport server after you configure public folder replication in Exchange Server 2007
  • 2536695  “Some items cannot be deleted” error message when you try to delete or modify an email message in a public folder in an Exchange Server 2007 environment
  • 2536697  DBCS characters in a rule name are converted to question marks after you move a mailbox from Exchange Server 2003 to Exchange Server 2007
  • 2537783  The EdgeTransport.exe process crashes occasionally after you install Update Rollup 2 for Exchange Server 2007 SP3
  • 2538958  Extended Protection Warning Displayed in Exchange Management Console and Exchange Management Shell After Installing RU2 for Exchange 2007 SP3
  • 2554575  Items accumulate in the MRM submission folder when managed folder assistant journal items in an Exchange Server 2007 environment
  • 2556751  The EdgeTransport.exe process crashes when processing certain email messages on an Exchange Server 2007 Hub Transport server
  • 2557304  The Store.exe process may consume excessive CPU resources and memory resources intermittently when a user opens a calendar item by using OWA in an Exchange Server 2007 SP3 environment

When running ForeFront Protection for Exchange, make sure you disable ForeFront before installing the rollup and re-enable it afterwards, otherwise the Information Store and Transport services may not start. You can disable ForeFront using fscutility /disable and enable it using the fscutility /enable command.

Note that update rollups are cumulative, i.e. they contain fixes released in earlier update rollups for the same product level (RTM, SPx). This means you don’t need to install previous update rollups during a fresh installation but can start with the latest rollup available right away.

You can download Exchange 2007 SP3 Rollup 5 here.

Managing Remote IP Ranges of Receive Connectors

When managing receive connectors in Exchange, you probably had to configure IP addresses or IP ranges on those receive connectors. This may be required when limiting access to a certain receive connector for applications to drop their mail using SMTP. Of course this can be done using the Exchange Management Console, but this may become tedious when lots of addresses are involved. Also, when multiple Hub transport servers are involved you may need to keep those IP ranges in sync on those Hub Transport servers in which case mismatches are likely.

As you’ve probably guessed, a little PowerShell makes life more easier. To configure the allowed IP ranges we need to use Set-ReceiveConnector and configure the RemoteIPRanges attribute. We’ll use a text file to maintain the list of allowed IP ranges and a PowerShell one-liner to set RemoteIPRanges.

The file should contain IP ranges in a RemoteIPRanges acceptable format, e.g.:

  • 192.168.1.10
  • 192.168.1.20-192.168.1.29
  • 192.168.2.0/24

When we have prepared the file, we can use the following cmdlet to set RemoteIPRanges:

Get-ReceiveConnector *\APPRELAY | Set-ReceiveConnector -RemoteIPRanges (Get-Content RemoteIPRanges.txt)

This will configure all receive connectors named APPRELAY on all Hub Transport servers in the organization using IP ranges defined in the file RemoteIPRanges.txt. Be advised that this cmdlet overwrites the current configuration of RemoteIPRanges; if you need to add it to the current configured set of IP ranges on each receive connector, use the following cmdlet:

Get-ReceiveConnector *\Appl-Relay | ForEach { Set-ReceiveConnector -RemoteIPRanges ($_.RemoteIPRanges+ (Get-Content ipranges.txt) | Sort -Unique) }

By adding the Sort -Unique filter, we make sure each range is only specified once. This prevents errors caused by setting a range using the RemoteIPRanges.txt file when that range has already been configured in the current value of RemoteIPRanges.

Note that when inspecting the results you can set $FormatEnumerationLimit to a value higher than the default (16) to have Get-ReceiveConnector * | fl RemoteIPRanges display all its values. Also, keep in mind when configuring connectors that the connector with the most specific matching IP address wins.

Exchange 2007 SP3 Update Rollup 4

Today the Exchange Team released Rollup 4 for Exchange Server 2007 Service Pack 3 (KB2509911). This update raises Exchange 2007 version number to 8.3.192.1.

Here’s the big list of changes included in this rollup:

  • 2531208  You cannot synchronize a folder hierarchy by using Outlook for Mac 2011 in an Exchange Server 2007 SP3 environment
  • 2528437  EWS applications cannot connect to Exchange Server 2007 servers after you make changes on accepted domains
  • 2521063  You are incorrectly displayed as a meeting organizer after you synchronize the meeting by using your mobile device in an Exchange Server 2007 environment
  • 2517337  You cannot open a mailbox that has a “#” character in the primary SMTP email address by using OWA in an Exchange Server 2007 environment
  • 2515428  The MSExchangeMailboxAssistants.exe process crashes when the managed folder assistant tries to journal a message in an Exchange Server 2007 environment
  • 2508872  The W3WP.exe process in the Autodiscover application pool on the Exchange Server 2007 Client Access servers consumes excessive CPU resources
  • 2507374  “Cannot open this item” error message in Outlook online mode in an Exchange Server 2007 environment
  • 2506827  An UM auto attendant times out and generates an invalid extension number error message in an Exchange Server 2007 environment
  • 2502276  A meeting request series are deleted unexpectedly from the calendar in an Exchange Server 2007 environment
  • 2498924  “Could not connect to a directory server” error message when you click the last page button in the search results in Exchange Server 2007 OWA
  • 2498156  OLM/OLD incorrectly runs against databases in a RSG in an Exchange Server 2007 environment
  • 2496806  A mobile phone times out when you use ActiveSync to synchronize the calendar on the mobile phone with an Exchange Server 2007 mailbox
  • 2543879  A PDF attachment sent from a Mac Mail client does not display when you open the email message by using Outlook 2010 in an Exchange Server 2007 SP3 environment
  • 2491751  Spell checking does not function correctly in OWA when an S/MIME control is used and SSL Offloading is enabled in Exchange Server 2007
  • 2484147  “HTTP Error 400 Bad Request” error message when you use OWA to log on to a newly created Exchange Server 2007 mailbox
  • 2466220  Question mark (?) characters appear in the subject of a reply email message in an Exchange Server 2007 environment
  • 2223294  A new feature is available to disable the “No end date” check box in OWA when you create a recurring meeting item in an Exchange Server 2007 environment
  • 977906  You receive an error message when you run certain commands in the EMS on an Exchange Server 2007 server
  • 2495010  The EdgeTransport.exe process consumes 100% CPU usage on an Exchange Server 2010 Edge Transport server or an Exchange Server 2007 Edge Transport server
  • 2484817  A mailbox does not show in certain address lists after you run commands on an Exchange Server 2007 mailbox

When running ForeFront Protection for Exchange, make sure you disable ForeFront before installing the rollup and re-enable it afterwards, otherwise the Information Store and Transport services may not start. You can disable ForeFront using fscutility /disable and enable it using the fscutility /enable command.

Note that update rollups are cumulative, i.e. they contain fixes released in earlier update rollups for the same product level (RTM, SPx). This means you don’t need to install previous update rollups during a fresh installation but can start with the latest rollup available right away.

You can download Exchange 2007 SP3 Rollup 4 here.

Microsoft Office Filter Pack 2010 SP1

Right after launching Office 365, Microsoft released Service Pack 1 for Microsoft Office 2010 (KB2460049) which includes Office 365 support besides a big list of fixes. For those interested, Office 2010 SP1 can be downloaded here (x86) and here (x64). For a full list of changes, check out Microsoft Office 2010 Service Pack 1 Changes Excel sheet.

More interesting for Exchange folks is that there’s also an SP1 for the Microsoft Office Filter Pack 2010, which of course you install during Exchange setup as one of the prerequisites. As you probably know, the Filter Pack is used to index Office documents stored in Exchange databases to speed up queries.

You can download Service Pack 1 for Microsoft Office Filter Pack 2010 x64 Edition here. The related knowledgebase article is KB2460041.

Exchange Environment Report

A quick post on Exchange fellow Steve Goodman who created a nice PowerShell script which generates a basic HTML report on your Exchange environment. When required, you can also e-mail the report, which is nice if you want to schedule the script to run on a daily basis for example.

The script is provided as-is so you can tailor it to your needs. It’s still work in progress, so if you got any requests just send Steve a message.

You can find the post and script here.

Exchange 2007 SP3 RU3 potential database corruption (update)

Update: Exchange 2007 SP3 Rollup 3 has been re-released. Version 2 of the Rollup can be downloaded here. Rollup 3 version 2 raises Exchange 2007′s version number to 8.3.159.2 (initial release was 8.3.159.0). The related knowledgebase article is kb2530488.

Update: The related knowledgebase article kb2531163 can be found here.

A quick notice on a potential issue with Exchange 2007 SP3 and database corruption after installing Rollup 3. The issue may occurs in the following situations:

  • When transaction log replay is performed by the Replication Service as part of ensuring the passive database copy is up-to-date;
  • When a database is not cleanly shut down and recovery occurs.

Because of this, Rollup 3 for Exchange 2007 was pulled and you’re advised to uninstall Rollup 3 on Exchange 2007 Mailbox and Transport servers. For more information, consult the post on the Exchange Team’s blog here.

Note that the issue may affect all Mailbox servers, clustered or standalone, so you’re also advised to uninstall RU3 on standalone Mailbox servers. For those with the issue on CCR or SCC setups, it requires reseeding or restoring from backup so plan accordingly.

Looking at the issues with latest Rollups for Exchange 2010 and 2007, it gives to think about the quality control process or if the Exchange team’s priorities perhaps lay somewhere else (Office 365?).

Potential for database corruption as a result of installing Exchange 2007 SP3 RU3