HTTP Proxy TargetBackEnd limits

powershellLast Update: February 4th, 2016

When deploying Exchange 2013 or Exchange 2016 in co-existence with a legacy version of Exchange, there comes a point where all traffic is routed through Exchange 2013/2016. Traffic for mailboxes hosted on legacy Exchange versions will be proxied by Exchange 2013/2016 to the back end.

This proxy process has some built-in limits for certain protocols, which you could encounter. Symptoms of these limits are Event 2022’s being logged in the Application log by the MSExchange Front End HTTP Proxy service:

image

Per Exchange 2013 CU7, this message should be considered a notice, despite the confusing event description. No connections are being blocked. However, the events create noise in your logs, which can be prevented by raising these limits. To accomplish this, you need to dive in to the web.config of the applicable HTTP Proxy protocols:

  • $ExInstall\FrontEnd\HttpProxy\sync\web.config (for ActiveSync, EAS)
  • $ExInstall\FrontEnd\HttpProxy\rpc\web.config (for OA, RPC/http)

In those files, create or adjust the entry in the <appsettings> configuration node, where <value> is the limit you want to configure (default is 150):

<add key=”HttpProxy.ConcurrencyGuards.TargetBackendLimit” value=”<value>” />

After adjusting these values, recycle the relevant application pools, e.g. MSExchangeSyncAppPool and MSExchangeRPCProxyAppPool.

The above steps need to be performed on all Exchange 2013/2016 Client Access Servers.

To automate this process of tedious editing in web.config files, I have created a small script which lets you alter these values for EAS and RPC against the local server or remotely. The script, Configure-HTTPProxyTargetBackEnd.ps1, has the following parameters:

  • Server to specify server to configure. When omitted, will configure local server.
  • AllServers to process all discoverable Exchange Client Access servers
  • TargetBackEnd specifies Target Backend limit (default 150).
  • NoRecycle to prevent recycling the MSExchangeSyncAppPool and MSExchangeRPCProxyAppPool

For example, to configure the local server with a limit of 2000 for Exchange Active-Sync and RPC access, use:

.\Configure-HTTPProxyTargetBackEnd.ps1 -TargetBackEnd 2000

image

Note that the script will create a backup copy of the web.config files before editing, using the current timestamp.

Download
You can download the script from the TechNet Gallery here.

Feedback
Feedback is welcomed through the comments. If you got scripting suggestions or questions, do not hesitate using the contact form.

Revision
See TechNet Gallery page.

Outlook for iOS adds Contacts support

imageA short notice on an update received today for Outlook for iOS 2.09. This update adds the much requested feature of integrating Outlook for iOS with the (native) Contacts in iOS:

“Your Office 365 and Exchange Contacts can now be saved to the iOS Contacts app. This will allow you to easily see the name of a contact when you receive a call or text message from them. Head to your Advanced Settings to turn on this feature.”

This does away with the requirement of resorting to setups like having the iOS Mail app sync with your Office 365 or Exchange On-Premises account, just to sync those contacts with your device. To disable syncing contacts through the Mail app, go to Settings > Mail, Contacts, Calendars and click the account you wish to disable syncing contacts for. Then, disable syncing its Contacts by toggling its switch:

image_thumb.png

You will get a warning contacts synced through this contact will be removed from Contacts, but since we are going to use Outlook for this, you can proceed.

Next, open up the updated new Outlook app, and go to Settings. Click the account from which you want to sync contacts to your device, and select Advanced Settings. In there, you will find a new switch, Save Contacts to Device. Behind it is the number of contacts available on this account:

image

Toggle the switch to start syncing contacts directly from your Office 365 or Exchange On-Premises account to Contacts, giving the Outlook app permissions to access your Contacts when requested. After this, you’re ready to go.

Note that all synced contacts will contain a line in the Notes field, stating:

Exported from Microsoft Outlook (Do not delete) [outlook:..:..]

This is to indicate this is a synced contact, and you must not edit or remove it using the device, rather remove it from the originating source as it might get recreated or overwritten during synchronization.

Finally, the sync is one-way, so although you can edit properties on your phone through the Contacts app, they won’t be synced back to the originating source. Also, when editing properties through Contacts, those edits are not propagated to the People view in the Outlook app, as those are the contacts from your Office 365 / Exchange On-Premises accounts. This can be confusing, but having to set up an e-mail account just once with a one-way sync seems more efficient and less confusing to me than having to configure the Mail app only to get your contacts on your phone.

Exchange 2013 Cumulative Update 11

Ex2013 LogoThe Exchange Team released Cumulative Update 11 for Exchange Server 2013 (KB3099522). This update raises Exchange 2013 version number to 15.0.1156.6.

  • KB 3120594 Appointment on the Outlook calendar isn’t updated to a meeting when attendees are added
  • KB 3108345 “The app couldn’t be downloaded” error occurs when you try to install an application from the Intranet in Exchange Server 2013
  • KB 3108011 Error message occurs in Outlook after you change a single instance of a recurring meeting by using an iOS device
  • KB 3107781 Exchange ActiveSync device doesn’t keep messages for 30 days as configured
  • KB 3107379 Noderunner.exe consumes excessive CPU resources by parsing an attached document in Exchange Server 2013
  • KB 3107337 Mailbox migration from Exchange Server 2007 to Exchange Server 2013 is very slow
  • KB 3107291 Exception occurs when you run the Invoke-MonitoringProbe cmdlets to set probes for IMAP and POP3 in Exchange Server 2013
  • KB 3107205 “Custom error module does not recognize this error” error when OWA web parts fail to load
  • KB 3107174 Pages that use the People pop-up URL don’t load in Chrome when you access OWA or the Exchange Server Administration Center
  • KB 3106613 Outlook Web App shows partial contacts in an Exchange Server 2013 environment
  • KB 3106475 POP3 and IMAP4 are not supported to use TLS protocol 1.1 or 1.2 in Exchange Server 2013
  • KB 3106421 Very long URLs in an email message do not open in OWA in Internet Explorer
  • KB 3105760 Exchange Server 2016 mailbox server can be added to an Exchange Server 2013 DAG
  • KB 3105690 Outlook clients that use MAPI over HTTP to connect to Microsoft Exchange Server 2013 mailboxes are intermittently disconnected
  • KB 3105685 The lsass.exe process leaks an amount of handles in Exchange Server 2013
  • KB 3105654 Cannot edit Inbox rules in Outlook Web App by using Chrome
  • KB 3105625 ActiveSync device downloads emails while it’s in quarantine in an Exchange Server 2013 environment
  • KB 3105389 WSMan-InvalidShellID error when you create remote PowerShell sessions in an Exchange Server 2013 environment
  • KB 3100519 No responses are sent from a room mailbox when a booked meeting extends beyond the date you set in Exchange Server 2013
  • KB 3093866 The number of search results can’t be more than 250 when you search email messages in Exchange Server 2013
  • KB 3088911 Inline attachments are sent as traditional when you smart forward an HTML email in an iOS device in Exchange Server 2013
  • KB 3088487 IOPS Write increase causes email delivery delays in an Exchange Server 2013 environment
  • KB 3076376 IMAP clients that use Kerberos authentication protocol are continually prompted for credentials in Exchange Server 2013
  • KB 3068470 “Something went wrong” error in Outlook Web App and ECP in Exchange Server 2013
  • KB 3048372 Exchange Calendar items are shifted incorrectly when some Windows DST updates are applied
  • KB 2968265 OWA cannot be accessed after you upgrade Exchange Server 2013

 

Notes:

  • This CU introduces an important change in the mechanism how Exchange Management Shell sessions will be initiated as of Exchange 2013 CU11 (and to be introduced in Exchange 2016, as well), called Mailbox Anchoring. More on this later in this article.
  • When using Exchange hybrid deployments or Exchange Online Archiving (EOA), you are required to stay current (version N) or be one version behind (N-1).
  • Cumulative Update may include schema or Active Directory changes (e.g. Role-Based Access Control). Make sure you run PrepareSchema /PrepareAD.  If you want to speed up the Cumulative Update installation process, you can temporarily disable certificate revocation checking as described here.

Note that Cumulative Updates can be installed directly, i.e. no need to install RTM prior to installing Cumulative Updates. Note that once installed, you can’t uninstall a Cumulative Update nor any of the installed Exchange server roles. The order of upgrading servers is irrelevant, unlike with previous generations of Exchange.

Finally, and I can’t emphasize this enough: For any Hotfix, Rollup, Service Pack or Cumulative Update, I’d recommend to thoroughly test this in a test and acceptance environment first, prior to implementing it in production. When you lack such facilities, hold out a week or two and monitor the comments on the release article or TechNet forum for any issues.

You can download Exchange 2013 Cumulative Update 11 here; UM Language Packs can be found here.

MAILBOX ANCHORING
This CU introduces an important change in the administrative model. In short, you need to home your administrative mailbox on the Exchange platform level you want to administer Exchange from (mailbox anchoring), as you will connect (or be proxied) to an Exchange Management Shell (EMS) session on that host. In other words, use an administrative account with a mailbox on Exchange 2013 to administer Exchange 2013, use an admin mailbox on Exchange 2016 for Exchange 2016. The logic behind this is to work around mixed-version environment issues, as newer Exchange versions may introduce changes, like new or enhanced cmdlets but also deprecated functionality. New general recommendation is to keep arbitration mailboxes as well as administrative mailboxes on the most current version.

If the admin has no mailbox, or if it’s unavailable, arbitration mailboxes – primarily SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c} – are considered for hosting your EMS session. Also, that ‘Connected to <Server>’ message when you open up an EMS session will no longer always mean your EMS session is hosted on that server; it could mean your EMS session is being proxied through there, which can create challenges when you’re running multiple sites with low bandwidth links – you may need to move your admin mailbox around or create one for local administration to enjoy better response times. You can only discover which host your session runs on by inspecting the local environment, using elements like the env:COMPUTERNAME variable or [System.Net.Dns]::GetHostName().

Also, it might be wise to spread administrative mailboxes over different servers or databases, in case your arbitration mailboxes become unavailable together with that one administrative mailbox, as you need to recover one of those just so you can set up an EMS session. The last resort for running an EMS cmdlets – against all best practices and recommendations, as it bypasses Role-Based Access Control for example – is  to load the Exchange module using Add-PSSnapIn. But be advised, you may not have all required permissions, for example your admin account may not have direct Active Directory permissions (and which is one of the reasons you shouldn’t just load the snap-in under normal circumstances).

The Exchange Team put up a separate blog to explain this change in behavior here.

Exchange 2010 SP3 RU12 & Exchange 2007 SP3 RU18

Exchange 2010 LogoThe Exchange Team released Rollup 12 for Exchange Server 2010 Service Pack 3 (KB3096066) as well as Rollup 18 for Exchange Server 2007 Service Pack 3 (KB3078672). These update raise version numbers to 14.3.279.2 and 8.3.445.0 respectively.

Apart from a Daylight Savings Time update, documented here, these Rollups contain the following fixes:

Exchange 2010 SP3 Rollup 12:

  • KB 3048372 Exchange Calendar items are shifted incorrectly when some Windows DST updates are applied
  • KB 3096125 CryptographicException error when Edge Transport service crashes in an Exchange Server 2010 environment
  • KB 3097219 Organizer’s name isn’t displayed in the subject of the recurring meeting requests in Exchange Server 2010
  • KB 3106421 Very long URLs in an email message don’t open in OWA in Internet Explorer
  • KB 3115809 Mailboxes can be accessed when the DefaultNetworkCredentials option is selected when you use Exchange Web Services Managed API to connect to Exchange Server

Exchange Server 2007 SP3 Rollup 18:

  • KB 3106421 Very long URLs in an email message don’t open in OWA in Internet Explorer

Notes:

    • If you want to speed up the update process for systems without internet access, follow the procedure described here to disable publisher’s certificate revocation checking.
    • If you got an Exchange 2010 DAG, and want to properly update the DAG members, check the instructions here.
    • As for any Hotfix, Rollup, Service Pack or Cumulative Update, apply this update to a acceptance environment first, prior to implementing it in production. When you lack such facilities, hold out a certain period and monitor the comments on the release article or TechNet forum for any issues.

Rollups are cumulative per service pack level, i.e. they contain fixes released in earlier update Rollups for the same product level (RTM, SP). This means you can apply the latest Rollup after installing a fresh installation of RTM or SPx version, for that product level.

You can download Exchange 2010 SP3 Rollup 12 here and Exchange 2007 SP3 Rollup 18 here.

Exchange Server Role Requirements Calculator 7.8

Exchange 2010 Mailbox Role Sizing Calculator 16.4The Exchange team today published an update for the Exchange 2013 Server Role Requirements Calculator as well. The new version number is 7.8. This version incorporates sizing for Exchange 2016 as well and includes support for ReFS (default for Exchange 2016). The version number is also dropped from the calculator.

More or less complementary to the calculator is the updated sizing guidance for Exchange 2016, which was also published today here. No big changes here, apart from multi-role only option and a slight increase in CPU requirements to cover for unforeseen circumstances as the team is still learning from real-world behavior. This makes sense, looking at the speed in which the calculator was released compared to the one for Exchange 2013. Kudos to the Exchange team!

New and enhanced functionality since version 7.6:

  • Added support for Exchange 2016
  • Included CPU utilization guidance changes for Exchange 2016
  • Diskpart.ps1 and CreateDAG.ps1 now support ReFS
  • Moved DataMoveReplicationConstraint setting from CreateMBDatabases.ps1 to CreateMBDatabaseCopies.ps1
  • Revised all of the Distribution dialog controls to load their defaults from variables rather than use hard-coded values
  • The DAG name from the Input tab now flows through as the default on the Export DAG dialog
  • Updated Distribution tab dialog controls to persist the global catalog value during a session
  • Added conditional formatting for ReplayLagTime and SafetyNetThreshold
  • Removed 2013 from the name of the calculator

Fixes since version 7.6:

  • Fixed inaccuracies with “Number of Exchange Data Volumes per Server” input
  • Fixed calcActDBPDCWorst formula to take into account non-HA deployments
  • Fixed multiple dbs / volume calculation to take into account ReplayLagManager
  • Fixed calcNumDBCopyInSDC formula to take into account proper number of lagged copies
  • Fixed MaxPreferredActive not being displayed for A/A (Single DAG) site resilient solutions
  • Fixed an issue with Fail* buttons on Distribution tab when using some regional settings
  • Fixed an issue with volume path persistence on the Distribution tab Mount Points dialog

You can download the calculator here. For more information, please consult the list of changes here or Read Me here.

Blocking Mixed Exchange 2013/2016 DAG

Ex2013 LogoIn the RTM version of Exchange 2016, there’s an issue in that it is allows you to add Exchange 2016 Mailbox servers to Exchange 2013 Database Availability Groups, and vice-versa. As stated in the Release Notes (you do read those?), creating such a mixed version DAG is not supported. In theory, you could even jeopardize your Exchange data, as database structures from both versions are different. This action is also not prevented from the Exchange Admin Center, requiring organizations to have very strict procedures and knowledgeable Exchange administrators.

If you are worried about this situation and you want to prevent accidently adding Mailbox servers to an existing DAG consisting of members of a different Exchange version, there is a way (until this is blocked by the product itself, of course). Cmdlet Extension Agents to the rescue!

The Scripting Agent not only allows you to add additional instructions to existing Exchange cmdlets, but also to provide additional validation before cmdlets are executed. I did two short articles on Cmdlet Extension Agents’ Scripting Agent here and here, so I will skip introductions.

First you need to download a file named ScriptingAgentConfig.xml from the location below. If you already have Scripting Agents, you need to integrate the code in your existing ScriptingAgentConfig.xml files. The code checks if the server you want to add using the Add-DatabaseAvailabilityGroup cmdlet is of a different major version than one of the current DAG members.

Next, you need to copy this ScriptingAgentConfig.xml file to $ENV:ExInstallPath on every Exchange 2013 and Exchange 2016 server in your organization, e.g. C:\Program Files\Microsoft\Exchange Server\V15\Bin\CmdletExtensionAgents\ScriptingAgentConfig.xml.  To help your with this process, Exchange fellow Paul Cunningham made a small script to push this XML from the current folder to every Exchange server in your organization, PushScriptingAgentConfig.ps1.

Last step is to enable the Scripting Agent using:

Enable-CmdletExtensionAgent ‘Scripting Agent’

After distributing the scripting agent file and enabling the scripting agent, when you try to add an Exchange 2016 (version 15.1) server to an Database Availability Group consisting of Exchange 2013 Mailbox servers, using Add-DatabaseAvailabilityGroupServer, you will receive an error message:

DAGCheck

This also works vice-versa, thus when you inadvertently try to add Exchange 2013 servers to an Exchange 2016 Database Availability Group, provided you distributed the XML on the Exchange 2013 servers as well. The error is also thrown when you try to perform this action using the Exchange Admin Console.

You can download the ScriptingAgentConfig.XML for blocking Mixed Exchange 2013/2016 DAGs from the TechNet here.

Exchange 2010-2013 Migration and OAB

Ex2013 LogoLast year, Exchange fellows Andrew Higginbotham, Paul Cunningham as well as the Exchange Team reported on checking, and when necessary configuring, your Offline Address Book (OAB) in your current Exchange Server 2010 environment, prior to installing Exchange Server 2013. Not doing so could result in a complete download of the Offline Address Book created by Exchange Server 2013, titled ‘Default Offline Address List (Ex2013)’.

Today I received a report that there is a different symptom of configuration absence. In this case, the customer reported on the inability to download the offline address book, and upon further inspection the Autodiscover server did not report back on the offline address book URL to use. In other words, OAB information was absent from the Autodiscover response, and Outlook gets confused. Note that this issue was reported in Outlook 2010 after installing Exchange Server 2013 Cumulative Update 10. I’m not sure if this change in behavior was introduced in these later builds of Exchange 2013 or Outlook, but it’s still a good thing to know.

The remedy here of course is to configure any (Exchange 2010) mailbox database with unconfigured Offline Address Book setting, and point them to the default offline address book using:

Get-MailboxDatabase | Where-Object {$_.OfflineAddressBook -eq $Null} | Set-MailboxDatabase -OfflineAddressBook (Get-OfflineAddressBook | Where-Object {$_.IsDefault -eq $True})

Knowledgebase RSS feeds

rss[1]Note: This is an update of an article from January, 2010.

Like most people I still use RSS feeds to keep track of news and updates from various sources. But did you know you can also keep track of Microsoft’s knowledgebase articles per product using RSS feeds? Great for keeping track of updates in RSS readers like Outlook or sites like Feedly, or creating triggers on sites like IFTTT (If-This-Then-That) to automatically send e-mail notifications.

Here are some RSS feeds on knowledgebase articles that might be of interest to you:

Exchange Server

Outlook

Office 365

Lync/Skype for Business

There is no RSS feed for Exchange Server 2016 yet.

For a complete list of the knowledgebase articles RSS feeds check here.

Exchange 2016 and IM Integration

Ex2013 LogoThose configuring IM integration for OWA and Lync or Skype for Business know the drill of editing the web.config files on your Exchange servers and configuring the certificate thumbprint and Lync/SfB pool? That especially became a nuisance as after each Cumulative Update those settings needed to be reconfigured, for which I wrote a Configure-IMIntegration script.

The Exchange team has obviously listened to feedback from customers and made this setting persistent in Exchange 2016. No longer is it required to dive in those web.config files after installing each CU. Instead, you now configure these settings using the Set-Override cmdlet, which will store the setting in Active Directory.

For example:

New-SettingOverride -Name '<Description>' -Server <Server/Wildcard> -Component OwaServer -Section IMSettings -Parameters @("IMServerName=<Server/Pool FQDN>","IMCertificateThumbprint=<Certificate Thumbprint>") -Reason "<Reason>" -MinVersion "<Minimum Version To Apply To>" -MaxVersion "Maximum Version to Apply To"
Get-ExchangeDiagnosticInfo -Process Microsoft.Exchange.Directory.TopologyService -Component VariantConfiguration -Argument Refresh

For example, to configure the override for all servers with a name starting with EX16, configuring lync.contoso.com as pool FQDN and a specific thumbprint, only for Exchange builds starting at 15.1.225.42 (Exchange 2016 RTM), you could use:

New-SettingOverride -Name 'IM Integration' -Server EX16* -Component OwaServer -Section IMSettings -Parameters @("IMServerName=lync.contoso.com","IMCertificateThumbprint=12345678123412341234567812345678123126789") -Reason "Configure IM" -MinVersion "15.01.0225.42"
Get-ExchangeDiagnosticInfo -Process Microsoft.Exchange.Directory.TopologyService -Component VariantConfiguration -Argument Refresh

Finally, restart the OWA App pool to have OWA reread the new settings:

Restart-WebAppPool MSExchangeOWAApppool

Exchange 2016 goes RTM!

Ex2013 LogoUpdate (4nov2015): You can block creating mixed DAGs using Cmdlet Extension Agents, I blogged about that here.

Today, the Exchange Team reached a milestone for the On-Premises by releasing Exchange Server 2016. The official announcement contains information on new features and enhancements.The version number of Exchange 2016 RTM is 15.1.225.42. After extending it, the schema version should report 15317, and the forest and domain versions after preparing Active Directory should read 16210 and 13236, respectively.

Much of what’s new or requirements for coexistence scenarios were already announced during the release of the Exchange 2016 Preview, a little over 2 months ago. I did a write-up on that here. However, some features didn’t make it for the RTM release. For example, the feature that makes Search Indexer use Passive Database Copies for indexing, instead of copying indexes from the active copy, is to be expected in a later Cumulative Update. Also, the auto-expanding Archive feature, available in the Preview, has not made it in the RTM version.

Also make sure you read the Release Notes, which contain important information on potential issues. For example, Exchange 2016 does not prevent you from adding Exchange 2013 Mailbox servers to an Exchange 2016 Database Availability Group, or vice-versa. This ability is also not blocked by the Exchange Admin Center console. This is totally unsupported (the database structure is different), but more importantly also puts your data at risk. Just don’t.

Some links to get you started:

The first Cumulative Updated is to be expected in Q1’16.

Accompanying the launch, Microsoft also published a number of videos highlighting certain aspects or features. One of them is the ever charming Greg Taylor talking about Exchange Server 2016 – Performance, architecture and compliance updates:

Other videos from the Exchange Team and Office Garage: