Exchange 2010 SP3 RU12 & Exchange 2007 SP3 RU18

Exchange 2010 LogoThe Exchange Team released Rollup 12 for Exchange Server 2010 Service Pack 3 (KB3096066) as well as Rollup 18 for Exchange Server 2007 Service Pack 3 (KB3078672). These update raise version numbers to 14.3.279.2 and 8.3.445.0 respectively.

Apart from a Daylight Savings Time update, documented here, these Rollups contain the following fixes:

Exchange 2010 SP3 Rollup 12:

  • KB 3048372 Exchange Calendar items are shifted incorrectly when some Windows DST updates are applied
  • KB 3096125 CryptographicException error when Edge Transport service crashes in an Exchange Server 2010 environment
  • KB 3097219 Organizer’s name isn’t displayed in the subject of the recurring meeting requests in Exchange Server 2010
  • KB 3106421 Very long URLs in an email message don’t open in OWA in Internet Explorer
  • KB 3115809 Mailboxes can be accessed when the DefaultNetworkCredentials option is selected when you use Exchange Web Services Managed API to connect to Exchange Server

Exchange Server 2007 SP3 Rollup 18:

  • KB 3106421 Very long URLs in an email message don’t open in OWA in Internet Explorer

Notes:

    • If you want to speed up the update process for systems without internet access, follow the procedure described here to disable publisher’s certificate revocation checking.
    • If you got an Exchange 2010 DAG, and want to properly update the DAG members, check the instructions here.
    • As for any Hotfix, Rollup, Service Pack or Cumulative Update, apply this update to a acceptance environment first, prior to implementing it in production. When you lack such facilities, hold out a certain period and monitor the comments on the release article or TechNet forum for any issues.

Rollups are cumulative per service pack level, i.e. they contain fixes released in earlier update Rollups for the same product level (RTM, SP). This means you can apply the latest Rollup after installing a fresh installation of RTM or SPx version, for that product level.

You can download Exchange 2010 SP3 Rollup 12 here and Exchange 2007 SP3 Rollup 18 here.

Exchange Server Role Requirements Calculator 7.8

Exchange 2010 Mailbox Role Sizing Calculator 16.4The Exchange team today published an update for the Exchange 2013 Server Role Requirements Calculator as well. The new version number is 7.8. This version incorporates sizing for Exchange 2016 as well and includes support for ReFS (default for Exchange 2016). The version number is also dropped from the calculator.

More or less complementary to the calculator is the updated sizing guidance for Exchange 2016, which was also published today here. No big changes here, apart from multi-role only option and a slight increase in CPU requirements to cover for unforeseen circumstances as the team is still learning from real-world behavior. This makes sense, looking at the speed in which the calculator was released compared to the one for Exchange 2013. Kudos to the Exchange team!

New and enhanced functionality since version 7.6:

  • Added support for Exchange 2016
  • Included CPU utilization guidance changes for Exchange 2016
  • Diskpart.ps1 and CreateDAG.ps1 now support ReFS
  • Moved DataMoveReplicationConstraint setting from CreateMBDatabases.ps1 to CreateMBDatabaseCopies.ps1
  • Revised all of the Distribution dialog controls to load their defaults from variables rather than use hard-coded values
  • The DAG name from the Input tab now flows through as the default on the Export DAG dialog
  • Updated Distribution tab dialog controls to persist the global catalog value during a session
  • Added conditional formatting for ReplayLagTime and SafetyNetThreshold
  • Removed 2013 from the name of the calculator

Fixes since version 7.6:

  • Fixed inaccuracies with “Number of Exchange Data Volumes per Server” input
  • Fixed calcActDBPDCWorst formula to take into account non-HA deployments
  • Fixed multiple dbs / volume calculation to take into account ReplayLagManager
  • Fixed calcNumDBCopyInSDC formula to take into account proper number of lagged copies
  • Fixed MaxPreferredActive not being displayed for A/A (Single DAG) site resilient solutions
  • Fixed an issue with Fail* buttons on Distribution tab when using some regional settings
  • Fixed an issue with volume path persistence on the Distribution tab Mount Points dialog

You can download the calculator here. For more information, please consult the list of changes here or Read Me here.

Ignite 2016: September 26-30, Atlanta

image

Out of nowhere, news came yesterday from the Chicago Tribune that Microsoft cancelled the Microsoft Ignite event at Chicago for 2016. Originally, the Ignite event, replacing former events like MEC, LyncCon, MMC and SPC, was said to be held for 4 consecutive years in Chicago, starting in 2015. Even at Ignite 2015, it was confirmed Ignite 2016 was going to take place in Chicago from May 9th to May 13th, 2016.

Without any reason given for this change of plans, although rumors are that either bad feedback on this year’s event or product release schedules could be the reason for changing plans, today Microsoft announced that Ignite 2016 will take place in the Georgia World Congress Center in Atlanta, from September 26th to 30rd, 2016.

This date is perhaps a bit too close for comfort to that other well-known event, the independent IT/DEV Connections which is scheduled for September 19th to 22nd, 2016 in Las Vegas. It remains to be seen if Penton – organizer of the IT/DEV Connections – moves their event or not.

Apart from potential schedule issues, though there are worse things than potentially staying half the month September in the US, it could pressure budgets for organization who want to have people attend both events, without the option to spread those investments.This is of course also true for those that are self-employed.

Other Microsoft events lined-up for 2016 are:

  • Build: Spring 2016 in San Francisco
  • Convergence: April 4-7, 2016 in New Orleans
  • WPC: July 10-14, 2016 in Toronto

You can pre-register for Ignite 2016 here. More information on Ignite is available here.

Blocking Mixed Exchange 2013/2016 DAG

Ex2013 LogoIn the RTM version of Exchange 2016, there’s an issue in that it is allows you to add Exchange 2016 Mailbox servers to Exchange 2013 Database Availability Groups, and vice-versa. As stated in the Release Notes (you do read those?), creating such a mixed version DAG is not supported. In theory, you could even jeopardize your Exchange data, as database structures from both versions are different. This action is also not prevented from the Exchange Admin Center, requiring organizations to have very strict procedures and knowledgeable Exchange administrators.

If you are worried about this situation and you want to prevent accidently adding Mailbox servers to an existing DAG consisting of members of a different Exchange version, there is a way (until this is blocked by the product itself, of course). Cmdlet Extension Agents to the rescue!

The Scripting Agent not only allows you to add additional instructions to existing Exchange cmdlets, but also to provide additional validation before cmdlets are executed. I did two short articles on Cmdlet Extension Agents’ Scripting Agent here and here, so I will skip introductions.

First you need to download a file named ScriptingAgentConfig.xml from the location below. If you already have Scripting Agents, you need to integrate the code in your existing ScriptingAgentConfig.xml files. The code checks if the server you want to add using the Add-DatabaseAvailabilityGroup cmdlet is of a different major version than one of the current DAG members.

Next, you need to copy this ScriptingAgentConfig.xml file to $ENV:ExInstallPath on every Exchange 2013 and Exchange 2016 server in your organization, e.g. C:\Program Files\Microsoft\Exchange Server\V15\Bin\CmdletExtensionAgents\ScriptingAgentConfig.xml.  To help your with this process, Exchange fellow Paul Cunningham made a small script to push this XML from the current folder to every Exchange server in your organization, PushScriptingAgentConfig.ps1.

Last step is to enable the Scripting Agent using:

Enable-CmdletExtensionAgent ‘Scripting Agent’

After distributing the scripting agent file and enabling the scripting agent, when you try to add an Exchange 2016 (version 15.1) server to an Database Availability Group consisting of Exchange 2013 Mailbox servers, using Add-DatabaseAvailabilityGroupServer, you will receive an error message:

DAGCheck

This also works vice-versa, thus when you inadvertently try to add Exchange 2013 servers to an Exchange 2016 Database Availability Group, provided you distributed the XML on the Exchange 2013 servers as well. The error is also thrown when you try to perform this action using the Exchange Admin Console.

You can download the ScriptingAgentConfig.XML for blocking Mixed Exchange 2013/2016 DAGs from the TechNet here.

Exchange 2010-2013 Migration and OAB

Ex2013 LogoLast year, Exchange fellows Andrew Higginbotham, Paul Cunningham as well as the Exchange Team reported on checking, and when necessary configuring, your Offline Address Book (OAB) in your current Exchange Server 2010 environment, prior to installing Exchange Server 2013. Not doing so could result in a complete download of the Offline Address Book created by Exchange Server 2013, titled ‘Default Offline Address List (Ex2013)’.

Today I received a report that there is a different symptom of configuration absence. In this case, the customer reported on the inability to download the offline address book, and upon further inspection the Autodiscover server did not report back on the offline address book URL to use. In other words, OAB information was absent from the Autodiscover response, and Outlook gets confused. Note that this issue was reported in Outlook 2010 after installing Exchange Server 2013 Cumulative Update 10. I’m not sure if this change in behavior was introduced in these later builds of Exchange 2013 or Outlook, but it’s still a good thing to know.

The remedy here of course is to configure any (Exchange 2010) mailbox database with unconfigured Offline Address Book setting, and point them to the default offline address book using:

Get-MailboxDatabase | Where-Object {$_.OfflineAddressBook -eq $Null} | Set-MailboxDatabase -OfflineAddressBook (Get-OfflineAddressBook | Where-Object {$_.IsDefault -eq $True})

OWA vulnerable to backdoor hack?

fudLast Update: October 10th, 2015

Yesterday, news rose of a security vulnerability in Outlook Web Access (OWA). A company called Cybereason claimed to have discovered an OWA backdoor hack of which they published in a report, “Webmail Server APT: new persistent attack methodology targeting Microsoft Outlook Web Application (OWA)” (APT stands for Advanced Persistent Threat). Supposedly, an OWA backdoor in ‘OWA Server’, the term used for Exchange Server in the report, allows a hacker to collect clear text usernames and passwords.

News sites quickly picked up the story, with catchy headlines such as:

  • New Outlook mailserver attack steals massive number of passwords (Arstechnica)
  • Microsoft OWA falls victim to password-pinching APT attack (Inquirer)
  • Potent OWA backdoor scores 11000 corporate creds from single biz (The Register)
  • Hackers Breach Microsoft OWA Server, Steal 11,000 User Passwords (SoftPedia)
  • Researchers find credential-stealing webmail server APT attack (ComputerWeekly)

The news was copied a lot without fact checking, and Microsoft felt the need to publicly make a statement: “No new security vulnerability in Outlook Web Access (OWA)”. Unfortunately that doesn’t stop media from reporting, as they are driven by a model based on page views and clicks. And such headlines most certainly will attract viewers.

Looking closer at the report, I’m inclined to think the company wanted to push for business and free publicity by spreading FUD (Fear, Uncertainty and Doubt), not uncommon in the security world. The report states that it is required to have installed (report does not disclose how) a malicious ISAPI filter on the ‘OWA Server’, without details on how this was achieved. Most likely they have used (or are referring to) the OWAAuth ISAPI filter also mentioned in a threat report (TG-3390) from Dell, dated August, 2015. The OWAAuth.dll filter authenticates users through Forms-Based Authentication against Active Directory.  Capturing and decoding client traffic is what these ISAPI filters can do, so that’s not worrying. Unfortunately, Cybereason report does not state the version of the ‘OWA Server’ or operating system. Was it current, and fully patched?

Key question is how did this filter get on the Exchange server in the first place? A properly managed environment does not allow for this type of access. So, the problem is likely not with the ‘OWA Server’ or the operating system. In a response on a blog reporting on this issue, Cybereason clarified that, “The hackers managed to obtain access to this server using stolen credentials.” Well, there is the confirmation of the real issue at hand: This is not an ‘OWA Server’ issue. The person could in theory have done anything with those stolen credentials.

In their response, the Cybereason spokesperson also stated that:

“The problem is that this server was in a very unique position. On one hand it’s completely internet facing and on the other hand, it is a focal point for the full credentials of all employees in the organization. Companies should be wary of using this server without requiring VPN (although this is usually its biggest advantage) and at the very least, require 2FA (2 factor authentication).”

I agree on the multi-factor authentication statement, especially for administrative or high profile accounts. However, claiming that VPN would prevent the issue is strange, as with most typical organizations that same set of stolen credentials would allow for setting up a VPN connection, maybe requiring some guesswork on the endpoint, but in the end enabling access to the same environment and practicing the same malicious behavior. Also, it is best practice to use a  more regular account for e-mail and connectivity, requiring another set of credentials for administrative privileges.

So, while the report may be based on a real world scenario, always have a healthy dose of common sense when reading these ‘research reports’ from companies selling security products and services. Manage your Active Directory and Exchange environment properly, use MFA for privileged accounts and remote access, and life should be good.

Other Exchange fellows also debunked the report:

Update (Sep9): If you are nevertheless still concerned, and want to do a quick scan of the currently loaded ISAPI modules on your Exchange servers, you can run the cmdlet below (be advised it’s a one-liner!). You should be able to spot ISAPI modules loaded from unusual locations or reporting an unexpected version number:

Get-ExchangeServer | ForEach-Object { Invoke-Command -ComputerName $_.Name -ScriptBlock { Get-WmiObject -Namespace 'Ro
ot\MicrosoftIISv2' -Class IISFilterSetting -Authentication 6 | ForEach-Object { (Get-Item $_.FilterPath | Select -ExpandPropert
y VersionInfo) } } } | Sort-Object PSComputerName,FileName | Format-Table -AutoSize PSComputerName, ProductVersion, FileName

isapifilt1

Update (Sep10): Cybereason provided some more details through Twitter and will publish a FAQ next week. However, more details were already given in an interview with ThreatPost (by Kaspersky Lab), in which Cybereason states that:

  • The harvesting took place over a period of months.
  • Stolen credentials were used to load a malicious, unsigned ISAPI filter, OWAAuth.dll.
  • The malicious OWAAuth.dll was residing in a non-standard location.
  • The malicious OWAAuth.dll was persistently loaded by modifying the registry.
  • Other modules were loaded, amongst them PlugX which has been around for a while, and which is the actual backdoor providing remote control mechanisms.

There are lots of similarities with the Cybereason case and Dell CTU’s TG-3390 analysis (use of PlugX, OWAAuth.dll). Since the harvesting took place over a longer period, were administrators not aware of the theft or not paying attention. Could it be that there’s a sudden increase of organizations and administrators not properly dealing with stolen passwords and password policies in general?

Meanwhile, Cybereason also claims the report, “was a malware analysis report and never about an OWA exploit”. While they have no control over the media, wording like “Cybereason Labs Reports on OWA Backdoor Attack” implies something differently. They also state one of the main concerns is, “Corporate Microsoft OWA servers are high prevalence in financial institutions”, which seems odd statement. Possibly, it’s a clue on where they hope to push business from, but from my personal experience these organizations are the most likely to have implemented multi-factor authentication and provide limited – if any at all – remote access functionality.

Knowledgebase RSS feeds

Update: Added Exchange 2016 and Skype for Business 2016 feed.

rss[1]Note: This is an update of an article from January, 2010.

Like most people I still use RSS feeds to keep track of news and updates from various sources. But did you know you can also keep track of Microsoft’s knowledgebase articles per product using RSS feeds? Great for keeping track of updates in RSS readers like Outlook or sites like Feedly, or creating triggers on sites like IFTTT (If-This-Then-That) to automatically send e-mail notifications.

Here are some RSS feeds on knowledgebase articles that might be of interest to you:

Exchange Server

Outlook

Office 365

Lync/Skype for Business

For a complete list of the knowledgebase articles RSS feeds check here.

Exchange 2016 and IM Integration

Ex2013 LogoThose configuring IM integration for OWA and Lync or Skype for Business know the drill of editing the web.config files on your Exchange servers and configuring the certificate thumbprint and Lync/SfB pool? That especially became a nuisance as after each Cumulative Update those settings needed to be reconfigured, for which I wrote a Configure-IMIntegration script.

The Exchange team has obviously listened to feedback from customers and made this setting persistent in Exchange 2016. No longer is it required to dive in those web.config files after installing each CU. Instead, you now configure these settings using the Set-Override cmdlet, which will store the setting in Active Directory.

For example:

New-SettingOverride -Name '<Description>' -Server <Server/Wildcard> -Component OwaServer -Section IMSettings -Parameters @("IMServerName=<Server/Pool FQDN>","IMCertificateThumbprint=<Certificate Thumbprint>") -Reason "<Reason>" -MinVersion "<Minimum Version To Apply To>" -MaxVersion "Maximum Version to Apply To"
Get-ExchangeDiagnosticInfo -Process Microsoft.Exchange.Directory.TopologyService -Component VariantConfiguration -Argument Refresh

For example, to configure the override for all servers with a name starting with EX16, configuring lync.contoso.com as pool FQDN and a specific thumbprint, only for Exchange builds starting at 15.1.225.42 (Exchange 2016 RTM), you could use:

New-SettingOverride -Name 'IM Integration' -Server EX16* -Component OwaServer -Section IMSettings -Parameters @("IMServerName=lync.contoso.com","IMCertificateThumbprint=12345678123412341234567812345678123126789") -Reason "Configure IM" -MinVersion "15.01.0225.42"
Get-ExchangeDiagnosticInfo -Process Microsoft.Exchange.Directory.TopologyService -Component VariantConfiguration -Argument Refresh

Finally, restart the OWA App pool to have OWA reread the new settings:

Restart-WebAppPool MSExchangeOWAApppool

Exchange 2016 goes RTM!

Ex2013 LogoUpdate (4nov2015): You can block creating mixed DAGs using Cmdlet Extension Agents, I blogged about that here.

Today, the Exchange Team reached a milestone for the On-Premises by releasing Exchange Server 2016. The official announcement contains information on new features and enhancements.The version number of Exchange 2016 RTM is 15.1.225.42. After extending it, the schema version should report 15317, and the forest and domain versions after preparing Active Directory should read 16210 and 13236, respectively.

Much of what’s new or requirements for coexistence scenarios were already announced during the release of the Exchange 2016 Preview, a little over 2 months ago. I did a write-up on that here. However, some features didn’t make it for the RTM release. For example, the feature that makes Search Indexer use Passive Database Copies for indexing, instead of copying indexes from the active copy, is to be expected in a later Cumulative Update. Also, the auto-expanding Archive feature, available in the Preview, has not made it in the RTM version.

Also make sure you read the Release Notes, which contain important information on potential issues. For example, Exchange 2016 does not prevent you from adding Exchange 2013 Mailbox servers to an Exchange 2016 Database Availability Group, or vice-versa. This ability is also not blocked by the Exchange Admin Center console. This is totally unsupported (the database structure is different), but more importantly also puts your data at risk. Just don’t.

Some links to get you started:

The first Cumulative Updated is to be expected in Q1’16.

Accompanying the launch, Microsoft also published a number of videos highlighting certain aspects or features. One of them is the ever charming Greg Taylor talking about Exchange Server 2016 – Performance, architecture and compliance updates:

Other videos from the Exchange Team and Office Garage:

2015 Microsoft MVP Award

I am proud and happy to announce I got re-awarded the Microsoft MVP Award for Exchange Server for the third year in a row:

mvp2015

MVP awards are given to individuals by Microsoft in recognition of their contributions to the technical community, such as this writing on blogs or books, presenting, forum contributions or The UC Architects podcast.

I’d like to take this opportunity to thank my readers, followers, fellow MVPs and of course the Microsoft employees that have encouraged, helped and supported me over years.

My MVP profile can be found here.