Michel de Rooij, with over 25 years of mixed consulting and automation experience with Exchange and related technologies, is a consultant for Rapid Circle. He assists organizations in their journey to and using Microsoft 365, primarily focusing on Exchange and associated technologies and automating processes using PowerShell or Graph. Michel's authorship of several Exchange books and role in the Office 365 for IT Pros author team are a testament to his knowledge. Besides writing for Practical365.com, he maintains a blog on eightwone.com with supporting scripts on GitHub. Michel has been a Microsoft MVP since 2013.
Few days ago, the Exchange Product made several announcements related to Exchange Server and its future. The overall message throughout these announcements can be interpreted as that Microsoft is publicly declaring to be committed to developing and supporting the Exchange Server product. This is especially of interest to those customers running it as part of their on-premises infrastructure. It is also assuring those that believe the road ahead was a dead end, eventually forcing them to move to Exchange Online, or look for alternatives.
The announcements made were in the area of:
Lifecycle policies remain intact for current versions of Exchange Server.
The next version of Exchange Server, also known as Exchange vNext, will move to a continuous support model, but comes with requirements.
Upgrade path for Exchange vNext.
Modern Authentication support for non-hybrid Exchange 2019 deployments.
Exchange 2019 support for TLS 1.3.
Possibility to receive pre-release builds of Exchange server through Microsoft’s TAP program.
Exchange Admin Center will receive overview section for Exchange servers update status in Exchange hybrid deployments.
HCW will allow admins to skip configuration steps.
Script to remove obsolete mitigations from EEMS.
Microsoft Exchange Conference Community Virtual Airlift (MEC) for September 13-14! (register)
Updated: 1.2 adds default ExchangeOnlineManagement cmdlets scanning and authentication options.
Since the original announcement on deprecation of Basic Authentication, organizations had time to analyze their environment which may include Exchange-related procedures and tools. These usually also contain scripts or commands, which depend on the Exchange Online Management module. A previous blog on its history and how version 2 of this module lends itself for unattended operation with certificate-based modern authentication support can be found here.
The initial release of the Exchange Online Management v2 – or EXOv2 – module offered a an additional small set of cmdlets which utilized REST-based services. Apart from the functional discrepancies, such as having to specify a property set to indicate which properties to return, the big advantage of these added commands was that they did not depend on the Windows Remote Management (WinRM) client using Basic Authentication for token exchange. Disabling Basic Authentication on WinRM client lead to messages such as:
Connecting to remote server outlook.office365.com failed with the following error message : The WinRM client cannot process the request. Basic authentication is currently disabled in the client configuration.
This dependency makes it challenging for organizations to turn off Basic Authentication altogether, or lead to problems when they did. Fast forward to the present, where the Exchange Online Management module in its current release is offering nearly all Exchange cmdlets in REST-based form, with full functional parity.
While I expect Microsoft to reach full command parity before they flick the Basic Authentication switch to off, there are also other use cases for which analyzing scripts might be helpful:
Ths initial purpose was identifying commands which require RPS (Remote PowerShell), and thus thus require WinRM Basic Authentication enabled. Because the Exchange Team did an amazing job in catching up in the recent months, only few Exchange Online cmdlets are still lacking REST support in my tenant at this moment, e.g. New-ApplicationAccessPolicy. But then again, your mileage may vary, as the recent Preview 7 module removed few UnifiedGroup related cmdlets which had issues.
New Exchange Online commands may not receive immediate REST support.
Organizations might want to cross-reference commands with scripts.
Identifying Exchange Online commands and parameters in scripts helps in determining the minimum set of permissions required to run the script.
To analyze and report on Exchange Online scripts, I created a simple script Analyze-ExoScript.ps1. This script, which is available on GitHub here, does the following:
Connect to Exchange Online using RPS and inventory the commands available. Note that this requires the UseRPSSession switch when connecting, which is only available per 2.0.6-Preview3 of the module. If your organization only runs GA versions of the module, this script cannot be used.
Connect to Exchange Online using REST and inventory the commands available. It will re-use the account used for authenticating the RPS session, which should prevent receiving another authentication dialog or MFA challenge.
Cache cmdlet information in an external file to prevent having to connect to Exchange Online for every run. The file is named EXO-CmdletInfo.xml and will be stored in the same folder as the script.
Process the script and report on the Exchange-related commands used.
Usage Calling Analyze-ExoScript is straightforward:
File is the name of one or more files which you want to analyze. Note that the script accepted pipeline output, so you can also feed it filenames using Get-ChildItem for example.
The ShowAll switch tells the script to output all found commands, not only the Exchange ones.
The switch Refresh tells the script to ignore saved command information, trigger reconnecting to Exchange Online in order to refresh the command sets.
Credential specifies the (Basic Authentication) credential to pass to Connect-ExchangeOnline.
Organization and AppId can be used to specify the tenant ID (x.onmicrosoft.com) and registered application ID to use with Connect-ExchangeOnline using Modern Authentication. This also requires one of the following:
CertificateThumbprint of the certificate to use for authentication.
CertificateFile of the file containing the certificate to use, together with CertificatePassword to specify its password.
When asked to authenticate, make sure your role has the necessary Exchange-related permissions as that will determine the Exchange Online cmdlets available to you, and consequently also the commands which Analyze-ExoScript will recognize in scripts to process.
For example, to process a script Fix-MailboxFolders.ps1, use:
The output consists of objects, which allow for further filtering:
The returned properties are:
Command is the Exchange Online command identified
Type will tell you if the command supports REST or requires RPS.
Parameters are the parameters used together with the command. This includes common parameters, which might be less usable for role assignment purposes.
Alt contains alternative REST-based cmdlet you could consider using for performance reasons, e.g. Get-EXOMailbox instead of Get-Mailbox.
File and Line are the file containing the command and on which line it is located.
AST To analyze code, I leveraged PowerShell feature called Abstract Syntax Tree, which was an interesting exploration in itself. PowerShell AST can be used to decompose PowerShell code into tokens. This is way better than simply looking for strings, and does away with having to interpret code yourself to see if something is a command, comment or just some string. AST allows for analysis of these tokens, in this case filtering on commands which are related to Exchange Online. If you want to get started on AST, check out this article, or plunge in the PowerShell SDK straightaway.
Final Words When every Exchange Online command discovered is found to be offering REST support, you can turn off Basic Authentication on the client, for example through GPO or by reconfiguring WinRM:
winrm set winrm/config/client/auth @{Basic="false"}
Only thing you might need to refactor is if and how the script connects to Exchange Online, as Basic Authentication allowed for connecting to Exchange Online using (stored) credentials for example. Examples on how to use more secure Modern Authentication-based methods to connect can be found in an earlier article here.
The Exchange PG released May updates for Exchange Server 2013, 2016 and 2019.
Note that per this cycle, Security Updates will be packaged in an executable wrapper. This should trigger the running elevated prompt, thus preventing any potential issues from simply double-clicking the .MSP file. More about the new package format, options for logging and command-line switches are mentioned in an article dedicated to the change of distribution method here.
The vulnerability addressed in the Security Updates for May is:
KB5013118 Exchange Service Host service fails after installing March 2022 security update
Important: As mentioned in the announcement, you must run /PrepareAllDomains after deploying the SU because of hardening measures. Exception is when you have multiple domains and some of them are never prepped; in that case prepare the individual domains required. Using your currently deployed binaries, run the following command, where the /IAccept switch you need to use depends on the Exchange version deployed and whether you provide diagnostics information:
Be advised that these security updates are Cumulative Update level specific. You cannot apply the update for Exchange 2019 CU12 to Exchange 2019 CU11. Also, the security update download has the same name for different Cumulative Updates, and I would suggest tagging the file name with the CU level, e.g. Exchange2019-CU10-KBXXXXXX-x64-en.msp.
Exchange servers running as part of hybrid deployment are running services, and thus need to be included in the patch cycle. If you are running Exchange 2019 CU12 Management Tools-only (for recipient management), you do not need to deploy this SU.
On a final note, as with any patch or update, I’d recommend to apply this in a test environment first, prior to implementing it in production. However, it is not recommended to wait for regular maintenance cycles when it concerns security updates, and follow a more agile approach; the ratings are an indication of the urgency.
In the announcement of the most recent set of Cumulative Updates for Exchange Server 2019 and 2016, Microsoft introduced some changes – features if you will – as well, which were received with enthusiasm. An overview of these Cumulative Updates and the features introduced was given in an earlier article. In this article however, I would like to zoom in on one of those features, which also happens to be a popular topic among customers running Exchange Hybrid deployments, “The Last Exchange Server”.
Up to Exchange 2019 CU12 (2022 H1), customers that migrated to Exchange Online were still required to leave Exchange-related components running on-premises. Even today, with all the information published around this topic, I am surprised this still surprised customers. This Exchange server running on-premises is to be used for managing recipients which have their source of authority in Active Directory, leveraging Active Directory Connect to propagate objects to Azure Active Directory and thus Exchange Online. Also, when there is a need to relay messages from applications or multi-functional devices, customers often need to have an Exchange server on-premises to accept these messages, as Exchange is the only supported mail relay product for hybrid deployments.
The Exchange Team released the quarterly half-yearly Cumulative Updates for Exchange Server 2019 and Exchange 2016. You read that right, half-yearly updates are replacing the cadence of quarterly update servicing model for Exchange Server. Effectively, this will be Exchange 2019 only, as Exchange 2016 will be out of mainstream support in H2 of 2022, and will therefor only receive Security Updates after this round. Note that this change also alters the effective ‘current’ state (n-1 or later) of your Exchange Server environment from half year to one year.
And that’s not the only good news that comes with these sets of updates. In short:
If you run Exchange 2019 in Hybrid only for the purpose of managing recipients, you can now use Exchange 2019 CU12’s Exchange Management Tools to accomplish this; no more need to have an Exchange server running just for this. More details here.
Exchange 2019 CU12 will reintroduce the Hybrid Key option. Its Hybrid Configuration Wizard supports this licensing method.
Exchange 2019 CU12 support managing the Hybrid Agent with MFA-enabled accounts.
Exchange 2019 CU12 adds support for Windows Server 2022, both for its underlying operating system, as well as deployment in environments running Windows Server 2022 Domain Controllers.
Note that while Windows Server 2022 supports TLS 1.3, Exchange 2019 CU12 on WS2022 does not yet support it. Adding support is scheduled for somewhere next year.
The supportability matrix has been updated for the supported Windows Server 2022 scenarios.
Exchange Server is now also part of Microsoft’s Bounty Program, which is an indication of continued focus for customers still running Exchange Servers on-premises.
Links to the updates as well as a description of changes and fixes are described below. The column Schema and AD indicate if the CU contains Schema (/PrepareSchema) and Active Directory (PrepareAD) changes compared to the previous CU. Refer to the Exchange Schema page for schema and related versioning information.
Apart from DST changes and the fixes mentioned below, these Cumulative Updates also contain a change which will not allow using UNC paths with several cmdlets. More information about this change and cmdlets affected can be found here: KB5014278.
Exchange 2019 CU12 fixes:
5012757 “Migration user… can’t be found” error when using Start-MigrationUser after batch migration fails
5012758 Start-MailboxAssistant is not available in Exchange Server 2019
5012760 You can’t access OWA or ECP after installing the July 2021 security update
5012761 External attendees see “Send the Response Now” although no response was requested in Exchange Server
5012762 PST creation is unexpectedly triggered again during multiple mailbox export
5012765 Email stuck in queue starting from “2022/1/1 00:01:00 UTC+0” on all Exchange on-premises servers
5012766 Transport Services fail repeatedly because of * Accepted Domain
5012768 Start-MigrationUser and Stop-MigrationUser are unavailable for on-premises Exchange Server 2019 and 2016
5012769 Invalid New Auth Certificate for servers that are not on UTC time zone
5012770 No response from public folder for users migrating to Microsoft Exchange 2019
5012772 Items are skipped at the start of a new search page request
5012773 OWAMailboxPolicy is bypassed and high resolution profile images can be uploaded
5012774 Can’t change default path for Trace log data in Exchange Server 2019 and 2016
5012775 No additional global catalog column in the address book service logs
5012776 Exchange Server 2019 help link in OWA redirects users to online help for Exchange Server 2016
5012777 Can’t find forwarded messages that contain attachments in Exchange Server 2019
5012778 Exchange Server stops responding when processing PDF files with set transport rule
5012779 Invalid new auth certificate for servers that are not on UTC time zone
5012780 Disable-Mailbox does not remove LegacyExchangeDN attribute from on-premises Exchange 2019
5012781 Exchange Server 2019 and 2016 DLP doesn’t detect Chinese resident ID card numbers
5012782 MS ExchangeDiagnostic Service causes errors during service startup and initialization in Microsoft Exchange 2019
5012783 Can’t restore data of a mailbox when LegacyDN is empty in the database
5012784 Exchange 2016 CU21 and Exchange 2019 CU10 cannot save “Custom Attributes” changes in EAC
5012785 Read Only Domain Controllers (RODCs) in other domains do not get desired permissions
5012786 Forwarded meeting appointments are blocked or considered spam
5012787 Download domains created per CVE-2021-1730 don’t support ADFS authentication in OWA
5012789 Can’t use Copy Search Results after eDiscovery & Hold search
5012790 OWA doesn’t remove the “loading” image when a message is opened in Chrome and Edge browsers
5012791 MailboxAuditLog doesn’t work in localized (non-English) environments
Exchange 2016 CU23 fixes:
5012757 “Migration user… can’t be found” error when using Start-MigrationUser after batch migration fails
5012760 You can’t access OWA or ECP after installing the July 2021 security update
5012761 External attendees see “Send the Response Now” although no response was requested in Exchange Server
5012765 Email stuck in queue starting from “2022/1/1 00:01:00 UTC+0” on all Exchange on-premises servers
5012768 Start-MigrationUser and Stop-MigrationUser are unavailable for on-premises Exchange Server 2019 and 2016
5012769 Invalid New Auth Certificate for servers that are not on UTC time zone
5012774 Can’t change default path for Trace log data in Exchange Server 2019 and 2016
5012779 Invalid new auth certificate for servers that are not on UTC time zone
5012780 Disable-Mailbox does not remove LegacyExchangeDN attribute from on-premises Exchange 2019
5012781 Exchange Server 2019 and 2016 DLP doesn’t detect Chinese resident ID card numbers
5012782 MS ExchangeDiagnostic Service causes errors during service startup and initialization in Microsoft Exchange 2019
5012783 Can’t restore data of a mailbox when LegacyDN is empty in the database
5012784 Exchange 2016 CU21 and Exchange 2019 CU10 cannot save “Custom Attributes” changes in EAC
5012786 Forwarded meeting appointments are blocked or considered spam
5012787 Download domains created per CVE-2021-1730 don’t support ADFS authentication in OWA
5012789 Can’t use Copy Search Results after eDiscovery & Hold search
5012791 MailboxAuditLog doesn’t work in localized (non-English) environments
5012829 Group metrics generation fails in multidomain environment
Notes:
If these Cumulative Updates contain schema changes compared to the Cumulative Update you currently have deployed, you need to run Setup with /PrepareSchema. If they contain Active Directory changes, you need to run /PrepareAD. Alternatively, permissions permitting, you can let Setup perform this step. Consult the Exchange schema versions page for schema and related versioning information.
When upgrading from an n-2 or earlier version of Exchange, or an early version of the .NET Framework, consult Upgrade Paths for CU’s & .NET.
Don’t forget to put the Exchange server in maintenance mode prior to updating. Regardless, setup will put the server in server-wide offline mode post-analysis, before making actual changes.
When using Exchange hybrid deployments or Exchange Online Archiving (EOA), support requires you to trail at most one version (n-1).
If you want to speed up the update process for systems without internet access, you can follow the procedure described here to disable publisher’s certificate revocation checking.
Cumulative Updates can be installed directly; no need to install RTM prior to installing Cumulative Updates.
Once upgraded, you can’t uninstall a Cumulative Update nor any of the installed Exchange server roles.
The recommended upgrade order is internet-facing, non-internet-facing servers first, followed by Edge Transports.
Caution:
As for any update, I recommend to thoroughly test updates in a test environment prior to implementing them in production. When you lack such facilities, hold out a few days and monitor the comments on the original publication or forums for any issues.
EighTwOne (821) 