MEC 2022 Sessions Downloading


Update 9/29/2022: By popular request, I modified the Get-EventSession script so it is now able to also download MEC sessions (-Event MEC). See below for details.

A quick post for those that are looking for a simple way to download the Microsoft Exchange Community (MEC) Technical Airlift 2022 sessions for offline viewing, here’s a simple way to accomplish this:

  1. Get youtube-dl.exe here. Youtube-dl is a tool to download videos or playlists from Youtube.
  2. Get aria2c.exe here. Aria2c can be used to download media using multiple streams, reducing time it takes to download video content.
  3. Put the executables from both downloads in the same folder, and, using a a (PowerShell) command prompt, run the following:

.\youtube-dl.exe -o "C:\MEC2022\%(playlist_index)s-%(title)s.mp4" --external-downloader aria2c --external-downloader-args "-x 16 -k 1M" https://www.youtube.com/playlist?list=PLxdTT6-7g--2POisC5XcDQxUXHhWsoZc9

  • “C:\MEC2022” is the folder where the downloaded files will be stored. Change when needed. For file naming, variables are used with define the name of the downloaded files using a prefix of the sequence number (from the playlist) together with the title of the video (session).
  • –external-downloader tells youtube-dl to use specified download utility (aria2c) instead of its own engine. The external-downloader-args parameters define concurrency and chunk size.
  • The last part is the URL for the MEC 2022 playlist.

9/29/2022: Alternatively, you can now use Get-EventSession (version 3.7 and up) to download MEC sessions. The script will parse the information shared through the playlist, but some usual attributes are missing, but there also some new attributes, such as likes and views. To use the script to download MEC session videos:

Get-EventSession> .\Get-EventSession.ps1 -Event MEC -DownloadFolder c:\MEC20222 -Format 22 -Speaker 'Michel de Rooij'

Few notes:

  • As there are no session codes in the YouTube metadata, session code is set to equal the playlist index.
  • Speaker names will be extracted from the description when present.
  • The session timestamp will be the upload date of the video.
  • Likes, Views and Duration are YouTube specific properties returned.

Using views and likes, you can do cool things such as get a scoreboard of the Top 10 most viewed videos from MEC playlist:

.\Get-EventSession.ps1 -Event MEC -InfoOnly | Sort Views -desc | Select -First 10 Title,SpeakerNames, Views, Likes

Note: If you do not specify format, YouTube videos will be downloaded in ‘best’ possible quality, which will be .webm by default. You can prevent this, and download 1080p movies, by specifying -Format 22.

MEC: Bringing your Exchange Scripts into the Modern Age


Yesterday, I had the pleasure of presenting at the Microsoft Exchange Conference Community Technical Airlift 2022. I talked about the challenges that organizations are facing that use Exchange scripts in their work processes or run them scheduled unattended.

Some of the challenges I mentioned, apart from the upcoming demise of Basic Authentication, and resources to methodically assess and make the necessary changes, are:

  • Get your code more secure leveraging Certificate Based Authentication, especially for scheduled tasks.
  • Get current with the most recent version of the Exchange Online Management Module for PowerShell.
  • The same exercise with regards to AzureAD when using MSOnline or AzureAD modules, and the inevitable move to the PowerShell Graph SDK.

In the end I also quickly demonstrated how much easier and secure things can be when utilizing Azure Automation, which might especially appeal to organizations that want to totally get rid of any infrastructure for running jobs.

You can watch the presentation below. All sessions are you published on YouTube, and its playlist can be accessed at aka.ms/MEC2022.

The presentation as well as the deck and script used in the live demonstration can be retrieved from GitHub. The Analyse-ExoScript used in the demo can be found on GitHub as well, or look at the accompanying blog I wrote a while ago here.

Note that during MEC, it was announced that the next GA release of the Exchange Online Management module will be version 3. This jump is likely to prevent any confusion with earlier GA and preview releases. It was said the next GA release might be as early as next week, which should be good news for organizations who’s policy it is to not run Preview software in production environments.

If you have any questions, ask them in the comments or send me a message via the contact form.

Analyzing Exchange Online scripts


Updated: 1.2 adds default ExchangeOnlineManagement cmdlets scanning and authentication options.

Since the original announcement on deprecation of Basic Authentication, organizations had time to analyze their environment which may include Exchange-related procedures and tools. These usually also contain scripts or commands, which depend on the Exchange Online Management module. A previous blog on its history and how version 2 of this module lends itself for unattended operation with certificate-based modern authentication support can be found here.

The initial release of the Exchange Online Management v2 – or EXOv2 – module offered a an additional small set of cmdlets which utilized REST-based services. Apart from the functional discrepancies, such as having to specify a property set to indicate which properties to return, the big advantage of these added commands was that they did not depend on the Windows Remote Management (WinRM) client using Basic Authentication for token exchange. Disabling Basic Authentication on WinRM client lead to messages such as:

Connecting to remote server outlook.office365.com failed with the following error message : The WinRM client cannot process the request. Basic authentication is currently disabled in the client configuration.

This dependency makes it challenging for organizations to turn off Basic Authentication altogether, or lead to problems when they did. Fast forward to the present, where the Exchange Online Management module in its current release is offering nearly all Exchange cmdlets in REST-based form, with full functional parity.

While I expect Microsoft to reach full command parity before they flick the Basic Authentication switch to off, there are also other use cases for which analyzing scripts might be helpful:

  • Ths initial purpose was identifying commands which require RPS (Remote PowerShell), and thus thus require WinRM Basic Authentication enabled. Because the Exchange Team did an amazing job in catching up in the recent months, only few Exchange Online cmdlets are still lacking REST support in my tenant at this moment, e.g. New-ApplicationAccessPolicy. But then again, your mileage may vary, as the recent Preview 7 module removed few UnifiedGroup related cmdlets which had issues.
  • New Exchange Online commands may not receive immediate REST support.
  • Organizations might want to cross-reference commands with scripts.
  • Identifying Exchange Online commands and parameters in scripts helps in determining the minimum set of permissions required to run the script.

To analyze and report on Exchange Online scripts, I created a simple script Analyze-ExoScript.ps1. This script, which is available on GitHub here, does the following:

  • Connect to Exchange Online using RPS and inventory the commands available. Note that this requires the UseRPSSession switch when connecting, which is only available per 2.0.6-Preview3 of the module. If your organization only runs GA versions of the module, this script cannot be used.
  • Connect to Exchange Online using REST and inventory the commands available. It will re-use the account used for authenticating the RPS session, which should prevent receiving another authentication dialog or MFA challenge.
  • Cache cmdlet information in an external file to prevent having to connect to Exchange Online for every run. The file is named EXO-CmdletInfo.xml and will be stored in the same folder as the script.
  • Process the script and report on the Exchange-related commands used.

Usage
Calling Analyze-ExoScript is straightforward:

.\Analyze-ExoScript.ps1 [-File <FileName[]>] [-ShowAll] [-Refresh] [-Organization <String> -AppId <String> -CertificateFile <String> [-CertificatePassword <SecureString>] -CertificateThumbprint <String>] [-Credential <SecureString>]

Where:

  • File is the name of one or more files which you want to analyze. Note that the script accepted pipeline output, so you can also feed it filenames using Get-ChildItem for example.
  • The ShowAll switch tells the script to output all found commands, not only the Exchange ones.
  • The switch Refresh tells the script to ignore saved command information, trigger reconnecting to Exchange Online in order to refresh the command sets.
  • Credential specifies the (Basic Authentication) credential to pass to Connect-ExchangeOnline.
  • Organization and AppId can be used to specify the tenant ID (x.onmicrosoft.com) and registered application ID to use with Connect-ExchangeOnline using Modern Authentication. This also requires one of the following:
    • CertificateThumbprint of the certificate to use for authentication.
    • CertificateFile of the file containing the certificate to use, together with CertificatePassword to specify its password.

When asked to authenticate, make sure your role has the necessary Exchange-related permissions as that will determine the Exchange Online cmdlets available to you, and consequently also the commands which Analyze-ExoScript will recognize in scripts to process.

For example, to process a script Fix-MailboxFolders.ps1, use:

.\Analyze-ExoScript.ps1 -File .\Fix-MailboxFolders.ps1

The script can accept files via the pipeline. For example, to process multiple scripts use something like:

Get-ChildItem -Path C:\temp*.ps1 | Analyze-ExoScript.ps1

The output consists of objects, which allow for further filtering:

The returned properties are:

  • Command is the Exchange Online command identified
  • Type will tell you if the command supports REST or requires RPS.
  • Parameters are the parameters used together with the command. This includes common parameters, which might be less usable for role assignment purposes.
  • Alt contains alternative REST-based cmdlet you could consider using for performance reasons, e.g. Get-EXOMailbox instead of Get-Mailbox.
  • File and Line are the file containing the command and on which line it is located.

AST
To analyze code, I leveraged PowerShell feature called Abstract Syntax Tree, which was an interesting exploration in itself. PowerShell AST can be used to decompose PowerShell code into tokens. This is way better than simply looking for strings, and does away with having to interpret code yourself to see if something is a command, comment or just some string. AST allows for analysis of these tokens, in this case filtering on commands which are related to Exchange Online. If you want to get started on AST, check out this article, or plunge in the PowerShell SDK straightaway.

Final Words
When every Exchange Online command discovered is found to be offering REST support, you can turn off Basic Authentication on the client, for example through GPO or by reconfiguring WinRM:

winrm set winrm/config/client/auth @{Basic="false"}

Only thing you might need to refactor is if and how the script connects to Exchange Online, as Basic Authentication allowed for connecting to Exchange Online using (stored) credentials for example. Examples on how to use more secure Modern Authentication-based methods to connect can be found in an earlier article here.

Managed AutoResponder Notifications


A long overdue blog on a solution which was created per request of an Exchange fellow. The original scenario is an organization spinning off, thereby changing their primary e-mail domain. They wanted to inform relations and partners using the old e-mail addresses of this change. However, this solution might also be helpful to organizations after merger and acquisitions or rebranding.

For the sake of the example, let us suppose Fabrikam was acquired by Contoso. Targeted mailboxes are migrated from the Fabrikam tenant to the Contoso tenant. Fabrikam will contain Mail-Enabled Users forwarding messages to their Contoso mailbox counterparts. The migrated mailboxes now hosted in Contoso will be configured to send notifications to senders which sent mail to the old Fabrikam address.

While organizations can resort to 3rd party tools to set up an Exchange Transport Rule with a generic message, a little script might also do the job, offering a more granular and controlled solution.

Solution
The scripted solution is available on GitHub here. It will configure an Inbox rule on the targeted Exchange mailbox(es), which would be the new/migrated mailbox. The rule will trigger when messages land in the inbox which were sent to a specific e-mail address, i.e. the previous e-mail address. It will send out an automatic response, which can consist of a custom subject, message body with an optional embedded image. The configuration of the response is defined in an customizable external XML file:

<?xml version="1.0" encoding="ISO-8859-1"?>  
<config>
  <rule>Contoso Autoresponder</rule>
  <subject>Please update your email recipient to Contoso</subject>
  <body>Dear Sender,

Thank you for your message. Fabrikam is now a whole Contoso subsidiary, and the fabrikam.com e-mail address will change to contoso.com. Your e-mail was forwarded to my new e-mail address. 

Please update contact information, distribution lists, etc. to update [OldMail] e-mail references with my new [Identity] e-mail address. 
 
[logo] 
The Contoso Corporation is a multinational business with its headquarters in Paris.</body>
  <logo>Contoso.png</logo>
</config>

The elements that can be used in the config are:

  • <rule> defines the name of the Inbox rule to be created. When the script runs, it will update any existing previously created inbox rules of the same name.
  • <subject> defines the subject of the response.
  • <body> defines the message body. You can use the following macros here:
    • [OldMail] will get replaced with the original e-mail address.
    • [Identity] will get replaced with the new e-mail address.
    • [Logo] will get replaced with the embedded image.
  • Optionally, <logo> refers to the file name of the image to embed.

Requirements
To run the script, you need the following:

  • Exchange Server 2013 SP1 or later, or Exchange Online.
  • Exchange Web Services (EWS) Managed API 2.21 or later (how to, NuGet package exchange.webservices.managed.api).
  • When using modern authentication or OAuth2, the MSAL library is required (NuGet package Microsoft.Identity.Client). Also, you need to have registered an App in Azure Active Directory with sufficient permissions (e.g. full_access_as_app). After registering the app, Tenant ID, Application ID and certificate or secret is what you need to use with the script to run successfully.
  • In addition to installing the NuGet packages, you can also store their DLLs in the same folder as the script for portability.

Usage
The syntax to run the script is as follows:

.\Set-AutoResponderNotification.ps1 -Identity <Identity> -TemplateFile <File> [-TenantId <TenantId> -ClientId <ClientId> [-CertificateThumbprint <ThumbPrint>] [-CertificateFile <File> -CertificatePassword <SecureString>] [-Secret <SecureString>]] [-Credentials <PSCredential>] [-Clear] [-Overwrite] [-Impersonation] [-TrustAll]

The available parameters and switches are as follows:

  • Identity specifies one or more e-mail addresses of mailboxes to process. Identity supports the pipeline (see examples).
  • OldMail specifies one or more old e-mail addresses to use when configuring the autoresponder message. When specifying multiple identities,
    the number of OldMail entries need to match the number of identities.
  • Server specifies the Exchange Web Services endpoint, for example outlook.office365.com for Exchange Online. When omitted, Autodiscover will be used.
  • Impersonation to use impersonation when accessing the mailbox. When using modern authentication, impersonation is mandatory.
  • TrustAll to accept all certificates including self-signed certificates.
  • TenantId specifies the ID of the Tenant when using a mailbox hosted in Exchange Online.
  • ClientId to specify the Application ID of the registered application in Azure Active Directory.
  • Credentials to specify the Basic Authentication credentials for on-premises usage or against Exchange Online when modern authentication is not an option.
  • CertificateThumbprint is the thumbprint of the certificate to use for modern authentication. The certificate with the public key needs to stored with the registered application for authentication. The certificate with the private key should be present in the local certificate store.
  • CertificateFile and CertificatePassword can be used to specify a certificate file to use. The file should contain the private key; the password protecting the certificate file can be specified using CertificatePassword as a secure string.
  • Secret can be used to specify the secret to authenticate using the registered application. The secret needs to be provided as a secure string.
  • Template
    specifies the XML template file to use when configuring or clearing the autoresponder inbox rule. The format has explained above.
  • Clear specifies if any you want to remove the inbox rules with name specified in the template. Use this when you want to remove autoresponder rules from mailboxes. When using Clear, you don’t need to specify OldMail.
  • Overwrite specifies if any existing inbox rules with name specified in the template should be overwritten. When omitted, the script will skip processing
    mailboxes with inbox rules with conflicting names. Use this when you want to configure the autoresponder only on mailboxes which do not have the rule.

Note that usage of the Verbose, Confirm and WhatIf parameters are supported.

Examples
Nothing more explanatory than an example. When we want to configure the autoresponder on a single mailbox, we can use something like:

image

Here, the autoresponder will get configured on the specified mailbox, triggering when mail has been sent to michel@fabrikam.com using the configuration defined in template.xml. Modern authentication will be used for authentication, using variables for Tenant, Client and in this case secret.

When you want to configure autoresponder for multiple mailboxes, you can for example use a CSV file. It needs to contain two elements which will be passed through pipeline, Identity and OldMail:

Identity,OldMail
philip@contoso.com,p.mortimer@fabrikam.com
francis@contoso.com,f.blake@fabrikam.com

After defining the response in a file template.xml, we can use this CSV to configure inbox rules on the Contoso mailboxes identified by Identity, triggering when mail is sent to their OldMail addresses:

Import-CSV -Path Users.csv | .\Set-AutoResponderNotification.ps1 -Server outlook.office365.com -Impersonation -TemplateFile .\Template.xml -TenantId <TenantId> -ClientId <ClientId> -Overwrite -CertificateThumbprint <Thumbprint>

What will happen is that for every set of Identity and OldMail, the mailbox specified by Identity will be configured with an inbox rule. When an existing rule is found, which is determined using the <rule> element in the XML template, it will get overwritten. The specified certificate will be picked from the local certificate store to authenticate against Tenant with TenantId, as application specified by ClientId.

Note that when the user opens Manage Rules & Alerts in Outlook, the configured inbox rule will be visible. This allows the user to remove it when it is no longer required. After a certain period, administrators can centrally remove these rules as well running the script using the Clear switch.

image

And finally, an example of how this may looks to senders when they receive an autoresponse message, using the sample configuration from the beginning of this article.

image

Application Access Policy
When using the script with modern authentication, you can leverage features such as conditional access to set boundaries for script usage. Also, you can configure application access policies to scope the registered app (script) to a subset of mailboxes. To accomplish this, assign an ApplicationAccessPolicy to the app.

To be more convenient in managing permissions, you can define a distribution group and assign the Application Access Policy with group scope (PolicyScopeGroupId). You then only need to add/remove members as you need to configure mailboxes.

More information about Application Access Policies here. Note that the permissions mentioned to not include full_access_as_app as permission, but it works.

EWS, why not Graph?
Microsoft already announced back in 2018 that development on Exchange Web Services will halt in and focus will shift to Graph. As part of this move, a more recent statement announced deprecation of some least-used API per March 2022, stimulating organizations to switch to using Graph. However, the organization looking for a solution wished for an automatic response with HTML as well as an embedded logo. Because of this, I had to use a template as a reply action.

But where Exchange Web Services supports reply with a template, Graph does not offer this functionality. So, until there is more feature parity between Exchange Web Services and Graph, or EWS goes completely out of service, solutions may still be forced to have a look at EWS for certain tasks.

Unarchiving Mailbox Items


With the introduction of Exchange 2010 at the end of 2009, a native feature was added to Exchange Server for which organizations required 3rd party products before that. The feature which I am talking about is Exchange’s Personal Archives, Online Archives, or In-Place Archiving as it is called nowadays.

Background
Archives were introduced at a time when Office 365 was in its early days, many organizations were running Exchange on-premises with mailbox quotas as bandwidth and storage were limited or relatively expensive. It was up to end users to make sure their mailbox remained within its limits, either by removing either old items, large items or just move them out of their mailbox to those pesky .PST files.

Archives introduced benefits such as lowering disk footprint by taking infrequently used items out of the primary mailbox (which then could only synchronize in full) to the archive, which is basically an additional mailbox for long-term storage. Exchange’s built-in Messaging Records Management (MRM) through retention policies and tags can be used for automatic moving of older items to the archive.

Archives also come with few downsides, especially in the early days. Most notably are perhaps clients not supporting archives at all, or searches not spanning both mailbox and archive. Also, and this is not to be underestimated, end users do not always grasp the concept of archives and the impact on the tasks and tools they use. It’s not uncommon to see people panicking about “missing data” in service tickets, only to discover their “missing data” was moved to their archive by the company retention policy after some digging.

In recent years, I have seen archives becoming less relevant, and organizations adopting the large mailbox concept in favor of lean and mean mailboxes with archives. There are still exceptions of course, usually in the form of substantial – usually shared – mailboxes. For those, staying with Exchange Online archives – and when needed auto-expanding archives – is usually still an option due to the different type of mailbox interaction, or to circumvent Exchange’s storage limitations or Outlook for Desktop’s synchronizing of offline cache files before issues might be seen. The maximum number of items per folder is such a limit, however these have been raised or done away with in recent years. Non-stubbing 3rd party archive solutions taking data out of Exchange can also be a option.

The Problem
Switching to the large mailbox concept creates a problem for those organizations that have already enabled in-place archives for their end users: How to get that data back from those archives to the primary mailbox. While retention policies can move data in opposite direction, there is no such thing as a reverse-retention policy. Also, not every organization would like to instruct end users to unarchive this contents themselves, as it is prone to failure, blocks Outlook for Desktop from doing anything else and might result in abandoned operations which limits future actions as moves are still happening in the background.

When investigating a possible solution I found that there is no other way to accomplish this, than to programmatically move contents from the in-place archive to the primary mailbox. While there is a ‘archive’ operation for mailbox items (which moves it to the assigned Archive folder, not the in-place archive) there is no other single API call to perform this task. Also, the solution would have to use Exchange Web Services, as a limitation in Microsoft Graph makes it incapable of moving messages between multiple mailboxes.

Note: If I overlooked something in this area, please let me know.

Solution
To help organizations accomplish this task, I wrote a PowerShell script which requires the following:

  • Exchange Server 2013 SP1 or later, or Exchange Online.
  • Exchange Web Services (EWS) Managed API 2.21 or later (how to, NuGet package exchange.webservices.managed.api).
  • When using OAuth, the MSAL library is required (NuGet package Microsoft.Identity.Client). Also, you need to have registered an App in Azure Active Directory; the Tenant ID, Application ID and certificate or secret is what you need to provide the script with to operate successfully.
  • In addition to installing the NuGet packages, you can also store the DLLs in the same folder as the script.

Note: Untested with Primary mailboxes on-premises and Exchange Online Archives.

The script Invoke-Unarchive will perform the following tasks:

  • Invoke-Unarchive will move contents from the in-place archive back to the primary mailbox.
  • The most optimal operation will be chosen:
    • Folders present in archive but not in primary mailbox will be moved in one operation.
    • Folders present in archive and primary mailbox are merged. Items in those folders are moved in batches.
    • The same steps are repeated recursively per folder for the whole archive.
  • If, after moving, a folder in the archive is empty, and it is not a non-removable well-known folder, it will be removed.
  • Optionally, Invoke-Unarchive can also move contents stored in the Recoverable Items from the archive to the primary mailbox.
  • Invoke-Unarchive will handle throttling, either by honoring the returned back-off period or by adding delays between operations.
  • Moving items is asynchronous, and Invoke-Unarchive needs to wait for Exchange to complete the previous move to folder X before it can move the next set of items to folder X.

Do not forget to reassign retention policies causing archival, or you might have the run the script again at later moment.

Syntax
The parameters to call Invoke-Unarchive.ps1 are:

  • Identity to specify one or more mailboxes to unarchive items for.
  • Server to specify the FQDN of the Client Access Server to use. When omitted, Autodiscover will be used.
  • IncludeRecoverableItems to instruct the script to process deletions stored in the Recoverable Items as well.
  • Impersonation to use impersonation when accessing the mailbox. When using modern authentication (OAuth), impersonation is mandatory.
  • Force to force moving of items without prompting.
  • NoProgressBar to prevent progress status.
  • TrustAll to accept all certificates including self-signed certificates.
  • TenantId specifies the ID of the Tenant when using a mailbox hosted in Exchange Online.
  • ClientId to specify the Application ID of the registered application in Azure Active Directory.
  • Credentials to specify the Basic Authentication credentials for on-premises usage or against Exchange Online when OAuth is not an option.
  • CertificateThumbprint is the thumbprint of the certificate to use for OAuth. The certificate with the public key needs to stored with the registered application for authentication. The certificate with the private key should be present in the local certificate store.
  • CertificateFile and CertificatePassword to specify the file of the certificate to use. The file shoud contain the private key, and the password to unlock the file can be specified using CertificatePassword.
  • Secret can be used to specify the secret to authenticate using the registered application.

Note that Credentials, CertificateThumbprint, CertificateFile + CertificatePassword and Secret are mutually exclusive.

Example
Below shows an example run against a test-mailbox using modern authentication (OAuth). The common parameter Verbose is used to display additional output.

.\Invoke-Unarchive.ps1 -Identity michel@myexchangelabs.com -Server outlook.office365.com -Impersonation -Secret <Secret> -TenantId <Tenant> -ClientId <AppId> -Verbose
image

You can find the script on GitHub here.

Final Notes
The EWS operation – especially moving items – is not necessarily slow, but against Exchange Online processing large archives can take considerable amount of time due to throttling. When moving a significant number of items using Outlook for Desktop, you will likely run into Outlook abandoning the operation after which you need to wait for Exchange to finish pending moves before you can continue with this task. Using the script, you can take away this unarchiving task from end users by running the operation in the background in one or multiple runs.

EWS.WebServices.Managed.Api


A short blog on the EWS Managed API and using the latest version with scripts leveraging Exchange Web Services (EWS), such as my Remove-DuplicateItems script. The installable EWS Managed API library was last updated in 2014 (version 2.2, reports as v15.0.913.22), and there have been few enhancements since then. These are included in the EWS.WebServices.Managed.Api package, which carries version 2.2.1.2.

Although this library was last updated in 2019, you might still need it to successfully run EWS scripts. The EWS.WebServices.Managed.Api package supports some Exchange Web Services calls which are not supported in the 2.2 version, and may lead to error messages like Exception calling “FindFolders” with “2” argument(s) or other messages related to the (number of) arguments. This may be an indication a particular call was used to one of the EWS functions, but which is not supported by the installed EWS Managed API library. In those cases, installing this updated library might help.

The library is published on NUGet as a package. To install the package, we first need to register NuGet as a Package Source:

Register-PackageSource -provider NuGet -name nugetRepository -location https://www.nuget.org/api/v2

Next, install the package from the newly defined NuGet source:

Install-Package Exchange.WebServices.Managed.Api

The package installs by default under C:\Program Files\PackageManagement\NuGet\Packages. I have updated my scripts to include this location when searching for the required Microsoft.Exchange.WebServices.dll as well. Alternatively, you can copy this DLL from the ..\Exchange.WebServices.Managed.Api.<Version>\lib\net35 folder to the location where the script resides. When running my EWS-based scripts in Verbose mode, it will report as version 2.2.1.0.

Hopefully this blog will potentially save you some time troubleshooting, and myself answering some support messages. Enjoy!

Exchange Online Management using EXOv2 module


Exchange2019Logo

Update (3sep2022) Updated reflect Azure AD app roles.

Early June, Microsoft released a new PowerShell module
for managing Exchange Online. This module got announced at Ignite 2019 already, but it took
few months between going into preview end of last year before it finally reached Generally Available status. Usage of this module offers substantial improvements over the existing methods to connect to Exchange Online using Powershell, such as:

  • Leveraging the PowerShell module ecosystem to install and update the module. This as opposed to the click-to-run Microsoft Exchange Online Powershell Module or connecting through PowerShell remoting.
  • Support for Multi-Factor Authentication. This is something which the click-to-run module also offers but is not available when using PowerShell remoting.
  • Robustness. Existing sessions could easily timeout when you took a short break from the console. Or worse, your script could terminate in the middle of execution. This required you to reconnect or forced you to add resilience to your scripts by handling with these disconnects from the back end. The cmdlets of the EXOv2 module should be more robust and resilient.
  • Introduction of the Graph API support, which should show improvements in terms of speed. Microsoft indicated an 4-8 times improvement should be achievable, but your mileage may vary depending on the operation.
  • Support for PowerShell 6/7, core, and non-Windows operating systems is coming.

Exchange Online Management v2 module

The module has been baptized EXOv2 to indicate a major change compared to the click-to-run module (hereafter referred to as EXOv1), and also because it uses Graph API, just like the AzureAD v2 module. The module is available in the PowerShell Gallery, and installation is straightforward. Open a PowerShell 5.1 or later session in elevated mode and run:

Install-Module ExchangeOnlineManagement

The EXOv2 cmdlets which are REST-based and and leverage Graph API have their nouns prefixed with ‘EXO’, e.g. Get-EXOMailbox. Currently, there are 9 EXO cmdlets in the GA module, as well as few additional ones (more on those later). The regular commands such get Get-Mailbox are available as well after connecting to Exchange Online. This is similar behavior to the EXOV1 module, e.g.

Connect-ExchangeOnline [-UserPrincipalName <UPN>]

When required, satisfy the Multi-Factor Authentication logon process, and you
are done. Be advised that the EXOv2 module also supports
Delegated Access
Permissions
(DAP), allowing partners to connect to customer tenants by
specifying
-DelegatedOrganization <mycustomer.onmicrosoft.com>
when connecting.

Also note that apart from the EXO cmdlets, the current module also offers few other interesting commands and helper functions apart from the ones for housekeeping:

  • Connect-IPPSSession to connect to Security & Compliance center or Exchange Online Protection, depending on licensing. This command was also available in EXOv1.
  • Get-UserBriefingConfig & Set-UserBriefingConfig. These are a bit out of context, as these commands allow you to enable or disable the Cortana Briefing for users.
  • IsCloudShellEnvironment indicates if you are running from PowerShell or Azure Cloud Shell, which might be useful in scripts to determine the current context.

The EXOv2 cmdlets and their regular equivalents are shown in the table below:

EXO v1 or Remote
PowerShell
EXO v2
Get-MailboxGet-EXOMailbox
Get-MailboxFolderPermissionGet-EXOMailboxFolderPermission
Get-CASMailboxGet-EXOCASMailbox
Get-MailboxFolderStatisticsGet-EXOMailboxFolderStatistics
Get-MailboxPermissionGet-EXOMailboxPermission
Get-MobileDeviceStatisticsGet-EXOMobileDeviceStatistics
Get-RecipientGet-EXORecipient
Get-RecipientPermissionGet-EXORecipientPermission

What you might notice is the absence of any Set-EXO* cmdlets. This is true, and there is no word yet on if and when Set cmdlets will be introduced. That said, the biggest speed gain is often in bulk retrieval of data, not so much in altering one or more attributes. Until then, do not
de
spair though, as you can pipe output of the EXO cmdlets to their regular cmdlet,
e.g.

Get-EXOMailbox michel | Set-Mailbox -EmailAddresses @{Add='michel@myexchangelabs.com'}

This construction will also provide the additional benefit of parallel processing of objects as they pass through the pipeline, but more on that later.

Now comes another thing you should be aware of, and that is that these EXOv2 cmdlets might not use the same parameter sets as their v1 equivalent. Simply said, you cannot perform a simple Find and Replace operation in your script replacing Get-Mailbox with Get-EXOMailbox to start enjoying benefits of the new module.

When running a cmdlet like Get-EXOmailbox, you might notice that it returns only a subset of the attributes you might expect. Similar to what Properties does for Active Directory module, the EXOv2 module requires you to specify the individual Properties to return. Alternatively, you can use PropertySets to select a predefined set of attributes. For example, Get-EXOMailbox supports PropertySets such as All, Minimum (default), Policy, Quota and Retention to name a few. When needed, you can combine PropertySets, so something like the following is possible:

Get-EXOMailbox -Identity michel -PropertySets Quota,Policy

A small note on the PropertySet All: Just like Get-ADUser .. -Properties
*
is considered bad practice as you can impact resource usage and usually return more than what you need, using -PropertySets All
for every call is also a bad idea. All is convenient, but make sure you only return the data you need. Be a good person.

To see which EXOv2 cmdlets support PropertySets, use:

(Get-Command -Noun EXO* -Module ExchangeOnlineManagement).Where{$_.Parameters.propertySets}

Performance

Now, I suppose we want to get an indication of the performance enhancements by comparing EXOv2 and equivalent operation using v1 cmdlets. In this simple example we are returning quota information for some 50.000 mailboxes:

In this case, it is not the 4-8x improvement, but more than twice as fast is significant nonetheless. Especially if you are running interactively. To see the impact of parallel processing in the pipeline, we run the following:

As shown, there is a substantial increase in performance, but of course your mileage may vary depending on things like the number of objects, the attributes you require, and any filtering
applied.
Note that the PropertySet StatisticsSeed used in the example is a very minimal set of attributes which you can use if you only wish the refer to the objects, such as userPrincipalName, primarySmtpAddress and externalDirectoryObjectID.

Speaking of filtering, one would expect that server-side filtering (-Filter) would show an improvement in terms of speed over client-side filtering (Where), as filtering at the source is far more efficient in terms of result set and data to send over. However, it seems that due to the nature of a shared environment, sending superfluous data over the wire is less of a penalty than local filtering. Of course, your mileage may also vary here, so experiment what works best for your situation. Also, not every attribute is supported for filtering with these EXO cmdlets, which lies in how Graph exposes data. More information on that here.

When your session times out or disconnects, you will see that the module tries to reconnect your session; something which you would have to programmatically solve for the v1 module or regular remote PowerShell:

Certificate-based Authentication

Exchange administrators often have a requirement to run unattended scripts against Exchange Online, for example scheduled reports or as part of another process. In the past, this lead to setups where service accounts and stored credentials were used. Later this was improved by the ability to apply Conditional Access to limit these logons to on-premises infrastructure.

The problem with Multi-Factor Authentication is that it requires interaction with end-user to approve the sign-on. Of course, while your token is still valid, you can easily (re)connect to Exchange Online just by providing the Username Principal Name, which will reuse the token if it didn’t expire. But all in all, these solutions are high maintenance, and far from ideal from a security perspective.

Here comes certificate-based authentication, which is supported in version 2.0.3 and up of the EXOv2 module. In short, certificate-based authentication allows you to log on to Exchange Online using:

  • PowerShell
  • EXOv2 module
  • A (self-signed) certificate containing private key
  • Enterprise App registration in Azure Active Directory which contains the public key of this certificate, and proper assigned Azure AD role(s).

Note: Enterprise app registration may require Azure AD P1/P2 license.

To install the EXOv2 2.0.3 version of the module (preview at time of writing), use:

Install-Module ExchangeOnlineManagement -AllowPrerelease

Note that it might complain if you have the GA version of the module installed, in which case you need to uninstall the GA module first, or you can install them side-by-side by specifying -Force.

Next, we need to create a self-signed certificate. To accomplish this, we can use the script published here. To create the certificate, simply use:

.\Create-SelfSignedCertificate.ps1 -CommonName 'EXOv2' -StartDate 7/30/2020 -EndDate 7/30/2021

Note that you need to provide a password to protect the PFX file containing the private key. Also do not forget to import the PFX in your local certificate store. When importing, you can mark the certificate as non-exportable, which prevents admins to transfer the certificate to other systems.

Import-PfxCertificate -CertStoreLocation Cert:\CurrentUser\My -FilePath .\EXOv2.pfx -Password (Read-Host -AsSecureString)

After importing, you can check for the certificate’s presence using:

Get-ChildItem Cert:\CurrentUser\My | Where {$_.Subject -eq 'CN=EXOv2'}

The Subject should be the CommonName you used when generating the certificate. The thumbprint of our certificate is 49A4A73B4696718676770834BCD534DE35030D2C. We are going to use this later on to connect.

Now we need to set things up in Azure Active Directory:

  1. Open up the Azure Active Directory Portal, and navigate to Active Directory.
  2. Select App registrations, and click New registration.
  3. Give the App a meaningful Name, and select Accounts in this organizational directory only. Set Redirect URI to Web and leave the URL blank. Then, click Register.

    clip_image013[4]

    Note that our App has been assigned an Application (Client) ID. Make note of this value, as we will need it to connect later on.
  4. Next, we need to configure the App permissions. Select API permissions. User.Read should show up as default. Click Add a permission, and locate Office 365 Exchange Online from the APIs my organization uses tab. Select Application permissions, and in the next screen expand Exchange and check Exchange.ManageAsApp. We are done here, so click Add permissions.
  5. Only thing left now is to Grant admin consent, which can be done by clicking Grant admin consent for <tenant>. When done, the Status column for Exchange.ManageAsApp permission should have changed to Granted for <tenant>.

    API Permissions
  6. Now we need to associate this App with out certificate. Select Certificates & Secrets, and click Upload certificate. Pick the certificate file which we generated earlier, and select Add.

    clip_image017[4]
  7. Last step is to assign the App one of the built-in Azure AD roles. Go to the Azure Active Directory blade, and select Roles and administrators. To manage Exchange Online using PowerShell, you need to assign the Exchange Administrator role; for Security & Compliance, you can assign the Compliance Administrator role.

    Select the desired role(s), and click Add assignments in the assignments overview screen. Note that when picking security principals, the App might not show up initially, and typing its first few letters might help. Click Add to assign the role.

    clip_image019[4]
    Note that the UserName mentioned in the overview is the Application ID.

Now we are done configuring the back end, we can look again at connecting. This should now be as simple as running:

Connect-ExchangeOnline -CertificateThumbprint '49A4A73B4696718676770834BCD534DE35030D2C' -AppId '0d3f8f4c-34fb-4a22-8466-80fd7379593b' -Organization '<tenant>.onmicrosoft.com'

Where:

  • CertificateThumbprint is the thumbprint of the self-signed certificate you created earlier.
  • AppID is the Application (Client) ID of the registered App.
  • <tenant>.onmicrosoft.com the initial domain name of your tenant.

Note that you can also connect specifying the CertificateFile instead of Thumbprint, but then you need to provide the password as well via CertificatePassword. Having the certificate in the certificate store of the administrator account or account running the task and just specifying the thumbprint is more convenient and requires zero interaction.

If all steps above were followed correctly, you should now be connected to Exchange Online, without any MFA interaction.

A final note is that Connect-IPPSSession mentioned earlier does not support certificate-base authentication.

What about other Workloads

You can use the same certificate-based authentication to connect to several other workloads as well. That is, provided you have installed the required PowerShell module and the Azure AD role you assigned to the Application has adequate permissions. You can use the commands below to connect to these workloads. A small note that the commands to connect may use a different parameter names for AppId or Organization, e.g. AppId, ApplicationId or ClientId and Organization and TenantId are same things in the examples below.

AzureAD (2.x or Preview)

Connect-AzureAD -CertificateThumbprint '49A4A73B4696718676770834BCD534DE35030D2C' -ApplicationId '0d3f8f4c-34fb-4a22-8466-80fd7379593b' -TenantId '<tenant>.onmicrosoft.com'

MicrosoftTeams (GA or Test)

Connect-Microsoftteams -CertificateThumbprint '49A4A73B4696718676770834BCD534DE35030D2C' -ApplicationId '0d3f8f4c-34fb-4a22-8466-80fd7379593b' -TenantId '<tenant>.onmicrosoft.com'

Microsoft Graph

Connect-Graph -CertificateThumbprint '49A4A73B4696718676770834BCD534DE35030D2C' -ClientId '0d3f8f4c-34fb-4a22-8466-80fd7379593b' -TenantId '<tenant>.onmicrosoft.com'


Audit

The logons which are performed in the context of the Application are viewable in the Azure Sign-Ins at https://aka.ms/iam/rtsp

Note that this view is currently in preview, and there might be a slight delay before logon shows up.

Final Notes

It would be nice if there would be a way to incorporate Exchange granular Role-Based Access Control model into the permissions model. Granting Apps only the built-in Azure AD roles is somewhat limiting, and it would be nice to restrict accounts in only being able to run the cmdlets and parameters they need to use.

When running Exchange cmdlets, you will find these in the audit log but with the <tenant>\AppID as UserName. Therefore, best thing to do is to use a single App registration for each individual administrator or process, instead of using a single App registration and multiple certificates.

And finally, it would be nice if the various teams would align their cmdlet and parameter naming schemes for consistency.

 

Comparing Sets of Cmdlets


powershellWith the speed of development in Office 365, it is sometimes hard to track which changes have been made to your tenant. Of course, there is the roadmap and message board which you can use to keep up to date, but those are in general high level descriptions. Sometimes you may want to see what are the changes at the cmdlet level in your tenant, between tenants, or Azure Active Directory module. And there is also the occasional gem in the form of a yet undocumented cmdlet or parameter which could hint at upcoming features.

For this purpose I have created a simple script which has two purposes:

  1. Export information on the current cmdlets available through Exchange Online or Azure Active Directory.
  2. Compare two sets of exported information, and display changes in a readable way.

The script is in PowerShell (of course), and is called Compare-Cmdlets.ps1. To export information, you need to be already connected to either Exchange Online or Azure Active Directory (or both).

To export cmdlet information, use:

.\Compare-Cmdlets.ps1 –Export

For Exchange Online and Azure Active Directory, separate export files are created. The files are prefixed with a timestamp and postfixed with the Exchange Online build or Azure Active Directory module version, e.g. 201803121814-ExchangeOnline-15.20.548.21.xml or 201803121815-AzureAD-2.0.0.137.xml.

After a few days/week, or when connected to another tenant or using a new Azure Active Directory PowerShell module, run the export again. You will now have 2 sets of Exchange Online or Azure Active Directory cmdlets, which you can compare using the following sample syntax:

Compare-Cmdlets.ps1 -ReferenceCmds .\201801222108-ExchangeOnline-15.20.428.21.xml -DifferenceCmds .\201803120926-ExchangeOnline-15.20.548.21.xml

image

A progress bar is shown as comparison might take a minute. When the script has finished checking the two sets, you will see output indicating changes in cmdlets, parameters or switches, e.g.

image

Download
You can find the script on the TechNet Gallery or GitHub.

Exchange admins & PowerShell


imageMany people I encounter in the field of Office 365 or Exchange have an infrastructure background. That is, they know a lot about their product(s), how to make it work (or don’t), how to manage, deploy or troubleshoot, etcetera.

Then there is, the let us call it, the reality check of the cloud era, with a roller coaster of cloud-originating developments. This requires a different management focus for these products, resulting in products architected for scale, and introducing configuration and management instruments primarily designed to be ready for automation and operate on scale as well. PowerShell support in Microsoft products is such an instrument.

The introduction of PowerShell required folks with an infrastructure background to develop a new skill: instead of clicking buttons in an interface, they should also become a PowerShell practitioner. Not necessarily wizard level, but at least they need to know their way around when managing their environment using PowerShell, reading and interpreting scripts provided by Microsoft or other vendors prior to usage, or even make changes to make those scripts fit for their own environment.

Writing scripts is another matter. This requires a tad different mindset, where you make repeatable tasks repeatable (time-saving), less prone to error (job-saving), and reusable by your coworkers or even the community who may need to perform the same task. Of course, everybody also expects your scripts to be generic (no hard-coded elements), robust and resilient, adding 90% more code (a bit exaggerated, but you get the idea).

What most of administrators struggle with, is making the connection between managing the product using PowerShell, and how to start using PowerShell to develop their own set of scripts or tools to automate tasks their environment. Administrators wanting to learn such skills will usually find is great books about the product, and great books on learning (generic) PowerShell. Of course, existing scripts found using their favorite search engine can also be a great starting point, provided somebody already developed it for the task you are trying to accomplish.

With the Exchange Server 2016 administrator in mind, Exchange fellows Dave Stork and Damian Scoles tried to bridge that gap with their book, Practical PowerShell: Exchange Server 2016. It uses some practical Exchange-themed examples, how to approach the problem, and how to go from running a few cmdlets in sequence to developing small scripts which operate against one or multiple servers. Also, while this book aims at the on-premises Exchange administrators, the skills learned are not lost when the organization moves to Exchange Online as these scripting skills are compatible.

Knowing how difficult it can be to transfer knowledge to paper from my own experience, I think Dave & Damian did a respectable job. The timing of the book release is also interesting, as the product which introduced PowerShell to so many of us, Exchange Server 2007, is going End of Life soon, on April 2011, 2017 to be exact. Realizing PowerShell has been around now for so many years, there is no excuse to get your PowerShell skills going, unless you want to share the faith of dinosaurs.

More information on the book, including a sample chapter, is available at https://www.practicalpowershell.com. You can also order the book from Amazon here.

Results Install-Exchange15 survey


stats chartA short blog on a small survey I’ve been running for some time now on the usage of Install-Exchange15, the PowerShell script for fully automated deployment of Exchange 2013 or Exchange 2016.

I started the survey because I was curious on a few things:

  • How the script is used; do folks use it for deploying in lab environments, or also actual production environments.
  • What Exchange versions are deployed; only current ones (n-2 at most, i.e. lagging 2 Cumulative Update generations at most), or also older versions.
  • What operating systems are used to deploy Exchange using this script.

The second and last items are of most interest, as keeping backward compatibility in the script, for example like deploying Exchange Server 2013 SP1 on Windows Servers 2008, requires keeping a lot of ‘legacy code’ in there.

Fortunately, the survey shows many of you use the script to deploy recent Exchange builds on current operating systems. So, in time, you will see support for older builds and operating systems being removed, making the script more lean and mean as well.

Now, on to the results:

In what environments do you use the script to deploy Exchange?

Lab

Production

Yes

86%

72%

No

14%

28%

Do you use Install-Exchange15.ps1 for previous (N-2 or older) Exchange 2013/2016 builds?

Yes 28%
No 72%

On which Operating Systems do you deploy Exchange 2013/2016? (multiple options possible)

Windows Server 2008 0%
Windows Server 2008 R2 18%
Windows Server 2012 18%
Windows Server 2012 R2 100%
Windows Server 2016 8%

Finally, a summary of the feedback and requests send in by respondents through the open comments section:

  • Installation on Windows Server 2016. The survey was created before Windows Server 2016 was supported, so we used the feedback given on people deploying on WS2016 in the above results.
  • In general, positive feedback on having this script for automated deployment, as well as the SCP feature.
  • Request for having a GUI to create the answer file.
  • Request to having the option to configure the virtual directories after installation. However, the script allows for inserting custom (Exchange) cmdlets in its post-configure phase.
  • Request to output cause of failed Exchange setup to the screen. That however, is something I wouldn’t recommend; the Exchange setup log files contain the details.
  • Request to have some sort of visible clue if the installation was successful or not.