Removing Duplicate Items from a Mailbox

powershellLatest version: 2.06, October 4th, 2021

For those involved with Exchange migration projects or managing Exchange environments, at some point you probably have experienced the situation where people ended up with duplicate items in their mailbox. Duplicate items can be caused by many things, but most common are:

  • Synchronization tools or plug-in. Entries from the mailbox are treated as new entries and as a consequence are added to the mailbox when synchronizing information back to the mailbox, creating duplicates. In the past, I’ve seen this happening with Nokia PC Suite and Google Apps Sync for example;
  • Importing existing data. Accidental import from – for example – a PST file to a mailbox  can lead to duplicate entries.


When looking for a solution, you’ll probably encounter MSKB299349, “How to remove duplicate imported items in Outlook”. This article describes a manual procedure to remove duplicates entries from your calendar, contacts, inbox or other folders. Not a very helpful and labor intensive.

When continuing your search, you’ll find lots (I mean lots!) of tools and Outlook add-ins, like Vaita’s DIR or MAPILab’s Duplicate Remover. Not all this software is free (some even require payment per duplicate removal of appointments, contacts or e-mail) and some might not even work (MAPI-based tools may not work against Exchange 2013).

When you finally have selected a tool, in most cases they require installation of a piece of software and someone to perform the removal process using the tool or Outlook with add-in. When you’re an Apple shop you’ll require different tools, unless you’re running a Windows desktop somewhere (I’ll just pretend I didn’t hear you saying ‘Why don’t you install the tool on the Exchange server’).

Wouldn’t it be nice if you’d have a PowerShell script you can conveniently run from any workstation (or server) with PowerShell installed, removing those duplicate items from a user’s mailbox remotely? If the answer is yes, the Remove-DuplicateItems.ps1 script may be something for you.

Using the Remove-DuplicateItems.p1 script requires Exchange Web Services (EWS) Managed API and for OAuth authentication the Microsoft Authentication Library (MSAL) libraries. You can install these packages from NuGet, or place their DLL’s in the same folder as the script. For an example of how to install EWS.Managed.Api from NuGet, see this article; for MSAL follow the same process but with the package titled ‘Microsoft.Identity.Client’.

Also take notice that since you’ll be processing user mailboxes, you’ll need to have full mailbox access or impersonation permissions when using Basic Authentication; the latter is preferred. For details on how to configure impersonation for Exchange On-Premises or Office 365, see this blog post. Using a registered app with OAuth is always through Impersonation.

The script Remove-DuplicateItems.ps1 uses the following syntax:

Remove-DuplicateItems.ps1 [[-Identity] ] [[-Type] ] [-Retain ] [-Server ] [-Impersonation] [-DeleteMode ] [-Credentials ] [-Mode ] [-MailboxOnly] [-ArchiveOnly] [-IncludeFolders <String[]>] [-ExcludeFolders <String[]>] [-PriorityFolders <String[]>] [-NoSize] [-CleanupMode] [-NoProgressBar] [-Force] [-WhatIf] [-Confirm] [-Secret] [-CertificateThumbprint] [-CertificateFile] [-CertificatePassword] [-TenantId] [-ClientId] [-TrustAll]

A quick walk-through on the parameters and switches:

  • Identity is the e-mail address or name of the mailbox to process. If name is used, it is matched against cn/SAMAccountname/email address of local AD.
  • Type determines what folders are checked for duplicates. Valid options are Mail, Calendar, Contacts, Tasks, Notes or All (Default).
  • Retain determines which item to retain by comparing last modification times. Valid options are Newest (default) or Oldest.
  • Server is the name of the Client Access Server to access for Exchange Web Services. When omitted, the script will attempt to use Autodiscover.
  • When the Impersonation switch is specified, impersonation will be used for mailbox access, otherwise the current user context will be used.
  • DeleteMode specifies how to remove messages. Possible values are HardDelete (permanently deleted), SoftDelete (use dumpster, default) or MoveToDeletedItems (move to Deleted Items folder).
  • Mode determines how items are matched. Options are Quick, which uses PidTagSearchKey and is the default mode, or Full which uses a predefined set of attributes to match items, depending on the item class:
ItemClass Criteria
Contacts File As, First Name, Last Name, Company Name, Business Phone, Mobile Phone, Home Phone, Size
Distribution List FileAs, Number of Members, Size
Calendar Subject, Location, Start & End Date, Size
Task Subject, Start Date, Due Date, Status, Size
Note Contents, Color, Size
Mail Subject, Internet Message ID, DateTimeSent, DateTimeReceived, Sender, Size
Other Subject, DateTimeReceived
  • MailboxOnly specifies you only want to process the primary mailbox of specified users. You als need to use this parameter  when running against mailboxes on Exchange Server 2007.
  • ArchiveOnly specifies you only want to process personal archives of specified users.
  • IncludeFolders specifies one or more names of folder(s) to include, e.g. ‘Projects’. You can use wildcards around or at the end to include folders containing or starting with this string, e.g. ‘Projects*’ or ‘*Project*’. To match folders and subfolders, add a trailing \*, e.g. Projects\*. This will include folders named Projects and all subfolders. To match from the top of the structure, prepend using ‘\’. Matching is case-insensitive.
  • ExcludeFolders specifies one or more folder(s) to exclude. Usage of wildcards and well-known folders identical to IncludeFolders.
    Note that ExcludeFolders criteria overrule IncludeFolders when matching folders.
  • CleanupMode specifies to cleanup duplicates per folder (Folder, default), the whole mailbox (Mailbox), or multiple mailboxes (MultiMailbox, identities specified using Identity). The first unique item encountered will be retained. For Mailbox-level cleanup, PriorityFolders can be used to give priority to retaining items in specified folders before those found in other folders.
  • PriorityFolders specifies which folders have priority over other folders, identifying items in these folders first when using MailboxWide mode. Usage of wildcards and well-known folders is identical to IncludeFolders.
  • NoSize tells script to not use size to match items in Full mode.
  • NoProgressBar prevents displaying a progress bar as folders and items are being processed.
  • Report reports individual items detected as duplicate. Can be used together with WhatIf to perform pre-analysis.
  • TrustAll can be used to accept all certificates, e.g. self-signed certificates or when accessing Exchange using endpoint with a different certificate.

For authentication, the following parameters are available:

  • Credentials specifies credentials to use for Basic Authentication.
  • TenantId specifies the identity of the Tenant (OAuth)
  • ClientId specifies the Id of the registered application (OAuth).
  • CertificateThumbprint specifies the thumbprint of the certificate from personal store to use for authentication (OAuth).
  • CertificateFile specifies the external certificate file (pfx) to use for authentication (OAuth). This certificate needs to contain a private key; the registered application needs to contain the certificate’s public key.
  • CertificatePassword optionally specifies the password to use with the certificate file (OAuth).
  • Secret specifies the secret to use with the application (OAuth).

Few notes:

  • When MoveToDeletedItems is specified, the Deleted Items folder will be skipped;
  • When Type is omitted or set to All, all folders are scanned, including folders like Conversation History, RSS Feeds, etc.;
  • When Quick mode is used and PidTagSearchKey is missing or inaccessible, search will fall back to Full mode;
  • For more info on PidTagSearchKey, see Note that PidTagSearchKey will have duplicate values for copied objects.
  • You need to specify MailboxOnly when running against mailboxes on Exchange Server 2007 as the Exchange 2010 personal archive options in EWSare not support in Exchange 2007 mode.

Well-Known Folders
For IncludeFolders, ExcludeFolders and PriorityFolders, you can also use well-known folders using this format: #WellKnownFolderName#, e.g. #Inbox#. Supported are #Calendar#, #Contacts#, #Inbox#, #Notes#, #SentItems#, #Tasks#, #JunkEmail# and #DeletedItems#. The script uses the currently configured Well-Known Folder of the mailbox to be processed.

Here are some examples of using pattern matching in IncludeFolders, ExcludeFolders or PriorityFolders, based on the following tree structure:

+ TopFolderA
  + FolderA
    + SubFolderA
    + SubFolderB
  + FolderB
+ TopFolderB

The following filters will match folders from the above structure:

Filter Matches
FolderA \TopFolderA\FolderA, \TopFolderB\FolderA
Folder* \TopFolderA\FolderA, \TopFolderA\FolderB, \TopFolderA\FolderA\SubFolderA, \TopFolderA\FolderA\SubFolderB
FolderA\*Folder* \TopFolderA\FolderA\SubFolderA, \TopFolderA\FolderA\SubFolderB
\*FolderA\* \TopFolderA, \TopFolderA\FolderA, \TopFolderA\FolderB, \TopFolderA\FolderA\SubFolderA, \TopFolderA\FolderA\SubFolderB, \TopFolderB\FolderA
\*\FolderA \TopFolderA\FolderA, \TopFolderB\FolderA

So, suppose you want to remove  duplicate Appointments from the calendar of mailbox migtester1 using attribute matching, moving duplicate items to the DeletedItems, using Impersonation and you want to generate extra output using Verbose. In such case, you could use the following cmdlet:

Remove-DuplicateItems.ps1 -Identity migtester1 -Type Calendar -Impersonation -DeleteMode MoveToDeletedItems -Mode Full -Verbose


Alternative, you can use an e-mail address and specify credentials.  This allows the script to run against mailboxes in Office 365, for example:

Remove-DuplicateItems.ps1 -Identity -Type Mail -DeleteMode MoveToDeletedItems -Mode Full -Credentials (Get-Credential) -Retain Oldest

A more complex example using IncludeFolders, ExcludeFolders and PriorityFolders:

$Credentials= Get-Credential
 .\Remove-DuplicateItems.ps1 -Mailbox -Server -Credentials $Credentials -IncludeFolders '#Inbox#\*','\Projects\*' -ExcludeFolders 'Keep Out' -PriorityFolders '*Important*' -CleanupMode Mailbox

This will remove duplicate items from the specified mailbox in Office365, using the following options:

  • Fixed Server FQDN – bypassing AutoDiscover.
  • Limits operation against the Well-Known Inbox folder, top Projects folder, and all of their subfolders.
  • Excluding any folder named Keep Out.
  • Duplicates are checked over the whole mailbox.
  • Priority is given to folders containing the word Important, causing items in
    those folders to be kept over items in other folders when duplicates are found.

In case you want to process multiple mailboxes, you can use a CSV file which needs to contain the Identity field. An example of how the CSV could look:


The cmdlet could then be something like:

Import-CSV users.csv1 | Remove-DuplicateItems.ps1 ..

The script is available on GitHub here.

Feedback is welcomed through the comments. If you got scripting suggestions or questions, do not hesitate using the contact form.

354 thoughts on “Removing Duplicate Items from a Mailbox

  1. Pingback: NeWay Technologies – Weekly Newsletter #48 – June 20, 2013 | NeWay

  2. Pingback: NeWay Technologies – Weekly Newsletter #48 – June 21, 2013 | NeWay

  3. First, this is a VERY cool script, so thank you. But I can’t get it to work, and am hoping you can help. I get an error stating “Cant access mailbox information store”
    i’m running this from a management server (not an Exchange box). The account I’m using is an Exchange org admin. Thanks!!


      • Thank you, I had to grant impersonation rights. At first I granted Full Access, but I guess it didn’t work. I’m running Exchange 2010, in case anyone else has this issue. Thanks again for a VERY useful script!


  4. It looks like its running fine, but don’t find the dubs that i made by copying mails, why is this? and why does it not run on custom folders?


    • Depending on the way you copied, it could be the “duplicates” have a different PidTagSearchKey. Try running it in Full mode instead of the default Quick Mode (-Mode), in which case it’ll use a predefined set of attributes depending on the type of items to find matching items (see article for which attributes are checked). By default it will scan all folder types (-Type), looking for duplicates on a per folder basis.


      • Thanks, I found the solution. it does not work fully before i configure the impersonation rights. Im running it from a workstation with web services 2.0 against a exchange 2010 installation. Wonderful script you have made!


  5. I receive an error with the script when installed on Exchange 2012. I do have the dll in the folder with the ps1, and have the EWS installed. I downloaded again and verified the download.

    …..Remove-DuplicateItems.ps1:349 char:156
    + … ationsMode and $AffectedTaskOccurences:$AffectedTaskOccurences”
    + ~~~~~~~~~~~~~~~~~~~~~~~~
    Variable reference is not valid. ‘:’ was not followed by a valid variable name character. Consider using ${} to delimit the name.
    + CategoryInfo : ParserError: (:) [], ParseException
    + FullyQualifiedErrorId : InvalidVariableReferenceWithDrive


  6. I get the following error :
    You cannot call a method on a null-valued expression.
    At C:\psscripts\Remove-DuplicateItems.ps1:271 char:82
    + $key= $Item.DateTimeReceived.ToString <<<< ()
    + CategoryInfo : InvalidOperation: (ToString:String) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull


  7. Hello,

    When I run the script, everything looks fine, except for the fact that is removes 0 items. It’s for shure that the mailbox where I try it on is full of duplicate items. This is the best way I found on the net to remove duplicate items, but doesn’t work for me. Please Help!


  8. Hi
    Wonderful script!
    I´m doing it on exchange 2013 and has change some of the rows to get it working.
    1. First probably a typo:
    … and $AffectedTaskOccurences:$AffectedTaskOccurences”
    … and AffectedTaskOccurences:$AffectedTaskOccurences”
    2. Change all text: Exchange2007_sp1 to Exchange2013
    3. On 2 occurences replace:
    $ItemIds= [activator]::createinstance(([type]’System.Collections.Generic.List`1′).makegenerictype([Microsoft.Exchange.WebServices.Data.ItemId]))
    $type = (“System.Collections.Generic.List”+’`’+”1”) -as “Type”
    $type = $type.MakeGenericType(“Microsoft.Exchange.WebServices.Data.ItemId” -as “Type”)
    $Itemids = [Activator]::CreateInstance($type)

    Now the script works! But still getting an error at the end:
    C:\root\Remove-DuplicateItems.ps1 : Problem processing mailbox it (
    At line:1 char:1
    + .\Remove-DuplicateItems.ps1 -mailbox it -type all -impersonation -deletemode har …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Remove-DuplicateItems.ps1
    I Think it fails the error check at the end but cannot solve it.
    $RootFolder= [Microsoft.Exchange.WebServices.Data.Folder]::Bind( $EwsService, [Microsoft.Exchange.WebServices.Data.WellknownFolderName]::MsgFolderRoot)
    If(! ( Process-Mailbox $RootFolder)) {…

    Any ideas?



    • 1) Thanks. Display only, won’t affect script;
      2) No. Defines minimum level of EWS support required.
      3) PowerShell v3 thing – fixed in Fix-MailboxFolder script, still need to propagate to this script.

      Regarding the message, make sure you have permissions (impersonation – preferred – or otherwise).


      • Hi

        The script is doing its job with previous changes. duplicates are deleted.
        I have the impersonation rights.
        But I get another error message in the middle of the process to:

        Process-Mailbox : Detected problem removing items (loop)
        At C:\root\Remove-DuplicateItems.ps1:458 char:16
        + If(! ( Process-Mailbox $RootFolder)) {
        + ~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
        + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Process-Mailbox

        This leads me to the lines:
        $RootFolder= [Microsoft.Exchange.WebServices.Data.Folder]::Bind( $EwsService, [Microsoft.Exchange.WebServices.Data.WellknownFolderName]::MsgFolderRoot)
        If(! ( Process-Mailbox $RootFolder)) {…

        Any surgestions?


        • The script checks if the number of items to be deleted are in actually deleted. However, with larger batches Exchange may still be busy deleting items (we hand off a list of items to delete) so numbers don’t add up. In such cases I’d recommend using smaller batches ($MaxItemBatchSize= 100). I may parametrize this in future versions.


          • Hi!
            Thanks for the Quick reply.
            This script is so cool!!!!

            had to change $type to $typ cause it was used earlier in script:
            $typ = (“System.Collections.Generic.List”+’`’+”1″) -as “Type”
            $typ = $typ.MakeGenericType(“Microsoft.Exchange.WebServices.Data.ItemId” -as “Type”)
            $Itemids = [Activator]::CreateInstance($typ)

            After this and raising the max items all seems fine.



  9. Is there anyway to use this with office365/exchange online, or will it only work with an in house exchange

    As we have just moved to office365 from another hosted exchange and using the built in Microsoft tools to move the mail have managed to end up with every mailbox with duplicate items



    • Hi I have tried to use this against office365 but get the following error

      PS C:\Users\cnl_admin\Desktop> .\Remove-DuplicateItems.ps1 -Mailbox jamiec -Type
      Mail -DeleteMode MoveToDeletedItems -Mode Full -Verbose

      Security warning
      Run only scripts that you trust. While scripts from the internet can be useful
      this script can potentially harm your computer. Do you want to run
      [D] Do not run [R] Run once [S] Suspend [?] Help (default is “D”): r
      VERBOSE: Loading C:\Program Files\Microsoft\Exchange\Web
      Processing mailbox jamiec
      VERBOSE: Set to trust all certificates
      VERBOSE: Looking up EWS URL using Autodiscover for
      C:\Users\cnl_admin\Desktop\Remove-DuplicateItems.ps1 : A positional parameter
      cannot be found that accepts argument ‘Exception calling “AutodiscoverUrl”
      with “2” argument(s): “The Autodiscover service couldn’t be located.”‘.
      At line:1 char:1
      + .\Remove-DuplicateItems.ps1 -Mailbox jamiec -Type Mail -DeleteMode
      MoveToDeleted …
      + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      + CategoryInfo : InvalidArgument: (:) [Remove-DuplicateItems.ps1]
      , ParameterBindingException
      + FullyQualifiedErrorId : PositionalParameterNotFound,Remove-DuplicateItem


        • Hi, I get exactly the same error:

          A positional parameter cannot be found that accepts argument ‘Exception calling “AutodiscoverUrl” with “2” argument(s): “The Autodiscover service couldn’t be located.”‘.

          and have seen your reply which refers to version 1.3. Where can I find the latest version of your PS script?

          Kind regards,
          Sven Hoelbling


          • Sorry, I noticed that I already had version 1.4, but I still get this error.
            Can you let me please know what I’m missing?
            All mailboxes are located in Office 365, but it is a hybrid deployment with on-premise Exchange 2013 SP 1 servers. When I point the script to these servers using the -server switch the script returns:
            Can’t access mailbox information store
            Shouldn’t it use autodiscover as Outlook does (in a hybrid depoyment the DNS entry for autodiscover points to the same onpremise CAS server)

            Kind regards,
            Sven Hoelbling


            • 1) You will find the latest version of the script here.
              2) Hybrid shouldn’t matter. If your internal DNS is set up correctly for Autodiscover (or using domain-joined client?). Can’t access mailbox store when overriding the EWS web service URL might possibly indicate a permission issue (or other blockade, e.g. firewall).


              • Hi,

                I have version 1.4 but it still exhibits the same problem. I’ve confirmed my client has full connectivity to O365 and that auto discovery is working fine for the mailbox. Has any progress been made resolving this problem?



                • Just ran it succesfully against a hybrid deployment with Autodiscover and Impersonation. To exclude (Autodiscover) redirection from the equation, use “-Server” to address cloud-based mailboxes.


  10. Hi Michel

    I have now run this with great success in Exchange 2013. But the largest mailboxes seems to fail with the largest folders.
    inbox and sent items in the largest mailboxes get:

    Exception calling “FindItems” with “2” argument(s): “Unexpected end of file while parsing Name has occurred. Line 1, po
    sition 1244584.”
    At C:\root\Remove-DuplicateItems.ps1:253 char:25
    + $ItemSearchResults= $EwsService.FindItems($SubFolder.Id, …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : XmlException

    Have tried the Maxfolder and Maxitem but no success.
    Could it be some limit i EWS config?

    I also get an error deleteing in a few folders:
    Process-Mailbox : Detected problem removing items (loop)
    At C:\root\Remove-DuplicateItems.ps1:459 char:16
    + If(! ( Process-Mailbox $RootFolder)) {
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Process-Mailbox
    any ideas?



    • 1) Do you have a number so I can try reproduce the problem?
      2) I check if the number of items returned have decreased by the number of items removed. However, it’s asynchronous and Exchange – especially with large counts – may still be removing things in in the background. Suggest using low MaxItems.


      • Hi
        Not so happy today!
        10% of the mailboxes has not deleted their duplicates. Still there
        Instead they have deleted all mail from all folders between a week back and a year back
        Cannot se a pattern. but it´s kaos!
        😦 MrT


  11. Would REALLY be great if this worked against Archive mailboxes. I am about to undertake a project where I have a huge migration with import from .pst, we had hoped to go to archive mailbox, but cant, it will take an extra step to do this. But testing on regular Exchange 2013 mailbox works like a champ. Thanks


  12. Hi Micheal,

    I testing your script but this send me a this message….
    [PS] C:\Documents and Settings\acweinzierll\Desktop>Remove-DuplicateItems.ps1 -Mailbox nrgonzalezd -Type contacts -Impersonation -DeleteMode MoveToDeletedItems -Mode Full -Verbose
    The term ‘Remove-DuplicateItems.ps1’ is not recognized as a cmdlet, function, operable program, or script file. Verify
    the term and try again.
    At line:1 char:26
    + Remove-DuplicateItems.ps1 <<<< -Mailbox nrgonzalezd -Type contacts -Impersonation -DeleteMode MoveToDeletedItems -Mo
    de Full -Verbose

    I have Exchange 2007

    Can YOu help Me… Thanks…


  13. VERIFIED TO WORK WELL ON EXCHANGE 2010 SP3! – 3/10/2014

    Very powerful script! Once I installed Exchange Web Services (EWS) Managed API 1.2 and assigned the Hygiene Management role to the account I was running the scripts under I had no problems!

    Be sure to follow the formatting provided in the examples as I noticed weird issues when I specified a different order of the arguments than that of the examples.

    Below is what worked best for my environment!!!

    ./Remove-DuplicateItems.ps1 -Mailbox -johnsmith -Type Mail -Impersonation -DeleteMode HardDelete -Mode Full -Verbose

    **Used to remove duplicates on imported mail that orginated from Google Apps via Outlook 2007 and GASMO**


  14. Hello! Congrats for your script. It’s really fantastic!! I’ve added impersonation rights, everything runs OK on most of mailboxes. But I have a little problem in one of them. The scripts crashes when starts analyzin the folder “Sent Items”. I supose there is a problem on this folder, otherwise the script wouldn’t run in the other mailboxes. I’m wondering if there is any possibility to define which folders analyze or better, add a folder exception… Is it possible? If not, have you ever been in the same situation, same problem? Many thanks!!


  15. Hi Michel,
    Thank you for your wonderful script.

    I used the script to remove duplicate contacts from an Exchange 2013 SP1 folder where most of them were duplicates (800 unique contacts, 138,000 duplicates)

    I got this error:
    Process-Mailbox : Detected problem removing items (loop)
    At .\Remove-DuplicateItems.ps1:470 char:16
    + If(! ( Process-Mailbox $RootFolder)) {
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Process-Mailbox

    The lines that cause this error start at line 369:
    $OldOffSet= $ItemView.Offset
    $ItemView.Offset+= $ItemSearchResults.Items.Count – $ItemIds.Count
    If( $OldOffset -eq $ItemView.Offset) {
    Write-Error “Detected problem removing items (loop)”
    $ProcessingOK= $False
    For me, most of the time the number of items deleted ($ItemIds.Count) was the same as the number of items in the batch ($ItemSearchResults.Items.Count), so this test condition was usually matching, and halting the script.
    I removed the test, and Remove-DuplicateItems.ps1 ran ok.

    Are there any side effects that might occur without this test?

    PS: I also tested increasing the $MaxItemBatchSize to 1000 and it seemed to work ok.


  16. Hi Michel
    I would like to add Appointment RequiredAttendees to the mode=full duplicate-detection $key.
    Does FindItems populate RequiredAttendees with the [Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties view?
    If not, what changes are needed to retrieve the RequiredAttendees property?
    Thanks for your help,


  17. Hello,

    We are using this script to try to remove some duplicate contacts. We have a sync process that pushes out all of our contacts from a public folder to everyone’s contacts. The tool we use to sync isn’t great, so we have dupes. The script works great, but it is removing the most recently added contact and I’d like to remove the old one. Any ideas on modifiying the script to make this happen?


  18. Pingback: Script Updates | EighTwOne (821)

  19. Hi Michel,

    Great script and great that it now also works with EO / O365.
    Only issue I have now is that it doesnt search through folders created by the user.

    Any suggestions?



  20. Hi,

    I’m trying to run this script against an Office365 tenant that I’ve just migrated our customer to as the migration process created quite a lot of duplicate items (which we expected it might).

    I’ve added the Impersonation rights and full access mailbox permissions to the user i’m running the script as but I just can get it to execute?

    autodiscovering the EWS url seems to fail (even though autodiscover works fine when setting up outlook profiles etc.) and i’m not sure what format to enter the -server parameter as? I’ve tried the generic, I’ve tried the mailbox users exchange (as per the outlook profile) but nothing seems to work.

    Any suggestions???

    Many thanks in advance 🙂


  21. I keep getting a “Can’t access mailbox information store At Line:1 char:28 + .\remove-duplicateitems.ps1 <<<< -Mailbox

    I've got application impersonation set on the account


      • I’ve tried both, same results

        This is the script that I’m running….

        .\Remove-DuplicateItems.ps1 -Mailbox aliasname -Type Mail -Impersonation -DeleteMode MoveToDeletedItems -Mode Quick -Verbose

        This is the output that I get…..

        VERBOSE: Loading C:\temp\duplicates\Microsoft.Exchange.WebServices.dll
        VERBOSE: Loading module from path ‘C:\temp\duplicates\Microsoft.Exchange.WebServices.dll’.
        Processing mailbox aliasname (
        VERBOSE: Set to trust all certificates
        VERBOSE: Using for impersonation
        VERBOSE: Looking up EWS URL using Autodiscover for
        VERBOSE: Using EWS on CAS https://servername
        VERBOSE: DeleteMode is MoveToDeletedItems
        VERBOSE: Processing Mail items
        C:\temp\duplicates\Remove-DuplicateItems.ps1 : Can’t access mailbox information store
        At line:1 char:28
        + .\Remove-DuplicateItems.ps1 <<<< -Mailbox aliasname -Type Mail -Impersonation -DeleteMode MoveToDeleted -Mode
        Quick -MailboxOnly -Verbose
        + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
        + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Remove-DuplicateItems.ps1


  22. Hello Michel,

    Situation long version: I have a mailbox of 14gb and with a ton of folders/e-mails, something went wrong with the migration and in some folders there are six duplicate items.

    I ran this script with the following command: .\Remove-DuplicateItems.ps1 -Mailbox -Type Mail -DeleteMode MoveToDeletedItems -Mode Full -Credentials (Get-Credential) -Retain Oldest -Verbose

    – I then get a popup asking me for credentials and I enter the credentials of the specific mailbox and it started moving over 13k duplicated to the deleted items folder.

    After it was done doing this, i checked a couple of folders and it showed some still have duplicate items. Checking the log it shows it never told there are duplicates in there.

    Processing folder FOLDER NAME
    Procession folder FOLDER NAME

    when it did remove duplicates it shows:

    Processing folder FOLDER NAME
    Removing # items from FOLDER NAME

    I tried running the script again cause maybe that might help, but It removed 0 items the second time around.

    Any tips on getting the rest of the duplicates removed?


      • I’am having the same problem, the script It’s working fine, but he cant detect duplicated items, I did the debug and i saw two items that suppost to be equals they have a the field “modified” diferentes.

        I’am trying to cleam those duplicated items because when i migrated e-mails from Gmail to O365 he just duplicated e-mails to diferente folders, the same email are inbox, stared, important, all e-mails etc…


        • Feel free to modify the matching mechanism if the Search Key or predefined set of attributes to match don’t meet your requirements. However, make sure to test and backup, as hard deletes (if chosen) are hard deletes, and there is no backup in Office 365 🙂


  23. Have you ever thought about writing an ebook or guest authoring on other blogs?
    I have a blog centered on the same ideas you discuss and would
    really like to have you share some stories/information. I know
    my viewers would value your work. If you are even remotely interested, feel free to send me an email.


  24. Hi Michel:

    we have a problem w/CRM synching – it is creating dup contacts in Outlook and we cannot figure out why. Is there a way to have your script be triggered when the dup comes in and delete it right then and there? thanks


    • No and yes. No, the script is not created with that in mind. Yes, you can use EWS to subscribe to a folder and trigger actions when changes occur. I have something similar in the pipeline, now I only need to find some time or clone myself 🙂


      • Let me ask this – I am hoping to use your script to remove dups from our Outlook clients, but is there a way your script can be modified to remove dups from the CRM database master from which the dups originated?
        thank you.


  25. Hi, I also seem to be having some permission errors, but can’t seem to find the problem…..
    Working on an exchange 2007 server.
    Installed the EWS API and gave the user full control and impersonation rights:
    Get-Mailbox mailboxname | Add-ADPermission –User ADM_user –ExtendedRights ms-Exch-EPI-May-Impersonate
    Then ran the script:
    Remove-DuplicateItems.ps1 -Mailbox mailboxname -Type Calendar -Impersonation -DeleteMode MoveToDeletedItems -Mode Full -Verbose -mailboxonly -server EWSServer

    Error is:
    .\Remove-DuplicateItems.ps1 : Can’t access mailbox
    information store
    At line:1 char:28
    + .\Remove-DuplicateItems.ps1 <<<< -Mailbox mailboxname -Type Calendar -Imperson
    ation -DeleteMode MoveToDeletedItems -Mode Full -Verbose -mailboxonly -server
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio

    Could you please help?


  26. This script is a thing of beauty. I work in a shop heavy with Mac users. At some point in the past many of them were hit but the Outlook for Mac/iOS/sync/duplicate bug. I had one user with over 1.5 million items. I used ODIR and it worked well but it was a very tedious process to migrate messages into folders in batches <10,000 messages so that ODIR could work without crashing. This script just deleted 580146 duplicates from one mailbox while I slept. Thank you!

    We do have a 3rd party archiving solution that changes the appends the IPM.NOTE class with an extension. In this case there are 2 of each message as the sizes and classes are different. Is there an easy add to the script to compare 2 separate message classes? IPM.Note and IPM.Note.ExShortcut.owa and only keep the shortcut version?


  27. Pingback: Removing Duplicate Items from a Mailbox - League Team

  28. Hi Michel,
    The script sounds great, but I just can’t get past the “Can’t access mailbox information store” error.

    I’m on an Exchange 2007 environment here and I need to remove duplicate contacts (50k+) from a single mailbox (i.e. “contoso\user”)

    What I’ve done:

    1. created new account “contoso\me-tst” with no mailbox and no administrative privileges

    2. granted ms-exch-epi-impersonation permission on all CAS servers to “contoso\me-tst”

    Get-ExchangeServer | where {$_.IsClientAccessServer -eq $TRUE} | ForEach-Object {Add-ADPermission -Identity $_.distinguishedname -User (Get-User -Identity me-tst | select-object).identity -extendedRight ms-Exch-EPI-Impersonation}

    3. granted ms-exxch-epi-may-impersonate an fullaccess permission on “contoso\user” mailbox to “contoso\me-tst”

    Add-ADPermission -Identity user -User me-tst -extendedRight ms-Exch-EPI-May-Impersonate

    4. “$a = get-credential” -> “contoso\me-tst” & password

    5. executed script -> scrip fails with “Can’t access mailbox information store” / “(401)Unauthorized”

    [PS] D:\Scripting>.\Remove-DuplicateItems.ps1 -mailbox user -type contacts -retain Oldest -server CAS001.contoso.local -deletemode harddelete -credentials $a -mailboxonly -mode full -impersonation -verbose
    VERBOSE: Loading C:\Program Files\Microsoft\Exchange\Web Services\2.2\\Microsoft.Exchange.WebServices.dll
    Processing mailbox user (user@contoso.local)
    VERBOSE: Set to trust all certificates
    VERBOSE: Using credentials contoso\me-tst
    VERBOSE: Using user@contoso.local for impersonation
    VERBOSE: Using Exchange Web Services URL https://CAS001.contoso.local/EWS/Exchange.asmx
    VERBOSE: DeleteMode is harddelete
    VERBOSE: Processing contacts items
    D:\Scripting\Remove-DuplicateItems.ps1 : Can’t access mailbox information store At line:1 char:28
    + .\Remove-DuplicateItems.ps1 <<> Exception calling “Bind” with “2” argument(s): “The request failed. The remote server returned an error: (401) Unauthorized.”

    – Permissions are there
    https://CAS001.contoso.local/EWS/Exchange.asmx is accessible

    The script fails when first getting the root folder ($RootFolder= … line530 and on).

    Is it not weird that is says “VERBOSE: Using user@contoso.local for impersonation”? should this not be something like “Impersonating user@contoso.local” or “using me-tst@contoso.local“?

    Do you have any idea what could be wrong?

    Thank you!



    • 99% permissions / firewall / EWS / CAS config. Take into account waiting period for new permissions to become effective.
      ps: Wording thing. When using impersonation, you are effectively using the account specified (see line above mentioning account used for impersonation).


      • The waiting period is not it. I have wated considerably longer than it takes to replicate the permissions.

        … oh wait … as we speak …

        it was a certificate issue. I typed the external URL as the Server and now it works! AWESOME!


      • And now I get this:
        VERBOSE: Searching for folder class
        VERBOSE: Processing folder Contactpersonen
        D:\Scripting\Remove-DuplicateItems.ps1 : Can’t access mailbox information store
        At line:1 char:28
        + .\Remove-DuplicateItems.ps1 <<<< -mailbox user -type contacts -retain Oldest -server email.sdr.rot
        $a -mailboxonly -mode full -impersonation -verbose
        + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
        + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Remove-DuplicateItems.p

        Exception calling "FindItems" with "2" argument(s): "The request failed. The operation has timed out"

        since it just worked on another mailbox i'm thinking that it is simply taking too long to collect the data (55k+ contacts). Is there a way around this?


  29. Hi, I have this working against mailboxes in O365 but am finding that after it has completed, there are still duplicates, not as many but still lots.
    When it runs, it says it has deleted loads of duplicates but then when I check using OWA, there are still duplicates.
    How does it determine if the messages are duplicates or not? Is there a different detection method?


    • Default it uses a MAPI property named PidTagSearchKey to identify duplicates. However, depending on how those duplicates got in the mailbox, that property might not match. In those situations, you can use mode ‘Full’ which uses a predefined set of properties to identify duplicates. Be advised that some 3rd party products (i.e. archiving) may modify those attributes or contents – changing item size – in which case the properties will not match up. In those cases, only option is to alter the set of attributes in the script.


      • Hi, thanks for the reply.
        I did run it with mode Full and checked a couple of messages and see they have the same Message-ID, so I thought I’d try removing the other conditions from the criteria but thought I’d check running mode quick first to see what happens, and voila! It worked like a charm; no more duplicates 🙂 Thanks for your help.


        • Hi, also, I’m trying to run this against 80ish O365 mailboxes but can’t get the csv to work. I setup impersonation in O365 but when running the script it fails on autodiscovery. Is there a way to test if impersonation is working correctly? I followed your impersonation guide to set it up.
          I tried without impersonation by adding the admin account to have Full Access permissions to all the mailboxes but that also didn’t work, same autodiscover/mailbox access error.
          I also tried adding the -Credentials (Get-Credential) switch, then when prompted entered the credentials for the first mailbox in the csv but then those credentials were used for all subsequent mailboxes also, and although it appeared to be running through all the mailboxes, I noticed it was actually just doing the first mailbox over and over. Is there a way to make it ask for the credentials for each of the mailboxes in the csv? It’ll be time consuming to enter them all, but still quicker than running the Remove-Duplicates command on every mailbox.


          • If autodiscover is not resolving properly from your internal network (maybe running split DNS & Hybrid ?), you can use Server to specify the external Office 365 URL, e.g. But, to be honest, I never have to use that when Autodiscover and DNS is set up correctly, and the request is not blocked. That nice thing with impersonation is that you only need to arrange for 1 account which can impersonate the mailbox owners when accessing those mailboxes in that CSV.


            • Thanks for the help but I’ve resorted to creating a script that runs the delete-duplicates.ps1 for each individual mailbox which it prompts for the credentials for each. It’s a long-winded process but seems to be working.
              I did spend a few hours trying to get it working using impersonation and delegated access but couldn’t make it work.
              Autodiscover is working fine and I’m working from home, so no split DNS in the mix or hybrid setup. I tried using -server but still same issue. I have no outbound firewall so don’t think the request could be blocked, It works fine when running on a single mailbox when I put in -Credentials (get-credentials) but doesn’t work if I try and use impersonation or delegated permissions. If I try using the -credentials switch and put in the admin account, which has full access, with inheritance, it still can’t connect to the mailbox with error about accessing the information store.


  30. Pingback: Exportar detalle de Shortcuts de Enterprise Vault desde Exchange |

  31. Script looks fantastic. Is there a way to specify a certain date range of duplicates to check for? For example, only compare the last 30 days worth of items?


  32. I’ve needed a script like this for a long time, but my powershell and exchange dev skills were too limited. This script works perfectly with impersonation rights. It did not work without that role assigned though. It acted like it was working but ended up finding no duplicates when i could see the duplicates in the mailboxes i was searching.


    • So after running it a few times, I’ve run into issues which i believe may be related to limits in the number of items per batch or possibly an ews throttling limit. It errors and says that EWS is not available because none of the client access servers in the destination site could process the request.

      I have some mailboxes that have over 10,000 duplicates.


        • Hey Michel,

          Did you happen to throw up a blog for this? We’re currently dealing with a calendar that’s got 1.7m items we’re trying to deduplicate and your script is working phenomenally well except now we’re getting hit with EWS operation failed, server busy. Hoping you’ve got some info on helping with that throttling customization.


    • I was able to tweak the script to get it work for what i needed. I spoke to Microsoft about getting a tool from them to remove the duplicates and they were not able to provide me with anything so this script is EXTREMELY useful! Thank you for sharing it.


  33. Hello.

    I’m searching for a script which could process public folders on Exchange Server 2013. There are always many duplicates, because mails, which had been sent to more than one user, are moved to the public folder by each user after he recognized or processed the mail. Is it currently possible to process these public folder with your script? If yes: what is the syntax? If no: is there any hope that you implement such function?

    Many thanks for providing this script and kind regards


  34. Hello,

    I have been using this on Office 365 for a while now.

    to make sure this works with power-shell you have to specify the account you setup impersonation with in the command.

    For example :-

    .\Remove-DuplicateItems.ps1 -Mailbox -Type all -DeleteMode MoveToDeletedItems -Mode Quick -Verbose -Server -Credentials

    This will then bring up the input box & then will it cache the credentials…just running it without that string will present you with the infamous :-

    C:\Scripts\Remove-DuplicateItems.ps1 : Can’t access mailbox information store
    At line:1 char:1
    + .\Remove-DuplicateItems.ps1 -Mailbox
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Remove-DuplicateItems.ps1


    • For some reason the -Credentials property isn’t working for me, I get this error:

      Remove-DuplicateItems.ps1 : Cannot process argument
      transformation on parameter ‘Credentials’. Cannot convert the “” value of type “System.String” to
      type “System.Management.Automation.PSCredential”.
      At line:1 char:144
      + … m -Credentials
      + ~~~~~~~~~~~~~~~~~~~~
      + CategoryInfo : InvalidData: (:) [Remove-DuplicateItems.ps1], ParameterBindingArgumentTransformationExce
      + FullyQualifiedErrorId : ParameterArgumentTransformationError,Remove-DuplicateItems.ps1


  35. We have our mailboxes hosted on Office 365, due to some reasons few mailboxes have duplicate mails. These mailboxes have large size – average 25 GB.

    I found your script very useful and tried it. Thanks a lot for sharing it online.

    Now I m facing a problem, whenever I run this script against large mailboxes, the script breaks the execution and throws a exception. I had tried to get the exception error message, mentioned below:
    Exception calling “FindItems” with “2” argument(s): “The request failed. Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.”

    Any help will be greatly appreciable.



  36. Will this help me to delete the duplicate calendar folder named – Calendar which is present in the user mailbox.

    The affected user account has two calendar folder

    English calendar
    German Kalender

    The user doesn’t have a mobile device associated , not sure how this happened all of a sudden as the migration happened years ago.

    Any information is highly appreciated !!.




    • No, this script is to detect and remove duplicate items, not folders. If the folders should be merged, you can check out one of my other scripts, Fix-MailboxFolders ( However, that script will use default localized names for Well Known folders like Calendar. See related blog on how to customize or define a custom ‘locale’ to cover specific scenarios. Note however, only ‘source’ can be customized, the destination folder names are determined by the configured mailbox locale.


  37. Michel, how does your script compare to this one below which relies on iCALUID method to look for duplicates? I used both (yours and that one below) and it seemed that below one produces some false positives while your script reports 0 duplicates in such mailboxes. I just wanted to ask for your technical opinion which method is more reliable?


    • I won’t recommend relying on iCALUID, as this property is set by the client. Mine uses PidTagSearchKey by default, but for calendar items in your case it may be better to resort to mode Full to use property-based matching. However, mileage may vary items if duplicate items are processed in some way, seemingly staying identical but altering their size for example. The script from the MSFT guy has something similar, but in that regards it does not work differently from mine.


      • Michel – yes this is exactly what I am observing with iCALUUD vs. PidTagSearchKEy thanks! Will try Full Mode now.
        I have another question on using -WhatIf. I did not seem to find any -WhatIf logic in your code. Running with or without WhatIf produces the same output. Am I mistaking perhaps and logic is built into the core PS -WhatIf module which is not exposed in your code? I wanted to analyze appointments that script finds before letting it to delete anything.


  38. I nearly gave up on this script after running into the “Can’t access mailbox information store” error everytime. Impersonation, Get-Credential, setting the servername, writing the mailbox/usernames in all kind of formats…- nothing worked.
    The key to get the script running: using the “-MailboxOnly” option, although the server is running Exchange 2010.
    AUSFÜHRLICH: Removing 1908 items from Kalender
    AUSFÜHRLICH: Total number of items removed: 1908
    AUSFÜHRLICH: Processing finished

    Even with all the time lost tinkering- this is SO MUCH faster than every other “remove duplicates”-tool out there.
    Thank you very much!


  39. So i tried this script and it says it working but its not processing the duplicates.

    Exchange 2010 SP3 Any help is appreciated

    VERBOSE: Loading C:\Program Files\Microsoft\Exchange\Web Ser
    Processing mailbox User (
    VERBOSE: Set to trust all certificates
    VERBOSE: Using Exchange Web Services URL https://NLKVEXCMBX0
    VERBOSE: DeleteMode is MoveToDeletedItems
    VERBOSE: Processing All items
    VERBOSE: Processing primary mailbox User
    VERBOSE: Searching for all folder classes
    VERBOSE: Processing folder Calendar
    VERBOSE: Processing folder Contacts
    VERBOSE: Processing folder Lync Contacts
    VERBOSE: Processing folder Conversation Action Settings
    VERBOSE: Processing folder Conversation History
    VERBOSE: Processing folder Drafts
    VERBOSE: Processing folder Inbox
    VERBOSE: Processing folder Journal
    VERBOSE: Processing folder Junk E-Mail
    VERBOSE: Processing folder News Feed
    VERBOSE: Processing folder Notes
    VERBOSE: Processing folder Outbox
    VERBOSE: Processing folder Quick Step Settings
    VERBOSE: Processing folder RSS Feeds
    VERBOSE: Processing folder Sent Items
    VERBOSE: Processing folder Suggested Contacts
    VERBOSE: Processing folder Sync Issues
    VERBOSE: Processing folder Conflicts
    VERBOSE: Processing folder Local Failures
    VERBOSE: Processing folder Server Failures
    VERBOSE: Processing folder Tasks
    VERBOSE: Total number of items removed: 0
    VERBOSE: Processing finished


  40. Greetings.. We have very large mail boxes. We were able to get the script to work on our smaller mail boxes… but it failed on the larger mail boxes with the message “can’t access mailbox information store at line 1 char 1.” Do you have a solution for larger mailboxes. The duplicates items have resulted in some of our mailboxes exceeding 35 gigabytes.


  41. Hi, thanks for this great script. I’m giving it a go now for our public folders (in ‘WhatIf’ mode), but it seems to get stuck on a certain folder, even after leaving it overnight. Is there any way to get it to skip past when it gets stuck?


  42. Hey Michel, greate article & script.

    We struggle with duplicate contacts created by the MS Dynamics CRM “Server side sync” feature.
    CRM related Outlook contacts are flagged with an user-defined attribute “crmLinkstate” which can have the following values:
    2->contact is “tracked” in CRM
    1->track is pending

    Would it be possible to alter your script to query on this crmLinkstate Attribute to get rid of “untracked” Contact duplicates?

    Thanks for your Feedback,


  43. Write-Error : A positional parameter cannot be found that accepts argument ‘Exception calling “AutodiscoverUrl” with “2
    ” argument(s): “The Autodiscover service couldn’t be located.”‘.
    At C:\Windows\system32\Remove-MessageClassItems.ps1:434 char:28
    + Write-Error <<<< "Autodiscover failed: " $error[0]
    + CategoryInfo : InvalidArgument: (:) [Write-Error], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.WriteErrorCommand

    I can't remove the stubs..
    Can you help me pls?


  44. Hello,

    thank you for the script. Found a little bug, it doesn’t properly detect number of duplicated items. I used it on O365.

    So here is what i did:
    1) created random folder inside outlook
    2) take some mail from inbox and copy it
    3) paste it 11 times inside random folder

    Running script only detected 8 of them in random folder. Took new mail from inbox, copied it to random folde 2 times and run the script again. Script deleted previous copies that remain in random folder from 1st copy action. It didn’t remove 2 new copied mails.

    I tried running the script again in 1st and 2nd scenario after 1st run finished and it didn’t detect or delete remaining copies.

    I have a problem from Google migration to O365 so I just wanted to test before I execute it customer account.



          • What cmdline are you using? Note that default it utilizes PR_SEARCH_KEY to match duplicates; that key should be a unique and when copying items, that attribute should be preserved and identical on those items. Can you inspect with for example MFCMAPI if that is the case? Alternatively, you can use -Mode Full to match on a predetermined set of attributes (see help)


              • Do you have a utility to remove duplications from the online version of Office365 for contacts, calendars and mailboxes, tasks, notes? Thanks in advance for your response.


                • That’s where the script is for – not only mail-items. Have you checked with MFCMAPI to verify the PR_SEARCH_KEY attributes are indeed identical? The cmdline you use should work. On another note, those items are of type mail (IPF.Note); not some sort of stub or something else? (you can use MFCMAPI for this as well or add the Message Class field to the view)


                  • Found out why it didn’t want to work. Mails don’t have same PR_SEARCH_KEY, so for example same (content) mail is inside “all mails” and “important” folder but PR_SEARCH_KEY does not match even though all data of the mail that is displayed inside outlook client is the same. Would you be interested to modify the script to fix our problem (we can discuss about the payment)? I don’t know what I was comparing few days ago…


  45. Do you know how I could get this to work when the PID searches do not match, and the Received timestamp, size, and last modified properties are all a tiny bit different? (so, essentially relying on sender, subject, time sent which remain unique)


    • Below ‘# Use predefined criteria for matching duplicates depending on ItemClass’ is the logic to match items, depending on item type (different items provide different attributes). I can’t determine “a tiny bit different” for you. There is also the danger of removing valid items, for example it happens quite often people forget to state something or add an attachment, making things like subject, sender and recipients match and only the timestamp ‘a tiny bit’ different, yet both messages should be kept.


  46. Hello Michel, the script is GenieL.
    We migrated from Lotus to Exchange 2013 but this generate many duplicates process.
    I read carefully your blog and I changed the script and the parameters to work. But, now does not fail but gives 0 items to remove

    PS C:\Windows\system32> Remove-DuplicateItems.ps1 -Mailbox -Type Mail -DeleteMode MoveToDeletedItems -ArchiveOnly -Verbose -Server Exchange.local -Mode Full
    VERBOSE: Loading C:\Program Files\Microsoft\Exchange\Web Services\2.2\\Microsoft.Exchange.WebServices.dll
    Processing mailbox (
    VERBOSE: Set to trust all certificates
    VERBOSE: Using Exchange Web Services URL https://Exchange.local/EWS/Exchange.asmx
    VERBOSE: DeleteMode is MoveToDeletedItems
    VERBOSE: Processing Mail items
    VERBOSE: Processing archive mailbox
    VERBOSE: Searching for folder class
    VERBOSE: Processing folder (Custom Expiration\Manage Folders)
    VERBOSE: Processing folder Elementos eliminados
    VERBOSE: Processing folder FolderHiddenPublic
    VERBOSE: Processing folder Folders
    VERBOSE: Processing folder Data
    VERBOSE: Processing folder Personal
    VERBOSE: Processing folder Inbox
    VERBOSE: Processing folder Sent Items
    VERBOSE: Total number of items removed: 0
    VERBOSE: Processing finished


    • I am having this same issue only on Wave 15.1 of Office 365.

      Previous comments lead me to think that it may be a permissions issue. We have duplicates that came in from an IMAP sync and PST import but the script output looks very similar to what you posted – 0 items removed. I have granted the admin account full mailbox access as well as ApplicationImpersonationRights via the Discovery Management admin role but no joy.

      The above links regarding permissions appear to be dead so maybe there is something else I am missing with permissions here.


      • 15.1 is a bit too generic – what is you specific version, i.e. what does Outlook’s Connection Status / Version report? Meanwhile I’ll check against an tenant.

        I’ve update the link to information on how to configure impersonation.


        • Well, my admin display version is 15.1.549.26. I’m not sure if that correlates to the specific Exchange Online version. In any case, I have verified the impersonation as well as given full mailbox access, so I don’t think permissions should be an issue at this point.

          However, I have tried both the Full and Quick mode with no items being found but no errors showing either.

          Thanks very much for the script and for your assistance!



  47. Hi,

    Thanks for sharing such a wonderful post with us.
    I had this issue & your blog as well as one other blog, helped me to overcome this scenario of duplicate emails. Another link is as follows:
    [link removed]


  48. I’m using v1.41 The script runs fine but finds no duplicates. I even tried selected an item in my inbox, ctrl-c, ctrl-v to duplicate it. Still the script says:
    PS C:\Users\xxxx\Downloads> .\Remove-DuplicateItems.ps1 -Mailbox -Type Mail -DeleteMode MoveToDeletedItems -Mode Full -Verbose -MailboxOnly -Server -Credential (Get-Credential)

    cmdlet Get-Credential at command pipeline position 1
    Supply values for the following parameters:
    VERBOSE: Loading C:\Program Files\Microsoft\Exchange\Web Services\1.2\\Microsoft.Exchange.WebServices.dll
    Processing mailbox (
    VERBOSE: Set to trust all certificates
    VERBOSE: Using credentials
    VERBOSE: Using Exchange Web Services URL
    VERBOSE: DeleteMode is MoveToDeletedItems
    VERBOSE: Processing Mail items
    VERBOSE: Processing primary mailbox
    VERBOSE: Searching for folder class
    VERBOSE: Processing folder Clutter
    VERBOSE: Processing folder Drafts
    VERBOSE: Processing folder Inbox
    VERBOSE: Processing folder Junk Email
    VERBOSE: Processing folder Outbox
    VERBOSE: Processing folder VoiceOutbox
    VERBOSE: Processing folder Quick Step Settings
    VERBOSE: Processing folder Sent
    VERBOSE: Processing folder Sent Items
    VERBOSE: Processing folder Sync Issues1
    VERBOSE: Processing folder Conflicts
    VERBOSE: Processing folder Local Failures
    VERBOSE: Processing folder Server Failures
    VERBOSE: Processing folder Trash
    VERBOSE: Total number of items removed: 0
    VERBOSE: Processing finished


  49. I got it working for me. It wasn’t for for 4 different reasons:
    1) autodiscover isn’t working for my office365 account, I just bypassed the issue adding parameter -Server
    2) -Mode Quick doesn’t seem to be able to retrieve any useful identifier for emails, so I used -Mode Full
    3) Even in -Mode Full, most of my duplicates are not recognized because they have identical time, sender, messageid, etc, but different size. That is, even if I manually duplicate a message copying it to a different mailbox, and then back, it gets a different (larger) size. Since this script compare also size (in Full Mode) it finds no duplicates. Solved by commenting out this line:
    # if ($Item.Size) { $key+= “,”+$Item.Size.ToString()}
    4) The search for a duplicate is done using an arrayList which (is inefficient and) always fails for me. Maybe because even if the two compared strings are identical they are still different objects.
    Solved by replacing line
    $UniqueList= [System.Collections.ArrayList]@()
    $UniqueList = New-Object ‘System.Collections.Generic.HashSet[string]’

    I also added two more options to Retain: Largest and Smallest, in order to be able to retain the smallest.


  50. Hello Michel,

    sorry, I Need to come back to this post from early this year:

    Would it be possible to use your script to:
    1) Lookup only contacts within a Mailbox that have the flag “crmLinkState=2”? (This would secure to work only on contacts that have been synced from Dynamics CRM)
    2) Do a merge/cleanup with data from 1) and define a master record that has a certain value (e.g. “merged master”) in the field “Nickname”

    We found a way to merge all duplicate contact records within our CRM System. To have a filtering Option, we´d update all remaining records with the above mentioned string within the field “Nickname” which would also be synced to Exchange mailboxes.

    Thanks in advance for any Reply.



  51. I would like to be able to run the script from my computer without having the username password
    To use the office administrator 365


  52. Script seems to run without errors, the include folder is processed, but it doesn’t identify or remove any obvious duplicates. I have commented out the item.size lines for -Full, but obvious duplicates are still not removed. Tenant version is: 15.1.1341.23. Impersonation is enabled. Duplicates were copied with a third party migrator that we are trying to clean up but the same product did both runs so they are identical messages. Wonderful script if it will do what we need.


  53. Just completed a GSuite migration to Office365 and I’m using your script to clean up the duplicate mess.

    Small folders process without any issues, but larger folders fail after deleting a few thousand duplicates…

    VERBOSE: Removing 15596 items from Archive
    Process-Mailbox : Problem removing items: Exception calling “DeleteItems” with “4” argument(s): “The server cannot
    service this request right now. Try again later.”
    At D:\Users\XX\Downloads\Remove-DuplicateItems.ps1:675 char:16
    + … If(! ( Process-Mailbox $RootFolder $IncludeSearchCollection $Exc …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Process-Mailbox

    D:\Users\XX\Downloads\Remove-DuplicateItems.ps1 : Problem processing primary mailbox of (
    At line:1 char:1
    + .\Remove-DuplicateItems.ps1 -Identity -Cr …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Remove-DuplicateItems.ps1

    Usually, a new powershell is required to resume the operation.

    Do you have any suggestions on how to complete the operation in a single pass?


  54. Should this be able to run against a shared mailbox? I’ve a shared mailbox (ex-user mailbox that has been converted to shared, with assigned email address) on O365 that i specify in identity, using my own credentials. The script runs but from the verbose output it’s obvious it’s running on my own mailbox and archive.


  55. Hello Michel, Thanks for this script.

    But i have some problems…

    D:\Remove-DuplicateItems.ps1 : Can’t access mailbox information store: Exception calling “FindItems” with “2” argument(
    s): “The server cannot service this request right now. Try again later.”
    At line:1 char:1
    + .\Remove-DuplicateItems.ps1 -Mailbox xxxxxxxxx@xxxx.xx -Server out …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Remove-DuplicateItems.ps1


  56. Hi, I am getting an error: Problem removing items. Maybe you have any ideas what the problem is? Thank you!

    Are you sure you want to perform this action?
    Performing the operation “Remove-DuplicateItems.ps1” on target “Remove 152 items from \Inbox”.
    [Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is “Y”): Y
    VERBOSE: Removing 152 items from \Inbox
    Process-Mailbox : Problem removing items: Attempted to divide by zero.
    At C:\Program Files\Microsoft\Exchange\Web Services\2.2\Remove-DuplicateItems.ps1:1040 char:29
    + … If (! ( Process-Mailbox -Folder $RootFolder -IncludeFilter $Inclu …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Process-Mailboxting error


  57. I am getting a repeated warning. I getting so many that the log text leading up to it has scrolled out of reach for a cut and paste. Several lines after the message telling me the script has started processing the mailbox I get:
    VERBOSE:Adding folder \ (priority 0)
    WARNING: Error performing operation FindFolders with Search options in . Error: Exception calling “FindFolders” with “3” argument(s): “Value cannot be null. Parameter name: parentFolderId”
    VERBOSE:Adding folder \\ (priority 0)
    WARNING: Error performing operation FindFolders with Search options in . Error: Exception calling “FindFolders” with “3” argument(s): “Value cannot be null. Parameter name: parentFolderId”
    This repeats with another back slash added until there are well over 800 hundred slashes.
    The script then ends with the message: Cannot access mailbox information store, error: The script failed due to call depth overflow.
    At line:1 char:1
    + C:\Users\Me\Remove-DuplicateItems.ps1 -Identity -Server …
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId: Microsoft.PowerShell.Commands.WriteErrorException,Remove-Duplicates.ps1

    The command line (v1.81) I am using looks like this:
    C:\Users\Me\Remove-DuplicateItems.ps1 -Identity -Server -Type Mail -DeleteMode MoveToDeletedItems -Mode Full -Verbose -Credentials (Get-Credential) -Retain Oldest


  58. I updated to 5.0 and the scripted worked for me too. This is great. I was surprised I didn’t have more duplicates. I found about 100 in 150,000 emails. It is really nice to have this cleaned up. Thanks Michel


    • Yes – use IncludeFolders and the tag #SentItems#, e.g. -IncludeFolders ‘#SentItems#’. But it seems you found a bug; if you only specify one element, it processes all folders. Bug filed; workaround is to specify the same folder twice, e.g. -IncludeFolders ‘#SentItems#’, ‘#SentItems#’


  59. sorry your scri[t does not work on o365 I have scaned an mailbox qith cript 60000 mails and more an returned 0 duplicates and atleast 40 % are duplicates


  60. I have a Problem with Exchange 2016. The Skript detects duplicate contacts and removes them. But it does not detect duplicate Mails.
    It lists all the Folders of the Mailbox that exist. It Shows the correct number of items that are in the Mailbox but it Shows 0 items removed. I spedify “-Mode Full -NoSize” but the duplicates are not recognized. The items are created by multiple runs of a restore, so they should be the same.

    Using the user credentials or impersonation makes no difference.


    • For e-mail with ‘Full’ there is a set of attributes which should match for it to be considered a duplicate. With NoSize, you can exempt size from this comparison, but if those others are not *all* identical, it’s not considered a duplicate. You can use tool like MFCMapi (or Outlook itself) to inspect the following attributes: DateTimeReceived, Subject, InternetMessageId, DateTimeSent, Sender and optional Size.


  61. Hi Michel. I have been running this script against a fairly large mailbox with a fair amount of duplicates and am getting following errors:

    VERBOSE: Using Exchange Web Services URL
    VERBOSE: Constructing folder matching rules
    VERBOSE: Processing primary mailbox
    VERBOSE: Collecting folders to process, type All
    WARNING: Previous EWS operation failed, adjusted sleep timer to 300ms
    C:\Users\Redblade\Downloads\Remove-DuplicateItems (1).ps1 : Cannot access mailbox information store, error: Unable to find type [Microsoft.Exchange.WebServices.Data.ServerBusyException].
    At line:1 char:2
    + & ‘.\Remove-DuplicateItems (1).ps1’ -Mailbox -S …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Remove-DuplicateItems (1).ps1

    Have checked impersonation which is setup correctly for the $usercredential account. Thing is if I run this against smaller mailboxes this seems to run fine, potentially ruling out impersonation rights or connectivity. There is another mailbox in the tenancy which is >10gb which gives the same error, all other mailboxes run fine. Any ideas would be much appreciated?



  62. Michel,

    many thanks for this script. I run it on an Exchange 2016 server.

    .\Remove-DuplicateItems.ps1 -Mailbox -Type Tasks -Impersonation -DeleteMode SoftDelete -Mode Full -Verbose –WhatIf

    The script finds many duplicate mails and calendar entries but no duplicate tasks. However, there are many duplicate tasks in the mailbox.

    Can you give me a clue?

    Thanks a lot


      • Hello Michel,

        I have a similar problem with the processing of tasks in user mailboxes. I’ve used the following Parameters:
        .\Remove-DuplicateItems.ps1 -Mailbox -Type All -Impersonation -DeleteMode HardDelete -Verbose

        The Skript was able to remove evry duplicate item from the mailbox except for tasks. Even with specifing tasks as the inteded search type and playing aroud with the -DeleteMode parameter, I was unable to remove them.

        Does it have anything to do with the german translation for the task folder? Or is there anything else I can try?

        Thank you in advance!

        Best regards,
        T. Nieresel


  63. Hello!
    Can you please explain I have larger mailbox 90 GB and into folders about 80k items.
    Script didn’t work properly and didn’t all cleaned up msg. Used Full mode
    Why? And How are using pig tiles or maybe more parameters?


    • 1) What you mean by pig tiles?
      2) Duplicate checks are performed on multiple attributes which all need to be equal depending on the class to be considered duplicate. That is not always the case as 3rd party software may touch items or dehydrate stubs which then look the same, but are not. Read the help for info on attributes to be considered.


  64. Michel
    I have managed to modify your script in a fairly minor way to work on Public Folders.
    Would you like me to submit my version (1.83PF) to you as I dont feel publishing it myself is valid as 95% ( & probably more) of the work is yours !


  65. hi when Using mailboxwide switch, how to change the priority of the folders. As I want to keep the emails in all the subfolders instead of Inbox and Important Folders after migrating IMAP mailbox to Office 365.


    • Use PriorityFolders to give folders priority over others; items founds in these folders are retained over duplicates in other folders. The order of folders specified with PriorityFolders also impacts which items to retain; PriorityFolders A,B will have items in folder A retained over duplicates in B. PriorityFolders also accepts well known folders, wildcards etc.


  66. Somehow a user’s contacts got duplicated and then some. She’s got over 500,000. I ran this once and it picked up and removed ~105,000. It completely ignored most of them though. One for example, there appears to be hundreds of, and it only has a name (which is a phone number) and company name. That’s not being picked up in quick or full mode. Any suggestions?


      • Even in quick mode, it’s not finding any of them.

        End of the log…

        VERBOSE: Collecting folders to process, type Contacts
        VERBOSE: Adding folder \Contacts (priority 0)
        VERBOSE: Adding folder \ExternalContacts (priority 0)
        VERBOSE: Adding folder \PersonMetadata (priority 0)
        VERBOSE: Found 3 folders that match search criteria
        Processing folder \Contacts
        Processing folder \ExternalContacts
        Processing folder \PersonMetadata
        1752592 items processed and 0 removed in 02:36:45 – average 11181 items/min
        WARNING: Cannot bind to ArchiveMsgFolderRoot – skipping. Error: Exception calling “Bind” with “2” argument(s): “The specified folder could not be found in
        the store.”
        VERBOSE: Processing finished

        Still have ~390,000 that need to be removed, grateful for any advice you can provide.

        Thank you


  67. Somehow a user’s contacts got duplicated and then some. She’s got over 500,000. I ran this once and it picked up and removed ~105,000. It seemed to ignored most of them though. One contact for example there appears to be hundreds of, and it only has a name (which is a phone number) and company name. That’s not being picked up in quick or full mode. Any suggestions?


  68. Hi,
    nice work but when I run the command, only 1786 items get processed in inbox and no more. Is there a limit set somewhere? There are more than 150000 mails in inbox only.



  69. Hi Michel.

    First thanks, thanks a lot for this script and for all the information about Exchange in your blog.

    We have a problem with your script. The script run without problems but it detects a lot of emails duplicated, says at the end that they have been removed but the emails duplicated (and triplicated) are still in the mailbox.

    [PS] >.\Remove-DuplicateItems.ps1 -Mailbox -Type mail -Impersonation -DeleteMode harddelete -Mode full -IncludeFolders “#sentitems#” -mailboxonly -confirm:$false

    Processing mailbox (
    93158 items processed and 20903 removed in 00:10:50 – average 8597 items/min

    We tried with impersonation and with the user’s credentials but the results were the same.

    We tried to check only the folder or all the mailbox but again the results were the same.

    At the first execution the script removed some of the detected duplicated emails but didn’t remove all of them.

    After that it doesn’t remove more duplicated emails even though it detects them.

    I can’t imagine why the script detects but doesn’t remove them. Do you have any idea about it?

    Thanks in advance for your response.


      • Thanks for the quick response.

        I’m checking the mailbox folder statistics in server side and I’m trying to remove the items since some days ago. The duplicated emails are still there.

        I don’t know why it doesn’t remove the duplicated emails because there is no errors in the execution. Maybe I have to put some flags and modify the script to debug my problem and see where is the strange thing which produce this behavior.



      • At the end I distribute the duplicated email in 4 folders with 10.000 duplicated emails each one and it works as desired. The emails are analyzed, duplicates founded and removed (each folder contains 7.000 email duplicated more ore less)

        The problem seems to be when there a too much emails to remove.


  70. Great thanks go out to you Michel as you patiently navigate these questions and provide excellent detailed answer. I just wanted to pass along a tip on an issue I was having.
    All of my attempts to use this script were failing with authorization errors. I finally ran it against my own mailbox with the same results. Double and triple checked my credentials…still no luck.
    Knowing I had MFA enabled, I used my app password and it all started working.

    Really very nice of you to share and continue to develop a script like this. We as a community can not thank you enough.


  71. So I have a quick question. I have some shared mailboxes that I have full access to. I am identifying the shared mailbox with the identity parameter. I am using my credentials for authentication. In verbose mode, the script says its processing the shared mailbox. But the scan appears to be scanning my own mailbox. I do not have impersonation enabled. And if I try to use the switch it fails. Must I have impersonation even if I have full access?
    On the bright said, I have thoroughly cleaned dupes out of my own mailbox. Command line below is what I used (notwithstanding some anonymity.)
    $creds = get-credentials
    .\Remove-DuplicateItems.ps1 -Identity -type Mail -Server -Credentials $creds -mode Quick -DeleteMode SoftDelete -MailboxWide -MailboxOnly -Verbose

    Some of the verbosity was…
    VERBOSE: Processing folder \Inbox\FTC
    VERBOSE: Processing folder \Inbox\Incident emails
    VERBOSE: Processing folder \Inbox\Helpdesk
    32310 items processed and 37 removed in 00:03:41 – average 8750 items/min
    VERBOSE: Processing finished

    The folders mentioned above were in my mailbox not the shared mailbox.


      • I have to follow up on my use of this great script. I finally got application impersonation permissions to run this. (This is a difficult role to explain to someone even if they know PowerShell really well.)
        In short, this script works great for me now. I discovered that using the mode body yields the most dupes. I encourage anyone using this to try many combinations of the switches to get what you need form this.
        Michel-Thanks again! I can’t help notice what a great over all site you have. I am starting to really dig into in. You are indeed an MVP!


  72. Good afternoon; is it possible to delete duplicates only based on Subject Line and date?

    My used got the same email twice multiple times from the same user but from different alias so the only thing that it matches besides the body content is the subject line and date which are the same.



  73. So I have a significant update on my issue. I had been running this program on my workstation with my Outlook open. In retrospect, that was quite foolish. I am trying it on an exchange server (which also feels foolish.) I believe my issues are one of environment. I will report my findings as they develop.


  74. After my Remove-DuplicateItems.ps1 was executed and completed, some mailboxes got an automatic email about “Unread email has been deleted”. Some users did have unread duplicated emails and it looks like the script or Exchange-server send an email to the sender of all unread emails, so both internal and external recipients in other organisations got a notification about this. Do you know how to turn of this notification emails?


      • It was Exchange 2016 CU11. What I remember, it was NDRs back to the senders of those old emails saying “You got an unread email deleted by the recipient…”. We did not test to manually delete some of these old unread emails to see if that also gerenated NDRs. We did just close that ticket. Let’s see if it happens more times. And can reply here with more details in that case. Thanks for a good script btw!


  75. You are a life saver!! After my parent company switched from exchange to Gmail (losing a few years worth of emails in the process), I’m now switching back to Office 365 for my team and the labeling in gmail created so many issues. Thanks to your script I can merge the gmail inbox AND the original PST file from the exchange server and not have to panic about the duplicates. Incredibly well explained and detailed and reading through the responses above saved me a ton of time. (I had never used a script before in Powershell and with your direction was able to get this up and running in less than an hour) Thanks again!


  76. Great script. I am running into problems with mailboxes in a different language. I cannot do -DeleteMode MoveToDeletedItems because the DeletedItems for a German mailbox is named ‘Gelöschte Elemente’ (Error:Exception calling “DeleteItems”…


    • That shouldn’t be the problem, because we’re referring to the DeletedItems folder through something called the WellKnownFolders, which is language-independent. Care to elaborate a bit more on the issue you’re encountering?


  77. I’m still running into the EWS throttling issue on large mailboxes (error below; example mailbox is ~200,000 items). I’ve read some comments that this could be an issue with interrupted internet connectivity. So I re-ran the command from an Azure server to alleviate any local issues and ran into the same error.

    Cannot access mailbox information store, error:
    Exception calling “LoadPropertiesForItems” with “2” argument(s): “The request failed. The read operation failed, see inner exception.”

    I’ve tried multiple combinations of parameters and it seems to be only affecting running in Full mode (full script below).
    .\Remove-DuplicateItems.ps1 -Mailbox -Credentials $creds -Mode Full -NoSize -Impersonation -Server -Type mail -verbose


    • 1) What version of EWS Managed API are you using?
      2) Mailbox parameter has been renamed to Identity for a while now.
      First thing the script does is collect all folders and select the ones which pass the Include/Exclude filter. That should not be limited due to no. of items in a folder. That ‘Exception calling “LoadPropertiesForItems” with “2” argument(s): ‘ is the only thing reported?


      • I can’t speak for BDP, but I am facing the same exact issue across multiple different mailboxes. I am trying this against O365. The EWS Managed API I’m using is v15.00.0913.015. The exact error that I get is:

        TerminatingError(): “Exception calling “LoadPropertiesForItems” with “2” argument(s): “The server cannot service this request right now. Try again later.”

        It definitely seems to be struggling with large mailbox sizes with a couple folders that are 10GB+ (50k+ items). Unsure how to handle this situation. Is it best to try and split these across multiple folders in some way? Guess I’ll have to find a way to do that programmatically…


  78. Hi,

    First of all, thanks a lot for your help to the community.I’ve discovered your thread a few weeks ago. I can’t believe you’ve helped so many people. I’m also having issues on a migration from G Suite to O365.
    I’ve downloaded your script. I’ve been trying to use it for several days. I went through most of the issues people also had on this topic, and got them solved one after the other. (credentials, impersonation, etc).

    I’ve run the script in Full Mode and it did not find a lot of duplicates (but I know I have at least 17Gb of them :-). I’ve just launched a new run with the -noSize option, I’ll see if it works.

    I have one question : I have found manually a few examples of emails that are duplicated. Following example is :
    – the original email is folder “Important”
    – the first copy I can find is in folder “PARTNER A” (G suite label). On that email I can spot one difference : Office has considered that my forwarding this email was to be in this folder, not in important. So the mailchain is a bit different here in this folder
    – the second copy I can find is in folder “Archive”, and it looks pretty much like the one in folder “Important”.
    Do you know a way for me to check the matching criterias in the script on these 3 particular emails to understand why they were not picked up ?

    My current command line :
    .\Remove-DuplicateItems.ps1 -Mailbox -Type Mail -DeleteMode MoveToDeletedItems -Mode Quick -Server -Impersonation -Credentials (Get-Credential) -MailboxWide -NoSize -Report -WhatIf:$true

    Thanks in advance for your answer !


    • Just realized : better with Full Mode to use NoSize
      .\Remove-DuplicateItems.ps1 -Mailbox -Type Mail -DeleteMode MoveToDeletedItems -Mode Quick -Server -Impersonation -Credentials (Get-Credential) -MailboxWide -NoSize -Report -WhatIf:$true

      My question is still on 🙂


  79. Hi dear,
    It is an amazing script, really liked it. What if we could use an parameter to say what should be check as a duplicated, like “messageID”. In my environment, mails were duplicated due an replication during migration and with an forward message redirection. It means that mails were received with seconds in difference between the migration process and the forward process. If I had the parameter to say that the messageID is the same, the duplicated mails could be removed.

    Thank you for sharing it! 🙂


  80. Hi,

    Thanks for this handy script, I’m having an issue in full mode however where it says Cannot access mailbox information store, error: Exception calling “LoadPropertiesForItems” with “2” argument(s): “Requested value ‘GroupMailbox’ was not found.

    I am using exchange api 2.2 with the following command .\Remove-DuplicateItems.ps1 -Mailbox -Server -Credentials $Credentials -Mode Body -PriorityFolders ‘Holidays’,’Relief’,’IT Related Stuff’,'[Gmail]’,’Starred’ -MailboxWide -Report -Verbose

    Any ideas?



  81. Hi, any news on removing duplicate items for public folder? Or may anyone else has a fork which works with PF? Thank you very much for your work so far!


    • I have modified version that works on our public folder configuration….
      note it is based 99% on v1.86 of Michel’s script & requires a mailbox with access to the entire tree to be reviewed.

      I run it on a portion of the tree at a time.
      Write-Output “Start– $(get-date -Format “ddd dd MMM yyyy HHmmss”) —–”
      $Credentials= Get-Credential;

      & $RDup -Mailbox FileMailer -PublicFolders -PFStart “_Projects\2004” -Credentials $Credentials -Impersonation -Report -Force
      & $RDup -Mailbox FileMailer -PublicFolders -PFStart “_Projects\2005” -Credentials $Credentials -Impersonation -Report -Force
      & $RDup -Mailbox FileMailer -PublicFolders -PFStart “_Projects\2006” -Credentials $Credentials -Impersonation -Report -Force

      I will have to diff it with current v1.88


      • Hello Michel, would be great if you could implement this in nearest release, this feature is expected very much on our side, thanks a lot for your magic script!


      • Could you please also clarify, what I do wrong?
        PS C:\Users\User\Downloads\Remove-DuplicateItems-master> .\Remove-DuplicateItems.ps1 -Identity ‘’ -Server -Credentials $Credentials -IncludeFolders ‘#Inbox#/03 COOL/0 BPM’ -DeleteMode SoftDelete -MailboxOnly -verbose
        VERBOSE: Loading C:\Program Files\Microsoft\Exchange\Web Services\2.2\\Microsoft.Exchange.WebServices.dll
        VERBOSE: Loaded EWS Managed API v15.00.0913.015
        VERBOSE: Set to trust all certificates
        VERBOSE: Using credentials
        VERBOSE: Processing items of type All, delete mode is SoftDelete
        Processing mailbox (
        VERBOSE: Using Exchange Web Services URL
        VERBOSE: Constructing folder matching rules
        VERBOSE: Processing primary mailbox
        VERBOSE: Collecting folders to process, type All

        (I have changed user email intentionally) It stops on the latest message and nothing continue.


  82. Michel,
    I really look forward to getting this to work, however I seem to run across a similar issue as many others:

    Cannot access mailbox information store, error: Exception calling “.ctor” with “0” argument(s): “This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms.”

    I have given myself impersonation rights, as well as full permissions to the mailbox in question (a different user). I’ve tried both -Impersonation and the -Credential method.

    I see a lot of issues for people regarding “Cannot access mailbox information store” but that seems to be the permission issue that is supposed to be fixed by giving impersonation/full access (with inheritance). However, I don’t see anything regarding the follow up part of the error mentioning FIPS. Unfortunately I’m in an environment where I cannot simply “disable FIPS”, it is a security requirement. Does that mean I’m out of luck? Is there some sort of switch that bypasses what FIPS may be complaining about? Some sort of alteration in the script that will allow it to be FIPS “compliant”. It seems to scope the mailbox just fine in the debug, but fails right after it starts to process the folders. I can even see the first item of the folder that it hits. If you have any idea how to address this, I’d super appreciate it.

    .\Remove-DuplicateItems.ps1 -Mailbox user.samaccountname -Type all -Impersonation -DeleteMode MoveToDeletedItems -Mode Full -Verbose -Debug

    I’m on-prem with Exchange 2016 CU15
    Using Remove-DuplicateItems.ps1 Version 1.88

    Name Value
    —- —–
    PSVersion 5.1.14393.3866
    PSEdition Desktop
    PSCompatibleVersions {1.0, 2.0, 3.0, 4.0, 5.0, 5.1.14393.3866}
    BuildVersion 10.0.14393.3866
    CLRVersion 4.0.30319.42000
    WSManStackVersion 3.0
    PSRemotingProtocolVersion 2.3

    If, in the case I’m out of luck because of FIPS, I still appreciate your time, not only in responding, but creating what looks to be a superb script.


    • Sorry, scripts are untested with FIPS locked down environments. That said, it’s not that the script shouldn’t work, I assume it’s a matter of crypto settings and defaults. Not totally into this, but could you set this (I assume no it environment is locked down) and run the script?


      • Sorry for the delay. Unfortunately, manually setting the registry key (as you mentioned above) did not work. Nor did temporarily removing the GPO that enabled FIPS cryptography. Very unfortunate. I was going through your script to find what could potentially be causing this. I noticed this section:

        Function Get-Hash {
        $md5 = New-Object -TypeName System.Security.Cryptography.MD5CryptoServiceProvider
        $data = ([system.Text.Encoding]::UTF8).GetBytes( $string)
        return ([System.BitConverter]::ToString( $md5.ComputeHash( $data)) -replace ‘ – ‘, ”)

        When I run that function all by its lonesome in a PowerShell session, I get presented with the same error! I have changed the encoding from MD5 to SHA256 in that function and now it works (or so it seems, I’m still running a test). OOOH man, so excited lol. Thanks. Crossing my fingers 🙂


  83. This script looks perfect, especially since I was advised post-migration that users had been forwarding all their email from an old domain to their new one, creating duplicates of their entire mailboxes. I’m trying to run this from the admin context against Office365, but can’t get it to work. Do you need to use the credentials of the account that you’re accessing, or can a Global Admin’s account be used? That seemed to be the function of the impersonate, but I haven’t been able to get it to execute properly. I’ve managed to get as far as getting an auth error with the user mailboxes. Appreciate your help and your awesome script!


  84. Hello,
    first of all Thanks for the Script and the Script to remove Items from a Mailbox. Is there a Way to only prioritize Subfolders of a Folder and not the Folder itself?


  85. Pingback: Annual Report 2020 | EighTwOne (821)

  86. Can this script (or maybe this one with modifications) be used to remove duplicates between two separate non-archive mailboxes? I’m hoping to locate something that can identify e-mails with identical message IDs distributed between two mailboxes and allow the deletion of one set of messages. The ability to dump the IDs and message properties to a CSV for review would also be useful. Any thoughts or pointers to get me going in the right direction? Have strong Powershell skills but the EWS component has always featured a layer of complexity beyond what I’m familiar with. Thanks in advance!


    • This script, no. Can it be made, yes. However, that would basically mean going through both, collecting things uniquely identifying mail in one (e.g. message ID and path for reference), and looking for presence in the other (reporting ID and path in mailbox A and mailbox B). That is the business case, if I may ask?


  87. Thanks Michel. Business case is a one-off unique situation – a user’s mailbox was split into two due to high folder counts and at Microsoft support’s recommendation. It was discovered later that for a period of time, emails were delivered to both mailboxes. We want to dedupe these message before we archive his mailbox as the dupes will be a source of annoyance / confusion for the end user.

    I’ve opened a case with Microsoft about this as well, but thus far they’ve been less than helpful. Your suggestions are helpful though, and I’ve made some progress just binding to a folder and producing FirstClassProperties of items.


    • Hi Kevin, to circle back on this: Could be more simple on second thought: The entry which initializes the variable to keep track of entries ‘seen’ before is initialized at the start when processing the mailbox. It is re-initialized after processing the folder, depending MailboxWide switch. You could of course move initialization to start of script, and comment all reinitializations out. Then, pass the mailboxes you want to process (you can pass more identities) so it will ‘know’ items seen in the first mailbox, and identical items from second (and third etc). identity will be considered duplicate. Now, to save you work, in 2.03 I removed MailboxWide and introduced -CleanupMode where you can specify MultiMailbox (and fixed identity parameter accepting multiple entries). So, you can do this (added whatif & report to make example non-destructive):
      .\Remove-DuplicateItems.ps1 -Identity, .. -CleanupMode MultiMailbox -Verbose -WhatIf:$true -Report


  88. Tried executing this script. It doesn’t give error but is not deleting duplicate items from Office 365.
    I am trying to delete all duplicate mails from server in multiple folders. Have changed email for security purpose
    Am I missing any thing
    Script used is :
    $Credentials= Get-Credential
    .\Remove-DuplicateItems.ps1 -Mailbox -Server -Credentials $Credentials -DeleteMode HardDelete -ExcludeFolders ‘Keep Out’ -PriorityFolders ‘*Important*’ -MailboxWide


  89. Hi Micheal ,

    issue was created after migrating from Gmail to Office 365 of many duplicate emails.
    After running this script. Script is working and deleting duplicate emails when duplicates are in sub-folders. How ever it is not deleting emails if duplicate emails are present in folders as same level

    So if mails are duplicates between Inbox,Achive and Testfolder it is not working. Please help. Script I am running is below :

    $Credentials= Get-Credential
    .\Remove-DuplicateItems.ps1 -Mailbox -Server -Credentials $Credentials -Type Mail -DeleteMode HardDelete -Verbose -MailboxWide -Mode Full


  90. Hi there. This looks like a great script. Unfortunately it appears that i’m running into an issue just running this script. This may be a simple fix, but could you point me in the right direction as to how I can get this running?

    PS D:\Install\Scripts\Remove-DuplicateItems> .\Remove-DuplicateItems.ps1 -Identity -CleanupMode Mailbox -Verbose -WhatIf:$true -Report
    D:\Install\Scripts\Remove-DuplicateItems\Remove-DuplicateItems.ps1 : Parameter set cannot be resolved using the
    specified named parameters.
    At line:1 char:1
    + .\Remove-DuplicateItems.ps1 -Identity
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : InvalidArgument: (:) [Remove-DuplicateItems.ps1], ParameterBindingException
    + FullyQualifiedErrorId : AmbiguousParameterSet,Remove-DuplicateItems.ps1


    • It’s missing a paramater to dermine what mode it needs run in (and other parameters). In this case, Credentials (for Basic Auth) or one of the other params for OAuth are required. Credentials is also required now for running against local environment. Let me know if that poses problem.


      • Thanks. That got me closer:

        PS D:\Install\Scripts\Remove-DuplicateItems> $Credentials= Get-Credential

        cmdlet Get-Credential at command pipeline position 1
        Supply values for the following parameters:
        PS D:\Install\Scripts\Remove-DuplicateItems> .\Remove-DuplicateItems.ps1 -Identity -CleanupMode Mailbox -Verbose -WhatIf:$true -Report -Credentials $Credentials
        VERBOSE: Loading module D:\Install\Scripts\Remove-DuplicateItems\Microsoft.Exchange.WebServices.dll
        VERBOSE: Loading module from path ‘D:\Install\Scripts\Remove-DuplicateItems\Microsoft.Exchange.WebServices.dll’.
        Import-ModuleDLL : Problem loading module Microsoft.Exchange.WebServices: Could not load file or assembly
        ‘file:///D:\Install\Scripts\Remove-DuplicateItems\Microsoft.Exchange.WebServices.dll’ or one of its dependencies. The
        module was expected to contain an assembly manifest.
        At D:\Install\Scripts\Remove-DuplicateItems\Remove-DuplicateItems.ps1:1316 char:5
        + Import-ModuleDLL -Name ‘Microsoft.Exchange.WebServices’ -FileName …
        + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
        + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Import-ModuleDLL

        I have not installed the Exchange Web Services (EWS) Managed API tool, but I have copied the *.dll & *.xml files from your Github page to the same directory as the script. I also unblocked the files to no avail.


        • Since you already unblocked the file, “D:” is a local drive? (network mapped drives are likely not trusted)
          Likely a .NET Framework dependency – the DLLs I posted on GitHub for convenience depend on .NET Framework 4.0


  91. I’m trying out this stellar looking script for the first time. Got EWS installed, etc etc and tried to run it and got the following message:

    Import-ModuleDLL : Problem loading module Microsoft.Exchange.WebServices: Could not load file or assembly
    ‘file:///C:\RBS\Support\Remove-DuplicateItems-master\Microsoft.Exchange.WebServices.dll’ or one of its dependencies.
    Operation is not supported. (Exception from HRESULT: 0x80131515)
    At C:\RBS\Support\Remove-DuplicateItems-master\Remove-DuplicateItems.ps1:1316 char:5
    + Import-ModuleDLL -Name ‘Microsoft.Exchange.WebServices’ -FileName …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Import-ModuleDLL

    Here’s the command I was using:

    .\Remove-DuplicateItems.ps1 -Identity -Credentials $Credentials

    Any help is appreciated.


      • Hi Michel,

        We are in the process of moving to Office 365 and on our on-prem Exchange we use Import-CalendarCSV.ps1 and then Remove-DuplicateItems.ps1 for for creating and cleaning up Calendar items.

        Could you expand on your comment of ‘Likely a .NET Framework dependency missing. The DLLs from nuget package depend on .NET Framework 4.0’
        I can’t find out what i need to install to get this working, i have the same problem.

        I also can’t get impersonation working properly…it looks pretty straight forward from the guides I have found. You mention licensing in your article about set up of impersonation I can’t see anything about this anywhere else.

        Thanks in advance 🙂



        • The EWS Managed API is built for certain .NET Framework versions. Depending on your OS, this one might be installed or not. NuGet package depends on .NET Framework 4. Also make sure that if you manually copy files over from internet locations, those might be blocked – this includes loading of DLLs as well (Explorer > Properties and see if you need to Unblock)


          • Hey Michel,

            Thanks for that, I’ll check the DLLs, it’s when I try and install using powershell. In either Windows 10 or server 2012r2



  92. Hi Michel,

    Thanks for your help, I managed to get it working by copying the DLL’s into the script folder.
    I have a problem with the script not removing duplicates, how does it check for a duplicate?
    It is on the title and time?

    What’s really odd is that sometimes it will remove items from the calendar if I leave it for a day….but not all items.

    For testing i have a CSV file that I import twice, then I run the remove duplicates.

    Thanks for any help 🙂


  93. Hey Michel,

    I have a problem getting the script to work leveraging OAuth as a registered App.

    After migrating to O365 most mailboxes are full of duplicates and we would like to run your script against the tenant to clean up all mailboxes, without having to have all our users type in / give up their credentials. As I understood it, we should be able to achieve this, using OAuth / registered App and running the script against a tenant-id, providing a csv-file with all users and using a client-id and the corresponding secret.

    Individually, all these components seem to work fine, but when put together, as per your last example in the script, (altered according to our tenant / client-id / secret, of course), we still get the message:
    Cannot bind to MsgFolderRoot: Exception when calling “Bind” with 2 Arguments: “Credentials are required to make a service request.”

    What am I overlooking?

    Thank you in advance for your time and response.



  94. Hi Michel

    Was hoping for some assistance here, getting a generic error running the script against a single mailbox in Exchange 2013 CU 23 environment:

    “Parameter set cannot be resolved using the specified named parameters.”

    I am executing the following:

    .\Remove-DuplicateItems.ps1 -Identity %MailBox_Name% -type All -Impersonation -DeleteMode HardDelete -Mode Full -Retain oldest -Force -Server %ServerName%

    Am I missing any mandatory parameters here? Thanks in advance!


  95. Thanks so much for the prompt response, I did use the credentials parameter by declaring it in a variable then executing the script, so the following are now used:

    $Credentials = Get-Credential

    Password has been converted to securestring also

    .\Remove-DuplicateItems.ps1 -Identity %MailBox_Name% -Credentials $Credentials -type All -Impersonation -DeleteMode HardDelete -Mode Full -Retain oldest -Force -Server %ServerName%

    But when using the creds parameter I look to run into an issue with converting it back to a string value it appears:

    Invalid credentials provided: Cannot convert value “%cleartextPassword” to type “System.Int32”. Error: “Input string was not in a correct format.”

    Thanks again!


  96. Hello, I’m from France, nice script, I’m trying to get this work but I’m getting errors after launching the script. Thanks for your help

    .\Remove-DuplicateItems.ps1 -Identity Mail -DeleteMode MoveToDeletedItems -Mode Full -Credentials (Get-Credential) -Retain Oldest

    The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks (“&”) to pass it as part of a string.
    At C:\Users\OKIALY\Downloads\Remove-DuplicateItems.ps1:220 char:189
    + … ata-ga-click=”(Logged out) Header, go to Features”>Features <span cla …
    + ~
    The '→Mobile <span cla …
    + ~
    The '→</s …
    + ~


  97. Hi! thank you for script, but I execute on office365 account and I have this error
    “Cannot access mailbox information store for acount@xxx…. The
    property ‘DisplayName’ cannot be found on this object. Verify that the property exists.
    Whats is it?


      • Thank you for feedback, i solved problem change user language to English.

        But my problem not was solved i did migration from Google, users duplicate mail in two or more folders, don’t same folder, because on google if message remove inbox, and check with one or more labels, this message migrate to Office365 for multiple folders

        Message1 archived on Google without label, migrate to Archive on Office365
        Massage2 archived on Google and check label “A” migrate to Office365 folder ‘Archive’ and Folder “A”

        On Google massage2 appears only one time

        Sorry my english!


        • No worries, I understood you question. We can do per-mailbox removal of duplicate entries for which the GMail scenario you described would be an excellent example. You may want to look into the CleanupMode parameter, setting it to Mailbox. To control if folders get preference (items retained there before other folders), use the PriorityFolders parameter.


          • Hi! I execute cmd but don’t clean mailbox, i have this message, thank you for support

            PS C:\Tools> .\Remove-DuplicateItems.ps1 -Identity vallery.vier@….. -Type Mail -CleanupMode Mailbox -DeleteMode HardDelete -Mode Full -Server -Credentials (Get-Credential) -verbose

            cmdlet Get-Credential at command pipeline position 1
            Supply values for the following parameters:
            VERBOSE: Module Microsoft.Exchange.WebServices v2.2.1.0 already loaded
            VERBOSE: Module Microsoft.Identity.Client v4.25.0.0 already loaded
            VERBOSE: Using credentials ti@….
            VERBOSE: Cleanup Mode: Mailbox
            Processing mailbox vallery.vier@…. (vallery….@)
            VERBOSE: Using Exchange Web Services URL
            VERBOSE: Constructing folder matching rules
            VERBOSE: Processing primary mailbox vallery….@
            VERBOSE: Collecting folders to process..
            VERBOSE: Adding folder \Archive (priority 0)
            VERBOSE: Adding folder \ARQUIVO DGS (priority 0)
            VERBOSE: Adding folder \Deleted Items (priority 0)
            VERBOSE: Adding folder \Drafts (priority 0)
            VERBOSE: Adding folder \Important (priority 0)
            VERBOSE: Adding folder \Inbox (priority 0)
            VERBOSE: Adding folder \Junk Email (priority 0)
            VERBOSE: Adding folder \Outbox (priority 0)
            VERBOSE: Adding folder \Sent Items (priority 0)
            VERBOSE: Adding folder \Starred (priority 0)
            VERBOSE: Found 10 folders that match search criteria
            VERBOSE: Processing folder \Junk Email
            VERBOSE: Processing folder \Inbox
            VERBOSE: Processing folder \Outbox
            VERBOSE: Processing folder \Starred
            VERBOSE: Processing folder \Sent Items
            VERBOSE: Processing folder \ARQUIVO DGS
            VERBOSE: Processing folder \Archive
            VERBOSE: Removing 1 items from \Archive
            VERBOSE: Processing folder \Deleted Items
            VERBOSE: Processing folder \Important
            VERBOSE: Processing folder \Drafts
            VERBOSE: Cleaning unique list (finished mailbox)
            17536 items processed and 1 removed in 00:05:04 – average 3455 items/min
            WARNING: Cannot bind to ArchiveMsgFolderRoot: Exception calling “Bind” with “2” argument(s): “The specified folder
            could not be found in the store.”
            VERBOSE: Processing vallery….. finished


            • The default mode uses an attribute which Exchange uniquely assigns to every item. Since you’re coming from GMail, that might not work, and you may need to use attribute-level comparison (-Mode Full). Note that before you perform a run, you can report on duplicate items using -Report -WhatIf:$True


              • Hello! I run PS cmd with mode full, Do i need to do other conf at my line?

                .\Remove-DuplicateItems.ps1 -Identity -Type Mail -CleanupMode Mailbox -DeleteMode HardDelete -Mode Full -Server -Credentials (Get-Credential) -verbose



                • Looks good to go. If duplicates are still not detected, one of the attributes may differ between folderA and folderB. There is another option “Body” comparing only body messages, but there might be legal implications. Last resort is to tweak the comparison.


  98. What if the duplicated messages are in the deleted items folder? can you tell me how to make him scan only the deleted items folder?


    • Specify IncludeFolders parameter to limit operation to certain folders or folder structures matching patterns. As mentioned in the help for IncludeFolders, you can specify well-known folders such as Deleted Items using ‘##’, e.g. ‘#DeletedItems#’. Deleting items from deleting items doesn’t make sense, but DeleteMode by default moves them to the dumpster (Recoverable Items), so you don’t need to change that.
      Command would then be something like
      .\Remove-DuplicateItems.ps1 -Identity -Impersonation -IncludeFolders ‘#DeletedItems#’
      You can always specify -WhatIf:$true to see what it would do, or add -Verbose to see additional information.


  99. I am having issues with a 401 Unauthorized error.

    PS C:\Users\users\Downloads\Remove-DuplicateItems-1.84\Remove-DuplicateItems-1.84> .\Remove-DuplicateItems.ps1 -Mailbox -Server -Credentials $Credentials -IncludeFolders Inbox -MailboxOnly -Verbose
    VERBOSE: Loaded EWS Managed API v2.2.1.0
    VERBOSE: Set to trust all certificates
    VERBOSE: Using credentials
    VERBOSE: Processing items of type All, delete mode is SoftDelete
    Processing mailbox (
    VERBOSE: Using Exchange Web Services URL
    VERBOSE: Constructing folder matching rules
    WARNING: Cannot bind to MsgFolderRoot – skipping. Error: Exception calling “Bind” with “2” argument(s): “The request
    failed. The remote server returned an error: (401) Unauthorized.”
    VERBOSE: Processing finished


    • – Are credentials valid and do credentials provide access to this mailbox – full access (when granted properly, eg using InheritanceType on top of information store) or impersonation (yet impersonation not specified, so guess not)
      – Is Basic Authentication allowed?
      – Is Basic Authentication allowed for the condition, e.g. location and user (e.g. no Conditional Access blocking things)


  100. failing at the first hurdle, trying both ISE and exchange online PowerShell.
    Have tried different parameters and my mailbox has full access, although reading this error – this isn’t permissions.

    Remove-DuplicateItems.ps1 : Parameter set cannot be resolved using the specified named parameters.
    At line:1 char:1


  101. receiving the error, using both ise and powershell for exchange online.

    s.ps1 -Identity -Type contacts -Server -Verbose
    D:\Scripts\RBetts\Remove-DuplicateItems.ps1 : Parameter set cannot be resolved using the specified named parameters.
    At line:1 char:1


    • You need to provide one of the authentication methods:
      * Basic using -Credential
      * Modern Auth using TenantId, ClientId and one of the following: Secret, CertificateThumbprint, CertificateFile(+CertificatePassword as secure string)


    • Seem to be missing one or more parameters for one of the authentication methods, eg
      * Basic using -Credential
      * Modern Auth using TenantId, ClientId and one of the following: Secret, CertificateThumbprint, CertificateFile(+CertificatePassword as secure string)


  102. Hi!

    I get this error when running against o365

    Write-Verbose : A parameter cannot be found that matches parameter name ‘f’.
    At C:\Users\xxx\Desktop\Remove-DuplicateItems-master\Remove-DuplicateItems-master\Remove-DuplicateItems.ps1:1438 char:5
    + Write-Verbose ‘Using EWS endpoint {0}’ -f $EwsService.Url
    + ~~
    + CategoryInfo : InvalidArgument: (:) [Write-Verbose], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.WriteVerboseCommand


      • Thanks for the quick reply!

        FYI – if anyone gets this error :
        Cannot bind to MsgFolderRoot: Exception calling “Bind” with “2” argument(s): “The specified object was not
        found in the store., The process failed to get the correct properties.”
        WARNING: Cannot bind to ArchiveMsgFolderRoot: Exception calling “Bind” with “2” argument(s): “The specified folder
        could not be found in the store.”

        this means that user which credentials you used to connect doesn’t have permissions to read the mailbox. in my case I used this script against a shared mailbox and used global admin account (no MFA) to connect and it did not work until I gave the global admin read ann manage rights for the shared mailbox.


  103. Hi,

    I am getting following error. Any Ideas?
    [PS] C:\temp\Remove-DuplicateItems-master\Remove-DuplicateItems-master>.\Remove-DuplicateItems.ps1 -Identity -Credentials $Credent
    ials -Type Calendar -Mode Quick -MailboxOnly -Server -Verbose
    AUSFÜHRLICH: Module Microsoft.Exchange.WebServices v2.2.1.0 already loaded
    AUSFÜHRLICH: Module Microsoft.Identity.Client v4.25.0.0 already loaded
    AUSFÜHRLICH: Using credentials
    AUSFÜHRLICH: Cleanup Mode: Folder
    Processing mailbox ()
    AUSFÜHRLICH: Using Exchange Web Services URL
    AUSFÜHRLICH: Constructing folder matching rules
    AUSFÜHRLICH: Processing primary mailbox
    AUSFÜHRLICH: Collecting folders to process..
    C:\temp\Remove-DuplicateItems-master\Remove-DuplicateItems-master\Remove-DuplicateItems.ps1 : Cannot access mailbox information store for
    : Ausnahme beim Festlegen von “PropertySet”: “Der Wert “Microsoft.Exchange.WebServices.Data.PropertySet” vom Typ
    “Microsoft.Exchange.WebServices.Data.PropertySet” kann nicht in den Typ “Microsoft.Exchange.WebServices.Data.PropertySet” konvertiert werden.”
    In Zeile:1 Zeichen:1
    + .\Remove-DuplicateItems.ps1 -Identity -Credent …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Remove-DuplicateItems.ps1


  104. Hello,

    Looking over your script to see if this is something we can implement. During our testing with it, we are finding that it does identify duplicate items for the mailbox, but is not moving forward with deleting the items? We are noting seeing much in Verbos output that would lead us to any conclusions so thought I would ping you and pick your brain.

    Hybrid M365

    Thank you again!


    • By default is identifies duplicates based on PR_SEARCH_KEY. Depending on how you got the duplicates in, that may not work and you may need attribute-level comparison (with or without size using -NoSize). Also make sure proper permissions have been set up, impersonation is preferred.


  105. Parameter set cannot be resolved using the specified named parameters.
    + CategoryInfo : InvalidArgument: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : AmbiguousParameterSet


    • You need to provide the script the right parameters so it knows which authentication method to use (and what parameters to expect, as it’s now ambigious):
      1) BasicAuth : -Credentials
      2) Modern Auth:
      2a) -CertificateThumbprint -TenantId -ClientId
      2b) -Secret -TenantId -ClientId
      2c) -CertificateFile [-CertificatePassword ] -TenantId -ClientId


  106. I was having issues getting this to authenticate for the longest time, always getting 401 Unauthorised. I even setup a policy to allow EWS basic auth for the account that I was trying to use for impersonation. As it turned out, it was disabled at a tenant level, and per-user policies weren’t going to cut it.

    This fixed the issue permanently for me.


  107. Hi Not sure if i have missed something but i followed the details of this post and the one for the EWS.Websrvices.Managed.Api post but i am getting the following error when attempting to run the script:
    Exception calling “LoadFile” with “1” argument(s): “An attempt was made to load an assembly from a network location which would have caused the assembly to be sandboxed in previous versions
    of the .NET Framework. This release of the .NET Framework does not enable CAS policy by default, so this load may be dangerous. If this load is not intended to sandbox the assembly, please
    enable the loadFromRemoteSources switch. See for more information.”
    At C:\users\\Documents\PS\Remove-DuplicateItems-1.84\Remove-DuplicateItems.ps1:343 char:13
    + [void][Reflection.Assembly]::LoadFile( “$EWSDLLPath\$EWSD …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : NotSupportedException

    Load-EWSManagedAPIDLL : Problem loading Microsoft.Exchange.WebServices.dll
    At C:\users\\Documents\PS\Remove-DuplicateItems-1.84\Remove-DuplicateItems.ps1:994 char:5
    + Load-EWSManagedAPIDLL
    + ~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Load-EWSManagedAPIDLL

    Is there something im missing here?


    • Ok so I have found what i missed.
      There is the need to Unblock not just the DLL and XML files but also the PS1 file as well.
      This has allowed the script to run.

      I am now getting another error:
      C:\users\\Documents\PS\Remove-DuplicateItems-1.84\Remove-DuplicateItems.ps1 : Autodiscover failed, error: Exception calling “AutodiscoverUrl” with “2” argument(s): “The Autodiscover
      service couldn’t be located.”
      At line:1 char:1
      + .\Remove-DuplicateItems.ps1 -Identity -Typ …
      + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
      + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Remove-DuplicateItems.ps1

      Ill be looking furhter into this myself and if i find the answer to this also i will add this to my reply.


  108. This script looks incredible, so first off, thank you for the dedicated work on it over the years.

    Unfortunately I can’t get this to authenticate with Exchange Online. I’m not 100% sure how the App Registration in AAD interacts with EXOL. I’m using an existing App Registration created for BitTitan to migrate emails into our tenant, so it should already have impersonation permissions as BitTitan can write into any mailbox in our tenant and we use the impersonation option within BitTitan. The app has Office 365 Exchange Online – EWS.AccessAsUser.All delegated permission, as well as Microsoft Graph – User.Read. I’ve added a secret to the app to use with this script. I’m running the following:

    $secret = ConvertTo-SecureString “XXXXXXXXXXX” -AsPlainText -Force

    .\Remove-DuplicateItems.ps1 -Identity -Type Mail -DeleteMode MoveToDeletedItems -Mode Full -Retain Oldest -IncludeFolders ‘#Inbox#\*’ -Server -Secret $secret -TenantId -ClientId

    I get the following result:

    WARNING: Cannot bind to Inbox: Exception calling “Bind” with “2” argument(s): “The request failed. The remote server returned an error: (401) Unauthorized.”
    WARNING: Cannot bind to MsgFolderRoot: Exception calling “Bind” with “2” argument(s): “The request failed. The remote server returned an error: (401) Unauthorized.”
    WARNING: Cannot bind to ArchiveMsgFolderRoot: Exception calling “Bind” with “2” argument(s): “The request failed. The remote server returned an error: (401) Unauthorized.”

    We have Modern Authentication enabled in our tenant and Basic Authentication turned off for all options. I can see the app authentication in the AAD sign-in logs as being successful, so it feels like that part of the authentication process is working, but not something at the EXOL level maybe? I don’t know.

    Any pointers would be greatly appreciated! Thanks in advance.


    • I assume you also specified the proper ‘TenantId’ and ‘ClientId’ IDs with those parameters (not specified)?
      Since the script is not from a verified publisher, we need app permissions, not delegated permissions. Also, the permission I usually assign is Office 365 Exchange Online > full_access_as_app; if orgs worry about this blanket permission, they can restrict it to certain mailboxes using application access policies.


      • Thanks for the reply Michel.

        After a bit of research I came across that exact permission, which I applied to the Azure AD App Registration and then it all started working.

        Yes I did indeed have the Tenant & Client IDs specified, I just removed them from the command.

        Thanks again for the script, it was just what I needed!


  109. Great script. Question: I’m testing it now and it seems to remove the first of 2 emails matched (or, the first one displayed in Outlook, at any rate). What would need to be changed to make it remove the 2nd one instead? I realize this is a fringe case, but it is useful if you have to import emails exported from IMAP, but you already imported some of them via ActiveSync, and some of the headers are different, apparently, so internal duplicate detection in Outlook doesn’t work… *sigh*. Seems the first displayed email (from the ActiveSync export) retains some of the group name expansion and short names, whereas the 2nd one (from the IMAP export) does not, so it would make more sense to keep the first one in my case just for the user experience.


    • Defining a criterium based on headers is a challenge, apart from the potential performance as you have to inspect each item.
      Only thing you can use for for determining what gets deleted is Retain, where you can specify Oldest or Newest. Depending on the way on you those duplicates got in, you may want to use Mode Full to have attribute-level comparison instead of PidTagSearchKey, which is an attribute set by Exchange when the item gets stored, and is preserved when copying, exporting/importing etc.


  110. I receive this error when i launch the script…i don’t know how to solve.. Can you help me?

    PS C:\script> .\Remove-DuplicateItems.ps1 -Mailbox -Server -Credentials $Credentials -Type Mail -DeleteMode HardDelete -CleanupMode Mailbox -Verbose -WhatIf:$true -Report
    DETTAGLIATO: Loading module C:\Program Files\PackageManagement\NuGet\Packages\Exchange.WebServices.Managed.Api.\lib\net35\Microsoft.Exchange.WebServices.dll
    DETTAGLIATO: Caricamento del modulo dal percorso ‘C:\Program Files\PackageManagement\NuGet\Packages\Exchange.WebServices.Managed.Api.\lib\net35\Microsoft.Exchange.WebServices.dll’.
    DETTAGLIATO: Module Microsoft.Exchange.WebServices v2.2.1.0 loaded
    DETTAGLIATO: Loading module C:\Program Files\PackageManagement\NuGet\Packages\Microsoft.Identity.Client.4.43.0\lib\monoandroid10.0\Microsoft.Identity.Client.dll
    DETTAGLIATO: Caricamento del modulo dal percorso ‘C:\Program Files\PackageManagement\NuGet\Packages\Microsoft.Identity.Client.4.43.0\lib\monoandroid10.0\Microsoft.Identity.Client.dll’.
    Import-ModuleDLL : Problem loading module Microsoft.Identity.Client: Non è stato possibile caricare il file o l’assembly ‘Mono.Android, Version=, Culture=neutral, PublicKeyToken=84e04ff9cfb79065’ o
    una delle relative dipendenze. Impossibile trovare il file specificato.
    In C:\script\Remove-DuplicateItems.ps1:1379 car:5
    + Import-ModuleDLL -Name ‘Microsoft.Identity.Client’ -FileName ‘Mic …
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Import-ModuleDLL


  111. Great script 🙂 However I’m having an issue getting it to process Inbox subfolders.

    I use the following parameter -IncludeFolders ‘#INBOX#\*’ and the script process the Inbox OK but ignores the dozens of subfolders. The mailbox is hosted in Exchange Online BTW.

    Am I missing something?


    • I just found that if I create a new subfolder, the script finds it. But it isn’t finding the dozens of existing folders (that were imported from a PST file if that makes any difference).


      • Are you using Impersonation? If not, does the account provided have sufficient permissions on those imported subfolders.
        Also, have you specified -Type Mail by any chance? Could be the Folder type of those imported folders are not Mail (IPF.Note)


  112. Great script, doing what’s advertised, love it. Ran the script for the shitload of extra data that came to be due to the old registry setting to copy a sent email in the shared mailbox sent items, which as well was done by EAC rule… just love (not) what was done by my predecessors without documenting their actions and hiding it in obscure gpo’s…

    I made 1 (cosmetic) adjustment – line 1155 had ‘Processed folder {0} or {1}’ which should of course be ‘Processed folder {0} of {1}’ 😉


    • Perhaps it is not finding what you expected it to find. There could be many reasons, usually because properties not directly visible that could identify duplicates are not identical. Also, make sure you have proper permissions on mailbox and folder, preferably using Impersonation so you do not have that issue.


  113. Some reason the script is throwing below error in Ms365 online mailbox

    PS C:\Users\support\Desktop\RemoveDuplicateiteminmailbox> .\Remove-DuplicateItems.ps1 -Mailbox agnoc -Type All -Impersonation -DeleteMode HardDelete -Mode Full -Verbose
    At C:\Users\support\Desktop\RemoveDuplicateiteminmailbox\Remove-DuplicateItems.ps1:246 char:1
    + [cmdletbinding(
    + ~~~~~~~~~~~~~~~
    Unexpected attribute ‘cmdletbinding’.
    At C:\Users\support\Desktop\RemoveDuplicateiteminmailbox\Remove-DuplicateItems.ps1:251 char:1
    + param(
    + ~~~~~
    Unexpected token ‘param’ in expression or statement.
    At C:\Users\support\Desktop\RemoveDuplicateiteminmailbox\Remove-DuplicateItems.ps1:285 char:20
    + [string]$Type= ‘All’,
    + ~~~~~
    The assignment expression is not valid. The input to an assignment operator must be an object that is able to accept assignments, such as a variable or a property.
    At C:\Users\support\Desktop\RemoveDuplicateiteminmailbox\Remove-DuplicateItems.ps1:302 char:22
    + [string]$Retain= ‘Newest’,
    + ~~~~~~~~
    The assignment expression is not valid. The input to an assignment operator must be an object that is able to accept assignments, such as a variable or a property.
    At C:\Users\support\Desktop\RemoveDuplicateiteminmailbox\Remove-DuplicateItems.ps1:351 char:26
    + [string]$DeleteMode= ‘SoftDelete’,
    + ~~~~~~~~~~~~
    The assignment expression is not valid. The input to an assignment operator must be an object that is able to accept assignments, such as a variable or a property.
    At C:\Users\support\Desktop\RemoveDuplicateiteminmailbox\Remove-DuplicateItems.ps1:368 char:20
    + [string]$Mode= ‘Quick’,
    + ~~~~~~~
    The assignment expression is not valid. The input to an assignment operator must be an object that is able to accept assignments, such as a variable or a property.
    + CategoryInfo : ParserError: (:) [], ParseException
    + FullyQualifiedErrorId : UnexpectedAttribute


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.