We are living in a big data world which is both a blessing and a curse. Big data usually means a huge number of files such as photos and videos and finally a huge amount of storage space. Files are accidentally or deliberately moved from location to location without first considering that these duplicate files consumes more and more storage space. I want to change that with you in this blog post. We will search duplicate files and then move them to a different storage location for further review.

The Goal

With my script in hand you are able to perform the described scenario. Make sure your computer runs Windows PowerShell 5.1 or PowerShell 7.

1. Open PowerShell (Windows Key + X + A)
2. Navigate to the script location. Enter the full path to the destination folder. This folder is our target for searching for duplicate files
3. A window will pop-up to select duplicate files based on the hash value. All selected files will be moved to C:\DuplicatesCurrentDate
4. Afterwards the duplicate files are moved to the new location. You will again see a new window appearing that shows the moved files for further review

Which brings me to the code.

The Script

Here is the code for download.

find_duplicate_files.ps1

And here is the code in full length. Copy the code to your local computer and open it in PowerShell ISE, Visual Studio Code or an editor of your choice. Hit F5 (PowerShell ISE or VS Code).

# .SYNOPSIS
# find_ducplicate_files.ps1 finds duplicate files based on hash values.

# .DESCRIPTION
# Prompts for entering file path. Shows duplicate files for selection.
# Selected files will be moved to new folder C:\Duplicates_Date for further review.

# .EXAMPLE
# Open PowerShell. Nagivate to the file location. Type .\find_duplicate_files.ps1 OR
# Open PowerShell ISE. Open find_duplicate.ps1 and hit F5.

# .NOTES
# Author: Patrick Gruenauer | Microsoft MVP on PowerShell [2018-2020]
# Web: https://sid-500.com

############# Find Duplicate Files based on Hash Value ###############
''
$filepath = Read-Host 'Enter file path for searching duplicate files (e.g. C:\Temp, C:\)'

If (Test-Path $filepath) {
''
Write-Warning 'Searching for duplicates ... Please wait ...'

$duplicates = Get-ChildItem $filepath -File -Recurse `
-ErrorAction SilentlyContinue |
Get-FileHash |
Group-Object -Property Hash |
Where-Object Count -GT 1

If ($duplicates.count -lt 1)

{
Write-Warning 'No duplicates found.'
Break ''
}

else {
Write-Warning "Duplicates found."
$result = foreach ($d in $duplicates)
{
$d.Group | Select-Object -Property Path, Hash
}

$date = Get-Date -Format "MM/dd/yyy"
$itemstomove = $result |
Out-GridView -Title `
"Select files (CTRL for multiple) and press OK. Selected files will be moved to C:\Duplicates_$date" `
-PassThru

If ($itemstomove)

{
New-Item -ItemType Directory `
-Path $env:SystemDrive\Duplicates_$date -Force
Move-Item $itemstomove.Path `
-Destination $env:SystemDrive\Duplicates_$date -Force
''
Write-Warning `
"Mission accomplished. Selected files moved to C:\Duplicates_$date"

Start-Process "C:\Duplicates_$date"
}

else
{
Write-Warning "Operation aborted. No files selected."
}
}
}
else
{
Write-Warning `
"Folder not found. Use full path to directory e.g. C:\photos\patrick"
}

Credits

Thanks to Kenward Bradley’s one-liner which sparks the idea in me to write this script. Here you go:

http://kenwardtown.com/2016/12/29/find-duplicate-files-with-powershell/

See also

Cool stuff? Take a look at my other scripts here:

Downloads Section sid-500.com