Remove Duplicate Files

Duplicate File Detective

Thursday, February 18, 2010

Duplicate File Detective v3.5 Released

We're proud to announce the release of Duplicate File Detective v3.5, available for immediate download from our product website.

New in v3.5:

- New "folder view" docking panel
- Search path protection
- File extension researcher tool
- Exclusion of protected file types during search
- Symbolic and hard link creation support
- Hard link identification during duplicate search
- New "group count" and "group size" report columns
- Improvements to search logging
- Many usability and performance enhancements
- Various bug fixes

For additional details on this release, please see the product release notes.

Thursday, December 17, 2009

Duplicate File Detective v3.0.1.69 Released

Duplicate File Detective v3.0.1.69 is now available for download.

This is a maintenance release, and is a free upgrade for all existing v3.0 license holders. It resolves a handful of bugs and provides minor feature enhancements. Full release notes here.

Tuesday, December 8, 2009

Actual vs. Allocated File Sizes

Did you know that Duplicate File Detective is capable of showing both actual and allocated space for the duplicate files it finds? Here's a screen shot (click to enlarge):

allocated duplicate file size

The "allocated" column shown above is hidden by default, but it's simple to make it visible. Just click the View tab in the ribbon bar, then click the Column Chooser button. This will cause the Column Chooser pop-up window to appear, and you can drag-and-drop columns into the report view in any position you like.

Another way to show this report column is to right-click on the duplicate search report header and choose it from the Columns pop-up menu.

So - what's the difference between allocated (also known as "size on disk") and actual disk space? To find out, please check out this blog article on

Monday, November 30, 2009

Create Shortcuts to Moved (or Removed) Duplicate Files

Duplicate File Detective v3.0 contains a large number of new features that are the direct result of user feedback.

One of the most important of these is the ability to create shell shortcuts when moving or removing (deleting) duplicate files. The idea here is quite simple - if you have (for example) a group of four duplicate files, it should be possible to move or delete three of them and create shell shortcuts in their place - those shell shortcuts will then resolve (or point) to the first unmarked file in the same duplicate group.

You can also create shell shortcuts when archiving duplicate files (e.g. adding them to a zip file), as long as you elect to remove the original duplicate files as they are added to the archive (the original file must no longer exist in order for its replacement with a shell shortcut to make sense).

Let's take a look at how these options are presented within Duplicate File Detective v3.0. Please click the screen shot below for a slightly larger image.

delete, move duplicate files with shell shortcuts

When creating shell shortcuts to moved or deleted duplicate files, the prerequisites are as follows:

  1. Duplicate files must be moved or deleted to make room for their replacement with shell shortcuts.
  2. At least one member of each duplicate group being processed must be unmarked. This is the target file to which duplicate shortcuts will resolve.
  3. If any of the above requirements are not met, shell shortcut creation will not be attempted for that duplicate group.

The benefit of this approach is clear - if you're working to dedupe a large document repository, for example, you can archive a series of duplicate files into a single large zip file and still allow users to access those files through (newly created) shell shortcuts.

In future releases of Duplicate File Detective, we'll be expanding upon this concept even further. Please stay tuned for details.

Wednesday, November 25, 2009

Calculating File Hashes (Checksums) in Duplicate File Detective v3

Today I'd like to briefly illustrate some of the improvements made to the file hash calculator tool in Duplicate File Detective v3.0.

First, and perhaps most importantly, the file hash calculator has been re-designed to allow for the generation of multiple file hash types for multiple file paths - all in a single pass.

file hash / checksum generation with Duplicate File Detective v3As you can see in the adjacent screen shot, it is now possible to generate CRC32, ADLER32, MD5, SHA1, SHA256, and SHA512 file checksums with only a single read of each target file.

The output of the file hash / checksum operations has also been re-engineered to display results in a hierarchical (tree like) fashion. For each file hashed, a series of child nodes show the hash value for each type (e.g. CRC32, MD5, etc.) selected in the area above.

It is also possible to copy file hash results to the Windows clipboard, in which case they are "flattened" into a series of plain-text lines.

Here's a quick sample of what the hash results shown above look like on the Windows clipboard (folder names and hash values are truncated for easier viewing in this limited space):

C:\MoveTest\Sample Pictures\Lighthouse.jpg (CRC32): A3A2B75C
C:\MoveTest\Sample Pictures\Lighthouse.jpg (MD5): 8969288F4245120E7C3870287CCE...
C:\MoveTest\Sample Pictures\Lighthouse.jpg (Adler32): A6EA1A41
C:\MoveTest\Sample Pictures\Lighthouse.jpg (SHA1): 1B4605B0E20CECCF91AA278D10E...
C:\MoveTest\Sample Pictures\Lighthouse.jpg (SHA256): FF86372CE43519D675B8D8D29...
C:\MoveTest\Sample Pictures\Lighthouse.jpg (SHA512): 9BDD0C215A9BE94F6F677F8AD...
C:\MoveTest\Sample Pictures\Hydrangeas.jpg (CRC32): 03632F46
C:\MoveTest\Sample Pictures\Hydrangeas.jpg (MD5): BDF3BF1DA3405725BE763540D66...

The actual file hash / checksum generation process is extremely fast and efficient, and can also be canceled if needed.

Finally, it is also now possible to right-click any number of selected files within the main Duplicate File Detective result report and elect to generate hash values for them. This combination makes it very easy to generate and compare hash values of arbitrary strength (e.g. bit length) as needed.

Tuesday, November 24, 2009

Duplicate File Detective 3 is Released

Key Metric Software is proud to announce the release of Duplicate File Detective v3.0, a major upgrade to our professional-grade duplicate file management solution.

Duplicate File Detective v3.0 provides a number of new features that focus on extending the scalability and reach of the product. New features include full Unicode support, new duplicate file processing options (such as the ability to replace moved or deleted duplicates with shell shortcuts), duplicate search logging, support for Windows 7 and Server 2008, and scalability improvements that allow the product to report upon (and process) millions of duplicate file entries in a single session.

- More Info on What's New in v3
- Full Release Notes
- Download Duplicate File Detective v3.0 Now

We're extremely proud of this release and, as always, welcome your thoughts and feedback.

Thursday, March 27, 2008

Duplicate File Detective v2.2.0.0 Released

Duplicate File Detective v2.2.0.0 has been released, and it provides a number of feature enhancements and bug fixes including:

  • Support for SHA256 and SHA512 hashing algorithms
  • Support for encoding hash / checksum as base64 (in addition to hex)
  • Greatly improved SmartMark system support
  • Numerous improvements to the built-in file hash calculation tool
  • Improved scan filtering capabilities
  • A broad range of performance enhancements
  • Experimental support for low-priority I/O on Windows Vista and Server 2008
  • A handful of bug fixes and other improvements

Please take a moment to review the complete release notes, and then download v2.2 when ready. This is a free upgrade for all v2.x license owners.