Duplicate File Detective - Release Notes

Maintained by Mark Richards (mark@keymetricsoft.com), President & CEO of Key Metric Software.
To download the latest version of Duplicate File Detective, visit the product website.

Version 2.2.0.0 (Released March 25th, 2008)

• Feature: Added SHA256 and SHA512 hash algorithm support to Duplicate File Detective (accessible via advanced project settings window).
• Feature: Added the ability to alter the encoding type used to represent file hash strings. By default, hexadecimal encoding is used, but this can now be switched to Base64. Hex has the advantage of using only uppercase characters and numbers (and thus being slightly easier to read), whereas Base64 encoded hash strings are significantly more compact but use a broader character set that includes lowercase letters.
• Feature: Duplicate file marking actions taken through the duplicate report context menu (and other means) will no longer change row selection.
• Feature: New keyboard shortcuts, including Ctrl+M for marking selected files and Ctrl+U for unmarking selected files.
• Feature: Improved tooltips when using the Office 2007 theme (see View menu).
• Feature: Created an "Exclude parent folder" shortcut within the duplicate file report context menu.
• Feature: The file hash calculator tool has been improved in several ways. It now provides access to SHA256 and SHA512 hashing algorithms and a switch for selecting hash encoding type (Hex or Base64). The window is also now resizable and retains configuration state between uses.
• Feature: Added a "reverse current markings" function to the smart marking menu. This simply inverts whatever duplicate file markings currently exist. Markings can also be reversed for a specific group (via right-click context menu).
• Feature: Duplicate File Detective will now automatically mark all members of a duplicate file group, with the exception of the first. If you prefer the previous behavior (where no files were marked by default), see the new setting in the general tab of the preferences window.
• Feature: The image preview window will now show a shell icon for duplicate files that are not actually images.
• Feature: Expanded SmartMark capabilities with a "mark all but last file in each duplicate group" function, also available from the duplicate report context menu.
• Feature: The speed at which the Duplicate Result Manager window appears has been dramatically improved, especially when dealing with a large number of duplicate file markings.
• Feature: Added a new file size comparison mechanism to the search filtering docking window that allows for isolation of files "between" two specified sizes.
• Feature: Further extended the duplicate file context menu (e.g. the right-click menu) to allow the user to mark all duplicates of a file in the same folder - or all other folders. This is different from the existing "mark all files in same folder' option, which works across all duplicate groups.
• Feature: Added experimental support for low-priority disk I/O on Windows Vista / Server 2008 and later. This switch, available from the Advanced tab of the Global Settings window, causes file system enumeration and file hashing threads to operate in a special, low-priority mode that makes them less likely to interfere with other functions the host computer may be performing.
• Feature: The Duplicate Result Manager window file list is now fully sortable by any column.
• Feature: Added a simple "Search with Duplicate File Detective" Explorer shell context menu capability.
• Bug fix: Widened cumulative file size display field within Duplicate File Manager window.
• Bug fix: Resolved a problem where "an unnamed file was not found" would be displayed when attempting to access a recent project file (via the File menu) that no longer existed. The error message is now clearer and more descriptive.
• Bug fix: Resolved a bug in the file name filtering system that would cause wildcard expressions containing spaces to function incorrectly.
• Bug fix: Resolved a rare "R6002 - Floating point support not loaded" error that could occur in some environments. 
• Other performance optimizations, help file updates, bug fixes and improvements.

Version 2.1.1.0 (Released November 24th, 2007)

• Feature: Added a total file space metric to the Duplicate Result Manager window. Provides a better sense of how much space will be freed based upon the pending action.
• Feature: Duplicate file manager window can now be maximized.
• Feature: When files are removed from the Duplicate Result Manager window, the file list scroll position is no longer changed. This makes it easier to track the position of such changes relative to other files being processed.
• Feature: Added "mark all but first" duplicate file marking capabilities to the duplicate report context (right-click) menu, so that this feature can operate at the duplicate group level (in addition to existing SmartMark implementation, which operates on all duplicate groups).
• Feature: When using Duplicate Result Manager to zip files with the "delete originals" option engaged, the deletions will now honor the "delete to Windows recycle" bin option.
• Feature: Numerous improvements to application performance and error handling, particularly within Duplicate Results Manager.
• Bug fix: Resolved a problem with duplicate file project paths not being cleared between loading of projects (v2.1.0.1 regression). 

Version 2.1.0.1 (Released November 20th, 2007)

• Bug fix: Resolved a defect in which DFD could crash when attempting to generate a preview for a duplicate file that had been deleted by an external program.
• Bug fix: Fixed a bug in which the duplicate result manager window could have its protection flags out of sequence.
• Bug fix: Added code to guard against runtime error display (and subsequent abort) when DFD encounters invalid date/time stamps on files.
• Other very minor tweaks.

Version 2.1.0.0 (Released November 15th, 2007)

• Feature: Added support for regular expression syntax to the filename masking system utilized throughout Duplicate File Detective (e.g. scan-time search filtering, smart filename marking, etc.). 
• Feature: Significant enhancements to duplicate file report navigation performance, particularly when scanning network file paths. Image preview loading is now threaded, and operates on a slight delay to allow for smoother file selection transitions within the duplicate result report.
• Feature: Added a new switch to the performance section of the advanced setting dialog that allows users to disable retrieval of file ownership data during the duplicate file search process. Use of this switch can enhance duplicate search performance, especially when scanning network paths.
• Feature: Whenever Duplicate File Detective saves a project to file, it creates a backup copy of the original (if one exists). The location of this .bak file has been moved to the user's local Application Data system folder (previously it was saved to the same path as the project file).
• Feature: Added a switch to the general tab of global preferences that allows for the auto-completion feature of drop-down combo boxes within DFD to be disabled.
• Feature: Added a switch to the View | Toolbars submenu that allows you to "lock all toolbars" (prevent them from being moved / re-docked to other parts of the application frame).
• Feature: Added a warning / confirmation window that appears when duplicate files are processed and one or more groups are found to have all of their entries marked. The display of this window can be managed via the Protection tab within the Global Preferences window. 
• Bug fix: Resolved a memory leak that would occur during XML data import processes.
• Bug fix: Resolved an issue with adding invalid paths via the "search folders" panel where the folder browser dialog would be closed and the last browse path would be added. Now the folder browser path will stay on-screen when an invalid path is entered, allowing the user to correct the situation.
• Bug fix: In some specific cases, clicking the cancel button during a duplicate file comparison process would not immediately cancel pending search operations (e.g. it could take several seconds). Now, cancellation occurs much more quickly (nearly instantly).
• Bug fix: Status bar statistics were not being updated after the Duplicate File Results Manager tool was utilized. Now when duplicates are moved, deleted, or zipped, the status bar will be updated to reflect the change in duplicate file count and disk space consumed.
• Bug fix: File type classification panels are now updated after the Duplicate File Results Manager tool is utilized.
• Bug fix: The auto-completion feature of the drop-down box used to input file name masks (in various places within the application) was not operating in a case-sensitive manner.
• Bug fix: Resolved a crash that could occur on Win9x operating systems when a project is run a second time.
• Bug fix: File size and count values within the File Types detail report were not using a thousands separator character (comma).
• Significant performance enhancements in many areas of the application.
• Numerous minor improvements in usability and appearance. 

Version 2.0.0.0 (Released October 16th, 2007)

• Feature: Added an image preview docking window. Current supported image file formats are: BMP, EMF, GIF, ICO, JPEG, PNG, TIFF, and WMF.
• Feature: Numerous changes (to application file storage, etc.) to make Duplicate File Detective more Windows Vista friendly.
• Feature: The "mark all files in group" context menu option available at the duplicate group level now descends into any sub-groupings that may have been established through the use of the report grouping bar.
• Feature: It is now possible to shift+click and ctrl+click multiple duplicate report record selections and use the "mark selected file(s)" context menu function to mark them.
• Feature: Added the ability to ignore file extensions when performing file name based duplicate matching. When this feature is enabled, the resulting duplicate file report will group entries by file name (minus the extension).
• Feature: Added an option to automatically show the duplicate file summary report upon completion of scanning operations. This checkbox is available within the summary report window itself, and is turned off by default.
• Feature: Added the ability to perform byte-for-byte matching during file comparison operations. This feature is useful when even the smallest potential for false positive matches cannot be tolerated. Although strong hashing algorithms such as MD5 and SHA1 are *extremely* accurate, the only way to be 100% certain that two files match is byte-for-byte binary comparison. Note that use of this feature can slow the file comparison process significantly (especially when comparing large files).
• Feature: Paths in the "search paths" and "exclusions" docking panels can now be double-clicked for editing. A "browse for folder" dialog will appear with the path being edited visible. If a new path is chosen, it will replaced the one that was double-clicked.
• Feature: Numerous improvements to scan-time error handling throughout the application. For example, if a project contains an invalid starting path, it now notifies the user of such once scanning operations are complete. File access exceptions that occur during file content comparison operations are also better exposed now.
• Feature: Added "last accessed" and "created" date / time columns to duplicate file report results. Both new columns can be exported to any of the supported export formats (and can also be re-imported from an XML export). Also note that both columns are hidden by default (right click on the report header and use the field chooser to make them visible).
• Feature:  Created a new "Office 2007" visual theme for the application that closely resembles Microsoft Office 2007 (minus the ribbon bar). Users can freely switch between this and the older Windows system-based theme used by DFD v1.x (see view menu).
• Feature: Added an option to the "appearance" section the preference window that allows users to control whether date-based report columns (such as "created", "modified", etc.) include a time value. By default, they will show date only.
• Feature: Added "attribs" column to duplicate file report results. The column expresses common file attributes as a series of alpha flags. For example, "HSA" is representative of the file attributes "hidden", "system", and "archive". This new column is hidden by default (use the report field chooser to make it visible) and can be included in all export operations.
• Feature: Made extensive improvements to the DFD v1.x "smart marking" system (which is now referred to as "SmartMark"). New capabilities include marking of duplicates file file size (largest or smallest in each group), filename length (longest or shortest file name in each group), and filename pattern. Filename pattern marking supports compound wildcard masks with exclusions, and can also ensure that at least one duplicate file in each group is left unmarked.
• Feature:  A context (e.g. right-click) popup menu has been added to the "search paths" and "exclusions" path management panels. The context menu provides access to existing functions and adds new ones (including the ability to check or uncheck all paths at once).
• Feature: Enhanced the duplicate file scan progress dialog to include a "files compared" indicator. The progress bar control logic has also been revised (and is now more accurate) and is supplemented by progress percentage text.
• Feature: The duplicate scan summary data that is included with XML data exports is now more complete (it now includes files scanned / skipped, folders scanned / skipped, report start and end times, etc.). Scan summary data is also now brought back into the application during an XML data import, making the "Summary" window more useful in these situations.
• Feature: Made numerous performance enhancements, especially with regard to file content comparisons (which can now operate concurrently, making better use of modern hardware capabilities).
• Feature: Added a new "advanced" tab to the global settings window. Currently, it provides access to file hashing thread pool settings, and will be further extended in future releases.
• Feature: The default style sheet (which applies to HTML exports) has been completely revised, and now provides a simpler, more professional appearance. As before, the style sheet can be customized to the needs of the user.
• Feature: Search and exclusion path management intelligence has been expanded considerably. DFD will now attempt to resolve a broader range of conflicts when new paths are added (such as when a parent directory already exists, or the new path is a parent to an existing path) through warnings, confirmations, and additional actions.
• Feature: The duplicate result manager dialog window can now be resized, and will retain it's size and position between uses.
• Feature: Added new options to the duplicate report context (right-click) menu that allow all files in the same folder as the selected item to be marked (or un-marked).
• Feature: Added a new option to the general tab of the settings window allowing you to disable "save current project" prompts. Currently, if any unsaved changes exist to the project, you will be prompted to save them under certain circumstances. Use this new switch to disable these entirely (project is never saved unless you save it explicitly).
• Feature: The file properties docking window can now be switched between "categorized" (the default) and "alphabetized" property sorting orders. The alphabetical view is slightly more compact, for those who might find the extra space beneficial.
• Feature: The duplicate file scanning event (error) log window is now resizable and remembers its position between uses. It also provides intelligent column resizing within the event log list control that it contains.
• Feature: It is now possible to drag directory paths from Windows Explorer and drop them onto the "Search Paths" and / or "Exclusions" docking panels.
• Feature: Added an option to the Export window that allows style settings to be merged directly into HTML export files (rather than having the CSS file copied to the export target path and linked into the HTML). This helps to create a more portable HTML export document, and is engaged by default.
• Feature: Added an option to the general tab of the global settings which causes DFD to display a desktop notification whenever a duplicate scan process completes. You can also elect to show such notifications only when duplicate scan errors occur.
• Feature: Duplicate file result manager window path list is now sorted alphabetically. 
• Feature: New MSI application installer package.
• Bug fix: File name filter edit / combo history no longer stores duplicate entries.
• Bug fix: Auto group results function (see general tab of preferences window) would fail if the correlative "preferred column" was hidden by the user (through the field chooser window or some other means). This has been resolved. 
• Bug fix: Closing the main application window and then choosing "cancel" when prompted to save any changes to the active project would cause problems (and prematurely cancel any duplicate scans in progress). This has been resolved.
• Bug fix: Resolved a problem with file size filtering not always performing comparisons correctly.
• Bug fix: Shell icons displayed within the duplicate file report detail view could be incorrect in specific, rare situations (usually on the Win9x family of operating systems). This has been resolved.
• Numerous other minor improvements and bug fixes.

Version 1.5.3.0 (Released August 4th, 2007)

• Bug fix: Resolved a problem with a key binary sorting algorithm that could cause some duplicates not be identified.
• Bug fix: Resolved a file shell icon association bug in the main report view.
• Bug fix: Resolved issue with folder and file scan / skip counts being off under certain conditions.
• Bug fix: Resolved a problem with the search paths docking panel not showing vertical scrollbars when its contents exceeded available window space.
• Bug fix: Fixed a problem within the duplicate result manager window where radio button exclusivity was not properly enforced.
• Bug fix: Resolved a problem with large numbers of folder paths not being saved properly as part of the project. 
• Change: Default file name scan masks have been changed from "*.*' to "*", which matches a broader range of files (including those without extensions) by default.

Version 1.5.2.1 (Released October 10th, 2006)

• Improvement: Added an option (under global settings) allowing Duplicate File Detective to follow file system re-parse points (off by default).
• Bug fix: Resolved a problem with filename-based duplicate comparisons being case sensitive.

Version 1.5.2.0 (Released March 13th, 2006)

• Improvement: Created a separate toolbar for managing duplicate file search results.
• Bug fix: Resolved a problem with sorting file size report columns w/ very large file sizes.
• Other minor improvements and bug fixes.

Version 1.5.1.5 (Released August 21st, 2005)

• Improvement: Report fields are now quoted during export to CSV format.
• Bug fix: Resolved a problem with filter criteria file name masks not honoring upper case characters (when the case sensitive masking switch is engaged).
• Bug fix: Resolved a problem with small duplicate file result sets being ignored under specific, rare conditions.

Version 1.5.1.4 (Released August 5th, 2005)

• Improvement: Added file icon to the file properties docking window.
• Improvement: Implemented progress bar for XML file import process.
• Improvement: Added cancel function to XML file import process.
• Improvement: Made the read-only file properties grid a slightly darker shade of grey (to make them easier to read).
• Improvement: Implemented more robust error logging for duplicate import processing. If, during the XML report import process, any parsing errors or warning occur, they will now be logged to a file named "XmlParseErrs.txt" in the product installation root folder.
• Bug fix: Resolved a couple of problems with the XML report import process, mainly relating to report contents that reference remote (UNC) file paths. 
• Bug fix: Resolved a defect that prevented the application from closing correctly if duplicate search was in progress (search had to be cancelled first).
• Bug fix: Resolved problem with "mark all files in group" when accessed via the duplicate group context menu. 
• Other cosmetic bug fixes.

Version 1.5.0.4 (Released June 20th, 2005)

• New feature: Added a set of File Types docking panels to the bottom of the Duplicate File Detective window frame. These show the distribution of duplicate files (compared with total files scanned) of specific file types.
• New feature: Added a new "appearance" tab to the global options window. These options provide additional control over the visual appearance and behavior of certain aspects of the application.
• New feature: Added a new "file types" tab to the global options window. These are used to manage the behavior of the file types report and graph docking panels.
• New feature: Added a new "file type" column to the primary report view. Important: this new column is not visible by default. In order to see it, simply select the "Report Field Chooser" option from the View menu. You can then drag and drop the new column into the report view at whatever column position you prefer. The file type column is also included in all report export formats.
• Improvement: Numerous minor cosmetic improvements.
• Improvement: Improved application performance in a number of key areas, but particularly in the displaying of duplicate search results.
• Bug fix: Resolved a problem with report view context menu "unmark files in group" option not working.
• Bug fix: Resolved a problem with docking panes somtimes not re-drawing correctly when in auto-hide mode.
• Product documentation updates, other minor bug fixes.

Version 1.1.0.0 (Released May 24th, 2005)

• New feature: Added the ability to import duplicate file reports that were previously exported in XML format. Note that only XML export files can be re-imported with this feature (HTML and CSV exports can not).
• New feature: Added a file type group editor, which allows users to modify what's available from within the filename masks Presets button.
• Improvement: Added the ability to rename files using the report view context (right-click) menu.
• Improvement: Added a new checkmark column header image to the main report view, and added code to make any checked file names appear in bold. Both steps were designed to add clarity to file marking capabilities.
• Improvement: Added the ability to access product release notes from the Help menu.
• Improvement: Added a "Warn if target file already exists" checkbox option to the export dialog.
• Improvement: Added the exclusion path list to XML exports. 
• Improvement: Added the exclusion path list to summary report.
• Improvement: Added context help buttons to a number of major dialog windows. 
• Bug fix: Fixed a problem with the Search Paths and Exclusions docking panels closing prematurely when being used in "auto hide" mode.
• Bug fix: Certain characters within file names and folders were not being escaped properly during XML export operations. These are now wrapped in CDATA blocks to prevent parsing errors during consumption by third-party programs.
• Bug fix: Resolved an issue where invalid date/time stamps on certain types of files could cause a terminating exception within Duplicate File Detective.
• Bug fix: Resolved a bug with report column customizations causing duplicate result grouping problems. Specifically, automatic result grouping would engage the wrong column if the column positions were changed.
• Bug fix: Fixed an issue with file name grouping being case sensitive.

Version 1.0.1.1 (Released May 12th, 2005)

• First public product release