Directory Comparison and Merge with KDiff3

Introduction into Directory Comparison and Merge
Often programmers must modify many files in a directory to achieve their purpose. For this KDiff3 also lets you compare and merge complete directories recursively!
Even though comparing and merging directories seems to be quite obvious, there are several details that you should know about. Most important is of course the fact that now many files might be affected by each operation. If you don't have backups of your original data, then it can be very hard or even impossible to return to the original state. So before starting a merge, make sure that your data is safe, and going back is possible. If you make an archive or use some version control system is your decision, but even experienced programmers and integrators need the old sources now and then. And note that even though I (the author of KDiff3) try to do my best, I can't guarantee that there are no bugs. According to the GNU-GPL there is NO WARRANTY whatsoever for this program. So be humble and always keep in mind:
To err is human, but to really mess things up you need a computer.
So this is what this program can do for you: KDiff3 ...
... reads and compares two or three directories recursively,
... takes special care of symbolic links,
... lets you browse files on mouse double click,
... for each item proposes a merge operation, which you can change before starting the directory merge,
... lets you simulate the merge and lists the actions that would take place, without actually doing them,
... lets you really do the merge, and lets you interact whenever manual interaction is needed,
... lets you run the selected operation for all items (key F7) or the selected item (key F6),
... lets you continue the merge after manual interaction with key F7,
... optionally creates backups, with the ".orig" extension,
...

Starting Directory Comparison Or Merge
This is very similar to the single file merge and comparison. You just have to specify directories on the command line or in the file-open dialog.

Compare/Merge two directories:

   kdiff3 dir1 dir2
   kdiff3 dir1 dir2 -o destdir

If no destination directory is specified, then KDiff3 will use dir2.

Compare/Merge three directories:

   kdiff3 dir1 dir2 dir3
   kdiff3 dir1 dir2 dir3 -o destdir

When three directories are merged then dir1 is used as the base for the merge. If no destination directory is specified, then KDiff3 will use dir3 as the destination directory for the merge.
Note that only the comparison starts automatically, not the merge. For this you first must select a menu entry or the key F7. (More details later.)

Directory Merge Visible Information
While reading the directories a message-box appears that informs you of the progress. If you abort the directory scan, then only files that have been compared until then will be listed.
When the directory scan is complete then KDiff3 will show a listbox with the results left, ...
... and details about the currently selected item on the right:

The Name Column
Each file and directory that was found during the scan is shown here in a tree. You can select an item by clicking it with the mouse once.
The directories are collapsed by default. You can expand and collapse them by clicking on the "+"/"-" or by double-clicking the item or by using the left/right-arrow-keys. The "Directory"-menu also contains two actions "Fold all subdirs" and "Unfold all subdirs" with which you can collapse or expand all directories at once.
If you double-click a file item then the file comparison starts and the file-diff-window will appear.
The image in the name column reflects the file type in the first directory ("A"). It can be one of these:
Normal file
Normal directory (directory-image)
Link to a file (file-image with a link arrow)
Link to a directory (directory-image with a link arrow)
If the file type is different in the other directories, then this is visible in the columns A/B/C and in the window that shows the details about the selected item. Note that for such a case no merge operation can be selected automatically. When starting the merge, then the user will be informed of problems of that kind.

The Columns A/B/C and the Coloring Scheme
As can be seen in the image above the colors red, green, yellow and black are used in the columns A/B/C.
Black: This item doesn't exist in this directory.
Green: Newest item.
Yellow: Older than green, newer than red.
Red: Oldest item.
But for items that were identical in the comparison their color also is identical even if the age is not.
Directories are considered equal if all items they contain are identical. Then they also will have the same color. But the age of a directory is not considered for its color.
The idea for this coloring scheme I came upon in dirdiff. The colors resemble the colors of a leaf that is green when new, turns yellow later and red when old.

The Operation Column
After comparing the directories KDiff3 also evaluates a proposal for a merge operation. This is shown in the "Operation" column. You can modify the operation by clicking on the operation you want to change. A small menu will popup and allows you to select an operation for that item. (You can also select the most needed operations via keyboard. Ctrl+1/2/3/4/Del will select A/B/C/Merge/Delete respectively if available.) This operation will be executed during the merge. It depends on the item and on the merge-mode you are in, what operations are available. The merge-mode is one of
Three directory-merge ("A" is treated as older base of both).
Two directory-merge.
Two directory-sync-mode (activate via option "Synchronize Directories").
In three directory merge the operation proposal will be: If for an item ...
... all three directories are equal: Copy from C
... A and C are equal but B is not: Copy from B (or if B does not exist, delete the destination if exists)
... A and B are equal but C is not: Copy from C (or if C does not exist, delete the destination if exists)
... B and C are equal but A is not: Copy from C (or if C does not exist, delete the destination if exists)
... only A exists: Delete the destination (if exists)
... only B exists: Copy from B
... only C exists: Copy from C
... A, B and C are not equal: Merge
... A, B and C don't have the same file type (e.g. A is a directory, B is a file): "Error: Conflicting File Types". While such items exist the directory merge cannot start.
In two directory merge the operation proposal will be: If for an item ...
... both directories are equal: Copy from B
... A exists, but not B: Copy from A
... B exists, but not A: Copy from B
... A and B exist but are not equal: Merge
... A and B don't have the same file type (e.g. A is a directory, B is a file): "Error: Conflicting File Types". While such items exist the directory merge cannot start.
Sync-mode is active if only two directories and no explicit destination were specified and if the option "Synchronize directories" is active. KDiff3 then selects a default operation so that both directories are the same afterwards. If for an item ...
... both directories are equal: Nothing will be done.
... A exists, but not B: Copy A to B
... B exists, but not A: Copy B to A
... A and B exist, but are not equal: Merge and store the result in both directories. (For the user the visible save-filename is B, but then KDiff3 copies B also to A.)
... A and B don't have the same file type (e.g. A is a directory, B is a file): "Error: Conflicting File Types". While such items exist the directory merge cannot start.
When two directories are merged and the option "Copy newer instead of merging" is selected, then KDiff3 looks at the dates and proposes to choose the newer file. If the files are not equal but have equal dates, then the operation will contain "Error: Dates are equal but files are not." While such items exist the directory merge cannot start.

The Status Column
During the merge one file after the other will be processed. The status column will show "Done" for items where the merge operation has succeeded, and other texts if something unexpected happened. When a merge is complete, then you should make a last check to see if the status for all items is agreeable.

Statistics Columns
When the file comparison mode "Full Analysis" is enabled in the options, then KDiff3 will show extra columns containing the numbers of unsolved, solved, nonwhite and whitespace conflicts. (The solved-column will only show when comparing or merging three directories.)

Selecting Listed Files
Several options influence which files are listed here. Some are accessible in the settings dialog. The Directory-menu contains the entries:
"Show Identical Files": Files that have been detected equal in all input directories.
"Show Different Files": Files that exist in two or more directories but are not equal.
"Show Files only in A": Files that exist only in A, but not in B or C.
"Show Files only in B": Files that exist only in B, but not in A or B.
"Show Files only in C": Files that exist only in C, but not in A or B.
Activate only the "Show"-options for the items you want listed. If for example you only want to list all items that exist either in A or in B but not in both, you'll have to activate "Show Files only in A" and "Show Files only in B" and deactivate all others ("Show Identical Files", "Show Different Files", "Show Files only in C"). The list will be updated immediately to reflect the change.
These options also apply for directories with one exception: Disabling "Show Different Files" will not hide any complete directories. This will work only for files within.
Note that of these only the "Show Identical Files"-option is persistant. The others are enabled when starting KDiff3.

Doing A Directory Merge
You can either merge the currently selected item (file or directory), or all items. When you have made all your operation choices (in all subdirectories too) then you can start the merge.
Be aware that if you didn't specify a destination directory explicitely, then the destination will be "C" in three directory mode, "B" in two directory merge mode, and in sync-mode it will be "A" or/and "B".
If you have specified a destination directory also check that all items that should be in the output, are in the tree. There are some options that cause certain items to be omitted from the directory comparison and merge. Check these options to avoid unpleasant surprises:
"Recursive Directories": If this is off, then items in subdirectories will not be found.
"Pattern"/"Anti-Pattern": Include/exclude items that match
"Exclude Hidden Files"
"Show"-options (Show Identical/Different Files, Files only in A/B/C)
If you change the settings in order to list more files, you must do a rescan via menu "Directory"->"Rescan" yourself. (The reason for this is that for faster comparison-speed KDiff3 omits the comparison for files suppressed by these criteria.) If you changed your file and dir patterns to exclude files, then the file-list will immediately be updated on closing the options-dialog.
Note that when you write to a completely new directory then you usually also want to copy the identical files. In that case enable the "Show Identical Files"-option. If your destination-directory is one of the inputs, then this isn't necessary because the file is already there.
If you are satisfied so far, the rest is easy.
To merge all items: Select "Start/Continue directory merge" in the "Directory"-menu or press F7 (which is the default shortcut). To merge only the current item: Select "Run Operation For Current Item" or press F6.
If due to conflicting filetypes still some items with invalid operations exist, then a messagebox will appear and these items will be pointed out, so you can select a valid operation for the item.
If you merge all items a dialog will appear giving you the options "Do it", "Simulate it" and "Cancel".
Select "Simulate it" if you want to see what would be done without actually doing it. A verbose list of all operations will be shown.
Otherwise select "Do it" to really start merging.
Then KDiff3 will run the specified operation for all items. If manual interaction is required (single file merge), then a merge window will open (see the big screenshot).
When you have finished with manually merging a file, again select "Start/Continue directory merge" or the key F7. If you haven't saved it yet, a dialog will ask you to do so. Then KDiff3 will continue with the next item.
When KDiff3 encounters an error, it will tell you so and will show the verbose-status-information. At the bottom of this list, there will be some error messages which should help you to understand the cause of the problem. When you continue merging (F7 key) KDiff3 will give you the choice to retry or skip the item that caused the problem. This means that before continuing you can choose another operation or solve the problem by other means.
When the merge is complete, then KDiff3 will inform you via a message box.
If some items were merged individually before running the directorymerge then KDiff3 remembers this (while this merge-session goes on), and doesn't merge them again when later the merge for all items is run. Even when the merge was skipped or nothing was saved these items count as completed. Only when you change the merge operation the "Done"-status of the item will be removed and it can be merged again.

Options for Comparing and Merging Directories
The KDiff3-preferences (menu "Settings"->"Configure KDiff3") has a section called "Directory Merge" with these options:
Recursive Directories:	 Select whether to search directories recursively.
File Pattern(s):	 Only files that match any pattern here will be put in the tree. More than one pattern may be specified here by using the semicolon ";" as separator. Valid wildcards: '*' and '?'. (e.g. "*.cpp;*.h"). Default is "*". This pattern is not used on directories.
File Anti-Pattern(s):	 Files that match this pattern will be excluded from the tree. More than one pattern may be specified here via using the semicolon ";" as separator. Valid wildcards: '*' and '?'. Default is "*.orig;*.o;*.obj".
Directory Anti-Pattern(s):	 Directories that match this pattern will be excluded from the tree. More than one pattern may be specified here via using the semicolon ";" as separator. Valid wildcards: '*' and '?'. Default is "CVS;deps;.svn".
Use CVS-Ignore:	 Ignore files and directories that would also be ignored by CVS. Many automatically generated files are ignored by CVS. The big advantage is that this can be directory specific via a local ".cvsignore"-file. (See info:/cvs/cvsignore.)
Find Hidden Files and Directories:	 On some file systems files have an "Hidden"-attribute. On other systems a filename starting with a dot "." causes it to be hidden. This option allows you to decide whether to include these files in the tree or not. Default is on.
Follow File Links:	 For links to files: When disabled, then the symbolic links are compared. When enabled, then the files behind the links are compared. Default is off.
Follow Directory Links:	 For links to directories: When disabled, then the symbolic links will be compared. When enabled then the link will be treated like a directory and it will be scanned recursively. (Note that the program doesn't check if the link is "recursive". So for example a directory that contains a link to the directory would cause an infinite loop, and after some time when the stack overflows or all memory is used up, crash the program.) Default is off.
Case Sensitive Filename Comparison:	 Default is false on Windows, true for other operating systems.
File Comparison Mode:	
Binary Comparison:	 This is the default file comparison mode.
Full Analysis:	 Do a full analysis of each file and show the statistics information columns. (Number of solved, unsolved, nonwhite and white conflicts.) The full analysis is slower than a simple binary analysis, and much slower when used on files that don't contain text. (Specify the appropriate file-antipatterns.)
Trust the modification date:	 If you compare big directories over a slow network, it might be faster to compare the modification dates and file length alone. But this speed improvement comes with the price of a little uncertainty. Use this option with care. Default is off.
Trust the size:	 Similar to trusting the modification date. No real comparison happens. Two files are considered equal if their file-sizes are equal. This is useful when the file-copy operation didn't preserve the modification date. Use this option with care. Default is off.
Synchronize Directories:	 Activates "Sync-Mode" when two directories are compared and no explicit destination directory was specified. In this mode the proposed operations will be chosen so that both source directories are equal afterwards. Also the merge result will be written to both directories. Default is off.
Copy newer instead of merging:	 Instead of merging the proposed operation will copy the newer source if changes happened. (Considered unsafe, because it implies that you know, that the other file hasn't been edited too. Check to make sure in every case.) Default is off.
Backup files:	 If a file or complete directory is replaced by another or is deleted then the original version will be renamed with an ".orig" extension. If an old backup file with ".orig" extension already exists then this will be deleted without backup. This also affects the normal merging of single files, not only in directory-merge mode. Default is on.

Other Functions in Directory Merge Window

Split/Full Screen Mode
Usually the directory merge list view remains visible while a single file is compared or merged. With the mouse you can move the the splitter bar that separates the file list from the text-diff windows. If you don't want this, you can disable "Split Screen View" in the "Directory"-menu. Then you can use "Toggle View" in the "Directory"-menu to switch between the file list and the text-diff view that then occupy the full screen.

Comparing or Merging a Single File
Probably you will prefer a simple double mouse click on a file in order to compare it. Nevertheless there also exists an entry in the "Directory"-menu. You can also directly merge a single file by selecting it and choosing "Merge current file" in the "Merge"-Menu. On saving the result, the status will be set to done, and the file will not be merged again if a directory merge is started.
But note that this status information will be lost when you rerun a directory scan: "Directory"-menu: "Rescan"

Comparing or Merging Files with Different Names
Sometimes you need to compare or merge files with different names (e.g. the current file and the backup in the same folder).
Select the exact file by clicking onto the icon in the column A, B or C. The first file selected thus will be marked with an "A", the second and third with "B" and "C" regardless on what column they are in. Only up to three files can be chosen like this.
Proceed by choosing "Compare Explicitely Selected Files" or "Merge Explicitely Selected Files" from the "Directory"-menu. For your convenience these menu entries also appear as context menu when you right-click the last selected file.
The comparison or merge of a file will happen in the same window. If this method is used for directories a new window will be opened.