ln - command line hardlinks
Last Updated May 15th 2011, Version 2.6.5.0
Introduction The NTFS filesystem of NT4/W2K/WXP supports hardlinks functionality. This tool creates hardlinks on NTFS volumes in a convenient way from the command prompt.
Installation copy ln.exe to some directory referenced by your PATH environment variable. %systemroot% is a good place.

Using ln ln.exe is a typical command line utility, which can be found similiar within the resource kit. Its' highlights are as follows:
 
UNC Name
support 		A local path can be a UNC name, which maybe reached from outside the computer via share. Sometimes it is neccessary to use UNC names to refer to local pathes.
ln resolves local UNC names to local path names and does its job on those resolved path names. e.g.:

ln \\local_computer\ls\sourcefile.txt x:\dir_on_local_computer\destination.txt

In the above example there is a share called ls on your local_computer which conains a file sourcefile.txt. ln resolves the UNC name to a local path if possible, and creates a link between the resolved location and destination.txt in x:\dir_on_local_computer.

Recursive support ln.exe can recursivley hardlink all files found below a given directory location to a newly created directory structure under a different location.

In other words this means, the given directory structure is rebuilt under a different location, and all files from the given directory structure are hardlinked to the newly generated structure. Something like a xcopy but the files are hardlinked and not copied:

Hardlink Clones how to

e.g.:

ln --recursive x:\dir\dir2 x:\dir\newdir

In the above example the directories below x:\dir\dir2 are rebuilt below x:\dir\newdir, and all files from x:\dir\dir2 are hardlinked to their corresponding locations under x:\dir\newdir.

A folder tree might also contain Junctions or Symbolic Links. The clone process recreates inner junction/symbolic links at the destination location very similar as SmartCopy does.


Hardlink Clones the smart way


Outer Junctions/Symbolic links are recreated with respect to the specified Outer Junction/Symbolic Link handling. e.g.

ln --unroll --recursive x:\dir\dir2 x:\dir\newdir

In the above example the directories below x:\dir\dir2 are rebuilt below x:\dir\newdir, all files from x:\dir\dir2 are hardlinked to their corresponding locations under x:\dir\newdir and all junctions/symbolic links are unrolled.


With Windows Vista and Windows7 this cloning process is also available with Symbolic Links instead of Hardlinks.

Symbolic Link Clones the smart way



Output
See Output among Delorean Copy for a detailed description of HardlinkClone's Output

Shell Integration
The recursive support or HardlinkClone functionality is also available via Link Shell Extension


Symbolic Links When used under Vista or Windows7 ln.exe can create symbolic links

ln --symbolic Sourcefile.txt SymbolicLinkToSourceFile.txt

In the above example a symbolic link SymbolicLinkToSourceFile.txt is created which points to sourcefile.txt.

ln --symbolic SourceDirectory SymbolicLinkToSourceDirectory

In the above example a symbolic link SymbolicLinkToSourceDirectory is created which points to SourceDirectory.

ln.exe can also only show the target of a symbolic link.

ln --symbolic SymbolicLinkToSourceFile.txt

Please make sure that symbolic links can just be successfuly created from an administrative command prompt. ln.exe will fail on symbolic links from normal command prompts!


Absolute
Symbolic Links 		The target of a symbolic link can either be
  • a fully qualified path starting at the root of a drive, e.g e:\data\cpp\myfile.txt
  • or can be be specified relativeley, e.g ..\..\data\cpp\myfile.txt
ln.exe by default tries to create relative target path names for symbolic links as long as this is possible, e.g the file and its target are on the same logical drive. Having relative symbolic link targets is much smarter especially when the target of links is in the same directory. If a symbolic link and its target are on different drives, ln.exe uses absolute pathnames.

If the creation of absolute target pathnames should be forced use the --absolute option.

ln --absolute --symbolic sourcefile.txt destination.txt

In the above example sourcefile.txt gets symbolycally linked to destination.txt, but the target path in the symbolic link is absolute.

ln --absolute --symbolic SourceDirectory DestinationDirectory

In the above example a symbolic link directory SourceDirectory gets symbolycally linked to DestinationDirectory, but the target path in the symbolic link is absolute.


Normal linking Just for completeness: ln.exe can do the normal ln stuff too.

ln sourcefile.txt destination.txt

In the above example sourcefile.txt gets linked to destination.txt

Smart Copy Once hardlinks are used on a disk, copying a whole directory structure with files, some of them hardlinked together, to another hardisk or path and preserving the inner hardlink structure of this directory structure becomes difficult, because currently there is no copy program available, which can do this.

The --copy basically creates a copy of the directory structure from the source location to the destination, but it preserves the inner hardlink structure of the source, and recreates this inner hardlink structure in the destination location:

Drop Smart Copy

By closely looking at the above picture some can find three different types of files:


Normal Files The file B is a normal file. It gets copied as any other copy tool would do.
Saturated Hardlinks The files E and F are hardlinked together. In LSE naming universe they are called Saturated Hardlinks, because the reference count, which is here 2, matches the number of occurrences below 'Folder 1', which is here 2.

In General: A hardlink is called Saturated with respect to a folder F, if the number of occurrences below the folder F matches the reference count.

Saturated Hardlinks can be copied completely via Smart Copy.

Unsaturated Hardlinks The File A, C, D are hardlinked together. In LSE naming universe they are called Unsaturated Hardlinks, because the reference count, which is here 3, does not match the number of occurrences below 'Folder 1', which is here 2. Only C and D are below Folder 1.

In General: A hardlink is called Unsaturated with respect to a folder F, if the number of occurrences below the folder F is smaller than the reference count.

Unsaturated Hardlinks can only be partially copied by Smart Copy. In the above example C and D are hardlinked together in the destination location, but the hardlink to A is broken. This means that the reference count of C and D is 2 under the destination.

With junctions or symbolic link directories the default behaviour during smartcopy is as follows:

Behaviour for inner junctions

By closely looking at the above picture some can find three different types of folders/junctions:

Normal Folders The folder 'Folder 3' is a normal folder. It gets copied with its content as any other copy tool would do.
Inner Junctions
Symlinks 		The folder 'Inner Junction/Symlink' is targeted at 'Inner Junction/Symlink Target'. In LSE naming universe this kind of folder is called Inner Junction/Symlink, because its target points to a folder, which is below the common anchor 'Folder 1'.

Inner Junctions/Symlink are restored properly via Smart Copy in the destination location.

Outer Junctions
Symlinks 		The folder 'Outer Junction/Symlink' is targeted at the folder 'Outer Junction/Symlink Target'. In LSE naming universe this kind of folder is called Outer Junction/Symlink, because its target points to a folder, which is in paralell and thus outside the anchor 'Folder 1'.

Outer Junctions/Symlink can be handled in three different ways. Please see the section on Outer Junction/Symlink Handling.

Windows 7 and Windows Vista support Symbolic Links, which behave as follows during Smart Copy:

Behaviour for inner symbolic links

By closely looking at the above picture some can find three different types of files/symbolic links:

Normal Files The file A is a normal file. It gets copied as any other copy tool would do.
Inner Symbolic
Links 		The symbolic link 'Inner Symlink' is targeted at 'Inner Symlink Target'. In LSE naming universe this kind of symbolic link is called Inner Symlink, because its target points to a file, which is below the common anchor 'Folder 1'.

Inner Symlink are restored properly via Smart Copy at the destination location.

Outer Symbolic
Links 		The symlink 'Outer Symlink' is targeted at the folder 'Outer Symlink Target'. In LSE naming universe this kind of symbolic link is called Outer Symlink, because its target points to a file, which is in paralell and thus outside the anchor 'Folder 1'.

Outer Symlink are handled by Smart Copy depending on the Outer Junction/Symbolic Link handling.

e.g.:

ln --copy x:\dir\dir2 y:\dir\newdir

In the above example the directories and files below x:\dir\dir2 are copied to y:\dir\newdir, and all hardlinks/junctions/symbolic links within x:\dir\dir2 are copied/tied to their new locations under y:\dir\newdir.

The --copy operation only copies if the files in the destination are either not there, or if the files in the source are newer. Otherwise the item is skipped.

See also the note on quotes.

If you use ln.exe as a backup tool with the --copy option, please make sure you test that everything is copied. I tested ln --copy many times, and had no problems, but when it comes to backing up data, some has to be careful.

Output
See Output among Delorean Copy for a detailed descritption of Smart Copies Output

Shell Integration
The Copy or Smart Copy functionality is also available via Link Shell Extension


Crop/Unroll/Splice
Outer Junctions/
Symbolic Links 		During SmartCopy, SmartMirror and Delorean Copy so called Outer Junctions/Symlink directories may need processing. There are 3 different ways to deal with those Outer Junctions/Symlink directories:
Crop Crop breaks links to Outer Junctions/Symlink directories in the destination.


Crop also applies to Outer Symlink Files.

Croping outer Junctions


In the above example Folder1 is copied to Destination/Folder1, but Outer Junction/Symlink is not available in the destination, because Folder1/Outer Junction/Symlink pointed to Folder0/Outer Junction/Symlink Target, which is not below Folder1.

The objective behind cropping Outer Junctions/Symlink Directories is to get a pure copy during SmartCopy, SmartMirror, Delorean Copy and Clone without connections to the source.

Crop is the default behaviour for SmartCopy, SmartMirror, DeloreanCopy and Clone.

Unroll Unroll follows Outer Junctions/Symlink Directories and rebuilds the content of Outer Junctions/Symlink Directories inside the hierarchy at the destination location.

Unroll also applies to Outer Symlink Files, which means, that unroll causes the target of Outer Symlink Files to be copied to the destination location.


Unroll Outer Junctions


In the above example Folder1 is copied to Destination/Folder1, and Outer Junction/Symlink and all the files/directories below Outer Junction/Symlink Target are copied to the folder Outer Junction/Symlink in the destination.

The objective behind unrolling Outer Junctions/Symlink Directories is to get everything with which the source is connected and rebuild it as seperate copy in the destination. It resembles the 'hair of the elephant' pattern: Pull on a hair of an elephant, and get the whole elephant.


The command line invocation below unrolls all junctions/symbolic link directories below "c:\Folder 1"

ln --unroll --copy "c:\Folder 1" "c:\Destination\Folder 1"


In certain situations only certain junctions/symbolic link directories should be unrolled during SmartCopy, DeloreanCopy, SmartMirror or HardlinkClone. To accomplish this ln.exe can be run with wildcard expressions to specify certain directories for unrolling.

ln --unroll "MyJunc*" --copy "c:\Folder 1" "c:\Destination\Folder 1"



To specify junctions/symbolic link directories for unrolling more fine grained regular expressions can be given:

ln --unrollregexp "[a-z]*" --copy "c:\Folder 1" "c:\Destination\Folder 1"

The regular expressions used must conform to POSIX 1003.2 extended RE (ERE) syntax.


The regular expression specified under --unrollregexp is applied to the whole source path:

ln --unrollregexp "Folder 1\\MyJunc*" --copy "c:\Folder 1" "c:\Destination\Folder 1"


In this example only Outer Junctions/Symbolic Link Directories, matching "Folder 1\\MyJunc*" are unrolled. Please make sure that the back slash \ is escaped by a double \\ as shown in the above example.
Advanced thoughts on Unrolling

The picture above was just the simplest case, because Unroll does much more when it encounters complex situations. Think of a outer junctions/symbolic links, which itself contains junctions/symbolic links, which are inner with respect to the first outer junction symbolic link:


Unroll Inner Outer Junctions


In the above example Folder1 is copied to Destination/Folder1, and Outer Junction/Symlink and is unrolled as expected, but since Junction/Symlink is a inner junction with respect to Outer Junction/Symlink Target, the junction/symlink relation is restored in the destination.


This kind of nesting can be much more complex:


Unroll Inner Outer Junctions


In the above example Folder1 is copied to Destination/Folder1, and Outer Junction/Symlink and is unrolled as expected, but then it starts to get fascinating, because we have two levels of outer junctions/symlinks which all have respective inner junctions/symlinks, and which are restored properly. Once you digged yourself through the above picture, you got it. It is not simple I know, but it is neccessary to properly unroll.


And complexity increases if symbolic link files are within unrolled outer junctions/symbolic links:


Unroll Inner Outer Junctions


In the above example Folder1 is copied to Destination/Folder1, and Outer Junction/Symlink and is unrolled as expected, but it contains Inner Outer Symlink which points to Inner Outer Symlink Target and this is a inner junction/symbolic link with respect to Outer Junction/Symlink Target

But worth mentioning is the Symbolic Link Outer Symlink, which would be a definitive outer symbolic link, but since its targets parent-directory Outer Junction/Symlink Target is unrolled, Outer Symlink becomes an inner symbolic link with respect to Folder1.

File1 Symlink is also a outer symbolic link, but its target parent-directory Folder2 is not that lucky to get unrolled, so in the destination File1 Symlink is not a symbolic link any more, but a copy of the symbolic links' target.



The Unroll functionality also opens up the possibility to have circular Junction/Symbolic Link relations among a set of copied items:


Unroll Junctions Circularities


In the above example Folder1 is copied via the --unroll option to Destination/Folder1. Smart Copy/Smart Mirror and Delorean Copy operations can deal with the above shown circularities and break circularities by not following the affected Junction/Symbolic Link and posting an error message to the log output: !?j (0x00000423) d:\src\deny. 0x0423 is the Win32 Error Code for ERROR_CIRCULAR_DEPENDENCY



Junctions/Symbolic Links can also point to FAT drives or other NTFS drives requiring as a prerequisite unique Disk-IDs on all disks, which are chained together via Junctions/Symbolic links:


Unroll Junctions Unique DiskID


Hardlink siblings are found by matching the per NTFS volume uniqe file-id, but if more volumes are chained together it might happen that the same file-ids can be found on two different NTFS volumes. To address this all operations use the disk-id and the file-id to match hardlink siblings. If two disk-ids are equal the Win32 Error Code 0x0022 aka ERROR_WRONG_DISK is reported.

Furthermore it is not allowed to have the disk-id 0xffff-ffff, because the algorithms use this as internal indicator of a FAT drive.



The Unroll option also allows to point multpile junctions to the same target location, which causes the algorithms to traverse the same items many times:


Unroll Junctions Unique DiskID


At the first glance multiple traversation of items looks simple, but for files this means that multiple traversed files are the same in the destination and are hardlinked together. So don't be confused when you see hardlinks, which have never ever been there before.
Splice Splice reconnects Outer Junctions/Symlink directories in the destination to their original targets.


Splice outer Junctions


In the above example Folder1 is copied to Destination/Folder1, and Outer Junction/Symlink is available in the destination as junction, which points to the original location Outer Junction/Symlink Target.

The objective behind splicing Outer Junctions/Symlink Directories to its original location is to get a copy during smartcopy, but to reuse Outer Junctions/Symlink Directories source locations.


 The command line invocation below splices all junctions/symbolic link directories below "c:\Folder 1"

ln --splice --copy "c:\Folder 1" "c:\Destination\Folder 1"


In certain situations only certain junctions/symbolic link directories should be spliced during SmartCopy, DeloreanCopy, SmartMirror or HardlinkClone. To accomplish this ln.exe can be run with wildcard expressions to specify certain directories for splicing.

ln --splice "MyJunc*" --copy "c:\Folder 1" "c:\Destination\Folder 1"



To specify junctions/symbolic link directories for unrolling more fine grained regular expressions can be given:

ln --spliceregexp "[a-z]*" --copy "c:\Folder 1" "c:\Destination\Folder 1"

The regular expressions used must conform to POSIX 1003.2 extended RE (ERE) syntax.


Smart Mirror Smart Mirror is very similar to Smart Copy and not only copies but synchronises the folder Source to Destination:
  • Smart Copy only different items from Source to Destination.
  • Delete items not anymore in Source from Destination.
Smart Mirror takes care of Hardlink Relations, restores Inner Junctions or Inner Symbolic links or when issued unrolls or splices Outer Junctions or Outer Symbolic Links.

Different Items means: Either the file size changed, or the 'Last Write Date' changed. e.g.:

ln --mirror x:\source x:\destination

In the above example the directory x:\source is mirrored to x:\destination, and all symbolic links and junctions within x:\destination are updated with respect to their new locations under x:\destination.

See also the note on quotes.


Shell Integration
Smart Mirror functionality is also available via Link Shell Extension



Smart Move Smart Move enables folders with junctions and symbolic links beneath to be renamed, and the junctions and symbolic links' targets are updated below that folder. Without Smart Move renaming of such folders would end in dead junctions and symbolic links.

With junctions or symbolic link directories it behaves as follows:

Smart Move behaviour for junctions and symbolic links

By closely looking at the above picture some can find three different types of folders/junctions:

Normal Folders The folder 'Folder 3' is a normal folder. It gets moved with its content straight forward.
Inner Junctions
Symlinks 		The folder 'Inner Junction/Symlink' is targeted at 'Inner Junction/Symlink Target'. In LSE naming universe this kind of folder is called Inner Junction/Symlink, because its target points to a folder, which is below the common anchor 'Folder 1'.

Inner Junctions/Symlink are updated properly via Smart Move in the destination location.

Outer Junctions
Symlinks 		The folder 'Outer Junction/Symlink' is targeted at the folder 'Outer Junction/Symlink Target'. In LSE naming universe this kind of folder is called Outer Junction/Symlink, because its target points to a folder, which is in paralell and thus outside the anchor 'Folder 1'.

Outer Junctions/Symlinks are not touched by Smart Move and thus stay connected to their respective target. Please note that this is different to Smart Copy, which has 3 different ways to deal with Outer Junctions/Symlinks.

e.g.:

ln --move x:\dir\dir2 x:\dir\newdir

In the above example the directory x:\dir\dir2 is moved to x:\dir\newdir, and all symbolic links and junctions within x:\dir\dir2 are updated with respect to their new locations under x:\dir\newdir.

See also the note on quotes.


Shell Integration
Smart Move functionality is also available via Link Shell Extension



DeLorean Copy DeLorean Copy is a way of creating incremental backups by using a combination of hardlink clone and Smart Copy.

The following picture gives an overview what DeLorean Copy is about

DeLorean Copy Behaviour

In general a DeLorean Copy has 3 principals: Source(t), InitialBackup and Backup(n).

Phase 1: Intial SmartCopy The folder Source(t) is initially copied to InitialBackup. This is shown by the blue arrow.
Changes
happen 		During this phase the files under source change, and Source(t) becomes Source(t+1).

Phase 2: Hardlink Clone
 		The folder InitialBackup is Hardlink Cloned to Backup1, which ties InitialBackup and Backup1. This is shown by the red arrow.
Phase 3: Mirror
 		Mirror the folder Source to Backup1. This is shown by the green arrow:
  • Keep unchanged files as hardllinks to InitialBackup.
  • Delete files not anymore in Source(t+1) from Backup1.
  • Copy different files from Source(t+1) to Backup1.

With completion of this first round Backup1 contains the first lean and mean copy of Source only consisting of either hardlinks to InitialBackup, or of copied files if there was the need to copy them over from Source(t+1) because they were newer under Source(t+1).

The point is that all files in Backup1 are transparently accessible, but really little space is used, because not all files in the Source(t+1) changed, so that there was only the need to effectively copy over a few files from Source(t+1) to Backup1.

Different items means: Either the file size has changed, or the 'Last Write Date' has changed.


This can be repeated on and on. The second round would be using Source, Backup1 and Backup2 for DeLorean Copy:

Changes
happen 		During this phase the files under source change, and Source(t+1) becomes Source(t+2).

Phase 2: Hardlink Clone
 		The folder Backup1 is Hardlink Cloned to Backup2, which ties Backup1 and Backup2. This is shown by the red arrow.
Phase 3: Mirror
 		Mirror the folder Source(t+2) to Backup2. This is shown by the green arrow:
  • Keep unchanged files as hardllinks to Backup1.
  • Delete files not anymore in Source(t+2) from Backup2.
  • Copy different files from Source(t+2) to Backup2.


A real word example looks like this:

  ln --copy x:\dir\Source x:\dir\InitialBackup

This creates the initial Backup as shown above in Phase 1


  ln --delorean x:\dir\Source x:\dir\InitialBackup x:\dir\Backup1

This command does the first round of DeLorean copy as shown above in Phase 2 and Phase 3 creating a backup.


  ln --delorean x:\dir\Source x:\dir\Backup1 x:\dir\Backup2

This command creates another set of incremental DeLorean copy as shown above in Phase 2 and Phase 3 creating another backup.

See also the note on quotes.


Output
The output generated by DeLorean Copy/Smart Copy/Smart Clone/Smart Mirror looks like e.g.:

-f c:\backup\test\deleteme.dat
=d c:\backup\test\directorygone
+f c:\data\test\1.dat
+f c:\data\foo\3.dat
*h c:\data\foo\Hardlink of 3.dat
*h c:\data\foo\Another Hardlink of 3.dat
.h c:\data\already\x.dat
*h c:\data\already\Hardlink of x.dat
~f c:\data\failed\DidntMatch--splice.txt
!?f (0x00000002) c:\data\failed\AccessDenied.txt
Basically Delorean Copy protocols each action it did, and prefixes two characters to each item it processed for each line of the output. The first column of the output contains the Operation, which was performed, and the second column specfies the Type of item, which was processed.

Operation Description
= Item is already present in the target. Used during Smart Mirror
+ Copy/Create an item. Used during Smart Copy
* Hardlink a file
. Item is already present in the target and is used as the source of a newly created hardlink.
- Remove an item from the target that is not present in the source. Used during Smart Mirror
? Enumerate an item.
~ Item has been excluded by command line arguments.
! An error happened.

Item Description
f A File is processed.
h A Hardlink is processed.
s A Symbolic link file or Symbolic Link Directory.
j A Junction is processed.
d A Directory is processed.

Sample Description
+f c:\data\myfile.txt A normal file is copied.
*h c:\data\hardlnk.txt c:\data\hardlnk.txt is hardlinked to c:\data\myfile.txt.
+s d:\dest\symlink.txt A Symbolic link has been created.
+j d:\dest\junc01 A Junction has been created.
+d d:\dest\mydir A Directory has been created.
-f d:\dest\delfile d:\dest\delfile has been removed during e.g. Smart Mirror.
--junction d:\dest\junc02 The Junction d:\dest\junc02 has been removed.
=d d:\dest\mydir The directory d:\dest\mydir is the same as in the source.
=s d:\dest\symlink The symlink d:\dest\symlink is the same as in the source.
.h d:\dest\file.txt means that the name of this file is only printed, because some hardlinks have been newly tied to it. The file itself is already in Backup available, but the files following it are prefixed with * and are the newly tied hardlinks.
*h d:\dest\new_hlink If the line above is prefixed with . it means that d:\dest\new_hlink is tied to e.g. the already existing file d:\dest\file.txt.
~d d:\source\mydir The directory d:\source\mydir has been excluded intentionally by either e.g. --exclude.
~f d:\source\aFile The file d:\source\aFile has been excluded intentionally by e.g. --exclude. Items are also listes with ~ if the --unroll option is not used and outer Junctions/Symbolic Links are cropped.
!-d (0x00000002) d:\source\aFile The ! operator indicates that there was an error. It is followed by the operation, the item which were processed and an Win32 error code.
!?d (0x00000005) d:\src\deny The ? operator means the current operation was unable to enumerate all available files. It is always prefixed with ! to signal, that during enumeration of items something went wrong.
!?j (0x00000423) d:\src\deny Circularities are detected during Smart Copy/Smart Mirror/Delorean Copy and are flagged as error with the code 0x0423, which means ERROR_CIRCULAR_DEPENDENCY.
!?s (0x00000780) d:\src\deny The error ERROR_CANT_ACCESS_FILE aka 0x0780 is thrown only under Windows XP if a symbolic link is among the files to operate on, because Windows XP can not process Symbolic Links.



Statistics
After DeLorean Copy is finished it shows a statistics e.g like this one:

  Total Copied Linked Skipped Removed Excluded Failed
Folder: 10 1 - 6 1 3 0
File: 1050 1026 4 0 3 0 20
Junction: 7 - 2 0 2 3 0
Symlink: 8 - 2 5 1 0 1
Byte: 458200 272600 185600 0 1392 0 0
 
  Overall Clone Mirror
Times: 00:00:02.621 00:00:02.559 00:00:00.062

The statistics of DeLorean Copy only counts the operations, which are performed during the Mirror phase between Source and Backup(n+1). The operations which are performed during cloning between Backup(n) and Backup(n+1) are not counted in the final DeLorean Copy statistics.

Total: The total number of items as they were found under the Source
Copied: The number of items which were copied from Source to Backup
Linked: The number of items which were either hard or symbolic linked within Backup
Skipped: The number of items which were skipped during copy, because they were already there.
Removed: The number of items which were removed from Backup because they are not the anymore under Source
Excluded: The number of items which were excluded from Backup because they were excluded from the Source
either via --exclude or they didn't match --unroll or --splice.
Failed: The number of items which failed during processing. Dead junctions or symbolic links are counted as failed.

Overall: Time used up for the whole DeLorean Copy including the Clone from Backup(n) to Backup(n+1) and the Mirror from Source to Backup(n+1)
Clone: Time used up for the Clone from Backup(n) to Backup(n+1)
Mirror: Time used up for the the Mirror from Source to Backup(n+1)



Limitations
It is little known, but NTFS has a limit to create a maximum of 1023 hardlinks to one file. For DeLorean Copy this means that it will display an error message if this limit is exceeded, because exceeding this limit means loss of data among the most recent backup sets:

Delorean Copy FAILED! The NTFS link limit of 1023 has been exceeded for:

The reason for exceeding this limit could either be, that there have been more than 1023 backup sets but no hardlinks within the Source, or there are hardlinks within the Source and less than 1023 backup sets.


Shell Integration
DeLorean Copy functionality is also available via Link Shell Extension


Exclude
Wildcards
for files 		In certain situations not all files below a path should be used in Smart Copy, Smart Mirror, Delorean Copy or Clone. To accomplish this ln.exe can be run with wildcards specified to exclude certain files.

ln --exclude *.ocx --exclude *.dll --delorean c:\source d:\b\dest d:\b\bk1

Basically any arbitrary wildcard expressions can be used, because the wildcard expressions are translated into a regular expression. This means that e.g *file*.ext*.* is also a valid wildcard expression for --exclude.

The wildcard expression specified under --exclude is applied to the filename only.


Exclude
Wildcards
for directories 		In certain situations not all directories should be used in Smart Copy, Smart Mirror, Delorean Copy or Clone. To accomplish this ln.exe can be run with wildcards to exclude certain directories.

ln --excludedir *test* --delorean c:\source d:\b\dest d:\b\bk1

Basically any arbitrary wildcard expressions can be used, because the wildcard expressions are translated into a regular expression. This means that e.g *file*.ext*.* is also a valid wildcard expression for --exclude.

The wildcard expression specified under --excludedir is applied to the whole path, which means that e.g

ln --excludedir "fotos\\temp" --copy c:\source d:\b\dest

will exclude all directories containing 'fotos\temp' and their subdirectories. The above expample will e.g. exclude 'fotos\tempur\myfotos', 'fotos\temp\myfotos' or 'fotos\tempomat\myfotos'. Please make sure that '\' has to be escaped via '\\'

Exclude Regular
Expressions
for files 		In certain situations not all files below a path should be used in Smart Copy, , Smart Mirror, Delorean Copy or Clone. To accomplish this ln.exe can be run with regular expressions specified to exclude certain files.

ln --excluderegexp "[a-z]*" --copy c:\source d:\b\dest

The regular expressions used must conform to POSIX 1003.2 extended RE (ERE) syntax.

The regular expression specified under --excluderegexp is applied to the filename only.


Exclude Regular
Expressions
for directories 		In certain situations not all directories should be used with Smart Copy, Smart Mirror, Delorean Copy or Clone. To accomplish this ln.exe can be run with regular expressions to exclude certain directories.

ln --excluderegexpdir "[a-z]*" --delorean c:\source d:\b\dest d:\b\bk1

The regular expressions used must conform to POSIX 1003.2 extended RE (ERE) syntax.

The regular expression specified under --excluderegexpdir is applied to the whole path.


Enum hardlinks Sometimes it is convenient to find out which files are linked together, because hardlinked files might be spread around a NTFS volume. This is somehow the inverse hardlink functionality and it can be invoked as follows

ln --enum path

In the above example files below path are searched, and the files which are hardlinked are printed out.

The --enum option first prints so called saturated and afterwards unsaturated hardlinks. For saturated hardlinks the number of files found below path matches the hardlink reference count of these files, but unsaturated hardlinks have a reference count greater than the number found below path.

List Hardlink sibblings With Vista finding out the sibblings of a hardlink is very simple and needs no timeconsuming operations

ln --list filename

In the above example the hardlink sibblings for filename are printed out.

With XP, W2K or W2K3 finding the hardlink siblings now also works, but it takes more time, because in the worst case the whole logical volume must be searched.


Junctions ln.exe can also create junctions, and print the target of junctions

ln --junction JunctionName TargetDirectory

In the above example a junctionJunctionName is created, which point to TargetDirectory

ln --junction JunctionName

In the above example the junctionJunctionName is searched for its Target and the target is printed out.

rd JunctionName

In the above example the junctionJunctionName removed. This is accomplished with an simple built in command prompt command.

Log Level The amount of output during various operations can be controlled via the --quiet switch:

ln --quiet ...

The --quiet option without arguments totally disables any output during ln.exe' operations.
To specify the amount of output more granular an optional argument can be passed to the --quiet switch:

0 Completely suppress output. This is the same as no argument would have been passed to --quiet.
1 Only show severe errors.
2 Show the changes, e.g if a file is copied additionally during delorean copy. Furthermore show severe errors.
3 Show all output available. This is same as --quiet has been ommitted.

ln --quiet 2 ...

In the above example only changes are shown.


Long path
support 		ln.exe can handle path longer than 256 characters. This is usefully especially in the --copy and --recursive mode

Quotes
around path 		If there are blanks in the argument path to ln.exe, the pathname has to be put under quotes, e.g.

ln --copy "x:\dir with blanks\dir2" "y:\destination with blanks\newdir"

Please make sure not to end a quoted path with \ because ln.exe treats this as escaping of the quote (\") and can not parse the command line.

ln --copy "x:\" y:\dir\newdir




Useful .bat files ln.exe can be used also from .bat files to accomplish various tasks:

Delete all hardlink siblings. This .bat file uses the --list option to enumerate all hardlinks and successivley deletes them.
  Usage: DeleteAllHardlinks Filename

Create timestamped DeLorean Copies as Link Shellextension does: 'data - yyyy-mm-dd hh-mm-ss', e.g 'MyData 2010-09-11 17-04-03'.
  Usage: DeloreanCopy.bat SourceDir BackupDir
Return Values As a typical command line utility ln.exe exits with return values, so that it can be conveniently used from .bat files:
 
0 Success. Everything is fine.

-1 The source directory given via command line parameters, does not exist.
This happens in recursive mode if the first command line argument is not there

-2 The first command line argument in recursive mode specifies a directory, and it is not a directory, but some other kind of file.

-3 The second command line argument specifies a file, or in recursive mode a directory, which already exists.

-4 The destination directory could not be created in recursive mode.

-5 The source file aka first command line parameter does not exist in normal mode.

-6 The filesystem is not NTFS and does not support the creation of hardlinks.

-7 For some unknown reason the creation of a hardlink failed.

-8 Less arguments were given via command line.

-9 Ln was not able to enumerate the files in a directory. This might happen in recursive or in hardlink eunumeration mode if the file permissions for a directory are insufficient.

-12 Ln was able not to create a SmartCopy.

-14 Ln was able not to create a clone.

Backgrounders Hardlinks are a feature which are common to every Unix system, but are not directly available with NT4/W2K/WXP. It is a feature, which must be basically supported by the file system of the operating system.

So what are hardlinks in detail. It is common knowledge, that a file has a name and data associated to it. By opening the explorer, the NTFS filesystem of NT4/W2K/WXP can be browsed comfortably. There is a 1:1 relationship between the filename and the filedata, but this assumption does not hold for every filesystem.

Some filesystems, such as UFS, XFS, or NTFS have a N:1 relationship between filename and the filedata. So this means there is more than one directory entry for a file.

But how does one create those multiple entries? There is a command under Unix called ln, which creates many file entries for a file, so that there are many filenames or also called hardlinks for one filedata.

For each hardlink created, the filesystem increments a reference count stored with the filedata, so that the filedata stores how many filenames point to it. If a filename pointing to a hardlinked filedata gets deleted, the reference count stored with the filedata gets decremented by one. The filedata gets deleted when the reference count of the filedata decreases to zero.


Limitations Ln.exe can only be used under the supported plattforms
HardLinks can only be created on NTFS volumes, under the supported plattforms.
HardLinks can only be created within one NTFS volumes, and can not span accross NTFS volumes.
volumes.
Delorean Copy/Smart Copy currently does not copy alternative datastreams.
Delorean Copy/Smart Copy currently does not copy ACLs.

History
May 15th 2011 Version 2.6.5.0
  • Speed improvements during SmartCopy/SmartMirror/HardlinkClone and Delorean Copy.
  • The log level can be specified more granularily.
  • Introduced new Heap Manager Rockall for x64 and x86 builds to gain performance.
April 17th 2011 Version 2.6.0.4
December 20th 2010 Version 2.5.5.5
  • Added more powerful options to exclude files/directories via wildcards/regular expressions.
  • Changed the regular expression machine to tre-0.8.0.
  • Delorean Copy speed improvements by about 35% since 2.500 (without virus scanner).
  • Introduced DeLorean Copy, which is a way of creating incremental copies using hardlinks.
  • Added a .bat file section, which contains usefull .bat wrappers around ln.exe
  • Fixed a minor memory leak/crash.
  • When specifying a rootpath e.g.: x:\ as source with --copy and --delorean, junctions were to rebuild properly in the destination.
October 3rd 2010 Version 2.4.0.0
May 22nd 2010 Version 2.3.5.4
  • Introduced the --move option for Smart Move
  • The --copy option now also supports copying Symbolic Links under Vista/Windows7
  • Directories can now be symbolically linked
  • With W2K & NT4 readonly/hidden/system files could not be hardlinked
  • Hardlink Clone now restores the attributes of cloned folders.
March 9th 2009 Version 2.2.0.6 Enumerate hardlink siblings for Vista & Wxp, Symbolic link support vista
Under W2K it turned out, that CreateHardlink() from kernel32.dll with long pathnames (e.g. \\?\) was broken.

September 12th 2008 Version 1.997 Introduced the --copy option
Enabled the --symbolic option for Vista
Introduced the --list option under Vista
Added long (32k) path support
February 17th 2006 Version 1.810 Various small fixes related to commandline parsing.

February 18th 2005 Version 1.803 Fixed NT4 compatibility issues.

February 7th 2005 Version 1.800 Fixed a few minor internal flaw related to hardlink enumeration. Improved html documenation and added description for return values.

November 26th 2004 Version 1.711 decides between saturated and unsaturated hardlinks when used with the --enum option.
November 18th 2004 Version 1.600 supports the enumerate functionality via --enum switch
June 24th 2004 Version 1.502 released
Status The 2.7.0.x version is a stable version.

Acknowledgements I wish to thank those who have contributed significantly to the development of ln.exe. Those include:.

Endre Both for beeing a great tester with endless patience and for excellent constructive feedback on features during dvelopment.
Open Issues Maybe bugs?
Feature requests?
Links

License
  • This program is provided as is. See license.txt from this distribution for legal issues.
  • ln.exe uses tre as the regular expression machine. See the tre license.
  • ln.exe uses ultragetopt for command line parsing. See the ultragetopt license.
  • ln.exe uses the Rockall v4.0 heapmanager for fast heap operations. See the EULA.

Contact / Donations Bug reports, or feature requests send to Hermann Schinagl..

ln.exe is and will be freeware, but if ln.exe was really helpful for you and saved lots of your time please think of donations either via PayPal



or by flattring me



or by sending me a gift certificate from

amazon.de .

Download
Windows 2000
Windows XP
Windows Server 2003
Windows Server 2008
Windows Vista
Windows 7
 		Please make sure that the necessary runtime .dlls are installed on your system. This prerequisites package can be downloaded from Microsoft:

vcredist_x86.exe for Vs2005 Sp1 (2.6 Mb)
Afterwards install the
ln.zip (1.14Mb)
 
Windows XP64
Windows Vista64
Windows 7 64bit
 		Please make sure that the necessary runtime .dlls are installed on your system. This prerequisites package can be downloaded from Microsoft:

vcredist_x64.exe for Vs2005 Sp1 (3.1 MB)
Afterwards install the
ln64.zip (1.17Mb)


Windows Itanium
Windows 7 Itanium
 		Please make sure that the necessary runtime .dlls are installed on your system. This prerequisites package can be downloaded from Microsoft:

vcredist_IA64.exe for Vs2005 Sp1 (6.1 mb)
Afterwards install the
lnItanium.zip (1.26Mb)