|
|
Home | Switchboard | Unix Administration | Red Hat | TCP/IP Networks | Neoliberalism | Toxic Managers |
| (slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and bastardization of classic Unix | |||||||
|
|
|
|
Windows has xcopy is a useful built-in bulk file copy tool. But in Windows XP it has limitations and can't copy, say 20K files from one disk to another. It just chokes. So you need another took in case you often have to copy an entire disk. This defect was fixed in Windows 7. I successfully copied 32GB drives with 30K files using it.
Microsoft's Robocopy, a command line directory replication command, gradually replaced the older xcopy. It rocks a whole host of features that make is the logical choice for transferring a large number of files. It's multi-threaded, has a ton of options, and generally outperforms vanilla Windows copy operations. Best of all, it's built right in to Redmond's operating system. Especially useful for network copy operations and backups, Robocopy doesn't stop to ask you one hundred questions while it copies over your music collection, either.
The reality of benchmarking file copy performance is that you need something fast to copy from and something fast to copy to. This is most important with SSDs. It doesn't matter if your drive can write sequentially at 500 MB/s if the source files are hosted on a USB 2.0-attached external hard drive. We're copying our test files from an Intel SSD DC S3700 to the drives in the chart below, taking source speed out of the equation.
There are 9065 files comprising the 16.2 GB payload. Some of the files are huge (up to 2 GB), while others are best described as tiny. On average, that's around 1.8 MB per file. The files are a mix of music, program, pictures, and random file types.
It's fair to say that this chart would look much different if we were copying from a hard drive to a SSD. Even if the disk drive's sequential throughput wasn't a bottleneck, it'd still choke on the smaller files.
There are several choices that can improve the situation. Among them:
Xcopy. Windows 7 xcopy utility has improved capabilities to copy large number of files. 20K is not a problem anymore. With options /r and /d can also be used for synchronization:
xcopy y:\Public_html f:\Public_html /s /r /c /d
Robocopy. robocopy is a standard windows utility which is present in Windows 7, Windows 8 and Windows 10. Part of Windows server 2003 resource kit. Looks like a replacement of xcopy with better capabilities. Has Mirror mode /mir. In this mode can sincronize large partitions. See also Robocopy SS64.com
XXCOPY. This is commercial tool which is also available for free for personal use. The latest version which I tried is 3.21.8.. It is a command like that preserves XCOPY syntax and extend XCOPY functionality.
When we designed XXCOPY, one of the key design principles was to keep the command syntax and the behavior of XXCOPY as close to those of the original XCOPY program. Although we did not agree with all the idiosyncrasies of XCOPY, modifying anything XCOPY already implements would cause substantial confusion to the users.If we change things arbitrarily, any incremental improvements we may add would be more than offset by the burden of remembering the specifics of changes we make. That would not benefit anyone.
It performs file-copy, delete, search and many common file management operations. It
can select files and directories by the date, age, and size, as well as the comparison between the
source and destination directories. It accepts wildcards in any place and with any number in the
source specifier. Its ability to preserve the SFN makes it ideal for volume cloning.
xxcopy online manual
In my test it took approximately 40 min to clone 32GB rive drive with approximately 20GB filled with 40K files.
Example:
XXCOPY.EXE f:\Public_html n:\Public_html /clone
|
|
Switchboard | ||||
| Latest | |||||
| Past week | |||||
| Past month | |||||
 |
 |
 |
Oct 19, 2017 | stackoverflow.com
Michael Stum , Aug 4, 2008 at 15:28
How about good old Command-Line Xcopy? With S: being the source and T: the target:xcopy /K /R /E /I /S /C /H /G /X /Y s:\*.* t:\/K Copies attributes. Normal Xcopy will reset read-only attributes.
/R Overwrites read-only files.
/E Copies directories and subdirectories, including empty ones.
/I If destination does not exist and copying more than one file, assumes that destination must be a directory.
/S Copies directories and subdirectories except empty ones.
/C Continues copying even if errors occur.
/H Copies hidden and system files also.
/Y Suppresses prompting to confirm you want to overwrite an existing destination file.
/G Allows the copying of encrypted files to destination that does not support encryption.
/X Copies file audit settings (implies /O).
(Edit: Added /G and /X which are new since a few years)
Miles , Oct 15, 2008 at 16:55
this works alot better than the regular windows copy. We had to copy a document server from one partition to another one time. I think we tried the windows copy first (said it was gonna take 20something hours). When we tried X-copy, it took less than 10 (and kept the windows permissions intact). – Miles Oct 15 '08 at 16:55Oliver Zendel , Mar 11, 2014 at 9:29
xcopy has a serious bug if the path of a single file while copying has more than 256 characters. xcopy will abort the whole operation with an 'insufficient memory' error -> don't use xcopy! As an alternative, this will do pretty much what xcopy does without the bugs:
robocopy c:\srcdir d:\dstdir /XJ /SL /B /E /S /R:1 /W:1 /NFL /NS /NC /MT > mylog.txt 2>&1huseyint , Aug 4, 2008 at 15:21
Use Robocopy (Robust File Copy) .NOTE:
In Windows Vista and Server 2008 when you type:
xcopy /?you get:
NOTE: Xcopy is now deprecated, please use Robocopy.
So start getting used to robocopy :)
Gordon Bell , Oct 3, 2008 at 5:40
The primary complaint I've always had with XCOPY is that it always re-copies the files even if the destination file already exists with the same file size and modification date. – Gordon Bell Oct 3 '08 at 5:40
|
|
|
I find RoboCopy is a good alternative to xcopy. It supports high latency connections much better and supports resuming a copy.
References
Wikipedia - robocopy
Downloads
Edit Robocopy was introduced as a standard feature of Windows Vista and Windows Server 2008.
•Robocopy is shipped as part of the Windows Server 2003 resource kit and can be download from the Microsoft download site.
•A very simple GUI has also been release for RoboCopy on technet http://technet.microsoft.com/en-us/magazine/cc160891.aspx
|
|
|
Copies files and directories, including subdirectories.Syntaxxcopy Source [Destination] [/w] [/p] [/c] [/v] [/q] [/f] [/l] [/g] [/d[:mm-dd-yyyy]] [/u] [/i] [/s [/e]] [/t] [/k] [/r] [/h] [{/a|/m}] [/n] [/o] [/x] [/exclude:file1[+[file2]][+[file3]] [{/y|/-y}] [/z]
Top of pageParameters
Source : Required. Specifies the location and names of the files you want to copy. This parameter must include either a drive or a path.
Destination : Specifies the destination of the files you want to copy. This parameter can include a drive letter and colon, a directory name, a file name, or a combination of these.
/w : Displays the following message and waits for your response before starting to copy files:
Press any key to begin copying file(s)
/p : Prompts you to confirm whether you want to create each destination file.
/c : Ignores errors.
/v : Verifies each file as it is written to the destination file to make sure that the destination files are identical to the source files.
/q : Suppresses the display of xcopy messages.
/f : Displays source and destination file names while copying.
/l : Displays a list of files that are to be copied.
/g : Creates decrypted destination files.
/d [ : mm-dd-yyyy ] : Copies source files changed on or after the specified date only. If you do not include a mm-dd-yyyy value, xcopy copies all Source files that are newer than existing Destination files. This command-line option allows you to update files that have changed.
/u : Copies files from Source that exist on Destination only.
/i : If Source is a directory or contains wildcards and Destination does not exist, xcopy assumes destination specifies a directory name and creates a new directory. Then, xcopy copies all specified files into the new directory. By default, xcopy prompts you to specify whether Destination is a file or a directory.
/s : Copies directories and subdirectories, unless they are empty. If you omit /s, xcopy works within a single directory.
/e : Copies all subdirectories, even if they are empty. Use /e with the /s and /t command-line options.
/t : Copies the subdirectory structure (that is, the tree) only, not files. To copy empty directories, you must include the /e command-line option.
/k : Copies files and retains the read-only attribute on destination files if present on the source files. By default, xcopy removes the read-only attribute.
/r : Copies read-only files.
/h : Copies files with hidden and system file attributes. By default, xcopy does not copy hidden or system files.
/a : Copies only source files that have their archive file attributes set. /a does not modify the archive file attribute of the source file. For information about how to set the archive file attribute by using attrib, see Related Topics.
/m : Copies source files that have their archive file attributes set. Unlike /a, /m turns off archive file attributes in the files that are specified in the source. For information about how to set the archive file attribute by using attrib, see Related Topics.
/n : Creates copies by using the NTFS short file or directory names. /n is required when you copy files or directories from an NTFS volume to a FAT volume or when the FAT file system naming convention (that is, 8.3 characters) is required on the destination file system. The destination file system can be FAT or NTFS.
/o : Copies file ownership and discretionary access control list (DACL) information.
/x : Copies file audit settings and system access control list (SACL) information (implies /o).
/exclude: filename1 [ + [ filename2 ]][ + [ filename3 ]] : Specifies a list of files containing strings.
/y : Suppresses prompting to confirm that you want to overwrite an existing destination file.
/-y : Prompts to confirm that you want to overwrite an existing destination file.
/z : Copies over a network in restartable mode.
/? : Displays help at the command prompt.
Top of page
Remarks
Top of page
- Using /v
Windows XP does not use this command. It is accepted only for compatibility with MS-DOS files.
- Using /exclude
List each string in a separate line in each file. If any of the listed strings match any part of the absolute path of the file to be copied, that file is then excluded from the copying process. For example, if you specify the string "\Obj\", you exclude all files underneath the Obj directory. If you specify the string ".obj", you exclude all files with the .obj extension.
- Using /z
If you lose your connection during the copy phase (for example, if the server going offline severs the connection), it resumes after you reestablish the connection. /z also displays the percentage of the copy operation completed for each file.
- Using /y in the COPYCMD environment variable
You can use /y in the COPYCMD environment variable. You can override this command by using /-y on the command line. By default, you are prompted to overwrite, unless you run copy from within a batch script.
- Copying encrypted files
Copying encrypted files to a volume that does not support EFS results in an error. Decrypt the files first or copy the files to a volume that does support EFS.
- Appending files
To append files, specify a single file for destination, but multiple files for source (that is, by using wildcards or file1+file2+file3 format).
- Default value for Destination
If you omit Destination, the xcopy command copies the files to the current directory.
- Specifying whether Destination is a file or directory
If Destination does not contain an existing directory and does not end with a backslash (\), the following message appears:
Does destination specify a file name or directory name on the target (F = file, D = directory)?Press F if you want the file or files to be copied to a file. Press D if you want the file or files to be copied to a directory.
You can suppress this message by using the /i command-line option, which causes xcopy to assume that the destination is a directory if the source is more than one file or a directory.
- Using the xcopy command to set archive attribute for Destination files
The xcopy command creates files with the archive attribute set, whether or not this attribute was set in the source file. For more information about file attributes and attrib, see Related Topics.
- Comparing xcopy and diskcopy
If you have a disk that contains files in subdirectories and you want to copy it to a disk that has a different format, use the xcopy command instead of diskcopy. Because the diskcopy command copies disks track by track, your source and destination disks must have the same format. The xcopy command does not have this requirement. Use xcopy unless you need a complete disk image copy.
- Exit codes for xcopy
To process exit codes returned by xcopy, use the errorlevel parameter on the if command line in a batch program. For an example of a batch program that processes exit codes using if, see Related Topics. The following table lists each exit code and a description.
Exit code Description 0 Files were copied without error. 1 No files were found to copy. 2 The user pressed CTRL+C to terminate xcopy. 4 Initialization error occurred. There is not enough memory or disk space, or you entered an invalid drive name or invalid syntax on the command line. 5 Disk write error occurred.
Examples
To copy all the files and subdirectories (including any empty subdirectories) from drive A to drive B, type:
xcopy a: b: /s /e
To include any system or hidden files in the previous example, add the/h command-line option as follows:
xcopy a: b: /s /e /h
To update files in the \Reports directory with the files in the \Rawdata directory that have changed since December 29, 1993, type:
xcopy \rawdata \reports /d:12-29-1993
To update all the files that exist in \Reports in the previous example, regardless of date, type:
xcopy \rawdata \reports /u
To obtain a list of the files to be copied by the previous command (that is, without actually copying the files), type:
xcopy \rawdata \reports /d:12-29-1993 /l > xcopy.out
The file Xcopy.out lists every file that is to be copied.
To copy the \Customer directory and all subdirectories to the directory \\Public\Address on network drive H:, retain the read-only attribute, and be prompted when a new file is created on H:, type:
xcopy \customer h:\public\address /s /e /k /p
To issue the previous command, ensure that xcopy creates the \Address directory if it does not exist, and suppress the message that appears when you create a new directory, add the /i command-line option as follows:
xcopy \customer h:\public\address /s /e /k /p /i
You can create a batch program to perform xcopy operations and use the batch if command to process the exit code if an error occurs. For example, the following batch program uses replaceable parameters for the xcopy source and destination parameters:
@echo off rem COPYIT.BAT transfers all files in all subdirectories of rem the source drive or directory (%1) to the destination rem drive or directory (%2) xcopy %1 %2 /s /e if errorlevel 4 goto lowmemory if errorlevel 2 goto abort if errorlevel 0 goto exit :lowmemory echo Insufficient memory to copy files or echo invalid drive or command-line syntax. goto exit :abort echo You pressed CTRL+C to end the copy operation. goto exit :exitTo use this batch program to copy all files in the C:\Prgmcode directory and its subdirectories to drive B, type:
copyit c:\prgmcode b:
The command interpreter substitutes C:\Prgmcode for %1 and B: for %2, then uses xcopy with the /e and /s command-line options. If xcopy encounters an error, the batch program reads the exit code and goes to the label indicated in the appropriate IF ERRORLEVEL statement, then displays the appropriate message and exits from the batch program.
|
|
|
Stack Overflow
/D may be what you are looking for. I find it works quite fast for backing-up as existing files are not copied.xcopy "O:\*.*" N:\Whatever /C /D /S /H /C Continues copying even if errors occur. /D:m-d-y Copies files changed on or after the specified date. If no date is given, copies only those files whose source time is newer than the destination time. /S Copies directories and subdirectories except empty ones. /H Copies hidden and system files also.
|
|
|
Feb 16, 2010 | PCWorld
When last I visited TeraCopy (the Pro version) I loved its features, but I discovered that version was not 100% stable. For a task as critical as copying and moving files, there's no substitute for rock-solid. Happily, I had no problems with this more recent free Home version of the program and can honestly say that TeraCopy seems ready for prime time.
TeraCopy inserts itself in place of the Windows file handling functions, adding to the usual right-click option. TeraCopy's pause function alone is worth the download if you regularly copy large amounts of files and need to stop the process to perform another disk-intensive task. Windows only lets you cancel--not pause--which makes it difficult to ascertain exactly where you left off. You may also choose up front whether you want to overwrite all files, older ones only, skip files that already exist, or rename the file being copied to avoid overwriting the existing files. TeraCopy seemed a tad faster than Windows in my tests with a large mix of small and large files though a little slower than the Windows native routine when copying a single very large file.
TeraCopy provides a lot more information about the files being copied, such as their size and original location. It may be invoked as a standalone so you can select files from diverse locations .
|
|
|
IntroductionHard drives are becoming larger with each advance made in the hardware industry. With the increase in storage size available the amount of files the average user keeps on their computer has also greatly expanded. But when you want to backup those files safely to either another partition or an external hard drive you need a copy utility that will not run into problems. I used my own music collection at 21.7GB and used several free copy utility programs to see which of them was the fastest, most reliable, and easy to use.
Other File Copiers
- FastCopy is my choice for the best file copy manager. When it comes to stability and speed, FastCopy gave me consistent results and beat every other manager in the running by several seconds. I also noted that during the transfer, even with the 64MB cache that it used, I was still not seeing a significant jump in resource usage. My RAM usage stayed consistently low throughout the entire transfer. The program is excellent which made it my top choice of the file managers I tested.
It also includes shell integration with the program and x64 capabilities. The program can also be run without being installed which is a very nice touch if you only need it for certain tasks. It also includes an installer for shell integration and for use as your default copier. It will not override your Windows file copier even when installed.
There are a lot of great things about this program but I cannot give this program a 10/10 for the following reasons. First, the uninstalling utility would probably be confusing for the average user since you have to enter the C:\Program Files\FastCopy folder in order to run setup again and choose to uninstall the program. With that said it did not show up in the "Uninstall Program" section of Windows 7 Control Panel which would be the optimal place for the program to be uninstalled. Secondly, shell integration must be configured via the "Options" menu and in order to choose what FastCopy options you would like added to the right-click menu since it does not automatically override your Windows copier.
These to me are minor gripes as the shell extensions added manually are excellent and you can even choose to hide the transfer window in the system tray and hover your mouse over to see the transfer speeds. Also, built into the program is a file wiping utility that performs up to three passes over the file it is deleting for those wanting secure deletion of their files. A big thank you to the commenters for bringing this program, and its benchmarks, to my attention.
- UltraCopier has some pretty exciting features including an excellent file resume option, which allows for resuming of a file transfer from a disconnected USB key. So, for those of you playing at home, if you're in the middle of a file transfer from a USB key and it somehow becomes disconnected, when you plug it in again all you have to do is to hit resume and it will pick up where you left off! A great innovation, especially if you have a finicky Windows install that randomly disconnects your drive.
It has also other useful options such as speed limitation and searching through the copy list. And as far as looks go, UltraCopier is a much more visually attractive copier replacement. It's open source, runs on multi-platforms including Windows, Mac and Linux.
All of the software I reviewed here is excellent and it is up to the user to try these out and decide which one they prefer. For now, I'll be using FastCopy for large transfers and UltraCopier for daily transfers side by side.
- The next program that I tested was Unstoppable Copier. This one wins hands down in the reliability category. Not only does it transfer faster than the normal Windows copier, it is also able to recover partial files from damaged media or corrupt areas of a hard disk drive. This program is not a miracle worker but I gave it several scratched DVDs that were denied by my Windows copier and it was able to get me a complete backup of the files on the disk. I am very impressed by the software and its uses.
I do not believe it is the intention of the project to do so, but I think this could be an even better program if it was able to be used in place of the Windows copier. It is portable and installable but both versions must be launched in order to do file transfers. The output of the program is outstanding and gives you a very good idea of which files have copied successfully and which files will be re-attempted. With its unique ability to recover damaged files, this copying program is the most reliable of the programs I tested.
- I recently used the 2.22 version of TeraCopy. I've seen a lot of complaints about stability with this program and so I decided to push it to the limits. Full on 21.7GB transfer of small files, pausing and resuming of said files, follow-up verification of the total transfer, and killing and re-starting of the same transfer. I will say that these features worked as intended on my setup which is Windows 7 x64 Home Premium.
While this program is a beta, the features I tested performed as they should on my setup. My results are as individual as they come because obviously I cannot account for every variation but the method I used involved TeraCopy as the only program running. Even when I wasn't 'harassing' it by pausing and resuming, my setup is RAID 0, I tried copying small files and one ridiculously large file several times. Each time it did exactly what I asked it to do. The file verification also reassured me that everything transferred correctly even if I did manually verify later.
I personally switch file copiers because I want something faster and more reliable than the Windows copier. In this respect as a Windows copier replacement TeraCopy still shines. I will have some people I know test this program and see if they run into any problems that perhaps I could not. But I still have to say that this is the most actively developed and impressively integrated file copier out right now.
The fact that they have a freeware version that works with Windows 8 x64 (seriously, look at their page) and smooth total integration still makes this one of the best file copiers I've used to date. Which is why it remains a contender for the top spot. If a new release ends up equalling FastCopy on speed, resource usage and stability, then you all will be seeing a new top spot recommendation from me.
Besides the recommended file copy utilities, I'd also mention SuperCopier and KillCopy since both are stable and reliable. Both are faster than the built-in Windows copier. They run from the system tray and are light on resources. KillCopy is quite a bit faster on transfers and has a plethora of options for the advanced user. It gets a hard time for its looks but it has many skins available on its webpage if you do decide to go that route. None that make it as attractive as UltraCopier, in my opinion, but it does help.
|
|
|
Microsoft's Robocopy, a command line directory replication command, gradually replaced the older xcopy. It rocks a whole host of features that make is the logical choice for transferring a large number of files. It's multi-threaded, has a ton of options, and generally outperforms vanilla Windows copy operations. Best of all, it's built right in to Redmond's operating system. Especially useful for network copy operations and backups, Robocopy doesn't stop to ask you one hundred questions while it copies over your music collection, either.
The reality of benchmarking file copy performance is that you need something fast to copy from and something fast to copy to. This is most important with SSDs. It doesn't matter if your drive can write sequentially at 500 MB/s if the source files are hosted on a USB 2.0-attached external hard drive. We're copying our test files from an Intel SSD DC S3700 to the drives in the chart below, taking source speed out of the equation.
There are 9065 files comprising the 16.2 GB payload. Some of the files are huge (up to 2 GB), while others are best described as tiny. On average, that's around 1.8 MB per file. The files are a mix of music, program, pictures, and random file types.
It's fair to say that this chart would look much different if we were copying from a hard drive to a SSD. Even if the disk drive's sequential throughput wasn't a bottleneck, it'd still choke on the smaller files.
Google matched content |
List of file copying software - Wikipedia, the free encyclopedia
TeraCopy for Microsoft Windows - Code Sector
TeraCopy - Wikipedia, the free encyclopedia
TeraCopy - Wikipedia, the free encyclopedia
FastCopy 2.11 Freeware Download - Freeware Files.com - Utilities Category
5 Free Bulk File Copy Software to Copy Files Fast Free Software
