Remote backups using dd

the dd   has certain features which make is uniquely suited for several types of backup and first of all damaged filesystems. 

You can use dd  as a backup command, since it can copy the bits in a file or raw device to another location. You even can pipe the bit stream through gzip, allowing you to store a compressed copy of the data. (dump, tar, and cpio  do not have this capability, although GNU tar  does.) The best example of using dd  as a backup command is the hot-backup script for Oracle, oraback.sh  Since Oracle can use both raw partitions and files for its database files, the script could not predict which command to use. However, dd  supports both of them!

Most other backup commands can only read or write from stdin, whereas dd  can do both at the same time. This makes dd  very versatile and the only native backup utility that can be used to pass a stream of data from one command to another or from one system to a device on another system, using rsh  or ssh. This can work either way.

Reading a backup on a remote device

The restore, GNU tar,  and GNU cpio  commands can read the remote device by simply giving it remote_host:remote_device  as the device name. However, the native versions of tar  and cpio  do not support such an option. Therefore, you need some way to do this with these commands. You simply ssh  a dd  command to the remote system and read its data stream on the local system:

# ssh  remote_host "dd if= device ibs= blocksize "| tar xvBf - 

Writing a backup to a remote device

This one is a bit trickier. You may have to create a subshell^[

# tar cvf - . | (ssh  remote_system dd of= device obs= block_size ) 

Putting parentheses around the remote command creates the subshell. Notice that you must specify the remote block size, and you need to be careful when doing so. If you want to create a volume that can be read by tar, make sure you use a block size that tar  can understand, such as 10,240. (This is usually the biggest block size tar  can read or write, and this is done by specifying a blocking factor of 20 in tar.)

To read tapes or backups on remote hosts

# ssh  remote_host "dd if= device bs= blocksize "| tar xvBf - 
# ssh  remote_host "dd if= device bs= blocksize "| restore rvf - 
# ssh  remote_host "dd if= device bs= blocksize "| cpio -itv 

To create backup  on remote hosts:

# dump 0bdsf 64 100000 100000 - \ 
  | ssh  remote_host  "dd if= device  bs=64k" 
# tar cvf - | ssh  remote_host  "dd if= device  bs=10k" 
# cpio -oacvB | ssh  remote_host  "dd if= device  bs=5k" 

Remote backup

It is often necessary to backup into a remote machine's tape drive. Here are the commands that can be used to achieve this. Execute this command on the machine you want to backup.

$ tar cvf - $DIRNAME | rsh $SYS dd of=$TAPEDEV

Substitute

$DIRNAME with the directory to backup,

$SYS with the machine name with the tape drive,

$TAPEDEV with the tape device.

Note: You must be able to rlogin into the remote machine without a password. To do this add the name of your local machine with your user name in the .rhost file in your home directory on the remote machine.

To retrieve the backed up info...

rsh $REM dd if=$TAPEDEV | tar xvf - Tips For Linux - How and when to use the dd command In this article, Sam Chessman explains the use of the dd command with a lot of useful examples. This article is not aimed at absolute beginners. Once you are familiar with the basics of Linux, you would be in a better position to use the dd command.

The ' dd ' command is one of the original Unix utilities and should be in everyone's tool box. It can strip headers, extract parts of binary files and write into the middle of floppy disks; it is used by the Linux kernel Makefiles to make boot images. It can be used to copy and convert magnetic tape formats, convert between ASCII and EBCDIC, swap bytes, and force to upper and lowercase.

For blocked I/O, the dd command has no competition in the standard tool set. One could write a custom utility to do specific I/O or formatting but, as dd is already available almost everywhere, it makes sense to use it.

Like most well-behaved commands, dd reads from its standard input and writes to its standard output, unless a command line specification has been given. This allows dd to be used in pipes, and remotely with the rsh remote shell command.

Unlike most commands, dd uses a keyword=value format for its parameters. This was reputedly modeled after IBM System/360 JCL, which had an elaborate DD 'Dataset Definition' specification for I/O devices. A complete listing of all keywords is available from GNU dd with

$ dd --help

Some people believe dd means ``Destroy Disk'' or ``Delete Data'' because if it is misused, a partition or output file can be trashed very quickly. Since dd is the tool used to write disk headers, boot records, and similar system data areas, misuse of dd has probably trashed many hard disks and file systems.

In essence, dd copies and optionally converts data. It uses an input buffer, conversion buffer if conversion is specified, and an output buffer. Reads are issued to the input file or device for the size of the input buffer, optional conversions are applied, and writes are issued for the size of the output buffer. This allows I/O requests to be tailored to the requirements of a task. Output to standard error reports the number of full and short blocks read and written.

Example 1

A typical task for dd is copying a floppy disk. As the common geometry of a 3.5" floppy is 18 sectors per track, two heads and 80 cylinders, an optimized dd command to read a floppy is:

Example 1-a : Copying from a 3.5" floppy

dd bs=2x80x18b if=/dev/fd0 of=/tmp/floppy.image
1+0 records in
1+0 records out

The 18b specifies 18 sectors of 512 bytes, the 2x multiplies the sector size by the number of heads, and the 80x is for the cylinders--a total of 1474560 bytes. This issues a single 1474560-byte read request to /dev/fd0 and a single 1474560 write request to /tmp/floppy.image, whereas a corresponding cp command

cp /dev/fd0 /tmp/floppy.image

issues 360 reads and writes of 4096 bytes. While this may seem insignificant on a 1.44MB file, when larger amounts of data are involved, reducing the number of system calls and improving performance can be significant.

This example also shows the factor capability in the GNU dd number specification. This has been around since before the Programmers Work Bench and, while not documented in the GNU dd man page, is present in the source and works just fine, thank you.

To finish copying a floppy, the original needs to be ejected, a new diskette inserted, and another dd command issued to write to the diskette:

Example 1-b : Copying to a 3.5" floppy
dd bs=2x80x18b < /tmp/floppy.image > /dev/fd0
1+0 records in
1+0 records out

Here is shown the stdin/stdout usage, in which respect dd is like most other utilities.

Example 2

The original need for dd came with the 1/2" tapes used to exchange data with other systems and boot and install Unix on the PDP/11. Those days are gone, but the 9-track format lives. To access the venerable 9-track, 1/2" tape, dd is superior. With modern SCSI tape devices, blocking and unblocking are no longer a necessity, as the hardware reads and writes 512-byte data blocks.

However, the 9-track 1/2" tape format allows for variable length blocking and can be impossible to read with the cp command. The dd command allows for the exact specification of input and output block sizes, and can even read variable length block sizes, by specifying an input buffer size larger than any of the blocks on the tape. Short blocks are read, and dd happily copies those to the output file without complaint, simply reporting on the number of complete and short blocks encountered.

Then there are the EBCDIC datasets transferred from such systems as MVS, which are almost always 80-character blank-padded Hollerith Card Images! No problem for dd, which will convert these to newline-terminated variable record length ASCII. Making the format is just as easy and dd again is the right tool for the job.

Example 2 : Converting EBCDIC 80-character fixed-length record to ASCII variable-length newline-terminated record
dd bs=10240 cbs=80 conv=ascii,unblock if=/dev/st0 of=ascii.out
40+0 records in
38+1 records out

The fixed record length is specified by the cbs=80 parameter, and the input and output block sizes are set with bs=10240. The EBCDIC-to-ASCII conversion and fixed-to-variable record length conversion are enabled with the conv=ascii,noblock parameter.

Notice the output record count is smaller than the input record count. This is due to the padding spaces eliminated from the output file and replaced with newline characters.

Example 3

Sometimes data arrives from sources in unusual formats. For example, every time I read a tape made on an SGI machine, the bytes are swapped. The dd command takes this in stride, swapping the bytes as required. The ability to use dd in a pipe with rsh means that the tape device on any *nix system is accessible, given the proper rlogin setup.

Example 3 : Byte Swapping with Remote Access of Magnet Tape
rsh sgi.with.tape dd bs=256b if=/dev/rmt0 conv=swab | tar xvf -

The dd runs on the SGI and swaps the bytes before writing to the tar command running on the local host.

Example 4

Murphy's Law was postulated long before digital computers, but it seems it was specifically targeted for them. When you need to read a floppy or tape, it is the only copy in the universe and you have a deadline past due, that is when you will have a bad spot on the magnetic media, and your data will be unreadable. To the rescue comes dd, which can read all the good data around the bad spot and continue after the error is encountered. Sometimes this is all that is needed to recover the important data.

Example 4 : Error Handling

dd bs=265b conv=noerror if=/dev/st0 of=/tmp/bad.tape.image 

The Linux kernel Makefiles use dd to build the boot image. In the Alpha Makefile /usr/src/linux/arch/alpha/boot/Makefile, the srmboot target issues the command:

Example 5 : Kernel Image Makefile

dd if=bootimage of=$(BOOTDEV) bs=512 seek=1 skip=1 

This skips the first 512 bytes of the input bootimage file (skip=1) and writes starting at the second sector of the $(BOOTDEV) device (seek=1). A typical use of dd is to skip executable headers and begin writing in the middle of a device, skipping volume and partition data. As this can cause your disk to lose file system data, please test and use these applications with care.

size="-2">Credits
size="-2">

GNU dd is found in the fileutils collection, with the current version at the URL ftp://prep.ai.mit.edu/pub/gnu/fileutils-3.12.tar.gz or a mirror near you.

Other major versions include SYSV and BSD, with the BSD source version 5.16 4/28/93 derived from software contributed to Berkeley by Keith Muller of the University of California, San Diego and Lance Visser of Convex Computer Corporation, Copyright © 1991 The Regents of the University of California.

NEWS CONTENTS

• 20101015 : Manual Backups in Linux, DD ( Eleven is Louder )
• 20100310 : mount_dd ( freshmeat.net )

Old News ;-)

[Oct 15, 2010] Manual Backups in Linux, DD

Eleven is Louder

So, a simple backup?

#dd if=/dev/sdX | pv | dd of=/dev/sdY

The above assumes that you have access to another hard disc of the exact same make, and that you wish to mirror your drive in use onto the other drive (pv is just the progress indicator). Not your cup of tea? Ok. Now, this is the command I use for backup. It's faster than bit by bit, though not quite as safe. It also will not quit when it encounters a disc error. I compresses your output, and instead of outputting to a disc, it will output to a file. The restore on this one is pretty much the same.

#dd if=/dev/sdX bs=64k conv=noerror,sync | pv | gzip -c -9 > sdX.img.gz
#gunzip -c sdX.img.gz | pv | dd of=/dev/sdX conv=sync,noerror bs=64K

Alright, so now you want to store your backup on your server? No problem, dd can handle networks too... with the help of SSH.

#dd if=/dev/sdX bs=64k conv=noerror,sync | pv | gzip -c -9 | ssh user@remote_server dd of=sdX.img.gz

Other gems of the Disk Destroyer:

Floppy copy: dd if=/dev/fd0 of=floppy.img bs=2x80x18b conv=notrunc
CD ISO copy: dd if=/dev/sr0 of=mycd.iso bs=2048 conv=notrunc
MBR Copy: dd if=/dev/sda of=mbr.img bs=512 count=1
MBR Wipe: dd if=/dev/zero of=/dev/sda bs=512 count=1

Disk Wipe: dd if=/dev/zero of=/dev/sda bs=64k
(could follow with if from random/urandom and then another zero, but you may not be paranoid)

... ... ...

Really nerdy stuff follows:

view filesystems

dd if=/proc/filesystems | hexdump -C | less

all loaded modules

dd if=/proc/kallsyms | hexdump -C | less

interrupt table

dd if=/proc/interrupts | hexdump -C | less

system uptime (in seconds)

dd if=/proc/uptime | hexdump -C | less

partitions and sizes in kb

dd if=/proc/partitions | hexdump -C | less

mem stats

dd if=/proc/meminfo | hexdump -C | less

There's been a number of questions regarding disk cloning tools (yes, I did search for it) :) and dd has been suggested at least once. I've already considered using dd myself, mainly because ease of use, and that it's readily available on pretty much all run of the mill bootable linux distributions. To my question, what is the best way to use dd for cloning a disk. I don't have much time to go around, and no test-hardware to play with, so I need to be prepared, and it's pretty much a one-shoot chance to get it done. I did a quick google search, and the first result was an apparent failed attempt. Is there anything I need to do after using dd, i.e. is there anything that CAN'T be read using dd? Thanks! dd disk-cloning flag edited Jul 17 at 19:13

Joseph 2,273●5●16 asked May 5 at 18:21

roe 520●1●2●10

I'm aware how dd works, my question was more in the direction of any known problems related to dd when cloning disks (as described by the link), maybe this wasn't very clear. What his answer contains and yours doesn't is "I've never once had any problems with it". I did upvote your answer too, as you did definitely present some interesting points (I like the one about no progress indication). – roe May 6 at 20:25

Looks like you got the Spolsky Bump: joelonsoftware.com/items/2009/05/29.html – Kyle Cronin May 29 at 18:04

didn't see this on here when I asked (and answered) a similar question on superuser - superuser.com/questions/11453/… – warren Aug 31 at 5:58 24 Answers oldest newest votes 23 dd is most certainly the best cloning tool, it will create a 100% replica simply by using the following command. I've never once had any problems with it. dd if=/dev/sda of=/dev/sdb

link|flag answered May 5 at 18:31

Adam Gibbins 3,356●6●22

5

Of course, as long as /dev/sdb is at least as large as /dev/sda... – Eddie May 5 at 21:09 5

add a "bs=100M conv=notrunc" and it's much faster in my experience. – Tim Williscroft May 6 at 22:14 3

@Eddie - and of course the partition table will be copied too, so if sdb is larger you'll have unused space at the end. – Alnitak May 7 at 8:41 1

notrunc means (according to the man page) : do not trunc the output file. I don't understand how it can be faster. You can follow the progression of the operation with : # dd if=/dev/sda of=/dev/sdb & pid=$! # kill -USR1 $pid; sleep 1; kill $pid – Manu May 29 at 19:09 12

just be very careful with the 'i' and 'o' letters... – bandi May 29 at 21:52 show 3 more comments

18 CAUTION: dd'ing a live filesystem can corrupt files. The reason is simple, it has no understanding of the filesystem activity that may be going on, and makes no attempt to mitigate it. If a write is partially underway, you will get a partial write. This is usually not good for things, and generally fatal for databases. Moreover, if you screw up the typo-prone if and of parameters, woe unto you. However, DD should accurately capture the bit state of an unmounted drive. Bootloaders, llvm volumes, partition UUIDs and labels, etc. Just make sure that you have a drive capable of mirroring the target drive bit for bit. link|flag answered May 5 at 20:20

jldugger 4,492●3●27

You can always use 'sync' to sync the file system to the hdd before running dd. – LiraNuna Jul 17 at 20:08 1

I suspect that sync is not the answer to file corruption problems. What happens if a deamon or something writes more files after the sync, during the dd operation? – Kent Aug 13 at 10:47

It's a good idea to umount the drive first (or remount as read-only) but it's not always possible – Bolotov Aug 22 at 9:36

In which case, you use rsync and let it do file handle magic to get a consistent file and let Copy On Write semantics handle the incoming writes. – jldugger Aug 22 at 21:18 13 To save space, you can compress data produced by dd with gzip, e.g.: dd if=/dev/hdb | gzip -c > /image.img

You can restore your disk with: gunzip -c /image.img.gz | dd of=/dev/hdb

To save even more space, defragment the drive/partition you wish to clone beforehand (if appropriate), then zero-out all the remaining unused space, making it easier for gzip to compress: mkdir /mnt/hdb mount /dev/hdb /mnt/hdb dd if=/dev/zero of=/mnt/hdb/zero

Wait a bit, dd will eventually fail with a "disk full" message, then: rm /mnt/hdb/zero umount /mnt/hdb dd if=/dev/hdb | gzip -c > /image.img

Also, you can get a dd process running in the background to report status by sending it a signal with the kill command, e.g.: dd if=/dev/hdb of=/image.img & kill -SIGUSR1 1234

Check your system - the above command is for Linux, OSX and BSD dd commands differ in the signals they accept (OSX uses "SIGINFO", I understand). link|flag edited May 7 at 8:24

answered May 6 at 22:47

David Hicks 634●1●10

Does this also work with "modern" fs such a BTRFS, NILFS, [whatever you can dream of] ? – Steve Schnepp Jun 26 at 14:41

DD works on block devices, a level of abstraction lower than the file system, so it should, yes. I haven't actually tried it, though. Hmm, NILFS looks interesting, I'll have to take a look at that. – David Hicks Jun 28 at 22:47

Sorry, just checked out NILFS' homepage and realised what you might have meant - can you use DD to copy a snapshot from a NILFS filesystem? I don't know, but it'd be interesting to find out. – David Hicks Jun 28 at 23:01 11 To clone a disk, all you really need to do is specify the input and output to dd: dd if=/dev/hdb of=/image.img

Of course, make sure that you have proper permissions to read directly from /dev/hdb (I'd recommend running as root), and that /dev/hdb isn't mounted (you don't want to copy while the disk is being changed - mounting as read-only is also acceptable). Once complete, image.img will be a byte-for-byte clone of the entire disk. There are a few drawbacks to using dd to clone disks. First, dd will copy your entire disk, even empty space, and if done on a large disk can result in an extremely large image file. Second, dd provides absolutely no progress indications, which can be frustrating because the copy takes a long time. Third, if you copy this image to other drives (again, using dd), they must be as large or larger than the original disk, yet you won't be able to use any additional space you may have on the target disk until you resize your partitions. You can also do a direct disk-to-disk copy: dd if=/dev/hdb of=/dev/hdc

but you're still subject to the above limitations regarding free space. As far as issues or gotchas go, dd, for the most part, does an excellent job. However, a while ago I had a hard drive that was about to die, so I used dd to try and copy what information I could off it before it died completely. It was then learned that dd doesn't handle read errors very well - there were several sectors on the disk that dd couldn't read, causing dd to give up and stop the copy. At the time I couldn't find a way to tell dd to continue despite encountering a read error (though it appears as though it does have that setting), so I spent quite a bit of time manually specifying skip and seek to hop over the unreadable sections. I spent some time researching solutions to this problem (after I had completed the task) and I found a program called ddrescue, which, according to the site, operates like dd but continues reading even if it encounters an error. I've never actually used the program, but it's worth considering, especially if the disk you're copying from is old, which can have bad sectors even if the system appears fine. link|flag edited May 30 at 18:07

answered May 5 at 18:26

Kyle Cronin 687●2●14

You can actually also use a read-only mount. A filesystem can be remounted with: mount -o remount,ro /path/to/device – Paul de Vrieze May 30 at 15:11

Good point, I added a note in my answer about that. – Kyle Cronin May 30 at 18:07

I used ddrescue to scrape data off a dying hard drive, and can confirm that it's awesome. – sleske Jun 23 at 0:45 7 When using dd to clone a disk which may contain bad sectors, use "conv=noerror,sync" to ensure that it doesn't stop when it encounters an error, and fills in the missing sector(s) with null bytes. This is usually the first step I take if trying to recover from a failed or failing disk -- get a copy before doing any recovery attempts, and then do recovery on the good (cloned) disk. I leave it to the recovery tool to cope with any blank sectors that couldn't be copied. Also, you may find dd's speed can be affected by the bs (block size) setting. I usually try bs=32768, but you might like to test it on your own systems to see what works the fastest for you. (This assumes that you don't need to use a specific block size for another reason, e.g. if you're writing to a tape.) link|flag answered May 7 at 2:42

TimB 748●2●5

7 Another nice thing you can do with dd and rescue disks is copy data over the network: remote_machine$ nc -l -p 12345 local_machine$ dd if=/dev/sda | nc remote_machine 12345

You can stick gzip in both these pipelines if the network is not local. For progress, use pv. To make local_machine's netcat quit after it's done copying, you might add -w 5 or something. link|flag answered May 29 at 18:09

gaal 81●1

5 Keep in mind that dd makes an exact copy, including all the blank space. That means: 2nd drive must be at least as big as first If 2nd drive is larger, extra space will be wasted (filesystem can be expanded mind you) If the source drive is not full, dd will waste alot of time copying blank space. You can copy either the entire drive, or a single partition this way. If this is a bootable drive, I'm pretty sure you need to install the bootloader after using dd Hope that is helpful link|flag answered May 5 at 18:38

Brent 6,936●1●5●22

1

If you're cloning the whole hard disk, you're also cloning the boot loader. – Cristian Ciupitu May 29 at 22:16 3 The source disk must not have any mounted filesystems. As a user able to read the block device (root works), run 'dd if=/dev/sda ....' Now, one of the neat things here is that you're creating a stream of bytes... and you can do a lot with that: compress it, send it over the network, chunk it into smaller blobs, etc. For instance: dd if=/dev/sda | ssh user@backupserver "cat > backup.img"

But more powerfully: dd if=/dev/sda | pv -c | gzip | ssh user@backupserver "split -b 2048m -d - backup-`hostname -s`.img.gz"

The above copies a compressed image of the source harddrive to a remote system, where it stores it in numbered 2G chunks using the source host's name while keeping you updated on progress. Note that depending on the size of disk, speed of cpu on source, speed of cpu on destination, speed of network, etc. You may want to skip compression, or do the compression on the remote side, or enable ssh's compression. link|flag edited May 29 at 19:28

answered May 29 at 19:23

retracile 816●7

3 To clone a disk, all you really need to do is specify the input and output to dd: dd if=/dev/hdb of=hdb.img

Of course, make sure that you have proper permissions to read directly from /dev/hdb (I'd recommend running as root), and that /dev/hdb isn't mounted (you don't want to copy while the disk is being changed). Once complete, hdb.img will be a byte-for-byte clone of the entire disk. There are a few drawbacks to using dd to clone disks. First, dd will copy your entire disk, even empty space, and if done on a large disk can result in an extremely large image file. Second, dd provides absolutely no progress indications, which can be frustrating because the copy takes a long time. Third, if you copy this image to other drives (again, using dd), they must be as large or larger than the original disk, yet you won't be able to use any additional space you may have on the target disk until you resize your partitions. You can also do a direct disk-to-disk copy: dd if=/dev/hdb of=/dev/hdc

but you're still subject to the above limitations regarding free space. The first drawback can be resolved by gzipping the data as you make the copy. For example: dd if=/dev/hdb | gzip -9 > hdb.img.gz

The second drawback can be resolved by using the pipeview (pv) tool. For example: dd if=/dev/hdb | (pv -s `fdisk -l /dev/hdb | grep -o '[0-9]*\{1\} MB' | awk '{print $1}'`m) | cat > hdb.img

I know of no way to overcome the third drawback. Additionally, you can speed up the copy time by telling dd to work with larger chunks of data. For example: dd if=/dev/hdb of=hdb.img bs=1024

link|flag answered May 29 at 22:03

jsumners 103●7

2 For future reference it might be of interest to check out ddrescue. It has saved my day a couple of times. link|flag answered May 6 at 21:06

Subtwo 61●1●3

2 If the source drive is damaged at all, you'll have more luck using dd_rhelp with dd_rescue (my personal preference) or GNU ddrescue. The reason behind this is that, on read errors, dd keeps trying and trying and trying - potentially waiting for a long time for timeouts to occur. dd_rescue does smart things like reading up to an error, then picking a spot further on on the disk and reading backwards to the last error, and dd_rhelp is basically a dd_rescue session manager - cleverly starting and resuming dd_rescue runs to make it quicker again. The end result of dd_rhelp is maximum data recovered in minimum time. If you leave dd_rhelp running, in the end it does the exact same job as dd in the same time. However, if dd encountered read errors at byte 100 of your 100Gb disk, you'd have to wait a long time to recover the other 9,999,900 bytes*, whereas dd_rhelp+dd_rescue would recover the bulk of the data much faster. link|flag answered May 31 at 2:12

Ben 21●1

1 dd does provide progress information - well most versions in linux. I've seen some which don't but don't recall the unix flavour. The man page says: Sending a USR1 signal to a running 'dd' process makes it print I/O statistics to standard error and then resume copying. I use this feature regularly. link|flag answered May 29 at 16:28

Steven 466●6

1 Someone had to say this: give Clonezilla a try (http:// clonezilla.org/) What do you get? To copy only used parts of the filesystem. Clonezilla uses dd, grub, sfdisk, parted, partimage, ntfsclone, and/or partclone. Depending on the options you choose. Decent documentation can be found at: http:// clonezilla.org/clonezilla-live/doc/ link|flag answered May 29 at 16:53

Roflo 11●1

I found the documentation a little rough, and cloning a linux PATA drive to a SATA drive did Not leave me with something I could boot (yet). But much faster to same result as dd, and it worked great for my laptop drive upgrades. – jbdavid May 30 at 0:43 1 Another grand feature is copying MBRs, partition tables and boot records. Just dd if=/dev/sda of=parttable bs=512 count=1

and the other direction around when you're writing it. Polish with fdisk after. You feel much more safe when you have your partition table backed up. Also, it makes migrating to another hard drive (while changing partion structure) a joy. link|flag answered May 29 at 18:18

alamar 11●1

1 This is kind of a cheap hack, but it's a quick and dirty way to monitor your DD process. Run your dd command. Open a new shell and do a ps awx to find your dd process' PID. Now in the new shell type watch -n 10 kill -USR1 {pid of your DD process} This will do nothing in the watch output window, but back in the original DD shell, DD will start outputting status reports every 10 seconds. You can change the -n 10 in the watch command to any other time frame of course. Tachyon link|flag answered Jul 14 at 20:13

Tachyon 11●1

1 For NTFS volumes, I prefer using ntfsclone. It's part of the ntfsprogs package. link|flag edited Jul 17 at 18:24

answered Jun 2 at 0:35

Ed Brannin 11●3

0 For some reason, dd fails when imaging CDs with audio tracks. You need to use cdrdao or something similar to get an image + TOC file. link|flag answered May 29 at 18:47

Matt 101●1

0 A note on speed: in my experience dd is twice as fast if you specify bs=1024 instead of the default bs=512. Using an even larger block size gives no noticeable speedup over bs=1024. link|flag answered May 29 at 19:26

mgedmin 1

1

disk clusters are generally around 4k now, so using 4096 is probably a good option, and even 8192 if you want to read 2 clusters at a time. Don't go too big though, as you run into fragmented memory problems – davejsmith May 29 at 19:32 0 One thing you must be aware of when dd-ing a full disk is that doing so will overwrite the master boot record of the receiving disk. This contains the partition table and other vital information. If the new disk is not the same as the old disk, this can create all sorts of tables. Copying over partitions is generally safer (and swap partitions do not have to be copied over) link|flag answered May 30 at 15:15

Paul de Vrieze 121●1●3

0 I've been out of the admin role for many years now, but I know that 'dd' is up to the job. I used this technique regularly in the late 80s on Sun Sparc and 386i computers. I had one client order over 30 386i systems running CAD software that was distributed on multiple QIC tapes. We installed on the first computer, configured the app, ran SunOS' sys-unconfig, placed the drive in a shoebox with a different SCSI address and then proceeded to 'dd' to the other 30 drives. link|flag answered Jun 1 at 11:23

pbrooks100 91●2

0 As others have mentioned above, one of the gotchas with cloning a mounted file system is potential data corruption. This obviously won't apply to full drive clones, but if you're using LVM you can Snapshot the LogicalVolume and dd from the snapshot to get a consistent image. link|flag answered Jun 2 at 0:51

Ophidian 984●1●8

0 You can create a compressed image file of the partition (or disk) on the fly using bzip2 or gzip instead of dd. This is nice for storing away images in removable media: bzip2 -c /dev/sdaX >imagefile.bz2 or gzip -c /dev/sdaX >imagefile.gz

If the disk has been heavily used before, you can enhance the compression by filling all unused space with zeros before the imaging: mkdir /mnt/mymountpoint mount /dev/sdaX /mnt/mymountpoint cat /dev/zero >/mnt/mymountpoint/dummyfile.bin (Wait for it to end with a "disk full" error) rm /mnt/mymountpoint/dummyfile.bin umount /mnt/mymountpoint

To restore the image into another disk, all you have to do is: bzcat imagefile.bz2 >/dev/sdbY or zcat imagefile.gz >/dev/sdbY

link|flag answered Jun 26 at 14:37

JCCyC 228●8

0 You could actually try something like this dd if=/dev/sda2 of=/dev/sdb2 bs=4096 conv=notrunc,noerror to skip all errors and have exact clone of a hard drive link|flag answered Aug 22 at 9:23

hirol 1

0 ddrescue worked great for me.

[Mar 10, 2010] mount_dd

freshmeat.net

mount_dd is a small tool for mounting a dd image with a GUI. You can mount it in read-write or read-only mode.

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March, 12, 2019