cpio command

• Introduction
• Options
• Simple backup script
• Examples
• Retrieving a Subset of Files

Introduction

The cpio  command is one of standard Unix backup utilities. It stands for "copy in/out." It is much less well known and more rarely used Unix utility in comparison with tar.

But in certain areas it is more powerful then tar due to its ability to handle links and special files, append backup volumes and span tapes, allowing you to create incremental backup sets and full systems backups without losing data integrity. The cpio  preserves permissions, times, and ownership of files and subdirectories. The data archive in RPM format is based on cpio archive and you can extract files from it.

cpio  works as a filter accepting standard input and writing to standard output. cpio  allows you to copy files into and out of a cpio  archive.

The input to cpio is the list of files. That means that results of ls  or find  command can be piped directly into cpio.  You can specify a device or file to which cpio will send its output, rather than sending it to stdout.

Piping find  command output into cpio allows to copy the entire directory tree while preserving the permissions, times, and ownership of every file and subdirectory. To do so, combine find  and the cpio  command, like this:

cd /path/to/source/dir
find . | cpio -pdumv /path/to/destination/dir

All the files you specify are copied to the current directory.

In this example, all files are restored from

cpio -icv < /mnt/backup/boot110725.cpi

Options

For creation of an archive

There are six options that should be used when making regular cpio backups. The first five usually are listed all at once (e.g., -oacvB), and the last one usually is listed as a separate argument (e.g., -C 5120).

• -o  Output - a backup should be created. The -o option is one of the cpio three modes of  (o, i, and p) and is the one used for creating a backup.

• -a Reset the access times of files after reading them to the initial value, so that it does not look like they have just been read. One of the differences between dump and cpio is that dump backs up directly using the disk device, whereas cpio must go through the filesystem.  In Unix reading a file changes its access time (atime). System administrators often use this value to see when a user has last used a file. Files that have not been accessed in a long time are not being used and typically can be removed from the system as part of a cleanup process.

• -c  The c option tells cpio to use the ASCII header format. Important for portability.
• -v The v option results in verbose output.
• -B, C  The B and C options let you specify the block size. In addition, you can specify a device or file to which cpio will send its output, rather than sending it to stdout. All of these options and more also are available in the GNU version of cpio, as is the ability to use remote devices.

For extraction

In order to extract files from a cpio  archive, you can do something similar to the following example:

cpio -icdB < /root/my_archive.cpio

where -i  extracts from the archive, c  writes header information in ASCII character form, d  creates directories as needed, and B  blocks input to 512 bytes to the record.

If you use the -o  option cpio archives pathname and file permissions information, as well as the contents for each name of the file it gets as input.

Note:

The -B and -C options are mutually exclusive—they cannot be used together.

Simple backup script

Here is the script that can help to backup system partition by partition. Here we assume that directories /root, /etc, /usr and /opt belong to root partition

#!/bin/bash -x
# a simple backup script that create cpio archives of each partition one by one
#target directory will not be backed up. It can be /tmp or /backup
TARGET=/tmp
for   d  in `df   -kl |  grep  "/dev/mapper"   |  cut   -c 59-`  ;  do 
   name=$d
   OPTIONS='-xdev -fstype ext3'
   if [[ "$d"  =  '/'   ]] ;  then
      name=/root
      OPTIONS='-xdev'
   elif [[ $d =  "$TARGET"   ]] ;  then
      continue
   fi
   echo Backup of $d to /$TARGET/$name`date +%y%m%d`.cpi
   find $d $OPTIONS | cpio -oav > /$TARGET/$name`date +%y%m%d`.cpi
   sleep 1
done

Examples

For example to back up the contents of a directory you can use ls command. The device file can be a regular file, for example, /tmp/mybackup:

ls | cpio -oacv > /tmp/mybackup

You can also backup all files from a partition (and that in Unix means the whole partition) to a file, for example

find /home -xdev -fstype ext3  | cpio -oacv >/backup/home130128.cpi

 find . -name .snapshot -prune -o \( \! -name *~ -print0 \)| cpio -pmd0 /dest-dir

You can also backup key second level directories of the root partition which give you the opportunity to restore the server from those files:

for d in /boot /bin /sbin /dev /lib64 /root /etc /home /opt /var ; do
   echo $d
   find $d -xdev -fstype ext3 | cpio -oav > /backup/$d`date +%y%m%d`.cpi
   sleep 10
done

In order to extract files from a cpio  archive, you can do something similar to the following example:

cpio -icdB < /root/my_archive.cpio

where -i  extracts from the archive, c  writes header information in ASCII character form, d  creates directories as needed, and B  blocks input to 512 bytes to the record.

If the archive was created using relative path names, the input files are built as a directory within the current directory. If, however, the archive was created with absolute path names, the same absolute paths are used to re-create the file. Here is how to copy a directory's contents

Retrieving a Subset of Files

You can retrieve a subset of the files from the archive by specifying a pattern to match using shell wildcard characters enclosed in quotation marks after the options:

1. Change to the directory where you want to put the files.

2. Insert the tape into the tape drive.

3. Type cpio -icv "*file  *" < /dev/rmt/n   and press Return. All the files that match the pattern are copied to the current directory. You can specify multiple patterns, but each must be enclosed in quotation marks.

NEWS CONTENTS

• 20020711 : -- Understanding CPIO by Dru Lavigne ( ONLamp.com )
• 190317 : Understanding cpio by Dru Lavigne ( Understanding cpio, )
• 190317 : cpio ( cpio, )
• 190317 : CPIO How-To ( CPIO How-To, )
• 190317 : OReilly - Safari Books Online - 078972250X - Practical UNIX ( O'Reilly - Safari Books Online - 078972250X - Practical UNIX, )
• 190317 : Backup using cpio command in linux ( Backup using cpio command in linux, )

Old News ;-)

[07/11/2002] -- Understanding CPIO by Dru Lavigne

ONLamp.com

In the previous article, I demonstrated the usage of the tar archiver utility. This week I'll continue by introducing the cpio archiver utility.

While both tar and cpio will achieve the same results, the cpio utility approaches things a little bit differently. The tar utility assumes that you want to recursively archive everything under the specified directory or directories, meaning that you have to explicitly tell tar if you want to exclude certain portions of that directory structure. In contrast, the cpio utility expects to be explicitly told which files or directories you wish to archive; this behavior is commonly referred to as "receiving from standard input." In other words, cpio expects to receive a list that contains one file per line, and if you remember from the "Finding Things in Unix" and "Find: Part Two" articles, that is exactly the type of list that the find utility creates. The ls utility can also create this type of list, meaning that you will see either of the ls or the find utilities used in conjunction with cpio. And since cpio archives a list of files it receives from standard input, you usually use a pipe (|) whenever you create an archive with the cpio utility.

The tar utility also assumes that you want to write the archive to your first SCSI tape drive, unless you explicitly specify a file using the f switch. In contrast, the cpio utility writes to what is known as standard output. This means that you will be using a redirector (either < or >) whenever you are creating, listing, or extracting a cpio archive file. Again, that file may be an actual file, or it may be your floppy, or it may be a tape device, since in Unix everything is a file.

This may sound a bit more complicated at first, but a few examples should convince you that it really isn't.

Let's start by creating a cpio archive. In the last article, I created a test user account and created a directory structure named www in this user's home directory so I would have some files on which to practice using the archiving utilities. I'll log in as the test user, cd into the www directory, and see what happens if I use the ls command with the cpio utility:

cd www; ls | cpio -ov > backup.cpio

You'll note that I first cded into the directory that contained the files I wished to archive. I used the ls utility to make a list of the files in the current directory and used a pipe (|) to send that list to the cpio utility. The o switch invokes what is known as "copy out mode," which tells cpio to create an archive. The v switch tells cpio to be verbose, meaning it will list each file as it archives it. Finally, I used the > redirector to write the results (the archive) to a file called backup.cpio. I can call this file anything I like; I chose to give it a cpio extension to remind me that it is a cpio backup file. I can verify the file type using the file utility:

file backup.cpio 
backup.cpio: cpio archive

Instead of using the redirector, I could have also used the F switch to specify which file to write the archive to. So the following command will achieve the same results:

ls | cpio -ovF backup.cpio

Once the archive was created, cpio told me how many blocks it wrote to the archive; in my case, it was 48 blocks.

So to create an archive, use the o switch or copy-out mode. To either view or extract the contents of the archive, use what is known as "copy-in mode." You invoke this mode by using the i switch. If you just want to view the contents of the archive, also include the t switch, which will list the contents of the archive without extracting them:

cpio -it < backup.cpio

You'll note that this time I used the other redirector (<), as I wanted the contents of the backup.cpio file to be sent to the cpio utility. I can also include the v switch, if I want to see a verbose listing of the backup:

cpio -itv < backup.cpio

Remember that it is important to view the contents of an archive before attempting to restore it, as you want to ensure that the files don't begin with a /.

To restore this archive, I simply cd into the directory to which I'd like to restore the archive, and repeat the above command without the t switch. I'll cd back into my home directory and create a directory named backupand do the restore there:

cd
mkdir backup
cd backup
cpio -iv < ~/www/backup.cpio

You'll note something interesting if you try this exercise yourself; if you use the ls -F command, you'll see that you did indeed restore all of the files and directories that were in the www directory. But if you cd into any of those subdirectories, you'll note that they are empty. Even more interestingly, if you try to remove any of those subdirectories, you still have to use the R switch, as they are still valid directories.

What happened here? Since the cpio utility received its file list from the ls utility (and the ls utility can only list the files in the current directory), cpio was unaware of all of the files that existed below the current directory. Remember, cpio will only archive the files that are sent to it in a list. This may seem odd at first, but it is an ideal way to archive just the files in the current directory. In order to do this with the tar utility, you would have to create an exclude file, as tar wants to recursively copy everything in and below the current directory.

This doesn't mean that cpio can't archive recursively; it simply means that if you want to just archive the current directory, you use ls and if you want to archive recursively, you use find instead.

Let's try that backup and restore again, this time using the find utility. First, I'll remove the old backup and empty out the backup directory:

rm www/backup.cpio
rm -R backup/*

Then I'll cd into the directory I wish to back up (www) and archive its contents:

cd www
find -d . -print | cpio -ov > backup.cpio

When using the find utility with cpio, it is always a good idea to include either the d or the depth switch. Remember from the find article that this switch prevented permissions from interfering with a backup. When using this switch, either put -d right after the word find and before the directory to search (in this case, "."), or put the word -depth after the directory to search, like so:

find . -depth -print | cpio -ov > backup.cpio

So as a recap on the find command, I told find to search the current directory (".") and to "print" its contents; the | was used to send those contents to the cpio utility, which created an archive (-o) and wrote that archive to a file called backup.cpio. When I created this archive, I noted that cpio wrote 43097 blocks, which is many more than the 48 I received with the ls command.

Now let's see what happens when I try to restore this archive:

cd ../backup
cpio -iv < ~/www/backup.cpio

I received an interesting message on my screen when I did this restore:

<snip>
cpio: mod_tsunami/Makefile: No such file or directory
cpio: mod_tsunami/distinfo: No such file or directory
cpio: mod_tsunami/pkg-comment: No such file or directory
cpio: mod_tsunami/pkg-descr: No such file or directory
cpio: mod_tsunami/pkg-plist: No such file or directory
mod_tsunami
Makefile
.
43097 blocks

It looks like cpio read all 43097 blocks but complained about missing files or directories. Indeed, if I do an ls on any of the restored subdirectories, I'll discover that they are once again empty! Don't worry, all of those files and directories are in that archive file; I've simply demonstrated the default extraction behaviour of cpio. Unlike tar, the cpio utility does not recreate any directories during the restore unless you specifically ask it to with the d switch. And, unlike tar, the cpio utility will not overwrite any existing files unless you specifically ask it to with the u switch.

So let's try that restore again, this time using the d switch to create the directories and the u switch to overwrite the files I've already restored:

cpio -ivdu < ~/www/backup.cpio

This time I don't receive any error messages and I've successfully restored all of the subdirectories and their files.

There're a few more switches you may consider using when backing up and restoring with cpio. If I compare the modification times of a file before it was archived and after it was restored, I will see this:

ls -l www/zope/Makefile	
-rw-r--r--  1 test  wheel  4308 May 11 09:53 www/zope/Makefile

ls -l backup/zope/Makefile
-rw-r--r--  1 test  wheel  4308 Jun  2 11:38 backup/zope/Makefile

ls -l www/backup.cpio
-rw-r--r--  1 test  wheel  22065664 Jun  2 10:39 www/backup.cpio

You'll note that the original file was created on May 11, that it was backed up on June 2 at 10:39, and that it was restored on June 2 at 11:38. If you want to preserve the file's original time, include the a switch when creating the archive, and the m switch when restoring the archive:

cd www
find -d . -print | cpio -ova > backup.cpio
cd ../backup
cpio -ivdm < ~/www/backup.cpio

If you try this and repeat the ls -l command, you'll see that the original times of the archived files were kept intact.

The nice thing about using the find utility with cpio is that you have all of find's switches available to you, to fine-tune which files you would like to back up. For example, if you'd like to do an incremental backup, use find's -newer switch. In this example, I'll back up all of the files in my home directory that have changed since 11 PM on June 1st:

cd
touch -t 06012300 June1
find -d . -newer June1 -print | cpio -ova > backup.cpio

Here I used the touch utility to create an empty file with a timestamp of month 06 day 01 time 2300, then I told find to use the time on that file as the reference point when searching the current directory. Alternatively, if I wasn't concerned so much about the time as the date, I could have used find's atime, ctime, or mtime switches. And if I only want to archive files of a certain size, I can use find's size switch.

Before ending today's article, I'd also like to demonstrate cpio's third mode, which is known as "copy-pass mode." This is an interesting mode, as it archives and extracts in the same command, making it ideal for copying one directory structure and recreating it in another location.

Let's say I want to copy the www directory structure from the home directory of the test user to the home directory of the user genisis. I'll have to become the superuser, as I'll be creating the archive in one user's home directory and recreating it in another user's home directory:

su
Password:
cd ~test/www
find -d . -print | cpio -pvd ~genisis/www

Note that I first cded into the directory I wanted to archive, in this case the www subdirectory of the test user's home directory. Then, with the cpio command, I invoked copy-pass mode with the p switch and specified that I wanted the archive recreated in the www subdirectory of the home directory of the user genisis.

If I run this command and then do an ls -l of genisis' home directory, I'll see that I've successfully recreated the entire www directory structure. However, I'll want to fine-tune that above command as those restored files still belong to the user "test." I'll repeat that command using the u switch so it will overwrite that last restore, and I'll include the R switch, which tells cpio to change the ownership of the files as it recreates them:

find -d . -print | cpio -pvdu -R genisis ~genisis/www

When using the R switch, follow it by the name of the user you wish to become the owner of the files, then follow that by the name of the directory to restore the files to.

Finally, if I want to keep the original times of the files instead of having them changed to the time the files were restored, I'd also add the a and m switches:

find -d . -print | cpio -pvduam -R genisis ~genisis/www

This should get you started with the cpio command. If you're planning on using cpio to copy between different computers, you'll want to read its manpage first, as there may be considerations, especially if the computers are running different versions of Unix or different architectures.

In next week's article, I'll continue the archiver series by introducing the pax command and, if space permits, the dd command.

Understanding cpio by Dru Lavigne

07/11/2002 | onlamp.com

Let's start by creating a cpio archive. In the last article, I created a test user account and created a directory structure named www in this user's home directory so I would have some files on which to practice using the archiving utilities. I'll log in as the test user, cd into the www directory, and see what happens if I use the ls command with the cpio utility:

cd www
ls | cpio -ov > backup.cpio

You'll note that I first cded into the directory that contained the files I wished to archive. I used the ls utility to make a list of the files in the current directory and used a pipe (|) to send that list to the cpio utility. The o switch invokes what is known as "copy out mode," which tells cpio to create an archive. The v switch tells cpio to be verbose, meaning it will list each file as it archives it. Finally, I used the > redirector to write the results (the archive) to a file called backup.cpio. I can call this file anything I like; I chose to give it a cpio extension to remind me that it is a cpio backup file. I can verify the file type using the file utility:

file backup.cpio
backup.cpio: cpio archive

Instead of using the redirector, I could have also used the F switch to specify which file to write the archive to. So the following command will achieve the same results:

ls | cpio -ovF backup.cpio

Once the archive was created, cpio told me how many blocks it wrote to the archive; in my case, it was 48 blocks.

So to create an archive, use the o switch or copy-out mode. To either view or extract the contents of the archive, use what is known as "copy-in mode." You invoke this mode by using the i switch. If you just want to view the contents of the archive, also include the t switch, which will list the contents of the archive without extracting them:

cpio -it < backup.cpio

You'll note that this time I used the other redirector (<), as I wanted the contents of the backup.cpio file to be sent to the cpio utility. I can also include the v switch, if I want to see a verbose listing of the backup:

cpio -itv < backup.cpio

Remember that it is important to view the contents of an archive before attempting to restore it, as you want to ensure that the files don't begin with a /.

To restore this archive, I simply cd into the directory to which I'd like to restore the archive, and repeat the above command without the t switch. I'll cd back into my home directory and create a directory named backupand do the restore there:

cd
mkdir backup
cd backup
cpio -iv < ~/www/backup.cpio

You'll note something interesting if you try this exercise yourself; if you use the ls -F command, you'll see that you did indeed restore all of the files and directories that were in the www directory. But if you cd into any of those subdirectories, you'll note that they are empty. Even more interestingly, if you try to remove any of those subdirectories, you still have to use the R switch, as they are still valid directories.

What happened here? Since the cpio utility received its file list from the ls utility (and the ls utility can only list the files in the current directory), cpio was unaware of all of the files that existed below the current directory. Remember, cpio will only archive the files that are sent to it in a list. This may seem odd at first, but it is an ideal way to archive just the files in the current directory. In order to do this with the tar utility, you would have to create an exclude file, as tar wants to recursively copy everything in and below the current directory.

This doesn't mean that cpio can't archive recursively; it simply means that if you want to just archive the current directory, you use ls and if you want to archive recursively, you use find instead.

Let's try that backup and restore again, this time using the find utility. First, I'll remove the old backup and empty out the backup directory:

rm www/backup.cpio
rm -R backup/*

Then I'll cd into the directory I wish to back up (www) and archive its contents:

cd www
find -d . -print | cpio -ov > backup.cpio

When using the find utility with cpio, it is always a good idea to include either the d or the depth switch. Remember from the find article that this switch prevented permissions from interfering with a backup. When using this switch, either put -d right after the word find and before the directory to search (in this case, "."), or put the word -depth after the directory to search, like so:

find . -depth -print | cpio -ov > backup.cpio

So as a recap on the find command, I told find to search the current directory (".") and to "print" its contents; the | was used to send those contents to the cpio utility, which created an archive (-o) and wrote that archive to a file called backup.cpio. When I created this archive, I noted that cpio wrote 43097 blocks, which is many more than the 48 I received with the ls command.

Now let's see what happens when I try to restore this archive:

cd ../backup
cpio -iv < ~/www/backup.cpio

I received an interesting message on my screen when I did this restore:

<snip>
cpio: mod_tsunami/Makefile: No such file or directory
cpio: mod_tsunami/distinfo: No such file or directory
cpio: mod_tsunami/pkg-comment: No such file or directory
cpio: mod_tsunami/pkg-descr: No such file or directory
cpio: mod_tsunami/pkg-plist: No such file or directory
mod_tsunami
Makefile
.
43097 blocks

It looks like cpio read all 43097 blocks but complained about missing files or directories. Indeed, if I do an ls on any of the restored subdirectories, I'll discover that they are once again empty! Don't worry, all of those files and directories are in that archive file; I've simply demonstrated the default extraction behaviour of cpio. Unlike tar, the cpio utility does not recreate any directories during the restore unless you specifically ask it to with the d switch. And, unlike tar, the cpio utility will not overwrite any existing files unless you specifically ask it to with the u switch.

So let's try that restore again, this time using the d switch to create the directories and the u switch to overwrite the files I've already restored:

cpio -ivdu < ~/www/backup.cpio

This time I don't receive any error messages and I've successfully restored all of the subdirectories and their files.

There're a few more switches you may consider using when backing up and restoring with cpio. If I compare the modification times of a file before it was archived and after it was restored, I will see this:

ls -l www/zope/Makefile	
-rw-r--r--  1 test  wheel  4308 May 11 09:53 www/zope/Makefile

ls -l backup/zope/Makefile
-rw-r--r--  1 test  wheel  4308 Jun  2 11:38 backup/zope/Makefile

ls -l www/backup.cpio
-rw-r--r--  1 test  wheel  22065664 Jun  2 10:39 www/backup.cpio

You'll note that the original file was created on May 11, that it was backed up on June 2 at 10:39, and that it was restored on June 2 at 11:38. If you want to preserve the file's original time, include the a switch when creating the archive, and the m switch when restoring the archive:

cd www
find -d . -print | cpio -ova > backup.cpio
cd ../backup
cpio -ivdm < ~/www/backup.cpio

If you try this and repeat the ls -l command, you'll see that the original times of the archived files were kept intact.

The nice thing about using the find utility with cpio is that you have all of find's switches available to you, to fine-tune which files you would like to back up. For example, if you'd like to do an incremental backup, use find's -newer switch. In this example, I'll back up all of the files in my home directory that have changed since 11 PM on June 1st:

cd
touch -t 06012300 June1
find -d . -newer June1 -print | cpio -ova > backup.cpio

Here I used the touch utility to create an empty file with a timestamp of month 06 day 01 time 2300, then I told find to use the time on that file as the reference point when searching the current directory. Alternatively, if I wasn't concerned so much about the time as the date, I could have used find's atime, ctime, or mtime switches. And if I only want to archive files of a certain size, I can use find's size switch.

Before ending today's article, I'd also like to demonstrate cpio's third mode, which is known as "copy-pass mode." This is an interesting mode, as it archives and extracts in the same command, making it ideal for copying one directory structure and recreating it in another location.

Let's say I want to copy the www directory structure from the home directory of the test user to the home directory of the user genisis. I'll have to become the superuser, as I'll be creating the archive in one user's home directory and recreating it in another user's home directory:

su
Password:
cd ~test/www
find -d . -print | cpio -pvd ~genisis/www

Note that I first cded into the directory I wanted to archive, in this case the www subdirectory of the test user's home directory. Then, with the cpio command, I invoked copy-pass mode with the p switch and specified that I wanted the archive recreated in the www subdirectory of the home directory of the user genisis.

If I run this command and then do an ls -l of genisis' home directory, I'll see that I've successfully recreated the entire www directory structure. However, I'll want to fine-tune that above command as those restored files still belong to the user "test." I'll repeat that command using the u switch so it will overwrite that last restore, and I'll include the R switch, which tells cpio to change the ownership of the files as it recreates them:

find -d . -print | cpio -pvdu -R genisis ~genisis/www

When using the R switch, follow it by the name of the user you wish to become the owner of the files, then follow that by the name of the directory to restore the files to.

Finally, if I want to keep the original times of the files instead of having them changed to the time the files were restored, I'd also add the a and m switches:

find -d . -print | cpio -pvduam -R genisis ~genisis/www

This should get you started with the cpio command. If you're planning on using cpio to copy between different computers, you'll want to read its manpage first, as there may be considerations, especially if the computers are running different versions of Unix or different architectures.

In next week's article, I'll continue the archiver series by introducing the pax command and, if space permits, the dd command.

cpio

In this month's column, Eric moves beyond find to cover duplicating files and directory trees using the versatile cpio command. cpio uses space on tape more efficiently than tar and is an excellent alternative for creating archives on platforms that do not have the GNU utilities available. Read on for a thorough discussion of cpio and its three modes of operation: Pass-through, Create and Extract.

CPIO How-To

Create an Archive

cpio creates archives differently than tar. Where tar automatically recurses into subdirectories, cpio reads from stdin a list of files & directories to archive; it does not automatically recurse into directories.

To create an archive, give cpio the -o (copy out) command line option. cpio will read a list of files & directories from stdin, create the archive, & write the archive to stdout.

A good way to generate the list of files is the find program.

To archive everything in a directory, compress it with bzip2, & write the results to a file, do this:

find dir -print |cpio -o |bzip2 >dir.cpio.bz2

That's the generic way to create an archive. On a Gnu/Linux system, you might get a lot of ugly warnings about i-node numbers being truncated. The archive will be fine, but it's never good to have unnecessary errors in the output; the eye-sore might prevent you from seeing important error messages. To prevent all those warnings, type this:

find dir -print |cpio -o -Hnewc |bzip2 >dir.cpio.bz2

A potential problem is that "-Hnewc" is not portable to all implementations of cpio. So either you must know when it's okay to use it or you must avoid using it & suffer with the gratuitous warning messages.

So far, we've created archives of all files in a directory tree. In other words, we've reproduced the functionality of tar but at the cost of more key strokes. Not very impressive. Since cpio reads a list of files from stdin, we can do a lot more.

If you want to create a distribution archive of your source code, leaving out object files (*.o), backup files (*~), and CVS & RCS directories, just take advantage of the features of find that you already know & love.

find dir \
     -name "*.o" -o \
     -name "*~" -o \
     -name CVS -prune -o \
     -name RCS -prune -o \
     -print \
|cpio -o -Hnewc |bzip2 >dir.cpio.bz2

(I've broken the example into multiple lines for readability. You'd either type the command on a single command line, or you'd break it into multiple lines, as I've done, by including the back-slashes (\) literally.)

Need to backup just the files that have changed since your last backup yesterday? Trivial!

find dir -ctime -1 -print \
|cpio -o -Hnewc |bzip2 >dir.cpio.bz2

By using find to generate the list of files, you can make cpio archive any combination of files you want. It's easy to use find from your own shell scripts, too, or you could even use your own programs to generate the list of file names. cpio achieves great flexibility by leaving the file-selection responsibilities to another program.

Advanced Features

Some (most? all?) cpio implementations are able to access file systems & tapes through a cpio server on another host. A benefit there is that you can use cpio to archive files from one host but write the archive file to, say, the tape drive on another host. I've found this useful in cases where I needed to backup large amounts of data to a tape drive, but the tape drive was on a server that didn't have enough disk space to hold a temporary copy of the entire archive, so I had to go directly to tape.

To use this feature, use the -O (that's a capital O) command line option in conjunction with the user@host:pathname method of specifying the destination file. See "man cpio" for details.

Similarly, you can use the -I command line option to extract files from tape archives mounted on servers.

As cool as it sounds, this feature has some draw-backs. System-specific command line options & device-file names are often necessary. For example, you might have to force special block sizes with -B or --block-size, or you might have to use system-specific device file names, such as /dev/st/n0a1bf00a or something similarly incomprehensible. Also, systems sometimes behave as though the communication between the client (your cpio process) & the server are treated as text, so non-text characters & end-of-lines get mangled. In other words, it sometimes just doesn't work.

In those cases, I've often made it work by using rsh and dd explicitly. In other words:

find . -print |cpio -o -Hnewc | rsh server dd bs=32kb of=/dev/st0

(The values for block size (bs) & output file (of) are system-specific, of course, & might differ for you.)

O'Reilly - Safari Books Online - 078972250X - Practical UNIX

Using cpio to Back Up and Restore Files

You will find that this section on cpio uses much of the same wording as in the previous section on tar. The two commands have similar functionality, and only some specific details and examples change the two sections. The tar and cpio sections are complete in themselves, without complicated references to one another, but if you read them both, you'll get a strong sense of déjà vu.

Backing Up a Directory Subtree

cpio stands for copy in/out and it enables you to back up files to tape, disk, or a disk file. Here is an example of a basic cpio command to back up one complete directory:

$ cd /usr/fred/projects
$ find . -print | cpio -ocvaB > /dev/XXX?

First use cd to change the directory to back up. Then use the find command to feed all the pathnames to cpio to back them up.

Backup using cpio command in linux

Backing Up using the cpio Command

To do a backup, use cpio with a search command, such as find .

The basic structure is: find -name string -print | cpio -o options > directory .

In this example:

The -name option for find lets you search for a string enclosed in double quotation marks. Metacharacters can be used. The bar character ( | ) redirects the output of find to cpio .

The -o flag sets cpio to create an archive file for backing up.

The target is a directory.

The > redirection operator redirects files to the location for the back up. Typically, this location is on a removable device.

CPIO Command Examples

Examples	What it does
1) cd /u/test 2) find . -print | cpio -ocv > /dev/fd0	Reads file names using the find command and copies to the floppy drive (/dev/fd0).
find . -cpio /dev/fd0 -print	Saves files in current directory and writes this info to floppy. Same command as above except much faster.
1) cd /u/test 2) cpio -icuvd < /dev/fd0	Restore files and directories saved on the floppy device. These files are restored under the current directory (/u/test) Only if relative pathnames (./<filename>) were used.
cpio -itvcC1 < /dev/rmt0	List the table of contents from a tape device.
1) cd /u/test 2) find . -print | cpio -dumpv /u/jerry	Copies all files FROM one directory TO another WITHOUT changing the permissions, owner/group or modification date of the file. Use the following command to verify that all files were copied:
find /u/test -print | wcfind /u/test1 -print | wc	If the number of files encountered is the same for both directories its safe to assume that the directories are identical. NOTE: that the number of blocks allocated to the SOURCE directory (/u/test) may be larger than the DESTINATION directory (/u/test1), since compaction of the directory structure will have occurred at the destination end.
cpio -imv /home/test/.profile < /dev/fd0	Selectively restore the /home/test/.profile file from floppy
cpio -i "*.f" "*.c" </dev/fd0	Selectively restore only the *.f and *.c files from floppy

Recommended Links

Softpanorama Recommended

Top articles

Sites

cpio - Wikipedia, the free encyclopedia

GNU cpio

AIX Commands Reference

CPIO How-To by Gene Michael Stover's

Red Hat Linux-Specific Information

Is tar-cpio a good backup program

Reference

GNU cpio

• Introduction
• Tutorial: Getting started.
• Invoking cpio: How to invoke cpio.
• Media: Using tapes and other archive media.
• Reports: Reporting bugs or suggestions
• Concept Index: Concept index.

--- The Detailed Node Listing ---

Invoking cpio

• Copy-out mode
• Copy-in mode
• Copy-pass mode
• Options

1 Introduction

GNU cpio copies files into or out of a cpio or tar archive, The archive can be another file on the disk, a magnetic tape, or a pipe.

GNU cpio supports the following archive formats: binary, old ASCII, new ASCII, crc, HPUX binary, HPUX old ASCII, old tar, and POSIX.1 tar. The tar format is provided for compatibility with the tar program. By default, cpio creates binary format archives, for compatibility with older cpio programs. When extracting from archives, cpio automatically recognizes which kind of archive it is reading and can read archives created on machines with a different byte-order.

2 Tutorial

GNU cpio performs three primary functions. Copying files to an archive, Extracting files from an archive, and passing files to another directory tree. An archive can be a file on disk, one or more floppy disks, or one or more tapes.

When creating an archive, cpio takes the list of files to be processed from the standard input, and then sends the archive to the standard output, or to the device defined by the -F option. See Copy-out mode. Usually find or ls is used to provide this list to the standard input. In the following example you can see the possibilities for archiving the contents of a single directory.

% ls | cpio -ov > directory.cpio

The -o option creates the archive, and the -v option prints the names of the files archived as they are added. Notice that the options can be put together after a single - or can be placed separately on the command line. The '>' redirects the cpio output to the file 'directory.cpio'.

If you wanted to archive an entire directory tree, the find command can provide the file list to cpio:

find . -print -depth | cpio -ov > tree.cpio

This will take all the files in the current directory, the directories below and place them in the archive tree.cpio. Again the -o creates an archive, and the -v option shows you the name of the files as they are archived. See Copy-out mode. Using the '.' in the find statement will give you more flexibility when doing restores, as it will save file names with a relative path vice a hard wired, absolute path. The -depth option forces 'find' to print of the entries in a directory before printing the directory itself. This limits the effects of restrictive directory permissions by printing the directory entries in a directory before the directory name itself.

Extracting an archive requires a bit more thought because cpio will not create directories by default. Another characteristic, is it will not overwrite existing files unless you tell it to.

% cpio -iv < directory.cpio

This will retrieve the files archived in the file directory.cpio and place them in the present directory. The -i option extracts the archive and the -v shows the file names as they are extracted. If you are dealing with an archived directory tree, you need to use the -d option to create directories as necessary, something like:

% cpio -idv < tree.cpio

This will take the contents of the archive tree.cpio and extract it to the current directory. If you try to extract the files on top of files of the same name that already exist (and have the same or later modification time) cpio will not extract the file unless told to do so by the -u option. See Copy-in mode.

In copy-pass mode, cpio copies files from one directory tree to another, combining the copy-out and copy-in steps without actually using an archive. It reads the list of files to copy from the standard input; the directory into which it will copy them is given as a non-option argument. See Copy-pass mode.

% find . -depth -print0 | cpio --null -pvd new-dir

The example shows copying the files of the present directory, and sub-directories to a new directory called new-dir. Some new options are the -print0 available with GNU find, combined with the --null option of cpio. These two options act together to send file names between find and cpio, even if special characters are embedded in the file names. Another is -p, which tells cpio to pass the files it finds to the directory 'new-dir'.

3 Invoking cpio

• Copy-out mode
• Copy-in mode
• Copy-pass mode
• Options

3.1 Copy-out mode

In copy-out mode, cpio copies files into an archive. It reads a list of filenames, one per line, on the standard input, and writes the archive onto the standard output. A typical way to generate the list of filenames is with the find command; you should give find the -depth option to minimize problems with permissions on directories that are unreadable. See Options.

     cpio {-o|--create} [-0acvABLV] [-C bytes] [-H format]
     [-M message] [-O [[user@]host:]archive] [-F [[user@]host:]archive]
     [--file=[[user@]host:]archive] [--format=format]
     [--message=message][--null] [--reset-access-time] [--verbose]
     [--dot] [--append] [--block-size=blocks] [--dereference]
     [--io-size=bytes] [--rsh-command=command] [--help] [--version]
     < name-list [> archive]

3.2 Copy-in mode

In copy-in mode, cpio copies files out of an archive or lists the archive contents. It reads the archive from the standard input. Any non-option command line arguments are shell globbing patterns; only files in the archive whose names match one or more of those patterns are copied from the archive. Unlike in the shell, an initial '.' in a filename does match a wildcard at the start of a pattern, and a '/' in a filename can match wildcards. If no patterns are given, all files are extracted. See Options.

     cpio {-i|--extract} [-bcdfmnrtsuvBSV] [-C bytes] [-E file]
     [-H format] [-M message] [-R [user][:.][group]]
     [-I [[user@]host:]archive] [-F [[user@]host:]archive]
     [--file=[[user@]host:]archive] [--make-directories]
     [--nonmatching] [--preserve-modification-time]
     [--numeric-uid-gid] [--rename] [--list] [--swap-bytes] [--swap]
     [--dot] [--unconditional] [--verbose] [--block-size=blocks]
     [--swap-halfwords] [--io-size=bytes] [--pattern-file=file]
     [--format=format] [--owner=[user][:.][group]]
     [--no-preserve-owner] [--message=message] [--help] [--version]
     [--no-absolute-filenames] [--sparse] [-only-verify-crc] [--to-stdout]
     [-quiet] [--rsh-command=command] [pattern...] [< archive]

3.3 Copy-pass mode

In copy-pass mode, cpio copies files from one directory tree to another, combining the copy-out and copy-in steps without actually using an archive. It reads the list of files to copy from the standard input; the directory into which it will copy them is given as a non-option argument. See Options.

     cpio {-p|--pass-through} [-0adlmuvLV] [-R [user][:.][group]]
     [--null] [--reset-access-time] [--make-directories] [--link]
     [--preserve-modification-time] [--unconditional] [--verbose]
     [--dot] [--dereference] [--owner=[user][:.][group]] [--sparse]
     [--no-preserve-owner] [--help] [--version] destination-directory
     < name-list

3.4 Options

-0

--null

Read a list of filenames terminated by a null character, instead of a newline, so that files whose names contain newlines can be archived. GNU find is one way to produce a list of null-terminated filenames. This option may be used in copy-out and copy-pass modes.

-a

--reset-access-time

Reset the access times of files after reading them, so that it does not look like they have just been read.

-A

--append

Append to an existing archive. Only works in copy-out mode. The archive must be a disk file specified with the -O or -F (--file) option.

-b

--swap

Swap both halfwords of words and bytes of halfwords in the data. Equivalent to -sS. This option may be used in copy-in mode. Use this option to convert 32-bit integers between big-endian and little-endian machines.

-B

Set the I/O block size to 5120 bytes. Initially the block size is 512 bytes.

--block-size=block-size

Set the I/O block size to block-size * 512 bytes.

-c

Use the old portable (ASCII) archive format.

-C io-size

--io-size=io-size

Set the I/O block size to io-size bytes.

-d

--make-directories

Create leading directories where needed.

-E file

--pattern-file=file

Read additional patterns specifying filenames to extract or list from file. The lines of file are treated as if they had been non-option arguments to cpio. This option is used in copy-in mode,

-f

--nonmatching

Only copy files that do not match any of the given patterns.

-F archive

--file=archive

Archive filename to use instead of standard input or output. To use a tape drive on another machine as the archive, use a filename that starts with 'hostname:', where hostname is the name or IP address of the machine. The hostname can be preceded by a username and an '@' to access the remote tape drive as that user, if you have permission to do so (typically an entry in that user's ~/.rhosts file).

--force-local

With -F, -I, or -O, take the archive file name to be a local file even if it contains a colon, which would ordinarily indicate a remote host name.

-H format

--format=format

Use archive format format. The valid formats are listed below; the same names are also recognized in all-caps. The default in copy-in mode is to automatically detect the archive format, and in copy-out mode is 'bin'.

'bin'

The obsolete binary format.

'odc'

The old (POSIX.1) portable format.

'newc'

The new (SVR4) portable format, which supports file systems having more than 65536 i-nodes.

'crc'

The new (SVR4) portable format with a checksum added.

'tar'

The old tar format.

'ustar'

The POSIX.1 tar format. Also recognizes GNU tar archives, which are similar but not identical.

'hpbin'

The obsolete binary format used by HPUX's cpio (which stores device files differently).

'hpodc'

The portable format used by HPUX's cpio (which stores device files differently).

-i

--extract

Run in copy-in mode. See Copy-in mode.

-I archive

Archive filename to use instead of standard input. To use a tape drive on another machine as the archive, use a filename that starts with 'hostname:', where hostname is the name or IP address of the remote host. The hostname can be preceded by a username and an '@' to access the remote tape drive as that user, if you have permission to do so (typically an entry in that user's ~/.rhosts file).

-k

Ignored; for compatibility with other versions of cpio.

-l

--link

Link files instead of copying them, when possible.

-L

--dereference

Copy the file that a symbolic link points to, rather than the symbolic link itself.

-m

--preserve-modification-time

Retain previous file modification times when creating files.

-M message

--message=message

Print message when the end of a volume of the backup media (such as a tape or a floppy disk) is reached, to prompt the user to insert a new volume. If message contains the string '%d', it is replaced by the current volume number (starting at 1).

-n

--numeric-uid-gid

Show numeric UID and GID instead of translating them into names when using the --verbose option.

--no-absolute-filenames

Create all files relative to the current directory in copy-in mode, even if they have an absolute file name in the archive.

--no-preserve-owner

Do not change the ownership of the files; leave them owned by the user extracting them. This is the default for non-root users, so that users on System V don't inadvertantly give away files. This option can be used in copy-in mode and copy-pass mode

-o

--create

Run in copy-out mode. See Copy-out mode.

-O archive

Archive filename to use instead of standard output. To use a tape drive on another machine as the archive, use a filename that starts with 'hostname:', where hostname is the name or IP address of the machine. The hostname can be preceded by a username and an '@' to access the remote tape drive as that user, if you have permission to do so (typically an entry in that user's ~/.rhosts file).

--only-verify-crc

Verify the CRC's of each file in the archive, when reading a CRC format archive. Don't actually extract the files.

-p

--pass-through

Run in copy-pass mode. See Copy-pass mode.

--quiet

Do not print the number of blocks copied.

-r

--rename

Interactively rename files.

-R owner

--owner owner

In copy-in and copy-pass mode, set the ownership of all files created to the specified owner (this operation is allowed only for the super-user). In copy-out mode, store the supplied owner information in the archive.

The argument can be either the user name or the user name and group name, separated by a dot or a colon, or the group name, preceeded by a dot or a colon, as shown in the examples below:

          cpio --owner smith
          cpio --owner smith:
          cpio --owner smith:users
          cpio --owner :users

If the group is omitted but the ':' or '.' separator is given, as in the second example. the given user's login group will be used.

--rsh-command=command

Notifies cpio that is should use command to communicate with remote devices.

-s

--swap-bytes

Swap the bytes of each halfword (pair of bytes) in the files. This option can be used in copy-in mode.

-S

--swap-halfwords

Swap the halfwords of each word (4 bytes) in the files. This option may be used in copy-in mode.

--sparse

Write files with large blocks of zeros as sparse files. This option is used in copy-in and copy-pass modes.

-t

--list

Print a table of contents of the input.

--to-stdout

Extract files to standard output. This option may be used in copy-in mode.

-u

--unconditional

Replace all files, without asking whether to replace existing newer files with older files.

-v

--verbose

List the files processed, or with -t, give an 'ls -l' style table of contents listing. In a verbose table of contents of a ustar archive, user and group names in the archive that do not exist on the local system are replaced by the names that correspond locally to the numeric UID and GID stored in the archive.

-V

--dot

Print a '.' for each file processed.

--version

Print the cpio program version number and exit.

4 Magnetic Media

Archives are usually written on removable media–tape cartridges, mag tapes, or floppy disks.

The amount of data a tape or disk holds depends not only on its size, but also on how it is formatted. A 2400 foot long reel of mag tape holds 40 megabytes of data when formated at 1600 bits per inch. The physically smaller EXABYTE tape cartridge holds 2.3 gigabytes.

Magnetic media are re-usable–once the archive on a tape is no longer needed, the archive can be erased and the tape or disk used over. Media quality does deteriorate with use, however. Most tapes or disks should be disgarded when they begin to produce data errors.

Magnetic media are written and erased using magnetic fields, and should be protected from such fields to avoid damage to stored data. Sticking a floppy disk to a filing cabinet using a magnet is probably not a good idea.

---

Next: Concept Index, Previous: Media, Up: Top

5 Reporting bugs or suggestions

It is possible you will encounter a bug in cpio. If this happens, we would like to hear about it. As the purpose of bug reporting is to improve software, please be sure to include maximum information when reporting a bug. The information needed is:

• Version of the package you are using.
• Compilation options used when configuring the package.
• Conditions under which the bug appears.

Send your report to <[email protected]>. Allow us a couple of days to answer.

Concept Index

• command line options: Invoking cpio
• copying directory structures: Tutorial
• creating a cpio archive: Tutorial
• extracting a cpio archive: Tutorial
• invoking cpio: Invoking cpio
• magnetic media: Media
• passing directory structures: Tutorial

Table of Contents

• 1 Introduction
• 2 Tutorial
• 3 Invoking cpio
  • 3.1 Copy-out mode
  • 3.2 Copy-in mode
  • 3.3 Copy-pass mode
  • 3.4 Options
• 4 Magnetic Media
• 5 Reporting bugs or suggestions
• Concept Index

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019