Burning a RedHat CD mini-HOWTO Morten Kjeldgaard, mok@imsb.au.dk and Peter von der Ahé, pahe+rhcd@daimi.au.dk v1.18, 27 December 1998 This document describes how to make your own CDs from the Red Hat Linux distribution equivalent to the ones commercially available from Red Hat. The structure of the distribution is described, as well as the procedure needed to include updated RPMS into the distribution. Prerequisites are a good network connection, and a CD-writer. ______________________________________________________________________ Table of Contents 1. Introduction 2. Anatomy of the Red Hat FTP site 2.1 The top level directory 2.2 The "RedHat" directory -- the core of the distribution 3. RPM packages 4. Obtaining your local copy of the distribution 5. Incorporating the updates 5.1 Correcting the file protection modes 5.2 Replacing the updated RPMS 5.3 Generating a new hdlist file 5.3.1 Important note for RedHat 5.2 6. At last: burning the CD 6.1 Creating a bootable CD 6.1.1 Create disk image 6.1.2 Test the image 6.1.3 Burn the disk 7. Installing from the CD 8. This document... 8.1 Related documentation 8.2 Acknowledgements 9. DISCLAIMER ______________________________________________________________________ 1. Introduction There may be several reasons for making your own CD. Perhaps you're a cheapskate and want to save the $50 cost of the Red Hat distribution . Or, perhaps you want a distribution CD containing the latest distribution with all current updates. This is highly relevant, because after each major release of the Red Hat distribution, there have been loads of updates, several of which are security related. Just take a look at the updates/00README.errata file. A specific errata sheet also exists for each supported platform. See for example the Intel errata sheet 2. Anatomy of the Red Hat FTP site In the spirit of the Linux community, Red Hat Software has made available their Linux distributions for several platforms on their FTP site. These are all available from the top distribution directory. 2.1. The top level directory The toplevel directory for RedHat Linux release 5.1 (pub/redhat/redhat-5.1 ) contains distributions for the different platforms, and a directory containing updates and corrections to program packages that have been published since the release. SRPMS/ alpha/ i386/ sparc/ updates/ In this document, we use the i386 distribution as an example. The procedure given in this document is likely to work on all platforms supported by Red Hat (Alpha, SPARC, ppc, etc.), but we have only tested it on the i386 platform (the authors would be most interested in additional information). The root of the i386 directory looks like this: -rw-r--r-- 8 ftpuser ftpusers 19686 May 27 1997 COPYING -rw-r--r-- 1 ftpuser ftpusers 3023 May 7 09:58 README -rw-r--r-- 10 ftpuser ftpusers 2751 Sep 18 1997 RPM-PGP-KEY drwxr-xr-x 5 ftpuser ftpusers 96 Jul 15 08:34 RedHat/ drwxr-xr-x 5 ftpuser ftpusers 8192 Jul 15 08:35 doc/ drwxr-xr-x 5 ftpuser ftpusers 8192 Jul 15 08:35 dosutils/ drwxr-xr-x 5 ftpuser ftpusers 8192 Jul 15 08:33 gnome/ drwxr-xr-x 2 ftpuser ftpusers 96 Jun 7 02:47 images/ drwxr-xr-x 4 ftpuser ftpusers 96 Jun 5 12:24 misc/ The doc directory contains an abundance of information. Most importantly, the RedHat installation manual can be found in HTML format in the directory doc/rhmanual/manual/ . Next, there is a number of FAQs, and finally, the entire collection of HOWTOs and mini-HOWTOs. The images directory contains boot floppy images that must be copied to a diskette. In the most recent distribution (5.1), there are two disk images available. The boot image is called boot.img, which is required when installation is performed directly from a CD-ROM. If installation from a local hard disk, NFS mounted disk or FTP is required, the supplementary disk image supp.img might be needed. See section ``Installing from the CD'' and references therein for details. The misc directory contains source and executables of a number of programs needed for the installation. 2.2. The "RedHat" directory -- the core of the distribution The most important part of the directory tree is rooted in the RedHat directory: drwxr-xr-x 2 ftpuser ftpusers 24576 Jul 15 08:35 RPMS/ drwxr-xr-x 2 ftpuser ftpusers 8192 Jul 15 08:32 base/ -rw-rw-rw- 59 ftpuser ftpusers 0 Aug 15 14:21 i386 drwxr-xr-x 4 ftpuser ftpusers 96 Jun 5 12:24 instimage/ The RPMS directory contains the major part of the Red Hat distribution consisting of a set of RPM (Redhat Package Manager) files. An RPM package typically contains binary executables, along with relevant configuration files and documentation. See the section ``RPM packages'' for more information. The base directory holds different book-keeping files needed during the installation process, e.g. the comps file, which defines the components (groups of packages) used during the "Choose packages to install" phase. Another important file in the base directory is the hdlist file containing most of the header fields from all the RPMs in the RPMS directory. This means that all the interdependencies among RPM packages can be determined just by reading hdlist without having to read all the RPM packages which is quite convenient especially during FTP installs. Another use of hdlist is mapping package names to file name, eg. perl to perl-5.004-6.i386.rpm. This means that if you want to incorporate updates from RedHat (see section ``Incorporating the updates'') or add your own packages to the RPMS directory, you need to update hdlist. This is descriped later in ``Generating a new hdlist file''. The instimage directory contains a bare-bones live file system with a number of programs and shared libraries needed during the installation procedure. 3. RPM packages The major part of the Red Hat distribution consists of a set of RPM (Redhat Package Manager) files. An RPM package typically contains binary executables, along with relevant configuration files and documentation. The rpm program is a powerful package manager, which can be used to install, query, verify, update, erase and build software packages in the RPM format. Rpm convieniently maintains a database of all the software packages it has installed, so information on the installed software is available at any time. The binary RPM files in the distribution have been built on a system running the distribution itself. This is important, because most of the programs in the packages rely on shared libraries. From RedHat version 5.0, the new version 2 of the GNU standard C library (which is 64-bit clean) has been used. This version of the library is commonly referred to as glibc or in Linux: libc 6. All executables in the distribution have been linked against this library. If you attempt to install binary files from a different distribution, chances are that they will not work, unless you install the libc5 package for backwards compability. The names of the RPM packages contain the suffix .arch.rpm, where arch is the architechture, having the value i386 for Intel platform binaries. The packages you install must match the versions of the shared libraries available on the machine. The rpm program is usually quite good at ensuring that this is indeed the case, however, there are ways around this check, and you should be sure that you know what you are doing if you force installation of packages this way. However, using the RedHat installation boot disk, it is ensured that the correct set of RPM packages are installed on the machine. If you discover an RPM package that was not installed on your system during the installation process, don't despair. At any time, you may (as root) install RPM packages, for example: rpm --install WindowMaker-0.18-1b.i386.rpm You can even install directly from the Internet, if you know the URL of an RPM package: rpm --install ftp://rufus.w3.org/redhat-contrib/noarch/mirror-2.9-2.noarch.rpm Another version of the RPM packages contain the original sources used to build the binaries. These packages have the suffix .src.rpm and are situated in the SRPMS directory. These packages are not needed on the installation CD, and in fact, there is not even enough disk space on an 74 minute burnable CD to accomodate them. Of course, you can make a separate CD with the SRPMS. 4. Obtaining your local copy of the distribution You need a copy of the distribution on a writable disk which is accessible from the computer having the CD writer (duh!). If you want to incorporate the latest updates, this directory should (also) be accessible from from a Linux machine, either from a local disk, an NFS mounted disk on a different computer, or a JAZ disk. You could copy the distribution from a RedHat CD, or you could get it via FTP. If you choose to use FTP, the best way to get a correct copy of the distribution is to use the mirror package. Mirror is a sophisticated perl script that compares the content of a directory on a remote site with a local directory. It will use FTP to fetch the files that are on the remote site but not the local site, and delete files on the local site that are not on the remote site. The mirror program is configured with a configuration file. The mirror package is available as an RPM from rufus.w3.org . Make your local copy mirror.redhat of the mirror configuration file, and edit the relevant fields at the top of the file. After the default section, define these packages: package=updates site=ftp.sunsite.auc.dk exclude_patt=(alpha/|sparc/) remote_dir=/disk1/ftp.redhat.com/pub/redhat/redhat-5.1/updates local_dir=/jaz/redhat-5.1/updates package=dist site=ftp.sunsite.auc.dk exclude_patt=(alpha/|sparc/) remote_dir=/disk1/ftp.redhat.com/pub/redhat/redhat-5.1/i386 local_dir=/jaz/redhat-5.1/i386 The following command will download a copy of the entire RedHat tree on your local disk. *Think* before you do this, you are about to transfer approximately 350Mb of data. mirror -pdist mirror.redhat This will mirror the Red Hat FTP site on your local disk. The content of a Red Hat distribution does not change between releases, so you only need to download this package ONCE. All changes to the distribution are in the updates directory. Thus, if you want to keep an up-to-date mirror of the Red Hat distribution, you only need to keep the updates directory current. This is done using the command mirror -pupdates mirror.redhat You can run this regularly, say, once a week, through a cron script. The RedHat distribution is available on a great number of FTP servers around the world, which are updated daily from the master site at . You should choose an FTP site close to you, see the RedHat FAQ 5. Incorporating the updates To incorporate the updates, you need write access to the distribution directory from a Linux machine, with a working version of rpm installed. There are three steps involved: 1. Correct the file protection modes. 2. Replace updated RPMs. 3. Generate the hdlist file If you maintain a mirror of the updates directory, you can at any time produce a CD including the current updates by repeating these steps. 5.1. Correcting the file protection modes During the installation process, some programs are run directly off the CD. Unfortunately, the FTP program does not always preserve the protection modes of the files and directories that are copied. Therefore, it is necessary to make sure that execute permission is given to programs, shell scripts and shared libraries, before the directory is burned on the CD. This is done by running the updatePerm script on your local copy of the distribution: ______________________________________________________________________ #!/bin/bash RHVERSION=5.1 LIST=/tmp/er3hd3w25 CDDIR=/jaz/redhat-${RHVERSION} # Find all directories, and make sure they have +x permission find $CDDIR -type d -exec chmod -c 755 {} \; # Find all files that are executables, shell or perl scripts find $CDDIR -type f | file -f - | grep -v RPM \ | egrep -i 'executable|perl|bourne|shell' | cut -f1 -d: > $LIST # Find shared libraries find $CDDIR -name \*.so >> $LIST # Make them executable while read file do if [ ! -x $file ] ; then chmod -c 755 $file fi done < $LIST /bin/rm $LIST exit 0 ______________________________________________________________________ 5.2. Replacing the updated RPMS The following script called updateCD copies all files from the update directory to the RPMS directory. The script uses some nifty rpm tricks to determine what packages in the updates directory are more recent. Older packages are moved to the ${OLD} directory. ______________________________________________________________________ #! /bin/bash # This script updates rpms in a RedHat distribution found in $RPMDIR. # The old rpms will be placed in $OLDDIR. # The new rpms should be located in $UPDDIR. # The architechture is $ARCH. RHVERSION=5.1 ARCH=i386 CDDIR=/jaz/redhat-${RHVERSION} RPMDIR=${CDDIR}/${ARCH}/RedHat/RPMS UPDDIR=${CDDIR}/updates/${ARCH} OLDDIR=${CDDIR}/old if [ ! -d $OLDDIR ] ; then echo making directory $OLDDIR mkdir $OLDDIR fi allow_null_glob_expansion=1 for rpm in ${UPDDIR}/*.rpm ; do NAME=`rpm --queryformat "%{NAME}" -qp $rpm` unset OLDNAME for oldrpm in ${RPMDIR}/${NAME}*.rpm ; do if [ `rpm --queryformat "%{NAME}" -qp $oldrpm` = "$NAME" ]; then OLDNAME=$oldrpm; break fi done if [ -z "$OLDNAME" ]; then echo $NAME is new cp -pv $rpm $RPMDIR else if [ `basename $rpm` != `basename $OLDNAME` ]; then mv $OLDNAME $OLDDIR cp -pv $rpm $RPMDIR fi fi done # Copy new boot image files to the right place... for newfile in ${UPDDIR}/images/* ; do file=${CDDIR}$/${ARCH}/images/$(basename ${newfile}) if [ $newfile -nt $file ] ; then cp -pv $newfile $file fi done exit 0 ______________________________________________________________________ 5.3. Generating a new hdlist file When installing from the CD, the installation program on the CD relies on the file RedHat/base/hdlist describing what RPM packages are available on the CD. The hdlist file can be generated by the program misc/src/install/genhdlist. This program must be run with the root name of the distribution as the only argument. Here is the updateHdlist script which calls that program: ______________________________________________________________________ #!/bin/bash RHVERSION=5.1 ARCH=i386 echo generating hdlist... CDDIR=/jaz/redhat-${RHVERSION} GENHDDIR=${CDDIR}/${ARCH}/misc/src/install chmod u+x ${GENHDDIR}/genhdlist chmod 644 ${CDDIR}/${ARCH}/RedHat/base/hdlist ${GENHDDIR}/genhdlist ${CDDIR}/${ARCH} || echo "*** GENHDLIST FAILED ***" exit 0 ______________________________________________________________________ NOTE: After having incorporated the updates in the main RedHat/RPMS directory, your copy of the distribution is no longer a mirror of the Red Hat distribution site. Actually, it is more up-to-date! Therefore, if you attempt to mirror the distribution, older versions of the RPM's that have been updated will be downloaded once more, and the updates deleted. 5.3.1. Important note for RedHat 5.2 As distributed with RedHat version 5.2 and earlier, genhdlist CRASHES if there are files in the RedHat/RPMS directory which are not RPM files! This causes problems, because in the 5.2 distribution, there are a couple of non-RPM files named ls-lR and ls-lR.gz in RedHat/RPMS. Therefore, you must remove all non-RPM files from the directory. Alternatively, you can apply the following patch to misc/src/install/genhdlist.c and do a make. The patch will cause genhdlist to ignore any non-RPM files. *** genhdlist.c.orig Fri Nov 27 12:08:13 1998 --- genhdlist.c Fri Nov 27 12:08:20 1998 *************** *** 12,23 **** --- 12,26 ---- #define FILENAME_TAG 1000000 + /* Not used apparently... + int tags[] = { RPMTAG_NAME, RPMTAG_VERSION, RPMTAG_RELEASE, RPMTAG_SERIAL, RPMTAG_FILENAMES, RPMTAG_FILESIZES, RPMTAG_GROUP, RPMTAG_REQUIREFLAGS, RPMTAG_REQUIRENAME, RPMTAG_REQUIREVERSION, RPMTAG_DESCRIPTION, RPMTAG_SUMMARY, RPMTAG_PROVIDES, RPMTAG_SIZE, RPMTAG_OBSOLETES }; int numTags = sizeof(tags) / sizeof(int); + */ int main(int argc, char ** argv) { char buf[300]; *************** *** 26,34 **** --- 29,39 ---- struct dirent * ent; int fd, rc, isSource; Header h; + /* not used int count, type; int i; void * ptr; + */ if (argc != 2) { fprintf(stderr, "usage: genhdlist \n"); *************** *** 74,79 **** --- 79,85 ---- rc = rpmReadPackageHeader(fd, , , NULL, NULL); + if (!rc) { headerRemoveEntry(h, RPMTAG_POSTIN); headerRemoveEntry(h, RPMTAG_POSTUN); headerRemoveEntry(h, RPMTAG_PREIN); *************** *** 110,115 **** --- 116,122 ---- headerWrite(outfd, h, HEADER_MAGIC_YES); headerFree(h); close(fd); + } } errno = 0; 6. At last: burning the CD As we assume that you have a working CD-writer on your system, and you know how to use it, we wont go into much detail about burning the CD. If you are burning your CD on a Linux system, you can install the excellent XCDroast package. In XCDRoast, switch on the Rock Ridge extensions, and the creation of a TRANS.TBL file. Make sure the top directory of the CD contains at least the following files and directories: COPYING RPM-PGP-KEY README RedHat/ The following directories might come in handy: doc/ gnome/ misc/ dosutils/ images/ 6.1. Creating a bootable CD (This section, thanks to Dawn Endico dawn@math.wayne.edu). Since XCDroast doesn't support creation of bootable disks you'll need to use other tools, for example mkisofs and cdrecord . Get the latest RPMs for these packages from . You'll need to create an image file which will be written to the CD. This file will be 500Mb or more so find a partition with enough free space and change the path for redhat.img in the following commands if necessary. You may need to be root to use mount and cdrecord. 6.1.1. Create disk image Change directory to the place in your mirror that will be the root directory of the cd. For instance, redhat-5.2/i386. mkisofs -v -r -T -J -V "Red Hat 5.2" -b images/boot.img -c misc/boot/boot.cat -o /tmp/redhat.img . 6.1.2. Test the image If you're paranoid you can test your new disk image by mounting it. If you forgot to fix the file permissions or set the rock ridge extensions then the error will be obvious here since the file names and directory structure will be wrong. mount -t iso9660 -o ro,loop=/dev/loop0 /tmp/redhat.img /mnt/cdrom When you're done, don't forget to unmount it. umount /mnt/cdrom 6.1.3. Burn the disk Be sure to set the correct speed for your device. This command is for a 4X CDR. You may have a 1X or 2X drive. cdrecord -v speed=4 dev=0,0 /tmp/redhat.img 7. Installing from the CD When installing from the new CD, you first need to create a bootable installation diskette. IMPORTANT: use a NEW, freshly MS-DOS formatted diskette! Using an old, worn-out, faulty diskette can result in strange problems during the installation! On a Linux system, you can create the diskette using the dd command: dd if=/mnt/cdrom/images/boot.img of=/dev/fd0 bs=1440k On a system running DOS or Windows-9x, you need to use the RAWRITE.EXE program, which is found on the CD in the dosutils directory. Shut down the machine you want to install (or do a system upgrade) on, insert the boot diskette and your freshly burned CD, and let the machine boot from the diskette. For more information on the installation process, se the documents and the Installation-HOWTO or the Bootdisk-HOWTO which are on the CD in the doc/HOWTO directory. 8. This document... The SGML source of the most recent version of this document can be retrieved from . 8.1. Related documentation Ed Schlunder has written a utility called fix-rhcd to let you check your Red Hat Linux distribution mirror for matching file sizes, names, permissions, and symlinks against an "ls -lNR" listing from the offical Red Hat ftp site. Any permissions that are wrong are changed to match the ls listing. See the fix-rhcd homepage . 8.2. Acknowledgements Thanks to the following people for valuable input: · Lars Christensen · Thomas Duffy · Dawn Endico · Seva 9. DISCLAIMER While the given information in this document is believed to be correct, the authors assume no responsibility whatsoever for any damage to hardware and/or software, or any loss of data resulting from the procedures outlined in this document.