JeremyKatz JackNeely jjneely@pams.ncsu.edu 1.1 23 April 2001 jlk 1.2 09 September 2002 jjn Updated to reflect current process of updating the Realm Kit trees. Also began integration into a larger document.

The NCSU Realm Kit for Red Hat Linux cannot stay in an unchanging state due to the constant discovery of security problems and usability problems. These problems must be corrected by the addition of errata to the Realm Kit tree. This document describes the necessary steps to incorporate errata into the Linux Realm Kit trees. Regular system updates are vital to preventing security problems with any operating system. Mechanisms exist for machines running the NCSU Realm Kit to automatically update themselves every night. Unfortunately, this relies on the maintainer of the tree to first make the update available in the tree for retrieval. When you complete the steps described below, updates which you have added to the tree will be available for all machines running the NCSU Realm Kit for Red Hat Linux. This document assumes that you have some familiarity with the NCSU Realm Kit for Red Hat Linux. Some familiarity with bash scripting is helpful in reducing the amount of typing which must be done, yet is not a prerequisite for being able to update the tree.

Use Red Hat's errata advisories to keep up to date with what errata are available and what packages they affect. The best way I have found to do this is to keep tabs on the Red Hat Errata web page. Login to the build machine, either via ssh or on the console. As of this writing the host name is rk-devel.pams.ncsu.edu. First, check that the package is not one which has been modified with patches specific to campus. (If this is the case, the current package in the Realm Kit tree should have "ncsu" as part of the release number.) If the package has been modified, then you will need to update the package to contain the NCSU customizations. This is beyond the scope of this document. Also, its a good idea to install this package on a test machine and make sure it works and is sane. Good places for getting information about errata from Red Hat are the Red Hat mailing lists and by simply asking others that may have already looked at the package in question. Run sudo bash to become root. pams% sudo bash bash-2.05a# Or gain root privileges by using the ksu command or some other method of becoming root. Download the binary packages from the local mirror such as ftp://kickstart.linux.ncsu.edu/pub/redhat/linux/updates/ for the errata in question. Specific package names can be found in the Red Hat errata advisory. It can be more convenient to NFS mount this directory on the build machine by doing the following. bash-2.05a# mount -t nfs kickstart.linux.ncsu.edu:/export/redhat-updates /mnt/net After the above you should have all the packages that make up all the errata you would like to apply to the Realm Kit trees. You should know what packages apply to what versions of the tree (from the errata advisory), if the package needs have any NCSU customizations done, and that the package is relatively safe to apply.

At this point we are ready to actually apply the errata to the Realm Kit trees. This has gotten very easy with the addrpm.py script. This script is present in the realmkit-maintenance package version 0.11.1 and higher. The addrpm.py script does all the hard work for you for the most common case. It will find the packages that you are replacing, move them out of the way, drop in the new packages, and create symlinks as appropriate. To do this the program does need a bit of information from the user. The following is the help text displayed by addrpm.py. Usage:/usr/bin/addrpm.py -e dir -u dir -v ver -r arch [listofrpms] Required arguments: -v, --ver [version of distro] -r, --arch [architecture of distro] -e, --tree [where the top of the tree is] -u, --updatedir [subdir for updates ex: other-pkgs/updates Optional arguments: -t, --testmode Run in test mode -a, --addnew Add rpms to tree if they are not already there -d, --debug Run in debug mode -h, --help Print this message The end user does need to carefully understand most of the options here. The -e defines the directory that contains all the versions of the tree. The -v defines the version of the tree to apply the updates to. You must also specify the architecture of the tree with the -r option. All three of these arguments are concatenated together to form a path to a Red Hat install tree. (The directory that contains the RedHat/ directory.) The last required option is -u which specifies a subdirectory of the install tree to put updates. (The updates will be symlinked into the install tree.) The rest of the options are not required but can be very useful. If you need to add an RPM into the tree that did not previously exist you need to specify -a otherwise you will receive a warning and the RPM will not be added. To run in test mode use -t. This mode will not alter the tree in any way put will tell you exactly what it would have done. Debug mode is turned on with -d and will print out lots of extra information about what is happening. Lets look at a typical example. Red Hat released a Bind errata in August, 2002. From which you see several RPMs all starting with bind-, then followed by a sub-package name, then followed by the exact same version and release number. Here are the new packages for Red Hat Linux 7.3 on a i386: bind-9.2.1-1.7x.2.i386.rpm bind-devel-9.2.1-1.7x.2.i386.rpm bind-utils-9.2.1-1.7x.2.i386.rpm We take the location of the Realm Kit 7.3 tree from where it actually is on the current build server, that is /dist/7.3. The updates directory for the Realm Kit is always the relative path updates/. So, to apply this update you could use the following command. addrpm.py -u updates -e /dist -r i386 -v 7.3 bind-*9.2.1-1.7x.2*.rpm The addrpm.py will take a few moments to collect information about the existing packages and do a bit of error checking to make sure that these packages are appropriate. Finally it will print out some messages from RPM while it checks the MD5 checksums and GPG signatures. The packages are now applied to the install tree. Please check the outcome of the MD5 and GPG checks. Now that we have looked at the addrpm.py script let's go over the steps necessary to apply errata to an install tree. Obtain the errata as discussed in the previous section. Use addrpm.py to apply the packages to a specific tree. I normally use one run of the program per version I am updating per errata. Check to make sure the MD5 checksums and GPG signatures are okay. That's it. The errata are now applied. Next you need to rebuild the meta-information about the install tree. This is covered in the next section.

The meta-information in the Realm Kit trees does two things. First it includes the header information in hdlist files that the Red Hat installer, Anaconda needs to properly install the system. Secondly, it rebuilds the meta files for the tool(s) that apply RPM updates to every Realm Kit machine. This task is handled by a collection of scripts in the realmkit-maintenance package. Use the following instructions for rebuilding the meta-information. Run /usr/lib/rkmaint/meta-update.py <version> to update the meta-information for the specified version. (The scripts assume that the root of all trees is /dist.) bash-2.05a# /usr/lib/rkmaint/meta-update.py 7.0 Repeat the above for all affected versions. Now we have the errata applied to the trees and the meta-information has been updated. Now we are ready to test our updated trees.

One of the most important parts of any update is actually testing the update. Although updates released by Red Hat, Inc. have passed through their Quality Assurance Department, there is still value in at least a minimal smoke test of the package to ensure that nothing was missed. Install the update locally and ensure that programs affected by the update still work as expected after the upgrade. The degree of testing here should be relative to the importance of the package in our environment. For example, testing for packages not installed by default could be a simple check to ensure that the programs run. Test an update of a current machine using /usr/sbin/yup update to ensure that the automated update mechanism has no difficulties installing the package. To facilitate this you can point the RPM updating program (currently Yup) to ftp://rk-devel.pams.ncsu.edu/ <version>. For example, to reconfigure Yup you might replace or edit /etc/yup.conf to look like: .master http://www.linux.ncsu.edu/realmkit/yup.master.txt .dist-option NCSU Realm Kit for Red Hat Linux 7.3 .arch-option i386 .begin mirror-option .name Official Realm Kit Distribution Source .country USA .addr ftp://rk-devel.pams.ncsu.edu/7.3 .arch i386 .end .logfile /var/log/yup.log .checksig yes This example was taking from a Linux Realm Kit 7.3 machine. Run a test kickstart-based reinstall to ensure that the installer doesn't have any problems with the updated package. A manual install will also work here but does take more time. After we have tested the trees to make sure they are production quality, we are ready to do the proper notification and to push the new trees out to all the clients,

It is important, at a bear minimum, to inform the community what changes are about to happen to the Realm Kit trees. There are some people out there that take the base kit and do even more modifications. Also, this keeps the IT folk informed about what security updates have and have not taken place. This step is pretty closely related to pushing the trees to the public servers so I will include information about that here too. The template for the email notification has recently be changed to make them much more easier to handle, especially when you are updating multiple errata at a time. You will need a description, the errata type (such as "Security Advisory" etc.), affected versions or the Realm Kit, and the URL to Red Hat's errata. All of this information can be gotten from Red Hat's errata page. Also, a shorter message should be posted to SysNews containing a short description of the changes to the trees. Its good practice to send out the errata announcements before the new trees are pushed to the public servers. Normally, I try to wait at least 24 hours between sending out the updates and when the public servers will get the new trees. Below is a step by step method for notification and pushing the new trees. Fill out the errata template located at /dist/new-advisory with the current date, descriptions, affected versions, and URLs. Email completed errata advisory to realmlinux-announce@lists.ncsu.edu. Posting to this list requires moderator approval. Hopefully, if you are doing updates you'll not have a problem with this. Make a posting on SysNews under the Linux organization. I normally list the errata and versions and give a pointer to the archives of the realmlinux-announce mailing list. Wait at least 24 hours. Push the trees from the build machine to the main repository for the NCSU Realm Kit for Red Hat Linux. This can be done by SSH'ing into the repository master.linux.ncsu.edu . Use rsync to sync the trees from the build machine to their proper place in the /export/updates directory. Using the current configuration, as of this writing, if the trees are synced before 11:00 PM then they should propagate to the public servers that night. A few notes about the main repository server. It actually holds two copies of all the trees. The /export/updates is a staging area to upload updates to. At 11:00 PM that area is synced to /export/trees which is the public space that is shared out via rsync and used by other servers (such as kickstart.linux.ncsu.edu to sync their copies which, in turn, is the server that all clients get their updates from. Well, that's all folks. After following these instructions you should now have errata being installed automatically on clients and completely up to date install trees that you can use to install. Good luck!