Verification of installed files on a system is an important part of administration. Thankfully, we have tools like tripwire to make that task much easier. These tools allow us to determine which, if any, files have been altered on a system. These tools require that they have been setup to monitor files before the need to verify them comes up though. What can administrators do to find this information out if they were unable to setup something like Tripwire in advance ?
Fortunately, many package managers allow you to verify the files installed on a system using built in MD5 checksums created during the creation of the package. In this article, we'll discuss using both RedHat's RPM and Gentoo's Portage to verify installed software.
The RedHat Package Manager comes with many Linux distributions. It's a very powerful package manager allowing you to install both binary and source packages. As part of its installation process, RPM updates a local database with the names of what packages have been installed. Along with package names, RPM also stores MD5 checksums, permissions, time, size and other information for every file included in the package. To query what packages have been installed on a system, we will need to use the following flags with the rpm command.
rpm -qa
This command tells RPM to query all installed packages. Its output will look similar to the following:
# rpm -qa hdparm-5.4-1 words-2-21 gawk-3.1.1-9 gpm-1.19.3-27.2 pyxf86config-0.3.5-1 pam_smb-1.1.7-1 procmail-3.22-9 setarch-1.3-1 freetype-2.1.4-4.0 groff-1.18.1-27 irda-utils-0.9.15-1 pyOpenSSL-0.5.1-8 portmap-4.0-56 perl-URI-1.21-7 perl-XML-Twig-3.09-3 alchemist-1.0.27-1 libIDL-0.8.0-9 libgnomecanvas-2.2.0.2-2 redhat-config-rootpassword-1.0.6-1 --cut--
To verify an installed package, we can make use of the -V flag to rpm along with the name of a package as an argument.
rpm -V package name
This command will only produce output if the stored checksums for the package do not match the checksums of the files installed to the disk. For example
# rpm -V gawk-3.1.1-9 # rpm -V pam_smb-1.1.7-1 .......T c /etc/pam_smb.conf
Here we see that the installed files for gawk have not changed from what is included in the package itself. One of the files for pam_smb has been altered though. From the output, we see that the file /etc/pam_smb.conf has been altered, along with a "T" and "c". This additional information explains specifically what the file is, and also what exactly has changed about it . The "c" indicates that this is a configuration file. The "T" indicates that the modification time has changed for the file. The following are the possible results.
S file Size differs
M Mode differs (includes permissions and file type)
5 MD5 sum differs
D Device major/minor number mis-match
L readLink(2) path mis-match
U User ownership differs
G Group ownership differs
T mTime differs
In this example only a configuration file has changed. It is very common to see this since once installed, many packages need to be configured accordingly. If a binary file has been changed, the output would look similar to the following:
# rpm -V cpio-2.5-4.RHEL3 SM5....T /bin/cpio
From this output, we can see that the file /bin/cpio has been changed. Its size, mode, checksum, and modification time has all changed. This file is now highly suspect and needs to be examined.
Now that we can see how to verify individual files, we can put these commands together to verify the entire system. In this example we'll be also using the xargs command.
# rpm -qa | xargs -i sh -c 'echo {} ; sudo rpm -V {} ; echo ""'
iproute-2.4.7-11.30E.1
openldap-2.0.27-17
S.5....T c /etc/openldap/ldap.conf
xinetd-2.3.12-6.3E
jaf-20030319-1
javamail-20031006-1
cyrus-sasl-devel-2.1.15-10
--cut--
Running this command will give us a complete summary of what's been changed on the system, and what RPM package the files belong to.
Gentoo Portage also stores information about the packages it has been used to install on a system. Both a checksum and timestamp are recorded at installation time. To take advantage of this data though, we will need to install the gentoolkit.
# emerge gentoolkit
Once installed, we will be using the qpkg and equery commands.
To get a complete listing of all installed packages on the system, we will use the following flags with the qpkg command:
qpkg -I -nc
This tells qpkg to list, without using terminal colors, every installed package.
# qpkg -I -nc app-admin/fam app-admin/gkrellm app-admin/ide-smart app-admin/integrit app-admin/logrotate app-admin/perl-cleaner app-admin/sudo app-admin/syslog-ng app-admin/sysstat app-admin/watchfolder app-arch/bzip2 app-arch/cabextract --cut--
To verify an installed package, we can make use of the check function of equery along with the package name.
equery check package name
This command will produce output similar to the following:
# equery check sys-devel/bc [ Checking sys-devel/bc-1.06-r6 ] * 19 out of 19 files good # equery check app-admin/sudo [ Checking app-admin/sudo-1.6.8_p9 ] !!! /etc/sudoers has incorrect md5sum * 34 out of 35 files good
Here we see that the installed files for bc have not changed and that the file /etc/sudoers, part of the sudo package, has. In this case, like our example using rpm, the file is a configuration file. If this file were outside of /etc we would want to investigate why it has changed.
Now that we see how to check individual packages, we should make use of the xargs command again and string qpkg and equery together to check the entire system.
# qpkg -I -nc | xargs -i sh -c 'equery check {} ; echo ""'
[ Checking app-admin/fam-2.7.0-r2 ]
* 31 out of 31 files good
[ Checking app-admin/perl-cleaner-1.01 ]
* 7 out of 7 files good
[ Checking app-admin/sudo-1.6.8_p9 ]
!!! /etc/sudoers has incorrect md5sum
* 34 out of 35 files good
[ Checking app-admin/syslog-ng-1.6.8 ]
* 50 out of 50 files good
[ Checking app-arch/bzip2-1.0.3-r5 ]
* 46 out of 46 files good
--cut--
With this command, we now have a complete listing of everything that's been changed on the system.
Once a system check is completed and you discover that some of the installed packages may have changed, it's very important that you find out why there have been changes. If you find that numerous files have been changed, it is possible that an intruder may have replaced system binaries with commands of their own. In the event that you cannot explain the changes, there are groups here at IU that can assist you. The UNIX Systems Support Group is available to help with most UNIX questions, as well as the IT Security Office. Should you be unsure about the integrity of your system, do not hesitate to contact either of these groups. It's much better to be cautious and wrong, than reckless and compromised.