This article accompanies a talk given by the ITSO at LinuxFest 2005.
If you had a few bars of gold bullion stashed in a bank vault, would you carry the key on your regular keychain with your house and car keys? Or would you store it in a hidden strong box with a secret combination? Common sense tells us the latter is more secure. Carrying that important key around exposes it to the threat of theft or loss. And if you don't need the key handy (it's unlikely you'll be accessing your stash of gold every day), it's an unnecessary risk.
The notion of not carrying the important key around is known as the principle of least privilege. In computer security, that means granting a user or process the smallest amount of privilege necessary to do a job. Following this principle is considered a good security practice because the less frequently a privilege is used, the less likely it is to be abused.
We're always telling people not to use administrative privileges except when necessary. On a Windows computer, you should remove yourself from the Administrators or Power Users group and use Run As... to do administrative tasks. On Unix-like systems, you should never log in as root. Always log in as yourself and use su or sudo for specific administrative tasks. This article shows that not only users but services, specifically the Apache web server on a Linux or Unix system, can be made to follow the same principle.
The Apache web server already has a built-in feature to limit its own privileges. You start it as root, and it spawns a bunch of child processes to handle incoming connections. Each of these children runs not as root but as a less privileged user, usually apache. It's smart for Apache to drop privileges in this way, but it doesn't go far enough. Given the right vulnerability, root can still be gained by attacking operations that the parent process must perform.
In this case study, we present a technique to run the parent process as the apache user, just like the children. With root privileges, the parent would have been able to access any file on the system, become another user, bind to an IP port lower than 1024, and do many other things. But as the apache user, the parent's privileges are greatly reduced. To make Apache continue to work after this reduction in privileges, you'll have to give the apache user access to the logfiles and, for an SSL-enabled web server, the SSL certificate and private key. You'll also have to configure it to use unprivileged ports.
The paths and configuration directions in this article are for Apache 2.0.52 on Red Hat Enterprise Linux 4, but the principles are the same for other systems. If you try this technique and run into trouble, the following command can be very helpful:
strace -f -u apache httpd -X
Here's what each portion of the command line does:
As mentioned before, most web servers use TCP ports 80 and 443, which can only be bound to by root. You can change them to anything above 1023. Simply adding 10,000 to the standard port number makes it easy to remember what ports you chose and is unlikely to result in conflicts with other services, so try 10080 instead of 80 and 10443 instead of 443.
To make these changes, search for any Port, Listen, or VirtualHost directives in your Apache configuration file(s) and change the port number as appropriate. On our RHEL4 server, the default configuration file is /etc/httpd/conf/httpd.conf. Once you make the changes and restart Apache, it should be listening on the high-numbered ports. If any users were to visit your server now, they would have to use a URL like https://example.itso.iu.edu:10443/.
The most tedious part of this whole process is tracking down every file that a service wants to access. Fortunately, Apache doesn't usually want to access that many. The most common ones are log files, the PID file, and SSL certificate and key files.
Apache is going to need to write to its log files. But these files are typically only writable by root. You can use traditional Unix groups to allow the apache user, which is by default a member of the apache group, to write to these files. First, change the group ownership on the log directory and all the files therein to (apache):
# cd /var/log # chgrp -R apache httpd
Then you can make sure this group can access the log directory by using the chmod command to change its permissions. Adding read permission on a directory allows the group to get a listing of files therein. Adding write permission allows the creation of files. Adding execute permission allows the directory to be entered. And you can turn on one other bit, the setgid bit (represented by s), which causes all files created inside this directory to inherit the owner, group, and permissions of the directory itself.
# chmod g+rwxs httpd
Using the chmod command again, allow the group to write to the log files themselves.
# chmod -R g+rw httpd
When you're finished, a listing of your log directory and log files should look like
drwxrws--- root apache /var/log/httpd/ -rw-rw-r-- root apache /var/log/httpd/access_log -rw-rw-r-- root apache /var/log/httpd/error_log -rw-rw-r-- root apache /var/log/httpd/ssl_access_log -rw-rw-r-- root apache /var/log/httpd/ssl_error_log -rw-rw-r-- root apache /var/log/httpd/ssl_request_log
You may be familiar with the apachectl command, which you can use to start, stop, and restart the Apache, among other things. It performs these actions by sending signals to the Apache parent process, which keeps its process id (PID) in a file for this purpose. The file itself usually lives in the /var/run/ directory, but on our RHEL4 server there's a symbolic link to this directory called /etc/httpd/run:
lrwxrwxrwx root root /etc/httpd/run -> ../../var/run/ drwxr-xr-x root root /var/run/
The symlink is how Apache actually refers to the file.
It may be tempting to treat the PID file and its parent directory as you did the log files. But the /var/run/ directory could contain PID files for other services, and you don't want the apache user to be able to write to them. So it's best to make Apache its own subdirectory and set its permissions appropriately:
# mkdir -m 2775 /var/run/httpd # chgrp apache /var/run/httpd
The -m 2775 arguments to mkdir are a shorthand way of making this subdirectory look like the /var/log/httpd/ directory.
To really make Apache create the PID file in your new directory, you have to change the symlink:
# cd /etc/httpd # ln -sf ../../var/run/httpd run
You can control access to your SSL certificate and public key by altering the approach you took with the logs and PID files. You don't want the apache user to be able to write to the SSL files. If Apache gets compromised while running with reduced privileges, the attacker only gets into the apache account. But if that account can write to the SSL files, the attacker can replace or change those files. Instead, keep them owned by root and only readable by apache so they can't be tampered with even after a successful compromise.
The technique is still similar; just don't add the w permission bit:
# cd /etc/httpd/conf # chgrp -R apache ssl.* # chmod -R g+rX ssl.*
This should cause the structure to look like
drwxr-x--- root apache ssl.crl/ drwxr-x--- root apache ssl.crt/ -rw-r----- root apache ssl.crt/server.crt drwxr-x--- root apache ssl.csr/ drwxr-x--- root apache ssl.key/ -rw-r----- root apache ssl.crt/server.key drwxr-x--- root apache ssl.prm/
This step is the crux of the whole operation: modify apachectl so that when it is used to start Apache, it does so as the user apache rather than as root. Make the change in the definition of the $HTTPD variable in the apachectl script, using sudo to drop privileges:
HTTPD='sudo -u apache /usr/sbin/httpd'
From now on, when apachectl is run, it will start the httpd process as the user apache.
Now, when you start Apache, nothing should be running as root:
# apachectl startssl # ps -o user,pid,ppid,cmd -C httpd USER PID PPID CMD apache 4895 1 /usr/sbin/httpd -k start -DSSL apache 4896 4895 /usr/sbin/httpd -k start -DSSL apache 4897 4895 /usr/sbin/httpd -k start -DSSL apache 4898 4895 /usr/sbin/httpd -k start -DSSL apache 4899 4895 /usr/sbin/httpd -k start -DSSL apache 4900 4895 /usr/sbin/httpd -k start -DSSL apache 4901 4895 /usr/sbin/httpd -k start -DSSL apache 4902 4895 /usr/sbin/httpd -k start -DSSL apache 4903 4895 /usr/sbin/httpd -k start -DSSL
You can use the netstat program to see what ports your system is listening on. You should see Apache listening on high-numbered ports.
# netstat -lnpt Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 :::10080 :::* LISTEN 4895/httpd tcp 0 0 :::10443 :::* LISTEN 4895/httpd
So you still have the problem of these ugly high-numbered ports appearing in URLs, but you can eliminate that problem with a tool like iptables.
Hopefully, you already have iptables or some other packet filtering tool configured to drop all packets that aren't explicitly allowed. If so, you'll have to add rules to your iptables ruleset to allow packets in at the high-numbered ports. Here are some sample commands to insert the rules for Red Hat:
# iptables -I RH-Firewall-1-INPUT -p tcp --dport 10080 -j ACCEPT # iptables -I RH-Firewall-1-INPUT -p tcp --dport 10443 -j ACCEPT
The RH-Firewall-1-INPUT chain is specific to Red Hat; for other systems you'll probably want to use the default INPUT chain. These rules simply allow packets in to the high ports you've configured Apache to listen on. If you already have similar rules for ports 80 and 443, you can remove them.
The redirection is done by rules in a different table, the nat table:
# iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 10080 # iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-ports 10443
These rules use the PREROUTING chain to rewrite packets destined for ports 80 and 443 as soon as they come in. The -j REDIRECT and following arguments simply change the destination port to the appropriate high-numbered port. Now your web site should function as you expect. Normal URLs should seamlessly result in requests to the high-numbered ports.
After you've finalized your ruleset changes, don't forget to save them:
# /etc/init.d/iptables save Saving firewall rules to /etc/sysconfig/iptables: [ OK ]
You may have noticed that you still have to use explicit port numbers when connecting from the web server itself. That's because packets that don't actually come in from the network don't go through the nat table, so they don't get rewritten. The most common reason to connect to localhost is to get a status report using apachectl status. It simply connects to the URL in the $STATUSURL variable defined in the apachectl script. All you have to do is add an explicit port number to that URL:
STATUSURL="http://localhost:10080/server-status"
Once you've made this change, apachectl status should work again.
You might be concerned about the overhead caused by rewriting every HTTP packet traversing your web server, but it turns out not to be a big deal in most cases. Really, iptables is only inspecting two bytes of every packet and then changing those two bytes if it's an HTTP packet. Some simple tests with the Apache benchmarking tool (ab) on the reference web server for this article showed a performance hit of around 1.4%, which is not a bad trade-off considering the improvement in security. But this number doesn't mean much; you'll want to do your own benchmarks in your own environment to get a more accurate assessment.
Apache is not the only service that can benefit from this technique. Any service that runs as root only to bind to a low port and write to a few files is a good candidate. FTP, DNS, LDAP, NNTP, and various kinds of print servers come to mind.
An rsync server is a great example; you want the rsync process to be able to read the files it serves but certainly not write to them. But if the rsync service runs as root and an attacker can exploit an rsync vulnerability, the attacker could inject malware into any of the files distributed by the service, infecting all your customers. Limiting the privileges of the rsync process mitigates such an attack.
So there are many ways this technique can be used, and probably ways it can be improved. The key is to find ways to implement the principle of least privilege wherever possible.