Tuesday, 28 April 2009

Running a production IPCop in a Virtualbox

Why?

A friend of mine was running IPCop. He also wanted to have a Samba fileserver that would be available to store data. Not wanting two machines powered up all the time, he asked me if I could combine the two.

At first I said that I didn't think it was secure. I thought that perhaps he should run one of those combined firewall/server distros like Clarkconnect, Endian Firewall, etc. But IPCop has a nice web interface, and a small footprint.

When Virtualbox 1.6.2 came out, I decided to experiment to see if a production IPCop firewall could be run inside a Virtualbox VM on a minimal distribution such as Ubuntu or Debian server. I have since also repeated the experiment with VirtualBox 2.1.4, where host networking is much simpler. The sections that differ are noted as such.

Is this for me?

That depends. You would have to appreciate IPCop's flexibility to want to use it over an embedded router. Or perhaps you are disappointed in the throughput of the cheaper embedded routers, but don't want to go all the way with firewall/server distros like Clarkconnect, Endian Firewall and their ilk. This hack occupies a narrow niche, so outside of its boundaries, there may be no advantages to you.

How?

I will describe the setup on an Ubuntu/Debian host. It will work with other Linux distros with the appropriate adjustments to the commands. I have not tried it under non-Linux OSes, so that is for you to experiment.

The minimum specs for the host are:
  • 500MHz CPU
  • 192MB RAM
  • 2 NICs
  • 2GB disk space for host OS and IPCop, any extra can be for data
I'd be interested your experiences with lower or higher specs.

First do a standard Ubuntu 8.04 server install. Note that I picked a CLI server configuration to minimise footprint. That's also the reason we don't use the Virtualbox GUI setup tools, but use VBoxHeadless and VRDP instead, so as not to require X11 libraries. I will not describe this install. Choose the services you want, e.g. LAMP, Samba, etc. Remember to do an apt-get update and apt-get upgrade to get the latest fixed packages for your install. In particular I believe there has been a kernel update which you should do before fetching the Virtualbox package. (BTW I will not mention the implicit sudo or su required to do root-only operations.)
Next install the Virtualbox software, either the OSE version from the Ubuntu repositories or the Sun .deb version. If you are using the Sun version you may need to run:


/etc/init.d/vboxdrv setup

to compile the driver for the kernel version you are using. You will need gcc and the kernel sources to do this.

Next add the sysv-rc-conf and dhcp3-server packages.


apt-get install sysv-rc-conf dhcp3-server

Setup for 1.6.2

First add the bridge-utils package.


apt-get install bridge-utils

Set up the two NICs like the "wiring diagram" below by editing /etc/network/interfaces. eth0 is the inside NIC which responds to both the host and IPCop's addresses. eth1 is the outside NIC which is only attached to IPCop's external address.


# The primary network interface
auto br0
iface br0 inet static
        address 192.168.30.1
        netmask 255.255.255.0
        gateway 192.168.30.254
        bridge_ports eth0
        post-up chgrp vboxusers /dev/net/tun

# The secondary network interface
auto br1
iface br1 inet manual
        bridge_ports eth1
        post-up chgrp vboxusers /dev/net/tun

End of setup for 1.6.2.

Setup for 2.1.4

In 2.1.x, it is much simpler. You simply have to set up the host OS networking as normal, bind the guest virtual NICs to the host NICS and then configure the guest NICs as normal within the guest. So the /etc/network/interfaces files looks like this:


# The primary network interface
auto eth0
iface eth0 inet static
        address 192.168.30.1
        netmask 255.255.255.0
        gateway 192.168.30.254

# The secondary network interface
auto eth1
iface eth1 inet manual

End of setup for 2.1.4.

In both cases the wiring diagram is the same, although in 2.1.4, there are no intermediate interfaces names vbox? and br?.




One change from the normal IPCop setup I would recommend is not to run a DHCP server on it, but instead to run one in the host OS which is why we installed dhcp3-server. That way, if IPCop fails to start, at least the clients on your network can get an IP address to work with.
Edit /etc/default/dhcp3-server and set the interface to br0. Have something like this in /etc/dhcp3/dhcpd.conf. Tailor to suit your LAN.


ddns-update-style none;
option domain-name "localdomain";
option domain-name-servers 192.168.30.254;
default-lease-time 600;
max-lease-time 7200;
authoritative;
log-facility local7;
subnet 192.168.30.0 netmask 255.255.255.0 {
  range 192.168.30.128 192.168.30.191;
  option routers 192.168.30.254;
}

You will need to restart the host networking at this point. The clever way is:


/etc/init.d/networking restart
/etc/init.d/dhcp3-server restart

The blunt way is a reboot.

Now create a user to run the VM under and give it a password for logins.


useradd -u 65000 -g vboxusers -m ipcop
passwd ipcop

Now login as ipcop for the following steps.

Copy the ISO install image for IPCop to the host machine, in /tmp, as you can see below. Then set up a VM for IPCop from the command line. This VM has 32MB memory and a 500MB disk.


I=/tmp/ipcop-1.4.20-install-cd.i386.iso
VBoxManage createvm -name IPCop -register
VBoxManage modifyvm IPCop -ostype linux24 -memory 32
VBoxManage createvdi -filename IPCop.vdi -size 500 -register
VBoxManage modifyvm IPCop -hda IPCop.vdi
VBoxManage modifyvm IPCop -nic1 hostif
VBoxManage modifyvm IPCop -nictype1 82540EM
VBoxManage modifyvm IPCop -nic2 hostif
VBoxManage modifyvm IPCop -nictype2 Am79C973
VBoxManage registerimage dvd $I
VBoxManage modifyvm IPCop -dvd $I

Activate the interfaces in 1.6.2

Add the interfaces needed by the VM:


VBoxAddIF vbox0 ipcop br0
VBoxAddIF vbox1 ipcop br1
VBoxManage modifyvm IPCop -hostifdev1 vbox0
VBoxManage modifyvm IPCop -hostifdev2 vbox1

End of 1.6.2 activation.

Activate the interfaces in 2.1.4

Bind the interfaces needed by the VM:


VBoxManage modifyvm IPCop -hostifdev1 eth0
VBoxManage modifyvm IPCop -hostifdev2 eth1

End of 2.1.4 activation.

Now let's start up the IPCop VM.


VBoxHeadless -s IPCop -a 192.168.30.1 -p 4000

From another machine connected to the inside network running X, connect to the ipcop console with:


rdesktop -a 16 192.168.30.1:4000

From Windows you can use the supplied Remote Desktop Client. You specify the host as 192.168.30.1:4000.

You will see the "console" of IPCop.

Do a standard install except that choose noscsi in IPCop install so that it doesn't attempt to probe for SCSI cards. The SCSI probes fail non-fatally, but mess up the screen and slow down the install. I will not describe the install here, there are IPCop documents for this. You will have to select Intel Gigabit NIC (e1000) for the first ethernet card in IPCop and assign this to GREEN, so that the pcnet32 NIC can be assigned to RED. Remember not to activate the DHCP server in IPCop since we have one running in Ubuntu.

After IPCop has been installed, shutdown the VM. Then we detach the ISO image so that it will boot from the "disk" next startup.


VBoxManage modifyvm IPCop -dvd none

Now we need to make IPCop start at boot time. Here's a SysV init script to do this. This is for Ubuntu/Debian, you'll have to do the equivalent in other Linux distros. Put it in /etc/init.d/vbox-ipcop. Activate at runlevel 2 with the sysv-rc-conf utility and make sure it follows vboxdrv and vboxnet in the bootup sequence, and precedes vboxdrv and vboxnet in the shutdown sequence.


#!/bin/sh
# Start/stop the IPCop VM
#
### BEGIN INIT INFO
# Provides:          vbox-ipcop
# Required-Start:    $syslog $time vboxdrv vboxnet
# Required-Stop:     $syslog $time
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: IPCop in Virtualbox VM
# Description:       IPCop in Virtualbox VM
### END INIT INFO

VBOXUSER=ipcop
VARRUNDIR=/var/run/VBoxHeadless
PIDFILE=$VARRUNDIR/$VBOXUSER.pid
VRDPADDR=192.168.30.1
VRDPPORT=4000

test -f /usr/bin/VBoxHeadless || exit 0

. /lib/lsb/init-functions

case "$1" in
start) log_daemon_msg "Starting IPCop in Virtualbox" "VBoxHeadless"
 mkdir -p $VARRUNDIR
 chgrp vboxusers $VARRUNDIR
 chmod g+ws $VARRUNDIR
        USER=$VBOXUSER LOGNAME=$VBOXUSER start-stop-daemon --start --quiet --background --make-pidfile --pidfile $PIDFILE --name VBoxHeadless --chuid $VBOXUSER --startas /usr/bin/VBoxHeadless -- -s IPCop -a $VRDPADDR -p $VRDPPORT
        log_end_msg $?
 ;;
stop) log_daemon_msg "Stopping IPCop in Virtualbox" "VBoxHeadless"
 su -c '/usr/bin/VBoxManage controlvm IPCop keyboardputscancode 1d 38 53' $VBOXUSER
 sleep 30
 start-stop-daemon --stop --quiet --pidfile $PIDFILE --name VBoxHeadless
 rm -f $PIDFILE
        log_end_msg $?
        ;;
restart)
 $0 stop
 $0 start
 ;;
status)
 su -c '/usr/bin/VBoxManage showvminfo IPCop' $VBOXUSER | grep '^State:'
 ;;
*) log_action_msg "Usage: /etc/init.d/vbox-ipcop {start|stop|restart|status}"
        exit 2
        ;;
esac
exit 0

The shutdown part of the script sends a Ctrl-Alt-Del to the VM. However in the default IPCop installation, this reboots the machine. So login to the IPCop VM and edit /etc/inittab, changing the line with shutdown to halt the machine instead. It should look like this after the change.


ca::ctrlaltdel:/sbin/shutdown -h now

The shutdown part is not ideal, it assumes that the machine has been halted within 30 seconds before terminating it anyway. If you come up with a better way, please let me know.

You can now lock logins to the ipcop user account if you wish, since startup is from system scripts now. On the other hand it might be useful for debugging using manual startup. Then again, you can "become" ipcop using su -.


usermod -L ipcop

At the time of writing there is one update you should apply to IPCop to bring it up to 1.4.21. This is a standard IPCop update maneuver so I won't describe it here.

Caveat

Remember that Virtualbox relies on a kernel driver to run. If you update the kernel, the driver will probably be invalidated, and you will not be able to start Virtualbox. Since your firewall is run in Virtualbox, you have painted yourself into a corner. Therefore do not remove the previous version of the kernel until you have the packages needed to make Virtualbox run in the new kernel. For the OSE this will be a release of Virtualbox that is compatible with your new kernel. For the Sun version, it will be the kernel source package to the new version so that you can rebuild the driver with:


/etc/init.d/vboxdrv setup

Is it safe?

The question should be: Are any additional dangers introduced by the use of bridging through a non-firewall host? I believe they are minimal, as the packets only traverse the host firewall, which should be iptables and as good as IPCop's, and the tap driver, which is straightforward and does no interpretation of the packets. Just make sure you never assign an IP address to the outside ethernet interface or bridge in the host OS, otherwise services will bind to that address and will be reachable from the outside. Eventually, you have to decide for yourself. I'd be interested to hear if you have concrete arguments one way or the other.