It's the wireless switch, stupid

A friend of mine asked to see why a HP Mini 5101 netbook with Ubuntu Karmic on it wouldn't connect to a WiFi AP. As far as I could see the best advice was to install bcmwl-kernel-source, which contains a non-GPLed driver and has to be compiled into a non-free kernel module. But I could not get it to see any APs.


Since the release of Ubuntu Lucid is imminent, I suggested trying Lucid Beta2 on it and then installing the official release after April 29. It would provide a netbook remix and LTS install too. (The Karmic install was a vanilla desktop install, not the best use of the small screen.) Installing off a USB memory stick was painless and fast. I connected it up to a wired network and did:

apt-get install bcmwl-kernel-source

That did all the setup required. But it still wouldn't connect to any APs. Wait a moment, I thought, how do I know if the RF transceiver is turned on or not? I remembered that many notebooks come with a RF kill switch. An examination of the keyboard showed no such function key. Time to do an Internet search. Sure enough, there is a RF kill switch, and it's on the front edge. See the first picture in this photo review. If the RF is on, the LED glows blue.


After flicking the switch on, all the neighbourhood APs were visible in NetworkManager. Duh, that will teach me to at least glance through the manual before installing.


And thank you HP for making a RF kill switch that looks so much like a lid latch. (Sarcasm.)

Cloning a Linux distro inside a VirtualBox VM to a real machine

Recently I tried out LinuxMint Helena LXDE inside a VirtualBox virtual machine. Very nice it was too. I decided that I wanted it installed on a real machine. Being loath to do another install on the real machine I decided to do a network clone, a technique that I have refined over time.


The key idea is to boot both machines with a rescue CD and mount their respective disks. I use SystemRescueCD because it's very up to date with all kinds of filesystems and modules. Then we create a network pipe with netcat and tar the source machine filesystem onto the target machine. I won't specify every step in detail; you have to fill in some details yourself.


First boot the source machine and make a note of the partition filesystems and sizes. In the case of LinuxMint LXDE, it's one of the standard Ubuntu setups, / in /dev/sda1 and an extended partition /dev/sda2 containing a swap in /dev/sda5. I noted the size of the swap, and that / was ext4.


Then boot the target machine. First setup the networking and note the address assigned to the network interface. Then using fdisk, create the same layout. However you can make the disk larger or smaller, so keep the swap partition the same size (I assume your VM was created with the same amount of memory as your real machine; otherwise use the 2xRAM rule) but adjust /dev/sda1 to suit. This is one advantage of this cloning method, you can increase the size of the disk. Then create the target filesystems:

mkfs.ext4 /dev/sda1
mkswap /dev/sda5

Now on both systems, mount the root filesystem and any other filesystems on it. We have only / so this will do:

mkdir /disk; mount /dev/sda1 /disk

Go to the top of the target filesystem and start a netcat listener piping into tar:

cd /disk; netcat -l 2222 | tar zxvf -

On the source machine, mount the source filesystem.

mkdir /disk; mount /dev/sda1 /disk

Go to the top of the source filesystem and start a tar feeding into netcat:

cd /disk; tar zcvf - * | netcat 192.168.1.123 2222

Subsitute the real IP address for 192.168.1.123 above. You will see the files being packed up and sent over. On the target machine you will see the files extracting.


Netcat doesn't terminate at end of input so you have to judge when there are no more files to be tar'red and interrupt it. Usually it's the last file alphabetically, something like /var/xxx or /vmlinuz, depending.


Now you have to reinstall GRUB on the target machine. LinuxMint Helena uses GRUB2, so

grub2-install --root-directory=/disk /dev/sda

You also have to adjust the UUIDs of the partitions or it will not boot. Look at the UUIDs in /disk/etc/fstab, then use tune2fs and mkswap to fix:

tune2fs -U xxxx-xxxx /dev/sda1
mkswap -U yyyy-yyyy /dev/sda5

where the xxxx-xxxx and yyyy-yyyy are the long UUIDs. I recommend cut and paste from /disk/etc/fstab to avoid keying errors.


Another thing you might want to fix up is the label of the ethernet interface. Edit /etc/udev/rules.d/70-persistent-net.rules and comment out the entry for eth0 which will be for a pcnet32, the default "hardware" for VirtualBox VMs and promote the eth1 entry to eth0. This is cosmetic since the system will work fine with eth1 as the network interface.


Other distros may require more fixing up of sound or video configuration. Fortunately LinuxMint (and Ubuntu distros in general) work out these things and adapt more or less automatically.


Now you should be able to boot the target machine. Have fun.


It is also possible to do the reverse, but as the VM is inside a private network, you have to use bridging, or ssh forwarding to pipe the tar stream into the VM. This is one way to virtualise a real install.

Poor man's Model in PHP, Smarty and ADODB

Model-View-Controller (MVC) is a well-established architecture in web applications. If you are not using an established framework for your web app because it's a small app, it still pays to adopt some features of MVC because it makes your app cleaner.


If you are using a templating language like Smarty with PHP, you are getting some of the View aspect. I recently realised that there is a simple way to get a Model of sorts. Now typically with Smarty you would do a whole lot of assignments like this to pass values into the View:

$page->assign('empid', $empid);
$page->assign('empname', $empname);

I realised that with ADODB (or ADODBLite) I could turn on associative fetch, which makes the DB SELECT statement return rows of hashes, e.g.

$employee = Array('id' => '101', 'name' => 'Joe Bloggs', ...)

Then you pass the whole array into a Smarty template like this with one assignment, instead of one for each field.

$page->assign('employee', $employee);

Inside the template you can refer to the fields as:

{$employee.id} {$employee.name}

But that's not all. You can also get the input fields to return values in an array, like this:

<input name="employee[name]" title="Employee name" type="text" value='{$employee.name|escape:"quotes"}' />
In the PHP, you would get the array of values with:


$employee = $_REQUEST['employee'];
and it's an associative array similar to what you put in. You can even, with care, use that array to build an INSERT or UPDATE query for the database. What you have to watch out for is that the array has both numeric and symbolic keys when ADODB returns it.

You can also use that array to carry other fields such as error flags and messages, provided you are careful not to try to write those back to the database. One way would be to filter out keys of a particular form, say those starting with _.

In some of that HTML above you should take care to apply htmlspecialchars() or the Smarty modifier "escape" as necessary.

Access modem web admin page from inside IPCop protected LAN

I recently installed an ADSL2+ modem for a client. The previous ADSL1 modem had been setup by the ISP many years ago. Nowadays it's bring your own modem.


I used a patch cable to access the modem at the factory address 192.168.1.254 then put it in place as a transparent bridge in front of IPCop. I remembered that there was a trick to allow internal machines to access the web interface of the modem in its final position but it wasn't until later that I worked it out.


First of all, IPCop's eth1 needs to be on the same subnet as the modem. On a PPPoE interface on IPCop, the assigned IP address is 1.1.1.1. The exact address is irrelevant to the PPPoE discovery process, but it isn't one that can access the modem. We can either hack the PPPoE script to assign one that suits us, like 192.168.1.1. I preferred to leave that alone and instead assign an additional IP address to eth1, like this:

ip addr add 192.168.1.1/24 dev eth1 

This by itself is not enough, because packets intended for 192.168.1.254 from internal machines  will reach the modem, but the packets from the modem have no way to get back to the web browser. All we need is a masquerade rule in iptables, similar to the one that handles the NAT for traffic with the outside.

iptables -t nat -A REDNAT -o eth1 -j MASQUERADE

And that's it. These were the two lines I put in /etc/rc.d/rc.local. If you go to http://192.168.1.254/ from an inside machine, you get the modem's web admin page.

Adjust the IP addresses to suit your modem of course. Of course, if your LAN has the subnet 192.168.1.0 then you would have to change the modem's address first. Personally I don't understand people who, when they have a choice, use common subnets like 192.168.0.0 or 192.168.1.0 for their SOHO LANs. One of these days they will want a VPN with another LAN, and then... There are vast address spaces in 10.0.0.0/8 and 172.16.0.0/12 for the taking.

In some other tutes on the web you will see another method suggested and that is to establish a ssh forwarding to the modem. This has the advantage that you can control who gets access to modem admin page by who has ssh login on IPCop, assuming you don't trust the modem's login dialog to restrict access. The problem is that the default configuration of IPCop these days disallows ssh forwarding at the IPCop's sshd and you have to enable that by editing /etc/ssh/sshd_config and enabling AllowTcpForwarding.

Clone an IPCop installation over the network

Perhaps you need to clone IPCop to replace the disk, make a working copy for another machine, alter the sizes of the partitions, or for some other reason. This can be done over the network with nothing more than ssh access to IPCop. Remember, clone means clone and you get all the settings and logs replicated, so you should deal with those once you have the clone working.

Connect up the destination disk to a spare machine and boot up with a Puppy Linux CD. There is no reason some other distro couldn't be used, it's just that Puppy Linux runs off the CD and allows you to connect up the disk as the normal primary disk. Another rescue distro would work equally well.

Assuming you have Puppy Linux up and running, first get a network connection with the Network Wizard. Then partition your disk. IPCop creates three partitions, the first for /boot, the second for /var/log and the third for /. The first has only to be about 8-10MB. The third should be about 200-500MB, and the second can use up all the remaining space. You can look at your current IPCop to see the usage of space and tweak these numbers. Use fdisk to do the partition. I will not give a fdisk tutorial here, you can use the built-in help or find its documentation. If your Linux kernel is 2.6 and addresses the disk as sda then substitute sda for hda in the following instructions. In what follows, # is the root shell prompt, which you don't type and // starts a comment, which you don't type either.

# fdisk /dev/hda
// create primary partitions 1, 2 and 4 with the required sizes
// change the type to Linux (0x83)
// (optional) set partition 1 to bootable

Now format the partitions ext3. A gotcha is that recent mke2fs programs set various features on the filesystems that are not supported by the e2fsck program in IPCop. You need to avoid them.

# mke2fs -j -O none,filetype,sparse_super /dev/hda1
# mke2fs -j -O none,filetype,sparse_super /dev/hda2
# mke2fs -j -O none,filetype,sparse_super /dev/hda4

Mount the partitions

# mkdir /ipcop
# mount /dev/hda4 /ipcop
# mkdir -p /ipcop/boot /ipcop/var/log
# mount /dev/hda1 /ipcop/boot
# mount /dev/hda2 /ipcop/var/log

Create a swapfile. The size should be 2x RAM on the destination machine but this is not critical.

# dd if=/dev/zero of=/ipcop/swapfile bs=1M count=64
// e.g. for a 64MB swapfile
# mkswap /ipcop/swapfile

Make some required directories.

# mkdir /ipcop/proc

Now copy the files over. It doesn't take very long, IPCop is a small distro. The backslashes indicate either type it all on one line or type the backslash to the shell to continue the line. You will see the filenames as they are extracted.

# cd /ipcop
# ssh -p 222 root@first.ipcop.box \
 'cd /; tar cf - --numeric-owner bin boot dev etc fastboot home lib mnt root sbin tmp usr var' | \
 tar xvf -

Now you need to install GRUB. grub> is the GRUB prompt.

# grub
grub> device (hd0) /dev/hda
grub> root (hd0,0)
grub> setup (hd0)
grub> quit

GRUB should display messages about installing various files. Now umount the IPCop disk.

# cd /
# umount /ipcop/var/log
# umount /ipcop/boot
# umount /ipcop

Now take out the Puppy Linux CD and reboot with the primary disk. For some reason, only the bootloader has been installed, not the menu, so for the first time you need to run GRUB manually. You might want to disconnect the network cable so that you don't interfere with the network in case IPCop starts using the network cards.

grub> root (hd0,0)
grub> kernel /vmlinuz... root=/dev/hda4
grub> boot

You can use tab file completion to fill out the vmlinuz kernel filename. Once booted into IPCop you need to run grub-install to fix the GRUB menu. BTW, inside IPCop you do not substitute sda for hda because IPCop runs Linux 2.4.

# grub-install --no-floppy /dev/hda

If you have no floppy, you might want to remove the line with fd0 in /boot/grub/device.map otherwise the probing will take a long time.

Now you can login to your new IPCop box and fix up the hardware details and the configuration.

Running a production IPCop in a Virtualbox

Why?

A friend of mine was running IPCop. He also wanted to have a Samba fileserver that would be available to store data. Not wanting two machines powered up all the time, he asked me if I could combine the two.

At first I said that I didn't think it was secure. I thought that perhaps he should run one of those combined firewall/server distros like Clarkconnect, Endian Firewall, etc. But IPCop has a nice web interface, and a small footprint.

When Virtualbox 1.6.2 came out, I decided to experiment to see if a production IPCop firewall could be run inside a Virtualbox VM on a minimal distribution such as Ubuntu or Debian server. I have since also repeated the experiment with VirtualBox 2.1.4, where host networking is much simpler. The sections that differ are noted as such.

Is this for me?

That depends. You would have to appreciate IPCop's flexibility to want to use it over an embedded router. Or perhaps you are disappointed in the throughput of the cheaper embedded routers, but don't want to go all the way with firewall/server distros like Clarkconnect, Endian Firewall and their ilk. This hack occupies a narrow niche, so outside of its boundaries, there may be no advantages to you.

How?

I will describe the setup on an Ubuntu/Debian host. It will work with other Linux distros with the appropriate adjustments to the commands. I have not tried it under non-Linux OSes, so that is for you to experiment.

The minimum specs for the host are:

• 500MHz CPU
• 192MB RAM
• 2 NICs
• 2GB disk space for host OS and IPCop, any extra can be for data

I'd be interested your experiences with lower or higher specs.

First do a standard Ubuntu 8.04 server install. Note that I picked a CLI server configuration to minimise footprint. That's also the reason we don't use the Virtualbox GUI setup tools, but use VBoxHeadless and VRDP instead, so as not to require X11 libraries. I will not describe this install. Choose the services you want, e.g. LAMP, Samba, etc. Remember to do an apt-get update and apt-get upgrade to get the latest fixed packages for your install. In particular I believe there has been a kernel update which you should do before fetching the Virtualbox package. (BTW I will not mention the implicit sudo or su required to do root-only operations.)
Next install the Virtualbox software, either the OSE version from the Ubuntu repositories or the Sun .deb version. If you are using the Sun version you may need to run:

/etc/init.d/vboxdrv setup

to compile the driver for the kernel version you are using. You will need gcc and the kernel sources to do this.

Next add the sysv-rc-conf and dhcp3-server packages.

apt-get install sysv-rc-conf dhcp3-server

Setup for 1.6.2

First add the bridge-utils package.

apt-get install bridge-utils

Set up the two NICs like the "wiring diagram" below by editing /etc/network/interfaces. eth0 is the inside NIC which responds to both the host and IPCop's addresses. eth1 is the outside NIC which is only attached to IPCop's external address.

# The primary network interface
auto br0
iface br0 inet static
        address 192.168.30.1
        netmask 255.255.255.0
        gateway 192.168.30.254
        bridge_ports eth0
        post-up chgrp vboxusers /dev/net/tun

# The secondary network interface
auto br1
iface br1 inet manual
        bridge_ports eth1
        post-up chgrp vboxusers /dev/net/tun

End of setup for 1.6.2.

Setup for 2.1.4

In 2.1.x, it is much simpler. You simply have to set up the host OS networking as normal, bind the guest virtual NICs to the host NICS and then configure the guest NICs as normal within the guest. So the /etc/network/interfaces files looks like this:

# The primary network interface
auto eth0
iface eth0 inet static
        address 192.168.30.1
        netmask 255.255.255.0
        gateway 192.168.30.254

# The secondary network interface
auto eth1
iface eth1 inet manual

End of setup for 2.1.4.

In both cases the wiring diagram is the same, although in 2.1.4, there are no intermediate interfaces names vbox? and br?.

One change from the normal IPCop setup I would recommend is not to run a DHCP server on it, but instead to run one in the host OS which is why we installed dhcp3-server. That way, if IPCop fails to start, at least the clients on your network can get an IP address to work with.
Edit /etc/default/dhcp3-server and set the interface to br0. Have something like this in /etc/dhcp3/dhcpd.conf. Tailor to suit your LAN.

ddns-update-style none;
option domain-name "localdomain";
option domain-name-servers 192.168.30.254;
default-lease-time 600;
max-lease-time 7200;
authoritative;
log-facility local7;
subnet 192.168.30.0 netmask 255.255.255.0 {
  range 192.168.30.128 192.168.30.191;
  option routers 192.168.30.254;
}

You will need to restart the host networking at this point. The clever way is:

/etc/init.d/networking restart
/etc/init.d/dhcp3-server restart

The blunt way is a reboot.

Now create a user to run the VM under and give it a password for logins.

useradd -u 65000 -g vboxusers -m ipcop
passwd ipcop

Now login as ipcop for the following steps.

Copy the ISO install image for IPCop to the host machine, in /tmp, as you can see below. Then set up a VM for IPCop from the command line. This VM has 32MB memory and a 500MB disk.

I=/tmp/ipcop-1.4.20-install-cd.i386.iso
VBoxManage createvm -name IPCop -register
VBoxManage modifyvm IPCop -ostype linux24 -memory 32
VBoxManage createvdi -filename IPCop.vdi -size 500 -register
VBoxManage modifyvm IPCop -hda IPCop.vdi
VBoxManage modifyvm IPCop -nic1 hostif
VBoxManage modifyvm IPCop -nictype1 82540EM
VBoxManage modifyvm IPCop -nic2 hostif
VBoxManage modifyvm IPCop -nictype2 Am79C973
VBoxManage registerimage dvd $I
VBoxManage modifyvm IPCop -dvd $I

Activate the interfaces in 1.6.2

Add the interfaces needed by the VM:

VBoxAddIF vbox0 ipcop br0
VBoxAddIF vbox1 ipcop br1
VBoxManage modifyvm IPCop -hostifdev1 vbox0
VBoxManage modifyvm IPCop -hostifdev2 vbox1

End of 1.6.2 activation.

Activate the interfaces in 2.1.4

Bind the interfaces needed by the VM:

VBoxManage modifyvm IPCop -hostifdev1 eth0
VBoxManage modifyvm IPCop -hostifdev2 eth1

End of 2.1.4 activation.

Now let's start up the IPCop VM.

VBoxHeadless -s IPCop -a 192.168.30.1 -p 4000

From another machine connected to the inside network running X, connect to the ipcop console with:

rdesktop -a 16 192.168.30.1:4000

From Windows you can use the supplied Remote Desktop Client. You specify the host as 192.168.30.1:4000.

You will see the "console" of IPCop.

Do a standard install except that choose noscsi in IPCop install so that it doesn't attempt to probe for SCSI cards. The SCSI probes fail non-fatally, but mess up the screen and slow down the install. I will not describe the install here, there are IPCop documents for this. You will have to select Intel Gigabit NIC (e1000) for the first ethernet card in IPCop and assign this to GREEN, so that the pcnet32 NIC can be assigned to RED. Remember not to activate the DHCP server in IPCop since we have one running in Ubuntu.

After IPCop has been installed, shutdown the VM. Then we detach the ISO image so that it will boot from the "disk" next startup.

VBoxManage modifyvm IPCop -dvd none

Now we need to make IPCop start at boot time. Here's a SysV init script to do this. This is for Ubuntu/Debian, you'll have to do the equivalent in other Linux distros. Put it in /etc/init.d/vbox-ipcop. Activate at runlevel 2 with the sysv-rc-conf utility and make sure it follows vboxdrv and vboxnet in the bootup sequence, and precedes vboxdrv and vboxnet in the shutdown sequence.

#!/bin/sh
# Start/stop the IPCop VM
#
### BEGIN INIT INFO
# Provides:          vbox-ipcop
# Required-Start:    $syslog $time vboxdrv vboxnet
# Required-Stop:     $syslog $time
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: IPCop in Virtualbox VM
# Description:       IPCop in Virtualbox VM
### END INIT INFO

VBOXUSER=ipcop
VARRUNDIR=/var/run/VBoxHeadless
PIDFILE=$VARRUNDIR/$VBOXUSER.pid
VRDPADDR=192.168.30.1
VRDPPORT=4000

test -f /usr/bin/VBoxHeadless || exit 0

. /lib/lsb/init-functions

case "$1" in
start) log_daemon_msg "Starting IPCop in Virtualbox" "VBoxHeadless"
 mkdir -p $VARRUNDIR
 chgrp vboxusers $VARRUNDIR
 chmod g+ws $VARRUNDIR
        USER=$VBOXUSER LOGNAME=$VBOXUSER start-stop-daemon --start --quiet --background --make-pidfile --pidfile $PIDFILE --name VBoxHeadless --chuid $VBOXUSER --startas /usr/bin/VBoxHeadless -- -s IPCop -a $VRDPADDR -p $VRDPPORT
        log_end_msg $?
 ;;
stop) log_daemon_msg "Stopping IPCop in Virtualbox" "VBoxHeadless"
 su -c '/usr/bin/VBoxManage controlvm IPCop keyboardputscancode 1d 38 53' $VBOXUSER
 sleep 30
 start-stop-daemon --stop --quiet --pidfile $PIDFILE --name VBoxHeadless
 rm -f $PIDFILE
        log_end_msg $?
        ;;
restart)
 $0 stop
 $0 start
 ;;
status)
 su -c '/usr/bin/VBoxManage showvminfo IPCop' $VBOXUSER | grep '^State:'
 ;;
*) log_action_msg "Usage: /etc/init.d/vbox-ipcop {start|stop|restart|status}"
        exit 2
        ;;
esac
exit 0

The shutdown part of the script sends a Ctrl-Alt-Del to the VM. However in the default IPCop installation, this reboots the machine. So login to the IPCop VM and edit /etc/inittab, changing the line with shutdown to halt the machine instead. It should look like this after the change.

ca::ctrlaltdel:/sbin/shutdown -h now

The shutdown part is not ideal, it assumes that the machine has been halted within 30 seconds before terminating it anyway. If you come up with a better way, please let me know.

You can now lock logins to the ipcop user account if you wish, since startup is from system scripts now. On the other hand it might be useful for debugging using manual startup. Then again, you can "become" ipcop using su -.

usermod -L ipcop

At the time of writing there is one update you should apply to IPCop to bring it up to 1.4.21. This is a standard IPCop update maneuver so I won't describe it here.

Caveat

Remember that Virtualbox relies on a kernel driver to run. If you update the kernel, the driver will probably be invalidated, and you will not be able to start Virtualbox. Since your firewall is run in Virtualbox, you have painted yourself into a corner. Therefore do not remove the previous version of the kernel until you have the packages needed to make Virtualbox run in the new kernel. For the OSE this will be a release of Virtualbox that is compatible with your new kernel. For the Sun version, it will be the kernel source package to the new version so that you can rebuild the driver with:

/etc/init.d/vboxdrv setup

Is it safe?

The question should be: Are any additional dangers introduced by the use of bridging through a non-firewall host? I believe they are minimal, as the packets only traverse the host firewall, which should be iptables and as good as IPCop's, and the tap driver, which is straightforward and does no interpretation of the packets. Just make sure you never assign an IP address to the outside ethernet interface or bridge in the host OS, otherwise services will bind to that address and will be reachable from the outside. Eventually, you have to decide for yourself. I'd be interested to hear if you have concrete arguments one way or the other.