Apache2 403 error but no clue in error_log, check rewrite rules

I had a directory inside an Apache2 website that I needed to open for indexed browsing.  But it always returned 403 errors. This old but still valid document explains how to solve 403 errors. Generally the first place you should look is /var/log/apache2/error_log. The usual reason is clients' IP address not allowed or file permission problems and there will be a line in the log. However in my situation there was absolutely no diagnostic.

Finally I figured the error must be generated by a RewriteRule. But which one? This page shows how to debug rewrite rules by enabling the log. Put it in say /etc/apache2/conf.d/rewrite.conf (Debian).

RewriteEngine On
RewriteLog "/var/log/apache2/rewrite.log"
RewriteLogLevel 3

It turned out to be a rule in Joomla!'s .htaccess that was blocking indexing on the directory. I also had to re-enable Options Indexes for that directory, as well modify .htaccess to comment out IndexIgnore *, since this cannot be overridden, and Options -Indexes blocks indexing already.

Remember to disable the rewrite log afterwards.

Gotchas setting up FreeBSD as a remote syslog server

I needed to migrate an OpenBSD remote syslog server to a VM. As FreeBSD is better supported under VMWare, I switched to this.

The configuration is well documented in section 12.7.3 of the online manual but I hit a couple of gotchas which I'll share with you to save you work.

The procedure in 12.7.3.1 asks to add a couple of lines to /etc/syslog.conf for each client. This is only necessary if you want to have a separate log file for that client. Otherwise received messages will be distributed to the default log files as configured. By the way I could not find a way to specify a wildcard for the client.

You also need to edit /etc/rc.conf, and specify syslogd_flags. Here you can use a hostname, netblock or domain, see man syslogd. But there is one gotcha. FreeBSD syslogd will only accept messages originating from port 514 on the client. This may not always be the case. The symptom is that tcpdump shows that the messages are arriving on the server, but syslogd is ignoring them. To allow any origin port, specify it with an asterisk after the host or netblock specification.

syslogd_flags="-a 10.10.10.0/24:* -v -v"

And to install open-vm-tools simply do pkg install open-vm-tools. It will pull in a lot of dependencies though.

Avoiding the remote host identification changed warning ssh'ing to localhost

If you run several virtual machines under VirtualBox and use port forwarding (Settings > Network > Advanced > Port Forwarding) to map its port 22 to a localhost port so that you can ssh to it, you will end up with several remote hosts all accessed from localhost but at different ports. In such a situation, if you have an older ssh client, you might end up with this well known warning when you try to connect:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!

The reason is that older ssh clients only store the domain name or the IP address of the server in the ~/.ssh/known_hosts file. So there can only be one host key for localhost and when you connect to a different port there is a key mismatch. Newer ssh clients store the port number also and don't have this problem. You can fix this temporarily by deleting the host key entry but when you connect to another server it happens again.

I've seen solutions that suggest suppressing the host key check. There is another way. All addresses in the 127.0.0.0/8 subnet are localhost so you can use distinct IP addresses for each VM. For example in ~/.ssh/config you could have something like this:

Host centos64

Hostname localhost

        Port 2201

Host debian64

Hostname 127.0.0.2

        Port 2202

This way there are distinct host key entries for the two VMs and you will not get the dreaded clash and warning.

Once again, this is only necessary when using an older ssh client.

Setting event timezone in Lightning (Thunderbird)

This has been a solved problem since 2008 but it may not be obvious to everybody so I'm publicising it here.

You can set the timezone of individual events in Lightning. When you edit an event, there are timezone links next to the start and end times. You can see it in this page's screenshot. The link colour may not always stand out. Clicking on the link brings up a timezone chooser map where you can change the timezone of the event. If the start and end times are linked, you only need to change one. Also if you have used a foreign timezone before, a shortcut is provided.

It seems however, from limited testing, that for synced calendars such as Gcal, existing events have to be edited at both sides for the timezone to stick, even though the time is correct.

An essential feature for those of us who work or communicate across timezones, or have deadlines in other timezones, e.g. taking a MOOC.

Now if only the ICS files sent to me always had the correct timezone.

ping: icmp open socket: Operation not permitted, and capabilities

Last night I installed a SSD on my openSUSE 13.1 system, copied my root filesystem to it, and made it the boot volume. Afterwards it booted up very quickly. I went to bed a happy person.

This morning, while checking some network issues I used ping and got the error in the title. What happened!?

Checking the permissions on both the old filesystem and the new one revealed no differences.

-rwxr-xr-x 1 root root 43480 Nov 16 09:47 /usr/bin/ping

Was it supposed to be setuid? Let's see whether the package manager thinks it's installed correctly.

$ rpm -qf /usr/bin/ping
iputils-s20101006-23.4.1.x86_64
$ rpm -V iputils
/usr/bin/ping should be root:root 0755 "= cap_net_raw+ep". (wrong missing capabilities)

Ah, something was lost in the copying. Searching for capabilities revealed that they allow finer grained privileges than setuid. To fix I was supposed to run setcap. But there was no such utility installed, so I repaired the situation by reinstalling iputils instead of installing the libcap-progs package.

$ sudo zypper install -f iputils

And ping was back to normal.

Incidentally some posts claim that rsync preserves capabilities. I did use rsync to copy and yet the capability did not come across. I can find nothing in the manual page of rsync about this.

Here's a good introduction to capabilities.

Connecting a USB 1.1 device to a USB 3.0 port

I upgraded my computer recently and the new motherboard has USB 3.0 ports throughout, which meant a great improvement in transfer speed for external hard disks.

However I have an ancient Canon flatbed scanner which I use for the occasional document. This was initialised by gscan2pdf, although very slowly, but scans failed with an I/O transfer problem.

Some probing with sane-find-scanner and scanimage, lower level utilities, showed that the scanner wasn't recognised properly, only the driver (plustek) could be ascertained (presumably from the USB vendor and device IDs), but no details about the scanner could be probed.

I remembered a similar situation a while back where a new workstation wouldn't recognise the keyboard and mouse at boot time although later the GUI system did. In that situation, interposing a hub solved the situation. So I put a USB 2.0 hub that I had handy between the motherboard port and the scanner, and lo and behold, everything worked fine from that point.

The specs at the Wikipedia page on USB 3.0 don't clarify whether 3.0 is always backward compatible with 1.1. So if you run into this problem try a hub.

CentOS/RHEL: dig axfr works but zone transfer fails?

I set up named as a secondary with a stanza like this:

zone "example.com" IN {
        type slave;
        masters { 10.0.0.1; };
        file "example.com.zone";
        notify yes;
};

When I restarted named the log messages in /var/log/messages told me that the transfer failed. However I could do:

dig @10.0.0.1 axfr example.com

just fine. What was the problem?

Looking in /var/named/chroot/var/named I saw that there was a directory named slaves. Should I be using that? Looking at the permissions, I saw:

total 88
drwxr-x--- 4 root  named 4096 Jan  7  2013 .
drwxr-x--- 6 root  named 4096 Jan  7  2013 ..
drwxrwx--- 2 named named 4096 Mar  4 23:23 data
-rw-r----- 1 root  named  208 Apr 28  2005 localdomain.zone
-rw-r----- 1 root  named  195 Apr 28  2005 localhost.zone
-rw-r----- 1 root  named  427 Apr 28  2005 named.broadcast
-rw-r----- 1 root  named  424 Apr 28  2005 named.ip6.local
-rw-r----- 1 root  named  426 Apr 28  2005 named.local
-rw-r----- 1 root  named 1892 Feb 26  2008 named.root
-rw-r----- 1 root  named  427 Apr 28  2005 named.zero
drwxrwx--- 2 named named 4096 Mar  5 00:59 slaves

Yes, I bet I'm supposed to use the directory slaves because that's writable by named but the top zone directory isn't, so I changed the line in the stanza to:

        file "slaves/example.com.zone";

and it worked.

You may see advice on the Web to change permissions of the top zone directory. Don't do that. Work with the structure and permissions that the distribution makers have developed.