Thursday, 11 January 2018

How to send email that will expire (Outlook only)

I get notification email from Linux cron jobs. I wanted some way to automatically mark mail as no longer relevant after a week. I recently discovered that there are a couple of mail headers defined for this purpose in RFCs: Expires: and the older, superseded Expiry-date: which is what Outlook uses. They are not honoured by most mail readers though so this tip is specific to Outlook.

I discovered sample code from Internet search. Here is my version in Perl which does the job.


use Net::SMTP;
...
my $expire_time = strftime "%d %b %y %T %Z", localtime(time() + 86400 * 7);
...

$smtp = Net::SMTP->new($mailserver);
...
$smtp->datasend("Expires: $expire_time\n");
$smtp->datasend("Expiry-date: $expire_time\n");

The Net::SMTP module allows one to send arbitrary headers. You can see the required format of the date time string from the strftime() call.

Expired emails are not deleted, but are displayed in desktop Outlook with strikethrough. No effect that I know of in other mail readers, not even the Internet Office 365.

Friday, 29 December 2017

How to change the video resolution in Raspbian PC/Mac Pixel

In case you are not aware, you can also get Raspbian Pixel for your PC or Mac. It's a 32-bit Debian Stretch distro with Raspbian enhancements, notably the Pixel desktop.

One problem is that the default video mode after install is 640x480, rather limiting. I searched a bit for how to change the video resolution but most articles were about Raspbian on the RPi. However this GRUB documentation was the key.

To find the modes available to you, interrupt GRUB's booting with c, then at the grub> prompt, type videoinfo. You will get a list of available modes. This depends on your (real or virtual) video card. In my case I was running under VirtualBox and had sufficient video memory configured. Note a video mode you want, then as root (either via sudo or getting a root shell) do the following:

Edit /etc/default/grub. Change the entry GRUB_GFXMODE which is commented out by default to for example:

GRUB_GFXMODE=1280x1024x32

Check that the file also contains

GRUB_GFXPAYLOAD_LINUX=keep

Run update-grub, which will rewrite the grub.cfg file. Reboot and enjoy your new video resolution.

Addendum: I've found that when the VirtualBox guest additions are installed, a different resolution is used. This can be configured with the program lxrandr. I have to investigate under what circumstances the display switches from the resolution inherited from grub to its own setting.

Saturday, 23 December 2017

Are all your blogger blogs using https?

Blogger now allows you to force all http access to redirect to https access. But if you have a lot of blogs how do you check which (historical) ones need to have this setting enabled in Settings > Basic? Wget to the rescue again. Assuming you have a list of http URLs in the file sites.

for s in $(cat sites)
do
  echo -n "$s " 1>&2
  wget --spider "$s" 2>&1
done | grep Location:


If the output is something like:

http://myblog.blogspot.com Location: https://myblog.blogspot.com/ [following]

that blog is fine.

The 1>&2 for the echo is so that its output isn't filtered out by the grep.

Monday, 27 November 2017

EFI System Partition in soft RAID1

One reason you might want to put the EFI System Partition (ESP) in a RAID1 array on a computer with Linux soft RAID is to have redundancy when booting. If one disk fails, you want the boot to continue from the other disk.

At first I thought this wasn't possible since a RAID1 partition wouldn't have the specific FAT filesystem and GUID required by the specification. However the fact that the CentOS 7 install media offered the choice of putting the ESP on a RAID1 array and that it actually works, made me doubt my hypothesis.

The key to this that the CentOS 7 installer uses RAID metadata format 1.0, which is located at the end of the partition. Thus it doesn't clash with the beginning of the partition, which is where the BIOS will check to see if the partition is an ESP. However most Linux partition tools will detect it first as a RAID member so it's not immediately obvious that it's an ESP.

There are some caveats to this scheme. All writing of the ESP must be done while it's mounted as a RAID array so that there is no discrepancy between the two members. If the only OS on the disks is Linux, this won't be a problem. But don't use this scheme if the ESP also boots other operating systems that don't know about Linux RAID.

For CentOS when you look at the choice of boot devices in the BIOS, you should see two disk boot candidates, both labelled CentOS.

On the machines I used, HP z230 workstations, I found that I had to disable Legacy Boot or errors reading the boot sectors would be triggered.

The bottom line is I now have workstations with soft RAID1 whose disks are fully redundant. If one disk fails, the other will continue to boot and run with degraded arrays for each of the partitions.

Thursday, 23 November 2017

grub2 error: failure reading sector 0x0 from 'hd0'. Press any key to continue

After I had installed CentOS 7 as the only OS on a HP z230 workstation in UEFI boot mode, I got this message before booting. It was actually the last of three errors:

error: failure reading sector 0xfc from 'hd0'. 
error: failure reading sector 0xe0 from 'hd0'. 
error: failure reading sector 0x0 from 'hd0'

Boot would resume from the hard disk after a timeout, but the pause was unacceptable and would worry users.

A search showed many articles like this but none solved my problem. I tried various things: refreshing grub.cfg, disabling the CD/DVD (thinking it might be trying to read the optical drive), checking if having the ESP in a EFI system partition in a RAID1 array was disallowed. (I figured out how ESP can work with RAID1, and its limitations, but that's for another blog entry.) None of my experiments worked.

However the linked to web page alluded to turning off Secure Boot so I went into that part of the BIOS setup. I found that it was already turned off but there was a setting there for Legacy Boot which was enabled. So I turned it off to see what would happen. Lo and behold, the error messages ceased, and UEFI boot worked as expected. Also the Boot Order menu stopped showing a Legacy section.

Since debugging the innards of the GRUB2 loader is beyond me, I can only surmise that the presence of Legacy Boot entries in the BIOS makes GRUB2 try reading the sectors in question but since the disk is formatted with GPT partitions and UEFI is in force, the sector reads fail, for some definition of fail. Maybe somebody can figure out the significance of the sectors 0xfc, 0xe0, and 0x0.

Thursday, 16 November 2017

Dos and Don'ts deploying sssd for authentication against Windows AD

New: For deployment on Redhat/CentOS 6, see here.

sssd (and realmd) in RedHat/CentOS 7 offers the chance to use Windows as a single authentication base. The RedHat manual was the most useful but there were also good debugging tips on stackoverflow and similar forums. However in deploying sssd I found some things worked for me and some things didn't.
  • Do harmonise all the Windows and Linux login IDs. If there are users with two different IDs, then they'll have to bite the bullet and accept the change of one ID. Unfortunately domain logins cannot have aliases.
  • When you join Linux to AD using the realm command and an unprivileged account, you may encounter this 10 machine limit. Here's how to raise the limit.
  • Do use ntpd to keep all the clients in time sync. Specify the domain servers as NTP servers in ntp.conf. I had an issue where one client wouldn't authenticate. All the config files were identical with a working client. Finally I realised I had not enabled and started ntpd. It turned out to be clock skew. Kerberos is sensitive to this.
  • Do enable GSSAPI with MIC in sshd. It really works and you can use putty to ssh to the server without specifying a password provided the Windows user has authenticated to the domain.
  • Do use AD security groups to restrict access to the Linux servers. Otherwise all AD users can login by default. This means that enrolling a new Linux user across all the servers is simply adding the user to your chosen security group. Create one if necessary. Oddjobd will take care of creating the home directory on first login, which is very nice. I used the simple access_provider. I couldn't get the ad access_provider and ad_access_filter to work, but this is probably because I couldn't work out the correct LDAP strings.
  • You can also use a security group to specify who can have extra privileges in sudo.
  • I used the deterministic hash scheme for mapping SIDs to UIDs because I didn't want to (and didn't have authority to) add attributes to the AD schema.
  • When migrating existing user accounts, make sure you find all the places a user might have a file. Not just /home but also /var/spool/cron and /var/spool/mail. Kick all the users off and kill all of their processes before you do the chown. Since after the switchover the names will map to the new UIDs, you can cd /home and run a loop: for u in * do; chown -R $u $u; done. Also the cron and mail directories.
  • If you have software that must have simple login IDs, i.e. fred and not fred@example.com, then you should set use_fully_qualified_names = False. This implies you cannot have a default_domain_suffix. If you have a single domain, then you don't need domain suffixes. If you have multiple domains, then this is beyond my knowledge. I found that some applications cannot handle usernames of the domain form. Even the crontab command will create and require cron files of the domain form if domain suffixes are enabled.
  • I couldn't get the sssd idmap to work with Samba so I chose winbind. Also you have to use winbind if you have to support NTLM authentication.
  • New: If you are running 32-bit applications, you should also install the 32-bit libsss* shared libraries corresponding to the 64-bit ones, otherwise those applications may not be able to get user account info via PAM. This showed up in icfb, an old 32-bit Cadence executable, that worked for local users (in /etc/passwd) but failed for SSSD authenticated users.
  • New: If oddjob_mkhomedir doesn't work, as evidenced by no home directory created for a new login, check /var/log/messages. SELinux is probably blocking this. Either make the policy permissive, or create a policy for this.

Friday, 15 September 2017

Use crontab to notify when a piece of software has been released

Sometimes I eagerly await the release of a distro version or a new version of a piece of software. But I don't want to remember to check constantly, so I wrote a script that can be run from cron to let me know.

#!/bin/sh
case $# in
0|1)
       echo Usage: $0 url message

       exit
       ;;
esac
url="$1"
shift
wget -q --spider "$url" && echo "$@"


As you can see, this runs a wget on a specified URL which does not produce any output, but if successful, will print the message. Put this in crontab and the message will be mailed to you.

This script depends on knowing a URL that will exist when the release happens. Often you can guess the URL from previous releases. Here are a couple of examples, each is all on a single line:

1. Check at 0808 every day to see if AntiX 17 has been released:

8 8 * * * watchurl https://sourceforge.net/projects/antix-linux/files/Final/antiX-17/ AntiX 17 released

2. Check at 0809 every day to see if VirtualBox 5.1.30 has been released:

9 8 * * * watchurl https://www.virtualbox.org/download/hashes/5.1.30/MD5SUMS VirtualBox 5.1.30 has been released