Avoid calling basename or dirname by using parameter expansion

These short cuts have been documented under parameter expansion for a long time but are not well known because people are used to writing basename and dirname.

Instead of using:

file=$(basename "$path")

use

file=${path##*/}

Instead of using:

dir=$(dirname "$path")

use

dir=${path%/*}

The advantage is not calling basename as an extra process. Although basename might be implemented as a builtin in some shells.

See the linked page for further short cuts.

Chrome/Chromium is storing the url and referrer of downloaded files in extended attributes

When I turned on extended attributes for a rsync command (-X option) I noticed that it proposed to copy attributes for a bunch of files it had not before. Most of those files appeared to be PDFs or zip files. So I wondered what those attributes were. Here is an example:

$ getfattr -d work/uC/asm7000.zip
# file: work/uC/asm7000.zip
user.xdg.origin.url="http://cd.textfiles.com/emspro1/ASMUTIL/ASM7000.ZIP"
user.xdg.referrer.url="http://cd.textfiles.com/emspro1/ASMUTIL/"

Oh! That's not good! The origin of this file is harmless, but what if it had been a file downloaded from a sensitive location or the URL contained sensitive parameters. I may want to give the file to somebody without revealing the URL I got it from, which may contain side channel information. You can imagine your own scenario.

I should add this is only an issue for you if you have enabled extended attributes on your filesystem and mounted with that option. Look for the option user_xattr in /etc/fstab lines. But maybe Android does enable extended attributes. That would be a big worry; your phone may be retaining information you didn't know about.

Doing a search found this recent Debian bug report with no resolution. Also these attributes are listed as in current use by freedesktop.org. And it looks like wget too stores the url. Update 2019-03-18: I have been informed that wget from 1.20.1 has options to control storage of the attributes.

This really should be behaviour that can be turned off in Chrome/Chromium. Until then, I'm not backing up extended attributes for my own files. You should also screen extended attributes in files you transfer to other systems.

It does have one use though. If you forgot where you got a file from, you can recover the URL.

Addendum: Here are some situations on Linux showing whether the extended attributes will be transferred and potentially revealed to others.

• Email as attachment: Extended attributes are not included. But see archive files further on.
• Zip archive: Extended attributes are not captured.
• Tar archive: Extended attributes could be copied, depending on command line options and the compiled default. It seems to be not on by default on my system (openSUSE).
• Transfer on DOS or Windows filesystem, e.g. flash storage: As far as I can tell these extended attributes are not stored on FAT or NTFS.
• Transfer on CD/DVD: ISO9660 can store extended attributes but mkisofs doesn't seem to store these despite mention of GETXATTR and SETXATTR error message suppression. I haven't found information on UDF regarding extended attributes yet.
• Transfer on external storage with Linux filesystem: This depends on whether the filesystem has user_xattr enabled and it is mounted with this enabled, and the copy command transfers it. cp -p doesn't but cp -a does. rsync without -X doesn't.
• Uploaded to the cloud: This would depend on the client doing the upload. Also see here.

More testing is required. Send additions and corrections by commenting on this blog post.

The true origin of "the NUXI Problem"

A bit of computer history today.

The Cathedral and Bazaar website defines the NUXI Problem and various other web pages have elaborated on how it is due to endianess of the CPU interacting with data transfer.

All well and good, but the name given to the problem dates from before data transfer. In those days, telecommunications was slow and data transfer was by magnetic tape. So it wasn't due to encountering problems with FTP or something like that, Internet Protocol had not come to Unix yet.

Rather it was named after the glitch observed on the console upon booting up a port of Unix V6 to, my unreliable memory tells me, the Interdata 7/32. The ports were done independently at two places, Bell Labs and University of Woolongong. Here is the story from the veteran of the UW port.

The PDP/11 is little-endian and the recipient of the port was big-endian so when Unix V6 booted up, the message on the console was supposed to be:

UNIX V6...

which came out as:

NUXI...

Obviously a fix had to be applied to the order of bytes within words in compiler generated code for C strings.

I don't know which team encountered this symptom first. The tale was told to me by Piers Lauder, the Unix guru at the time at Basser Dept. of CS at the U of Sydney.

Ancient EPROMS: 2516, 2716, 2532, 2732

I needed to burn a 16kb (2kB) EPROM as part of a restoration project on an old piece of equipment. When I couldn't burn the EPROMs on my TL-866 programmer, I started investigating. Here's what I found out, summarised for your benefit.

Programming voltage

Practically all EPROMs of this era require 25V programming voltage. That's why my TL-866 couldn't do it, it failed on the first byte, it can only go up to 21V. I can understand why it's not supported, only a tiny fraction of chips require 25V programming. Maybe your programmer supports 25V. However some parts suffixed with A are 21V programmable so try that voltage first. I didn't have any A parts.

Pinouts

This article goes into detail about the differences. To summarise:

For the 2716, the Texas Instruments equivalent is 2516. The TI 2716 is a different beast, fortunately quite rare.

For the 2732, this time TI came out with the 2532, with a different pinout from the more widespread 2732. So TI 2532 != other 2732. Again my programmer cannot program 2532s, not just insufficient voltage but different algorithm too. If you need to only read them, say you want to put 2732 EPROMs in a machine using 2532 EPROMs, it's possible to make an adaptor that reroutes three pins. The article is only for subscribers, but if you click on the PDF image of the PCB, it will be obvious what they have done to the three pins 18, 20 and 21.

From the 2764 onwards, TI fell in line.

EEPROM substitutes

So I had no programmable EPROMs in my spares box, but fortunately I had a Xicor 2816 which is an EEPROM with pinout compatible with the 2716. It speeds up my development too as I don't need to go through the UV erase step.

28pin to 24 pin adaptor

If you can't get hold of a suitable chip or programmer, another way out is to build an adaptor using two IC sockets and patch wire so that you can use the much more widespread 28 pin EPROMs (2764 and above). Fortunately I don't have to do this (yet).