Displaying problematic characters for Windows like the colon in Samba shares

I have a Samba share which contains music files which I want to play on my tablet (but could also be a Windows desktop). The colon is perfectly valid in Linux filenames and is used a lot in MP3 filenames. On the client side it falls back to DOS mapping because the colon is not legal in Windows filenames. So it's impossible to see what the song is. Here is how you can make the displayed names more readable.

Add these lines to smb.conf, either in the global or a share section:

vfs objects = catia 

catia:mappings = 0x22:0xa8,0x2a:0xa4,0x2f:0xf8,0x3a:0xf7,0x3c:0xab,0x3e:
0xbb,0x3f:0xbf,0x5c:0xff,0x7c:0xa6

And for good measure:

mangled names = no

Those changes map problematic characters to extended characters. For example colon : is now displayed as the division sign ÷ It's not ideal but at least you can read the filenames now.

Credit to:  https://unix.stackexchange.com/questions/299130/samba-how-to-display-files-with-colon-in-their-names/381639#381639

Reducing the size of virtual machines exported from VirtualBox

VirtualBox can export virtual machine images which is handy if you want to give someone a ready to use OS for example.

I had an old scanner which I was driving under XP, using the USB forwarding feature. I wanted to give a friend the scanner and an OVA image containing the ready to go OS. The problem is that unused blocks on the NTFS filesystem will affect the compression of the image. Even though I had defragmented the virtual disk and moved the used blocks to the front using a pre-8.0 version of Ultradefrag, the unused blocks after the used area still contained old random data.

Enter sdelete. This is a free utility from Microsoft that wll zero unused blocks. You use it like this:

sdelete -z c:

I ran this once on the virtual disk, then shutdown the virtual machine and exported it to OVA format. The result was quite satisfactory; what was once a 5.1 GB dump turned out more like 2.1 GB.

OpenVPN newer versions require new configuration to use longer Diffie-Hellman keys

I found that after an upgrade, my openvpn setup no longer worked with my client. The message in the log file was:

OpenSSL: error:1408518A:SSL routines:ssl3_ctx_ctrl:dh key too small

Long story short, you need to use dh2048 keys now. Run the command in the sample server.conf file in the sample-config-files of your OpenVPN distribution to generate a dh2048.pem, put it in the same directory where you have dh1024.pem and edit the config file for the parameter dh.

Other changes which I needed to make which you might or might not were:

I turned off comp-lzo compression as suggested. This change also needs to be made in the client config files.

I had the parameter: cipher AES-256-CBC Just let it autonegotiate to AES-256-GCM now. If you wish, turn that paramter to data-ciphers-fallback if you wish AES-256-CBC to be still considered.

Fortunately, aside from disabling comp-lzo, I didn't need to regenerate the client keys or change the rest of the config.

Prefer IPv4 addresses to IPv6

This started when I could not refresh a repo on my Linux installation. Doing a manual wget --spider on the metadata file that it could not retrieve revealed that it was trying to connect to the IPv6 address of the mirror site.

My Internet provider does offer a dual stack IP service but I have not activated the IPv6 portion yet. So the question was how to make sure any http/https clients prefer the IPv4 address.

A web search gave the answer, you edit the file /etc/gai.conf which controls the behaviour of the getaddrinfo(3) function. In it you will find a comment to uncomment the line:

precedence ::ffff:0:0/96  100

if you want IPv6 addresses to sort lower than IPv4 addresses. I also reloaded the nscd service to flush any cached entries.

I suspect I hit this problem because I run my own DNS resolver. If you are relying on a DNS relay from your Internet provider they may have filtered out IPv6 answers for the sake of compatibility.

Alias for hexdump for canonical output

The man page for hexdump from util-linux states:

-C, --canonical

Canonical hex+ASCII display. Display the input offset in hexadecimal, followed by sixteen space-separated, two-column, hexadecimal bytes, followed by the same sixteen bytes in %_p format enclosed in '|' characters. Invoking the program as hd implies this option.

This also happens to be my preferred format for working with microprocessor code. However my Linux distro's package chose not to provide a link or alias from hd to hexdump.

No matter, I just made a symlink called hd in my private bin directory to /usr/bin/hexdump and it works as described.

Using fail2ban with systemd and firewalld

I installed fail2ban to watch over an openvpn service on a system that uses systemd (and hence journald), as well as firewalld. These are the changes I had to make. Most of it is derived from this wiki entry.

The filter /etc/fail2ban/filter.d/openvpn.local unchanged from the wiki:

# Fail2Ban filter for selected OpenVPN rejections
#
#

[Definition]

# Example messages (other matched messages not seen in the testing server's logs):
# Fri Sep 23 11:55:36 2016 TLS Error: incoming packet authentication failed from [AF_INET]59.90.146.160:51223
# Thu Aug 25 09:36:02 2016 117.207.115.143:58922 TLS Error: TLS handshake failed

failregex = ^ TLS Error: incoming packet authentication failed from \[AF_INET\]<HOST>:\d+$
           ^ <HOST>:\d+ TLS Auth Error
           ^ <HOST>:\d+ TLS Error: TLS handshake failed$
           ^ <HOST>:\d+ VERIFY ERROR
           ^ <HOST>:\d+ Connection reset, restarting

ignoreregex =

The jail file /etc/fail2ban/jail.d/openvpn.local enables the jail for openvpn and the wiki missed out the .local extension without which it will not be registered:

# Fail2Ban configuration fragment for OpenVPN

[openvpn]
enabled  = true
port     = 1194
protocol = udp
filter   = openvpn
journalmatch = _SYSTEMD_UNIT=openvpn@yourhost.service  
backend  = systemd
logpath  = /var/log/fail2ban.log
maxretry = 3

The important line is the backend = systemd. The journalmatch makes scanning the journal more efficient. Usually the openvpn service has @hostname appended as there could be more than one instance. You could also log it to the system journal but here I use a separate log file.

And finally you need to link this jail to firewalld in /etc/fail2ban/jail.local:

# Do all your modifications to the jail's configuration in jail.local!
[DEFAULT]
banaction = firewallcmd-ipset

This calls firewall-cmd to use an ipset to ban the hosts.

Whole array operations on bash arrays

Today I wanted to take all the elements of a bash array and append /** to each one. This was for a script that syncs selected photo albums to cloud storage using rclone. As you know, rclone syncs directories, so to limit the transfer to a subset of of the directories, I used an --include pattern. Say the albums are cny13 fiji13 misc13 under archived. Then the command required is:

rclone sync archived mydrive:Photos --include "{cny13/**,fiji13/**,misc13/**}"

So the question is how to get this from an array containing:

declare -a ALBUMS=(cny13 fiji13 misc13)

Of course, I could run a small loop where I append /** to each element and accumulate in another array. This does work and efficiency isn't really an issue. But I guess the old APL fan in me was awakened and I wondered if I could transform the whole array in one fell swoop.

I tried:

echo "${ALBUMS[@]}/**"

but this only got me:

cny13 fiji13 misc13/**

The clue was supplied in an online tutorial on bash arrays in which an example was shown of parameter substitution. This in fact works just as well on arrays element by element.

echo "${ALBUMS[@]/%/\/**}"

This substitutes the end of line with /** escaping the leading slash, so we get:

cny13/** fiji13/** misc13/**

I haven't explained how the , is inserted between items, but the entire script using another trick to create a join function shows it:

#!/bin/bash

function join { local IFS="$1"; shift; echo "$*"; }

declare -a ALBUMS=(cny13 fiji13 misc13)
declare -a includes=("${ALBUMS[@]/%/\/**}")

cd ~/Albums || exit 1
albums=$(join , ${includes[@]})

rclone sync archived mydrive:Photos --include "{$albums}"