Get gitlab browsing statistics for your project

Unlike github, gitlab doesn't provide a web page to see the browsing activity on your project. But there is an API to fetch this information in JSON format. I explain how it's done so you can avoid the pitfalls I encountered.

In the documentation, all the API calls are with respect to a base URL, which is currently:

https://gitlab.com/api/v4

This StackOverflow post confirms this base URL. It also shows how the access token is passed to the GET request, as an extra header.

Getting a personal access token is described here with screenshots. You can set an expiry date for the token. Make sure you save a copy of it as you cannot see it again once generated.

Now if you look at the API documentation, the relative URL to get the statistics for the last 30 days is described here and is:

/projects/:id/statistics

This is where I got fooled. The : is not to be entered literally. Instead substitute the numeric ID of your project.

Putting it all together, here is the curl command to get the statistics for a fictional project ID of 1234.

curl -H 'PRIVATE-TOKEN: yourtokenhere' https://gitlab.com/api/v4/projects/1234/statistics

And here is typical output:

{"fetches":{"total":3,"days":[{"count":3,"date":"2020-11-23"}]}}

There is no newline at the end, JSON doesn't need it. But you'll be using some other utility to parse and display the statistics in a user-friendly form.

Now you can use this information to get all the other information exposed by the API. Some relative URLs don't require authentication if the project is public.

Upgrading unetbootin to remove qt4 dependency

When my openSUSE Leap 15.2 showed a libqt4 update I wondered what application needed these old libraries. Removing libqt4 removed a bunch of library dependencies and the one application that appeared was unetbootin. A search showed that Ubuntu 20 users had encountered missing libqt4 problems too, and the official solution is to upgrade to unetbootin 681. Even though libqt4 is still supported in Leap 15.2 I decided to upgrade. This can be done as a one-click install from software.opensuse.org and the "experimental" package is from the filesystems repo. I guess the next release of Leap will make unetbootin 681 official.

As a bonus this freed up about 150 MB of disk space now that nothing needs qt4 any more.

Ripping CDs to AAC (M4A) files

Since MP3 is officially obsolete and AAC is a better format, I have decided to rerip all the CDs I own to this format.

This is quite easy with k3b,  you just have to add another external encoder specification. I got most of the info I needed from this blog post. One problem was ffmpeg was spitting out too many messages, so I did this:

ffmpeg -v 24 -i - -b:a 256k %f

The -v 24 is to suppress the normal informative output from ffmpeg which is quite verbose.

I also wanted to add tags, so after ripping I run this kid3-cli command in the directory:

kid3-cli -c 'select all' -c 'totag 3' -c save -c exit

This puts the tags implied by the filenames into the tags.

Don't forget to specify Write WAV Header as the original blog post says.

Accessing Samba 4.11 shares from old clients, e.g. XP, Android apps, media players, etc

NB: Please do not do this if you are running a work network, as SMB1 has insecure authentication, and traffic is in clear. Get your clients updated. I did this only because I run Samba on a home network serving only a few of my applications, and only for legacy clients.

When I upgraded my Linux distro I discovered that starting with Samba 4.11 the minimum protocol support level has been raised to SMB2 by default. This made the shares I export inaccessible to an XP virtual machine driving a scanner and a chip programmer, and also an app on my Android tablet.

There are lots of confusing posts on the Internet some about older versions. This relates to these changes. You can still enable SMB1 support, using these lines in smb.conf:

min protocol = NT1
ntlm auth = Yes

Restart nmb and smb and the changes should take effect.

SMB1 support may be removed in future, so hopefully my applications will be also updated before then.

Discrepancies between CRMs and mailing lists

For a while now I have been updating my email address on online accounts and have discovered how sloppy the connection between customer details held in CRMs and email addresses in mailing lists are. Also how terrible some CRMs are.

Exhibit 1: I get a newsletter. There is a link at the bottom to update my preferences so I click on it and update my email address in a form. If they did it right there is a verification step. Otherwise one risks getting locked out if one enters a typo. Later on I still get email at the old address.

1a. It eventually resolves because the address list is only refreshed from the CRM at intervals. Not ideal but at least it works.

1b. I still get email at the old address. Evidently updates not set up correctly. Now I have to contact IT. Or unsubscribe and resubscribe with a new address.

1c. I get email at both the old and new addresses. This is terrible. Are the software developers or system admins competent?

Exhibit 2: I update my email address in the account. Later I get a newsletter to the correct new address. I click on the link to update my preferences. But the page is still showing my old address. When I try to change that to the new address it says it's taken, as it should be. Again some discrepancy between the CRM and the mailing system.

Exhibit 3: I update my email address in the account. But I cannot change the login, it has to be the first address I chose, forever. How silly.

Exhibit 4: I try to change my email address (and login) in the account. But it's read-only. A query reveals that I have to create a new account and ask IT to transfer all details, in particular any purchase history, to the new account. Aren't CRMs supposed to take care of this? Or is it to keep IT in work?

Exhibit 5: I try to change my email address (and login) in the account. Again it's read-only. A query reveals that it can only be resolved by contacting IT to modify it manually. A job preservation scheme. Sometimes the contact is not easily discovered via the website. Sometimes you have to post to a forum so that a moderator will notice.

Exhibit 6: Not really an email update issue, but I used to get newsletters that were sent by sales department of a legitimate company. But they were labelled as spam. Investigation revealed that the mailout was outsourced but they didn't register the outgoing servers in SPF. I tried to tell them about it, but they're still doing it and probably losing sales.

Putting an idle WiFi router to use as a LAN-WLAN bridge

In the current situation I have time to do the little chores I promised myself to do.

One was to set up an unused WiFi router as an auxiliary access point. In parts of my abode, the WiFi signal from the main router isn't strong, and besides it only handles the older standard. I have a spare router which handles the latest WiFI standards which I decided to put to use.

You may ask at this point why not swap the unused router in? Two reasons, the old router has DD-WRT on it and I have a particular set up for it, and the new router has gigabit Ethernet ports so I can use it as an Ethernet switch at the same time, and not have to run another Cat-6 cable to that area. Finally I use the Internet more from the new location.

Note that this is not about setting up a WiFi relay. You can buy such devices off the shelf to boost the signal. This is using a spare WiFi router as a LAN-WLAN bridge so you need to have wired Ethernet already. It also doesn't matter what the front end of the router is, whether ADSL or cable, as we don't use it at all. So ignore any Ethernet ports that say WAN or Internet. We only use the ones that say LAN or Ethernet. Here's a typical back panel. We want the yellow interfaces.

These are the steps you need to do:

1. Reset the router to factory settings.
2. Connect to it with a computer, usually a laptop, at the factory IP address and login.
3. Disable the DHCP server of the router.
4. Change the LAN address of the router to a free IP address in your subnet that isn't in the dynamic range of the main DHCP server.
5. Reconnect to the router at the new address.
6. Set up the WiFi using the same SSID and passwords as your main router.
7. Check tnat WiFi clients connect to your router.
8. Backup the configuration to a file on your computer.

Let's go through each of the steps.

1. Reset the router to factory settings

Usually this is done by poking a paper clip into a recessed reset button on the back or bottom of the router.

2. Connect to it with a computer, usually a laptop, at the factory IP address and login

Connect with an Ethernet patch cable and access the web interface. You need to know the factory default IP address and login/password. The manual will tell you or it may be on a sticker on the router. Often the address iis 192.168.0.1. You could either let the laptop get an address by DHCP, but since you will need to change it in step 5, you might as well learn how to configure it to a static address. Change the admin password after you have logged in.

3. Disable the DHCP server of the router

Somewhere in the settings, probably advanced, is where you turn off the DHCP server of the router. This is so that it won't clash with the main router. In my case it wasn't sufficient to disable it. I also had to shrink the range to a small one or it would say illegal address at the next step.

4. Change the LAN address of the router

You need to assign the router a fixed address that is on your LAN but not in the DHCP range of the main router. Say your main router gives out addresses from 192.168.100.100 to 192.168.200.200. In this example .99 or .201 is available. After you have changed this, your laptop cannot reach the router, so proceed to:

5. Reconnect to the router at the new address

For this you can either change the IP address of your laptop's Ethernet interface, or switch to the main router's WiFi which will be on the same subnet. For the former you would take another address on your LAN, say 192.168.100.202. The router address you give to your browser is for the last example: http://192.168.100.201

6. Set up the WiFI using the same SSID and passwords as your main router

Otherwise your devices will be confused which one to log onto. Naturally you should use the same standard, which is usually WPA2-AES. At the point you may wish to choose the wireless channel for least overlap with your neighbours, but if you are close enough to the router, it's not as important as your signal will be stronger. For this tuning task there are various WiFi analysis apps.

7. Check that WiFi clients connect to your router

The router may have a page that shows the WiFi clients connected to it. You should see entries in it when your phone or tablet connects.

8. Backup the configuration to a file on your computer

Most routers have a web page where you can download a dump of the configuration so that you can restore it quickly if you ever do a factory reset again.

The other Ethernet ports of the router (usually 3 left after you have used one) can be used to plug in other wired devices. In this case I connected an existing desktop to a free port, using the router as an Ethernet switch.

That's it. Enjoy faster Internet access from your secondary location.

How to convert text from one charset to another within vim

Sometimes when you edit a file with vim it sees the extended characters and guesses the charset of the file. If it isn't UTF-8 it usually guesses Latin-1 or ISO8859 and displays (converted) on the message line. In the case of a file of mine that had Chinese characters encoded in GBK, when I tried to convert the contents in situ with:

:1,$! iconv -f GBK -t UTF-8

I ended up with a buffer displaying the Chinese characters correctly, but when trying to write it out, vim indicated an error. Trying to force the write with w! gave me the original file.

The trick is you have to tell vim that the charset has changed. Do:

:set fileencoding=utf-8

before you write and quit and all will be well. You can check the current encoding at any point with:

:set fileencoding