Reliably recording Internet radio programs

In a previous blog, I've explained how to use mplayer to dump Internet radio programs to a file.

In practice the recording sometimes terminated before the time was up, which could be due to the server breaking the connection, Internet glitches and so forth. Interactive station tuners will attempt to reconnect. We want to do the same with our mplayer scripts.

Here is a solution I came up with, which I will explain below the code. Some code before the main code has been elided. Assume that the bash script is called with 3 arguments, the duration, the destination file, and the streaming URL.

#!/bin/bash

pipe="/run/user/$(id -u)/mplayerpipe$$"
READER_PID=''
MIN_DURATION=30
STRETCH_TIME=15         # seems to lose time when a new segment is started

cleanup() {
       rm -f "$pipe"
       # don't use builtin kill so that we can send to process group
       [[ -n "$READER_PID" ]] && echo "Killing group $READER_PID" && /usr/bin/kill -TERM -- "$READER_PID" $(pgrep -f "cat $pipe")
}

if [[ "$1" -lt "$MIN_DURATION" ]]
then
       echo Minimum duration "$MIN_DURATION" seconds
       exit 1
fi
trap cleanup EXIT SIGINT SIGTERM
mkfifo -m 600 "$pipe"
coproc READER { exec bash -c "while [[ -p '$pipe' ]]; do cat '$pipe'; echo 'Restart reader' 1>&2; done" > "$2"; }
now=$(date +%s)
later=$((now + $1))
left=$((later - now))
while [[ "$left" -ge "$MIN_DURATION" ]]
do
       # echo and sleep for testing
       echo mplayer -prefer-ipv4 -noconsolecontrols -slave -vo null -vc null -endpos "$left" -dumpaudio -dumpfile "$pipe" -really-quiet -loop 0 "$stream"
       # sleep 10
       mplayer -prefer-ipv4 -noconsolecontrols -slave -vo null -vc null -endpos "$left" -dumpaudio -dumpfile "$pipe" -really-quiet -loop 0 "$stream"
       now=$(date +%s)
       left=$((later - now + STRETCH_TIME))
done
sleep 2

The idea is we establish a named pipe in the standard location /run/user/<uid>/and mplayer writes to this pipe. At the same time we start a coprocess which is a bash inline script calling cat repeatedly to append to the destination file. Coprocesses are explained in the bash man page. Here we call it READER and its pid is put in READER_PID by convention. Note that we exec the bash coprocess so that the pid is that of the code fragment, not the bash parent.

When mplayer terminates prematurely, the amount of time left to go is computed and it's restarted with that duration. The cat process will receive EOF, and new one is started to continue to read from the pipe.

We need to clean up the named pipe at the end of the program so we install a signal handler cleanup() to do that. It removes the named pipe, and kills the group pid, which the kill command says can be done by prepending - to the pid (making it negative). Note that for this behaviour we use the kill command in /usr/bin/kill, not the bash builtin. However I couldn't get the group kill to work so I settled for using pgrep to find the cat process to get the pid.

You may ask why not try to trick mplayer into appending to a file by >> output redirection, then passing /dev/stdout as the dumpfile argument. It doesn't work. /dev/stdout is a symlink to the real file and mplayer will open it in overwrite not append mode.

After I wrote that script, I decided to try another tack. The script relies on Linux named pipes, so isn't cross-platform.

Here is the second attempt, which is a Python program. I'll show only the main routine, the needed auxiliary definitions and functions can be inferred.

def main():
   """ Main program """

   mplayer_cmd = sys.argv.copy()
   mplayer_cmd[0] = MPLAYER

   # find index of -dumpfile argument
   dumpfile_pos = find_index_of("-dumpfile", mplayer_cmd)
   if dumpfile_pos < 0:
       sys.exit("-dumpfile argument not found")
   dumpfile_name = mplayer_cmd[dumpfile_pos]
   insert_pos = dot_index(dumpfile_name)
   base = dumpfile_name[0:insert_pos]
   tail = dumpfile_name[insert_pos:]
   seq = 'aa'
   mplayer_cmd[dumpfile_pos] = first_name = base + seq + tail

   # find index of -endpos argument
   end_pos = find_index_of("-endpos", mplayer_cmd)
   if end_pos < 0:
       sys.exit("-endpos argument not found")
   if not mplayer_cmd[end_pos].isdigit():
       sys.exit("End posiiton not a number")

   left = int(mplayer_cmd[end_pos])  # get original endpos
   now = time.time()
   later = now + left                  # when we should stop
   # substitute endpos
   mplayer_cmd[end_pos] = str(left)
   while left >= MIN_DURATION:
       print(f'{" ".join(mplayer_cmd)}')
       before = time.time()
       # for debugging just sleep
       if TESTING:
           time.sleep(10)
       else:
           result = subprocess.run(mplayer_cmd, check=False)
           if DEBUG:
               print(result)

       if seq == 'zz':
           print("Can't handle more than 676 segments")
           break

       now = time.time()
       if bad_session(mplayer_cmd[dumpfile_pos], seq, before, now):
           continue

       # set up for next session
       seq = next_segment(seq)
       # substitute dumpfile
       mplayer_cmd[dumpfile_pos] = dump_name(base, seq, tail)
       # substitute endpos
       left = round(later - now) + STRETCH_TIME
       mplayer_cmd[end_pos] = str(left)

   # gather all the segments
   # rename first segment
   if DEBUG:
       print(f'{first_name} -> {dumpfile_name}')
   try:
       os.replace(first_name, dumpfile_name)
   except FileNotFoundError:
       print(f'Warning: {first_name} not found')
   # append remaining segments
   if seq > 'ab':
       append_segments(base, seq, tail, dumpfile_name)


if __name__ == '__main__':
   main()

We write a series of files called <basename>aa.aac to <basename>zz.aac and at the end of the program we rename the first one <basename>.aac and append all the others to it. Up to 676 segments are allowed. Hopefully you will not have so many connection breaks. All the string manipulations are to find the endpos and dumpfile arguments in the original invocation and replace those with new values for each run of mplayer. Thus this script which I called mplayer_retry is a plugin replacement for the original mplayer invocation.

This script has the advantage of not requiring more than standard file operations, nor named pipes, so is cross-platform.

Unfortunately it seems that in practice, some seconds of program are lost when the connection breaks. Only thing for it is to complain to the service provider to have more a robust streaming service.