Installing Linuxmint 18 on RAID 1

TL;DR: Can't be done easily.

I had a PC with dual disks that I was running under openSUSE Leap 42.1 in RAID 1. I wanted to install Linuxmint 18 on it before giving it away.

First problem I encountered was that the partitioner in the installer knows nothing about RAID.

After some reading I found that the solution is to use gparted to create the RAID partitions first. But gparted didn't know how to do this. I figured out that the mdadm tool was missing, so I did:

apt-get install mdadm

Note that this installs to RAM as it's a live filesystem so needs to be redone if the install is restarted. Also there are some post-install script errors but mdadm gets installed.

Now with gparted I could create the RAID partitions and assemble them. Alternatively it can be done from the command line:

mdadm --assemble --scan

Now md0 and md1 appeared on the list of "disks" in the partitioner and I could assign them to / and /home.

The system installation went fine until it was time to install GRUB. It failed when it couldn't do

grub-install /dev/sdb /dev/sda

Changing the target to just /dev/sda and clicking Continue did nothing. This seems to be an installer bug.

So I did this from the command line:

mount /dev/md0 /mnt
grub-install --root-directory=/mnt /dev/sda
umount /mnt

When I rebooted from disk it stopped at the GRUB prompt. So I tried to boot manually.

grub> linux /boot/vmlin.... --root=/dev/md0
grub> initrd /boot/initrd...
grub> boot

The ... are where you should use TAB completion to select the kernel and initramfs images respectively. Unfortunately this failed when systemd tried to mount the root partition. The reason is that the raid modules are not present in the stock initramfs file. This was built by the Linuxmint 18 distributors.

At this point I gave up. Obviously Linuxmint 18 isn't designed to support this kind of RAID installation. If you want to go further you should rebuild the initramfs with the RAID modules included. Do let us know if you overcome this next hurdle.

And remember if you get it to work, you should install the mdadm package permanently, as it's not part of the default package set.

RAID UUID != filesystem UUID

I had copied a RAID filesystem onto larger disks using a second computer and wanted to mount it on the original computer with minimal disruption. To this objective, I made the UUID of the new RAID array the same as the previous RAID using the assemble command of mdadm, like this:

mdadm -A /dev/md1 -U uuid --uuid=<old RAID UUID> /dev/sdb3 /dev/sdc3

(If the array is currently assembled, you have to disassemble it it first with mdadm -S /dev/md1)

This worked fine and I was able assemble the RAID on the second computer using the /etc/mdadm.conf from the first computer.

At the same time I decided to change to mount by UUID, so I edited /etc/fstab to look like this:

/dev/disk/by-uuid/<RAID UUID> /home xfs default 1 2

Alas when I booted, the RAID array wouldn't mount and I had to reboot in rescue mode to fix it up.

I wondered why the discrepancy between the symlink to /dev/md1 in /dev/disk/by-uuid and the RAID UUID. Eventually after some red herrings I understood. RAID UUID != filesystem UUID. They are distinct UUIDs. The filesystem UUID is that of the XFS (or ext4, btrfs, whatever) filesystem on it and is what is reported by blkid when udev runs it to populate the by-uuid directory. On the other hand the RAID UUID is what mdadm uses to match against entries in /etc/mdadm.conf when assembling the array.

The second lesson I learnt (again) is that you should not change two things in your config at once. I should have booted the machine up with /dev/md1 in /etc/fstab and then explored by-uuid later.

I could have kept the random RAID UUIDs generated by mdadm for the new disks and just run mdadm --examine --rescan > /etc/mdadm.conf in rescue mode to rebuild the metadata, but I want to be able to swap in the old RAID disk in an emergency and it's one less step to make the RAID UUIDs the same.