I recently upgraded tesla, my primary Linux desktop, to one of the new Ryzen processors from AMD.  So far it’s working very well, and the performance is sometimes even rather stellar.  One of the big performance boosters, in addition to the Ryzen CPU and doubling the memory, was I went SSD in a big way.

Now, it’s important to note, that I have a rather large music collection and some videos, a crap load of pictures, and other media I don’t want to store on SSD as it would be prohibitively expensive at this point in time.  So, the system, before upgrading had only a old smallish SSD that only had /boot and / on it.  And a 4 TB platter that had /var, /home, and swap on it.

The new configuration has a M.2 drive with /boot, /boot/efi, and / ( including /var this time ) on it. There is also a SATA 3 SSD with /home on it, swap, and what I’ll call /home2 is on the platter.

An aside note about swap. Yes, tesla now has 32 gigs of ram, but the way PC processors, and Linux memory management works, it’s better to have some swap than none at all, even if you don’t expect it to be used. General rule of thumb, is 2x Memory to 8 Gig, then same as memory until you go over 16 gig, the 16 gig. I had 16 gig memory, and the new on will have 32, so I already have a perfectly good 16 gig swap partition that is being used.

And a little disclaimer: This stuff worked very well for me. It may not for you. Make backups before trying ANY of this! I have backups. This was not a concern for me. If you don’t have backups and loose data, it’s NOT my fault. You’ve been warned.

Now, for the pre-work. The VG is the default fedora. I like to keep everything in on VG. It makes life easier.  First off, the home LV needs to be renamed to home2.  So boot the system into single-user mode, and rename the LV:

lvrename /dev/fedora/home /dev/fedora/home2

Next, the VG has to be exported, since its fedora, the system has to be booted into recovery mode, then export it:

vgexport fedora

This is what I came up with for a  filesystem layout:

As I was putting it all back together, I left the 4TB disk disconnected and put the M.2 drive and left the other SSD unplugged to prevent confusion during the install. I did a normal XFCE spin install, except I did a custom disk config only using an LVM scheme and only putting /boot, /boot/efi, and / on the M.2 drive as described in the table above.  A note here, if I plugged the 4 TB disk in at this point, the system would not boot.  You’ll have to plug it in hot when the time comes. Not ideal, but it worked for me.

Once that was done, I connected the SATA SSD for /home, and added it:

fdisk /dev/sdb
pvcreate /dev/sdb1
vgextend fedora /dev/sdb1
lvcreate -n home -l 100%PVS fedora /dev/sdb1
mkfs.ext4 -j /dev/mapper/fedora-home

That was the easy part, now to import the 4 TB disk, as I really don’t want to have to restore from disk.  Plug it in, and read some man pages.  Turns out, it’s actually easy.

First import it the existing PV:

vgimportclone --import /dev/sda1

This gives a VG called fedora1.  Close, but not quite what I want.

So offline it, then merge the two together:

vgchange -a n fedora1
vgmerge -v fedora fedora1

Now I have on nice big VG, fedora, that has all three drives in it. Perfect!

Now a little cleaning up to get rid of the old /var, expand home, and it’s probably a good idea to refresh the swap space:

lvremove /dev/fedora/var 
lvextend -l +100%PVS /dev/fedora/home2 /dev/sda1
resize2fs /dev/fedora/home2 
mkswap /dev/fedora/swap

And finally for this part, edit fstab, make the mount points, and mount everything:

mv /home /home.orig
mkdir /home
mkdir /home2
vi /etc/fstab
mount -a
swapon -a

NOTE: When configuring the fstab, make sure to add noatime to the mount options list for all the SSD’s.  It helps with their life and performance.

Next up: configure fstrim. This is as easy as making a script that runs */usr/sbin/fstrim -a -v* and putting it in /etc/cron.weekly.  There are other ways to do this, but this seems to be the preferred.  Then set issue_discards = 1 in /etc/lvm/lvm.conf. That should be picked up on the next reboot.

Now set up the elevators.  This is done with a script that sets the deadline queue in */sys/block/{ssd}/queue/scheduler*.  Then a systemd unit is created to run it at boot time.

You can find the scripts and systemd units as well as my script to run fstrim and other possibly interesting things on my system GitHub repo: https://github.com/lorddoomicus/system

Remember to enable and start them everything is installed.

And the final piece of the puzzle, using some spare space on the M.2 drive for LVM caching for the platter.  This isn’t that hard, but took some reading to figure it out.

The installer only allocated the space I requested to the PV for root. So the easiest way to fix that is just create another PV on the M.2 device using the rest of the space:

fdisk /dev/nvme0n1
partprobe
pvcreate /dev/nvme0n1p4
vgextend fedora /dev/nvme0n1p4

Now to create the cache pool and meta device.  Since I’m working on a 4 TB disk, it seemed to make sense to make the cache pool 40 G.  I made the Meta 256 Meg, probably over kill ( like my /boot/efi ), but I don’t want to have to worry about resizing it in the future:

lvcreate -L 256M -n home2_cache_meta fedora /dev/nvme0n1p4 
lvcreate -L 40G -n home2_cache fedora /dev/nvme0n1p4 
lvconvert --type cache-pool --poolmetadata fedora/home2_cache_meta fedora/home2_cache

Now just attach it the LV:

lvconvert --type cache --cachepool fedora/home2_cache fedora/home2

And after that, just give it a reboot and make sure everything comes up as expected.

You can use vgdisplay, vgs, and pvs to verify everything is the way you want it.