Thursday, May 4, 2017

Today is the day I hopefully finish the migration to Turbo.

Still some issues with Shuttle

I still have some issues with Shuttle. I've been trying to copy VM images over to Turbo, and it chokes everytime on the win10.qcow2 image. Each time the machine reboots. I'm suspecting a kernel panic, and since it happens during file copies, it's likely to be related to the filesystem. Yet another BTRFS issue?

I had some pretty interesting comments on my report about the BTRFS corruption. In particular, the fact that the qcow2 file format combined with copy-on-write leads to heavy filesystem fragmentation. And indeed, this is what I observed:

fedora25.qcow2: 28358 extents found
mac_hdd.qcow2: 79493 extents found
ubuntu14.04-64.qcow2: 35069 extents found
ubuntu14.04.qcow2: 240 extents found
ubuntu16.04-32.qcow2: 81 extents found
ubuntu16.04-64.qcow2: 15060 extents found
ubuntu16.10-64.qcow2: 228 extents found
win10.qcow2: 3438997 extents found
winxp.qcow2: 66657 extents found

The number of extents on the win10 machine (over 3M for a VM that was just installed) is just frightening. This is also the VM file that I have trouble copying.

But it's not always at the same spot:

 12,855,410,688  14%   72.91MB/s    0:16:19  packet_write_wait: Connection to 192.168.77.55 port 22: Broken pipe
ddd@turbo VMs> rsync -av --progress shuttle:/var/lib/libvirt/images/* .
receiving incremental file list
win10.qcow2
 71,308,574,720  82%   75.34MB/s    0:03:10  

Fragmentation of win10.qcow2 is nowhere as bad as on Muse:

win10.qcow2: 155203 extents found

Still not good.

After seven unsuccessful attempts, I decided to give up Shuttle for good for now, and transferred the TerraMaster to Turbo. After all, this is where I'll run most of my VMs. Then I'll try to convert all my VM images to raw files to avoid fragmentation and maximize performance.

An example to create a better backing store file would be:

touch windows.raw
chattr +C windows.raw
fallocate -l 50g windows.raw

Copied files over from the old Muse disk, which curiously still shows errors but now accepts to mount read-write.

Spice work

One of the patches I had for macOS enablement was related to building with clang. This issues warnings about misalingment that are fatal (-Werror). I had a workaround, but Victor Toso pointed out that this had been discussed before. So I decided to make another pass, taking into account the comments Christophe Fergeau had done at the time.

I ended up adding code to actually check for misalignments and report them.