The migration from ESXi to proxmox continues…
I have a large drive I’ve been using for backups. On ESXi, I had it installed in an external USB dock which was passed through to a VM. On my new server, I have hot swap SATA bays so I decided to eliminate the USB overhead and install it directly. I found the instructions to pass through a SATA drive, basically:
set <vmid> -sata<n> /dev/disk/by-id/<drive string>I typed that command, a drive was added to my VM. I saw it in drive manager. BUT no drive letter. I rebooted. Still no drive letter.
I looked close in drive manager. Instead of the partitions I knew were on the disk, I saw only a protected gpt partition.
Searching around I found the most common cause was some quirk of USB drives. Apparently a lot of them like to fake a 4k sector size for historical reasons. No problem says I. I make a backup of my backup drive, repartition the drive in Windows and it looks good.
But then I switch to Linux and take a look at the formatting. It’s trash. testdisk can make sense of it but it’s clear that something is still off.
I discovered that Proxmox (QEMU in general) doesn’t really pass drives through like ESXi does. It basically adds it’s own virtualization layer over the drive and that’s what the guest seems. It likes to use 512 byte sectors. It’s not too hard to request 4k sectors (like my drive uses natively) to QEMU but Proxmox doesn’t have such an option.
I found this page which described how somebody got creative in ramming the required options through to Proxmox. I just want to write down the process I used to find what needs adding since the commands I needed were very different than the ones he used.
First, I took a snapshot of the qm commands I had before adding the drive:
qm showcmd <vmid> >without-drive.txtThen I added my drive as I mentioned previously.
set <vmid> -sata<n> /dev/disk/by-id/<drive string>
qm showcmd <vmid> >with-drive.txtI then used ccdiff to find the additions and pasted them into a text file, sata<n>.txt. Be careful selecting the commands – ccdiff can be a bit naive in picking the character differences. These are the commands Proxmox adds to QEMU to get the drive added.
I then unlinked the drive from my VM:
qm unlink <vmid> --idlist sata<n>To add the drive back, I used:
set 103 -args "$(cat sata<n>.txt)"I then booted up the VM to check whether I was on track. As expected, the drive was passed through but still had the wrong sector size. Time to add the the options to set the sector size: logical_block_size=4096,physical_block_size=4096. When I looked at the commands in sata<n>.txt, I found 2 devices. One was a virtio and the other a ide-ht. These options go on that second device.
After editing the sata<n>.txt file and running the set args command again, I booted up the VM. The drive had the correct sector size.
BTW, you can clear out the args by calling the same command with an empty argument if you’re done with this hack:
set 103 -args ""One annoying thing is that the drive doesn’t show up on the Hardware tab in Proxmox. It doesn’t know what these commands do. I added a note on the Summary tab pointing me back to this blog post so I remember what I did. Hopefully Proxmox will support this at some point in the future.
P.S. The mount option -o big_writes really speeds up writing to NTFS drives from Linux.