Ok, so no posts for a couple of weeks. Why? Well I’ve been banging my head against various ZFS appliances, trying to make them do what I want 🙂 What I want is to migrate my old HP Microserver over to new hardware, and in the process, change out a bunch of the network services from custom Linux VM’s to something more Appliance-based, with Web management interfaces. Of course, sounds easy, but in practice, its turning out to be a nightmare. I’ve spent a couple of weeks fiddling with FreeNAS, NAS4Free, Nexenta(stor), and OpenIndiana/Napp-IT, coupled with Citrix’s XenServer, and Centos 6.4 (which now officially supports Xen 4.3). What I want to do is run a VM with:
- 2-4 cpu cores
- 4-8Gb of memory
- PCI Passthrough of LSI 9211-8i HBA disk controller (Dell Perc H310)
For the VM storage on the dom0, I’ve got a couple of RAID1 mirror pairs, one of SSD’s and one of 2.5″ SATA, so they’re reasonably fault-tolerant and I can use fast/slow storage as desired. The intent with the ZFS is to use it for network storage (NFS, Windows Shares, etc.), and enjoy simplified backups, error-detection and management, as well as provide common repositories for some of the other service VM’s to share (e.g. my FLAC music collection). Initial testing has got me to successful installs of domU’s for all of the above, although only Nexenta and OI have successfully managed to run as Paravirt domU’s.
- FreeNAS and NAS4free performance has been 3-legged-dog-like, largely due to running as HVM and using simulated IO controllers (disk/network).
- PCI Passthrough has failed to work on the Paravirt installs of OI and Nexenta (as OpenSolaris domU support doesn’t include the Pci-frontend drivers) but passthrough of individual disks and network performance are near-native.
So right now I’m stuck choosing between:
- poor IO performance creating massive CPU load on HVM instances, but sole control of LSI hardware
- good IO performance, but unable to dedicate the LSI controller to the storage domU.
I do have one more trick up my sleeve, but I need to spend some time fiddling, which is to use a combination of PV/HVM with the FreeNAS or NAS4Free setups, so I get the benefits of paravirt IO performance, and the convenience of PCI Passthrough. This is certainly possible with FreeBSD – so I’m hoping I can do it with these FreeBSD-derived Appliance setups, as if I’m forced to do it with FreeBSD, I may as well just go with ZFS-on-Linux. More on this as I get further with it, or hit roadblocks!