In my previous blog post about choosing the right dom0, I came to the conclusion that in the face of an upcoming RHEL7 which is based on Fedora 19/20, it made sense to set up my dom0 for using Fedora 20.
Given the similarity between Fedora 20 and RHEL7, it should be the same process for RHEL, except for the fact that RHEL7 beta has dom0 support in the kernel turned off, according to this post on the Centos mailing list. I’m hoping that the Centos people will see fit to provide a centos-plus kernel, or even turn on dom0 support in the default kernel, given that its already in the codebase. What I’d really like is for RHEL7 customers to ask Redhat to enable dom0 support!
So anyway, I digress. Here are the steps I went through to set up a working dom0 on Fedora 20. It wasn’t all smooth sailing for me, as I’m very used to Centos6 (and earlier), so the change to using the systemctl init system was a significant one. Its worth noting that my hardware is the following:
- Xeon X3450 (quad core w/ hyperthreading)
- Supermicro X8SIL-F (with 2 intel NIC’s onboard)
- 32GB RAM
- Intel X25-M 160GB SSD
- LSI 9211.8 HBA card (actually a Dell Perc H310 reflashed with LSI firmware)
- Dual port Intel PRO/1000 PT PCIe nic
I bring it up partly because hey, I like having “real” server hardware for the first time in um.. ever, and partly because you might have some minor variations to the rest of these notes depending on controller cards, network chipsets etc.
First up – installation.. here’s the basics I went with.
- 500 MB /boot partition (using ext4)
- LVM disk with
- 10gigabyte root partition (/)
- 4 gigabyte swap partition
- Lots of free space for VM’s
- Then selected the following software packages:
- XFCE Desktop (because sometimes its nice to have a vnc desktop off the dom0 – mostly for using virt-manager)
- Uncheck the Multimedia and Applications
- Check the “System Tools” and “Administrative Tools”
Once the installer has finished and you’ve rebooted the box, its time to make a few changes to the system to facilitate Xen.
First up, disabling the firewall, so that VNC will work (note – my dom0 sits behind a pfsense firewall and is not accessible from an external network):
systemctl stop firewalld.service systemctl disable firewalld.service
Yep, that’s right, no more “service firewall stop” or “chkconfig firewall off”.. welcome to the future!
Then disable that old chestnut, selinux:
Change the line starting with “SELINUX=” to:
Once that’s done, update the system packages:
yum update -y
And install xen, as we’ll probably need it 🙂
yum install xen
We also need to configure grub2 to load the xen kernel.. yes, more unfamiliar stuff from the older model. However, before we do that, here’s a small modification which improved performance and stability for me by taking away the need to balloon the dom0 memory:
And now lets run the commands to update the grub2 config file, and then set the default boot entry:
grub2-mkconfig -o /boot/grub2/grub.cfg grep ^menuentry /boot/grub2/grub.cfg | cut -d "'" -f2 grub2-set-default <menu entry title you want> (this one: "Fedora, with Xen hypervisor")
Next, disable the infernal NetworkManager service, and go with networking which is more xen-friendly.
systemctl disable NetworkManager.service || systemctl restart network.service chkconfig network on
DEVICE=xenbr0 TYPE=Bridge BOOTPROTO=dhcp ONBOOT=yes DELAY=0 NM_CONTROLLED=no
IPV6INIT="no" <-- I like to disable IPv6 as I don't use it (yet) BRIDGE=xenbr0 <-- this is the important line NM_CONTROLLED=no <-- this is handy in case NetworkManager rises from the dead like the vampire it is
systemctl restart network.service
ln -sf /lib/systemd/system/multi-user.target /etc/systemd/system/default.target
yum install -y libvirt virt-manager ntp chkconfig ntpd on systemctl start ntpd.service
And use contents similar to this in it:
options xen-pciback hide=(02:00.0)(03:00.0)(03:00.1)(05:00.0) install e1000e /sbin/modprobe xen-pciback ; /sbin/modprobe --first-time --ignore-install e1000e install mpt2sas /sbin/modprobe xen-pciback ; /sbin/modprobe --first-time --ignore-install mpt2sas # left from old hardware #install r8169 /sbin/modprobe xen-pciback ; /sbin/modprobe --first-time --ignore-install r8169
rmmod mpt2sas rmmod xen-pciback modprobe xen-pciback
Last round of tests.. first make sure the devices are withheld:
Should now return the devices which are reserved for pci passthrough. Lastly, reboot the box, just to make sure it all comes back up with reserved devices etc. And all done! Well ok, who am I kidding.. now the really hard work begins, installing, configuring and testing the various VM’s that I’ll be expecting to run 24/7 for a couple of years with minimal interruptions.