When I first started running Xen at home, it was on Fedora Core 6, and I was using hardware which didn’t include hardware-level virtualization support. Crazy huh?
Since the first iterations of my home server, I’ve been through Centos 5, and my current home server runs Scientific Linux 6 (SL6) for which I had to recompile a bunch of packages to get Xen working (using Fedora 14 SRPMS), since Redhat dropped Xen support from RHEL6.
As I’m still a big fan of bare-metal hypervisor concept, where even the host OS is running on top of the hypervisor, I’ve turned my attention towards choosing the next environment for my new home server build.
To start, there’s an oldish list of Xen dom0 capable kernels found on the Xenproject wiki. Of the ones available, I narrowed it down to these:
- Centos 6.4
- Opensuse 13.1
- Debian wheezy
- Xenserver 6.2
- Fedora 20
Centos 6.4
Lately, I was glad to discover that the Centos project have taken it upon themselves to bring back Xen support, with official packages and a really simple install, documented here on the Centos wiki. This is great news, and I can report the Centos 6.4 packages are good and reliable – I’ve played around with them for a month or two while building up this next generation home server. The 4-command install is just great, compared to the irritating juggle of rebuilding RPMs and dependencies I had to go through to get my current dom0 up and running 2 years ago on SL6.
OpenSUSE
I’ve had a soft spot for openSUSE for years – so I grabbed a copy of 13.1 and had a play, and I’ve been very impressed. However, it ultimately strikes me as being a desktop-focussed distro, with its well integrated UI and YAST control panels. Big points in openSUSE’s favour is that it comes with Xen 4.3.
The zypper package management is something I couldn’t get comfortable with – I ran into dependency issues, and whilst the fault is probably my lack of time spent reading man pages, I decided not to base my home server on openSUSE for the next couple of years.
Having said which, I will probably play around with openSUSE as a dom0 on my test machine, as there’s a lot to like.
Debian Wheezy
A lot of people really like Debian as a distro, so I had a quick look. I hadn’t looked at Debian since Lenny was on the horizon, and back then thought it looked pretty good. Unfortunately for me, I couldn’t get it to reliably install on my new server build – the bootloader was just not working, after either a net install or the DVD one. I suspect its my Supermicro motherboard’s BIOS at fault, but for now I can only report that I’ve read plenty of posts of people using Xen 4.3 successfully on Debian dom0’s.
Xenserver 6.2
This was a surprise! Citrix have made Xenserver 6.2 open source, and so it entered my consideration set at the last minute. I burned a usb stick (because who uses actual physical optical disks these days?) and installed it, along with the handy windows-based Xenserver tools on my desktop.
I was impressed – its stable, I got pci passthrough working – however its based on a 32bit hypervisor, and I also found it wasn’t using acpi states, so it consumed around 15W more at idle than the exact same hardware setup with a Centos 6.4 dom0 idling on it.
So for now Xenserver is on my “watching future releases carefully” as I liked the whole package and experience, but just found a couple of points which ruled it out for me.
Fedora 20
I’ve generally stayed away from Fedora since FC6, largely because it moves too fast and carries too many instabilities along with it as a result. In my experience, its been fine as a desktop platform, but not something I’d use for critical server infrastructure. I typically install and use every other version of Fedora.
This time around however, I was very taken by Fedora, despite my installing a pre-release copy of F20 (I installed RC5, then TC5, about a week before actual release). The Xen dom0 support is built into the kernel, and grub2 is easily updated to work with a Xen installation.
Now the real reason I was taken by Fedora 20.. RHEL7.
With the forthcoming RHEL7 being based upon Fedora 19 (actually a mix of 18/19/20 according to phoronix, whatever you might think of their site), that means Xen dom0 support is coming back to RHEL. yay! Also importantly, it means RHEL7 is going to be using systemctl rather than the antiquated (but functional) rc.d arrangement.
So instead of being all confused by RHEL7 when it gets here, I figure it makes sense to cut my teeth on F20 with my dom0, and then try something I’ve not tried before.. migrating my domU’s to a Centos7 dom0 once its released.
After a couple of months of doing waaay too many installs of various domU’s, dom0’s, and disk partitioning, I’ve reached a decision. Fedora 20 wins out – largely due to Redhat’s strategy for development of RHEL.
So my next few posts will detail the steps I take to set up my new home server. For insight, its going to be running Xen 4.3, and the hardware is loosely:
- Supermicro X8-SIL-F (with IPMI! oooh aaah!)
- Xeon X3450 cpu (kicking sand in the face of my old N36L microserver)
- 16G ECC ram
- 2x Intel X25-M 160G ssd’s
- 2x WD black 320G 2.5″ drives
And PCI passthrough of these things to various domU’s:
- Dell Perc H310 (LSI 9211-8i card) flashed to LSI HBA firmware 17
- 4x WD Red 2TB drives hanging off the LSI
- Second onboard Intel NIC
- Extra realtek gigabit ethernet PCI card
I’ve taken notes of all the commands and will be turning them into posts over the next few days, while I wait for more experimental stuff to compile (right now I’ve got Freenas compiling from source with the addition of the xenpci drivers and XENHVM stuff in the FREENAS.amd64 config file).
AlpineLinux on a read-only usb-stick http://alpinelinux.org/downloads/
fast, small, secure
Pingback: Fedora virtualization setup as Xen dom0 | My Wired House
Thanks for described results of your tests. I am happy to be able to confirm some of your findings. At the same time I learned a lot as well and got confirmations to some of my findings. I am caring for 3 64-bit servers for several years, 2 supermicro (AMD and Intel) and one ACER AMD based server. My first platform was OpenSuse for dom0 and later on I moved with dom0 to Centos to minimize the SW modules on domU.
I made very good experience with with running domUs on lvm partitions in combination with xm suspend. It is amazing function for online backup for domUs, combined with lvm snapshot and rsync copy. Not only backup, but simple migration when needed. Loss of communication is not a big issue in my view. Using FreeNX technology allows to reopen the online session when connection lost, independent on the cause of the loss. xm resume can restart domU on new Dom0 when needed setup made in advance.
The stability of Centos and level of support for XEN was very refreshing at time, when I started to doubt whether OpenSuse has serious interest to support XEN anymore when they are dependent on Novel/Microsoft.
But now I returned to OpenSuse 13.1 for support of domU installations (libvirt and VM) – it looks very good. By the way how did you setup grub2 for starting the XEN hypervisor? I am fighting with it, grub2 with the extending tools – a new obstacle on the XEN path for me. Golden Centos, grub2 without extensions and with simple direct editing.
Thanks for remark about XenServer 6.2 limited only to 32bit, I tried several installations on 64bit machine and missed this detail. I was not able to get a linux console and was too lazy or disgusted to install windows client.
Looking forward to your promised report of FreeNAS experiments.
Hynek