Don’t upgrade Xen domU’s to Ubuntu Karmic (9.10) on a Debian Stable (lenny) dom0!

Yeah, I found out the hard way.

If you’re running a Debian Lenny dom0 host, and you have some Ubuntu domU guests, you can happily dist-upgrade to the latest and greatest verison, Karmic (9.10) without any problems at all.

Until you reboot!

The current Debian Xen kernel (2.6.26-2-xen-amd64) is not new enough to contain a syscall (pipe2, if you were wondering) required by the mountall package in Karmic. There’s also has another stupid bug in it which causes mountall to loop infinitely on pipe2().

Basically it means that if you have a Debian Lenny dom0 and a Karmic domU, when you reboot your guest domain will boot up until mountall is called (around about the same time as /scripts/local-bottom) and then your guest domain hangs, forever, at 100% CPU usage.

Also, they’re not going to fix it in Karmic, apparently. Just Ubuntu Lucid 10.4. Great.

But wait! I notice there’s Ubuntu Lucid Alpha 1 just released! chroot into the guest root, quick lucid tweak to the sources file, apt-get dist-upgrade and… it’s still broken!

mountall:mountall.c:2938: Assertion failed in main: udev_monitor = udev_monitor_new_from_netlink (udev, “udev”)
init: mountall main process (721) killed by ABRT signal

It seems I need to wait for Alpha2 to get mountall 2 which apparently has this extremely stupid bug fixed.

Either that or go about custom-compiling a new Xen kernel for the domU that has pipe2() in it, but I’d rather have my genitals covered by flesh-eating fire ants and be forced to sit through a Belgian Opera than do that.

This is just awesome.

Edit: The Alpha 1 mountall dumps you out into an emergency shell, and although init is interupted and the system is not usable, all the drives are mounted and working fine, despite what mountall bitches about! You can get a workable system with (from the Xen console):

mount -n -o remount,rw /
mkdir /var/run/network
/etc/init.d/networking restart
mkdir /dev/pts
mount /dev/pts
/etc/init.d/ssh restart

At this point you should be able to shell in, and restart anything else not running! Note: as you’re still technically in an emergency shell DO NOT log out of the Xen console! Just leave it logged in as root and ^] out to the dom0.

I suppose this will have to do for a week or so until Alpha2 is released and I can get it working properly…