Xen images

From blag.wiki.aktivix.org

Contents 1 Xen_Images 2 Intro 3 Objective 4 Machine 5 Installation of Base Distro 6 Installation of Sub-Distros 6.1 Partition Table 7 creating the images 8 creating the domU distro 9 Umounting 10 Using the new domU 11 That should be all there is to it. 12 Notes

Xen_Images

(Mutli-Boot Partition Table for Xen-laptop)

Intro

This is a short report on the succcess of creating any flavour of Xen domU "guest images" for a Xen-enabled kernel. Use this method to create a domU for almost any distro to be used on almost any Xen host. It's the simplest and perhaps fastest method that I have found so far and involves nothing more than multi-booting a box and using your installed distros to create the directories required by the Xen domU. The directories are cp'd from the "non-primary" distro into images stored within a seperate /data partition used exclusively for images. [1] Alternatively one could use a seperate hard disk for the creation of the Xen images but I did not have any at the time of writing.

I am more comfortable with this manual process as the tools that I used to create guests were not simple to use or reliable at the time of writing. The added bonus is that guest distros can be removed once you have created your domUs from its directories. Also it's hard to guess in advance how a particular distro handles networking and package management. Install a distro into a small partition, configure it and use it's directories to make the files and directories for the domU.

Date:- Thu Sep 7 11:12:55 BST 2006 jayeola

Objective

• Build a Xen-kernel on a laptop
• Run Xen domUs from any GNU/Linux distribution as guests
• Ensure that the domUs are network ready

Machine

I've installed tools for a laptop for all of the distros, which included wireless, pcmcia and acpi tools. Most "proper" servers won't need those.

• Machine:- IBM Thinkpad X30 (laptop)
• Disk space:- 80gb
• Ram:- 1gb

Installation of Base Distro

For this excercise I installed a minimal Debian Etch (testing) system. On top of that I added stuff for a laptop. This included thinkpad tools, acpi and pcmcia stuff. The intention is to have the smallest number of daemons and services running as possible and only those that are deemed to be essential on the host system.

• Install your base distro with a minimal system. The minumum required to keep the box running.
• Ensure that one leaves sufficient free space for the sub-distro(s) to be installed on the system.
• Install Xen, (from source or binaries) and ensure that dom0 is working

Installation of Sub-Distros

These are the distros that one uses to create the guest images for XEN, otherwise known as the domUs. I had so much trouble with rpmstrap that I decided to find out how to create them manually. It's (possibly) tedious but it works.

• Base distro installed?
• Xen installed?
• Got that free space? Time to use it
• Install your sub-distro in the free space. Another bare-bones system.
• Place grub on the root partition of the sub-distro. We use this to chain load grub so that we can multi-boot [2].

Partition Table

Here follows the partition table for the laptop and a brief discussion of their use. [3]

acid80g:~# fdisk -l

Disk /dev/hda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
boot partition = 100mb
/dev/hda1   *           1          12       96358+  83  Linux

Create an extended partition immediately after creating /boot to overcome the four primary partition limitation. All partitions created from hereon used the extended part of the disk. We can have as many distros as we like, diskspace permitting.

/dev/hda2              13        7587    60846187+   5  Extended

The next two partitions are for the "non-base" distros. They were created just to build a file system for an image to be used in the Xen domU. I chose approx 8gb to start. I think you can get this down to 1gb if you only install a basic or minimal system. (No X-windows, bells and whistles, just the essentials).

This is GNU/BLAG Linux 30k - a classic. Installed as a minimal system on this partition

/dev/hda3            7588        8607     8193150   83  Linux

This is centos4.0 - a RHEL variant. Installed a minimal system on this partition. It has been upgraded to centos4.4 with no problems.

/dev/hda4            8608        9729     9012465   83  Linux

Here's my swap = 2 * ram. probably overkill but what the hell.

/dev/hda5              13         256     1959898+  82  Linux swap / Solaris

This is what I call the "base distro". It occupies the most space for the distros as this is where the most work is done. It needs disk space for the Xen kernel, base/default kernel, Xen-based tools and any other tools that one runs as a sys-admin. In this case it's Debian Etch, but it could be any other distro out there.

/dev/hda6             257        2700    19631398+  83  Linux

The following partition was created to share data. All of the Xen domU images are stored here. Use by copying directories in the "non-base"mounts and put them into the images created here, in /data. Call it what you like. Note that this is the largest partition on the disk as we store all of our guest images here.

/dev/hda7            2701        7587    39254796   83  Linux

In the following I've booted the machine and have listed the active partition. Remeber that I installed Etch on hda6 and /data is used to store the xen guests (domUs)

acid80g:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda6              19G  2.0G   16G  11% /
tmpfs                 232M     0  232M   0% /dev/shm
/dev/hda1              89M   19M   65M  23% /boot
/dev/hda7              37G  2.9G   33G   9% /data
tmpfs                  10M  112K  9.9M   2% /dev

I have modified /etc/fstab to allow for an automatic mounting of /data. Needed so that our xen daemon can start up any xen domus when thebox boots up. Handy for reboots and such

acid80g:~# mount
/dev/hda6 on / type ext3 (rw,errors=remount-ro)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
usbfs on /proc/bus/usb type usbfs (rw)
tmpfs on /dev/shm type tmpfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/hda1 on /boot type ext3 (rw)
/dev/hda7 on /data type ext3 (rw)
tmpfs on /dev type tmpfs (rw,size=10M,mode=0755)

creating the images

Better to use lvm than a file based system as it's faster for intensive input-output processes. I've used dd to make images for the file system and swap for each Xen domU. Change the paths to reflect your system!

#!/bin/bash

clear
IMAGEDIR=/home/jayeola/xen/images
mkdir -p $IMAGEDIR
for DISTRO in centos43 sarge heildelberg blag; do
 for X in $(seq 0 4); do
  dd if=/dev/zero of=$IMAGEDIR/$DISTRO$X.img bs=1M count=1 seek=2048
  dd if=/dev/zero of=$IMAGEDIR/$DISTRO$Xswap.img bs=1M count=1 seek=128
  echo "image $DISTRO$X created. creating filesystem"
  mkfs.ext3 $IMAGEDIR/$DISTRO$X.img
  mkswap $IMAGEDIR/$DISTRO$X-swap.img
  echo "filesystem and swap created for image $DISTRO$X "		 
 done
done	

I used this script to automate the process of creating images. It's normally here:- http://jayeola.org/bin/sys/xen/mkimages.sh Note that the images are stored on the separate /data partition. This allowed me to keep the files seperate from any distro partition which would be useful if I ever had to re-install, start over or recover the system.

creating the domU distro

This is where we create the domU. The previous step was just to create the file system and swap. The next step involves actually copying files from the "non-base distro" into the newly created image, making a few directories and tweaking the networking and /etc/fstab. Trust me - this has been the easiest method that I've found to date and it does not involve installing or building any new tools. If you are familiar with copying and editing files, creating directories this is for you.

This is a variantion on the xen-2.0 user guide. Works for me with no effort or sweat. First I create some mount points and give them names that have some meaning to me. I use /mnt/distro to mount the sub-distros from the base-distro.

# mkdir /mnt/{loop,disk,key,distro}

Mount the partition containing the base distro, say GNU BLAG/Linux.

# mount /dev/hda3 /mnt/distro

mount one of the images that you have created

# mount -o loop /data/xen/images/blag0.img /mnt/loop

Copy over the directories from the BLAG distro to the image.

# cp -ax /{root,dev,var,etc,usr,bin,sbin,lib} /mnt/loop

Create some directories for the new image. [4]

# mkdir /mnt/loop/{proc,sys,home,tmp,mnt}

One needs a minimum of two lines within the /etc/fstab of the new domU, / and /proc. There is no boot! [5]

# $editor /mnt/loop/etc/fstab

The following is an example of an /etc/fstab for a blag/rpm based Xen domU

/dev/hda1     /     ext3     errors=remount-ro     0     1
/dev/hda2     none  swap     sw                    0     0
proc          /proc proc     defaults              0     0
none          /dev/pts       devpts gid=5,mode=620 0     0
none          /dev/shm       tmpfs  defaults       0     0

Next, it's the networking

# $editor /mnt/loop/etc/hosts

and edit the hostname for the new domU accordingly.

Your networking devices have been copied over to the image in /mnt/loop so it should be working without the need to change anything. [6]

You may have seen some guides telling you to create some devices in /mnt/loop/dev/. This should have been taken care of already as we have cp'd files and dirs from the sub-distro but it does not hurt to check. Create the devices thusly [7];

# cd /mnt/loop/dev
# MAKEDEV null zero console

Check that the devices have been made with

# ls

Umounting

Normally I'd use umount -a but I'd like to keep the /data partition mounted as that's where I'm keeping the images to be used by Xen. We want that partition automounted - remember?

# umount /mnt/{distro,loop}

Using the new domU

Once /etc/fstab has been modified to automount /data containing the domU images and the xen daemon has been configured to start on boot you create a config file for it. This is typically in /etc/xen/ look at /etc/xen/xmexample{1,2} for clues.

That should be all there is to it.

Start the xen kernel on boot and use xm list to see the domUs.

Notes

• [1] I've also read that some folks use a sepereate /data, (or similar sharedp artition), to store config files as link then to /home. This would eliminate the need for repitition of many ~/.bashrc or similar config files.
• [2] Email me or ask at the Blag forums for how that's done.
• [3] I'm using a single disk in this laptop.
• [4] The Xen-2.0 user guide left out creating /mnt/foo/mnt. You'll soon notice this as your new domU image will bork, scream that it does not have a /mnt directory and have a hard time starting.
• [5] I'd love to see variations on this theme. I still get some errors when the domUs boot. Stuff like /dev/fd0 but I'll think about that later.
• [6] Another reason why I love this method.
• [7] It may be safer to chroot /mnt/loop and run it from /dev.