Virtualization with KVM

The task

As you probably read in the teaser, I need a lot of throw-away-machines and I got tired to set them up manually on real hardware every time I want to try something out.

Before we start, you need to check if your hardware actually supports hardware virtualization. You can check with:

grep -E '^flags.*\b(vmx|svm)\b' /proc/cpuinfo

If you get an output, you're fine. If you don't, you're hardware unfortunatelly doesn't support hardware virtualization. You will be able to use qemu, but you won't get satisfactory performance of your virtual machines.

To install qemu, simply do

sudo apt-get update && sudo apt-get install qemu

I use Ubuntu Server 14.4.3 LTS 64bit as the host's OS. My hardware is a Dell desktop computer with a Core 2 Duo CPU and 4GB of RAM. In addition to the on-board network interface card (1GBit), I added a 4-port D-Link network interface card (4 x 100MBit).

Creating the image

The first step to create a virtual machine is creating its hard disk drive image:

qemu-img create system.img 10G

This will create a 10G disk image file for your virtual machine. On some file systems, "ls" will show the file's size beeing 10G, but "df -h" will display no change. Why? The image is a "sparse file", not yet containing any data. It will dynamically grow when data is written to the virtual disk. Cool, isn't it?

Booting from an ISO image

When you first start the virtual machine, you probably want to boot it from an ISO image. This is how you can do that:

qemu-system-x86_64 -enable-kvm -smp 2 \
-cpu host -hda system.img \
-name vm1 -m 512 -boot d \
-cdrom ubuntu-14.04.3-server-amd64.iso \
-k de -vnc :1

Okay, this looks like a lot of parameters. Let's talk about them one by one:

qemu-system-x86_64
In my case, I'm using a Core 2 Duo CPU, so I use qemu-system-x86_64.

-enable-kvm
This actually enables kvm's hardware virtualization.

-smp 2
Provide two CPUs to the guest.

-cpu host
Present the host's CPU to the guest.

-hda system.img
Use the previously created disk image as first hard drive in the guest.

-name vm1
Set the name of the virtual machine.

-m 512
Give the guest 512MB of RAM.

-boot d
Boot from CD-ROM (c = hard drive, d = CD-ROM)

-cdrom ubuntu-14.04.3-server-amd64.iso
Use the specified ISO image as CD-ROM.

-k de
Use a german keyboard layout. Of course you have to specify your desired keyboard layout.

-vnc :1
Use VNC port 5900 + 1. This is very handy to see the VM's display. On Windows, you can use "TightVNC" to control your virtual machine. If you don't need the VNC access anymore, it's probably better to remove this parameter.

If you enter the command as shown, you're virtual machine should boot from the given ISO image. In the example given above, it will start the Ubuntu Server 14.4 LTS installation.

Networking

Because of the order, the OS is scanning the PCI bus, my interface names are not ordered properly:

eth0 = first port of the D-Link NIC
eth1 = second port of the D-Link NIC
eth2 = third port of the D-Link NIC
eth3 = on-board Intel NIC
eth4 = fourth port of the D-Link NIC

Keep this quirky naming in mind and don't get confused.

I use eth0 as management interface for the host. Because there is only management traffic, it doesn't matter that this is only a 100MBit port. eth3 is used as network access port for the virtual machines.

Before we start the VM's network configuration, I want to describe what we ware going to do: the VM will get a TAP interface on the host. This TAP interface will be bridget to eth3, this way the VM gets access to the network.

Because we need to create and configure a bridge interface, we're going to need the bridge-utils. You can install them with

sudo apt-get install bridge-utils

How to tell the VM about the interface? When we booted the VM the first time, we used this command:

qemu-system-x86_64 -enable-kvm -smp 2 \
-cpu host -hda system.img \
-name vm1 -m 512 -boot d \
-cdrom ubuntu-14.04.3-server-amd64.iso \
-k de -vnc :1

This time, we alter it to reflect two things: First, the VM now should boot from the disk image. Second, we want to connect the VM to the network. The new command looks like this:

qemu-system-x86_64 -enable-kvm -smp 2 \
-cpu host -hda system.img \
-name vm1 -m 512 -boot c \
-net nic,model=virtio,macaddr=02:00:00:00:00:01 \
-net tap,ifname=tap0,script=ifup.sh,downscript=ifdown.sh
-k de -vnc :1

-boot c
Boot from hard disk image.

-net nic,model=virtio,macaddr=02:00:00:00:00:01
Create a NIC for the guest (nic).
Use a paravirtualised device (model=virtio).
Give the NIC the MAC address "02:00:00:00:00:01" (macaddr=02:00:00:00:00:01). Be careful: You cannot use just a random MAC! There are specific "private" ranges, like there are private IP address ranges. One of these private ranges are MAC addresses starting with "02"

-net tap,ifname=tap0,script=ifup.sh,downscript=ifdown.sh
Create a TAP interface on the host (tap).
Name the TAP interface "tap0" (ifname=tap0).
Execute the script "ifup.sh" when bringen the TAP interface up (script=ifup.sh).
Execute the script "ifdown.sh" when bringing the TAP interface down (downscript=ifdown.sh).

To completely understand the network configuration, we need to get insights in "ifup.sh" and "ifdown.sh". Here they are:

ifup.sh

#!/bin/sh

BRCTL=/sbin/brctl
IFCONFIG=/sbin/ifconfig
BRNAME=br0
LAN_IF=eth3

echo "Bringing up $1"
sudo $IFCONFIG $1 0.0.0.0 promisc up

echo "Bringing up $LAN_IF"
sudo $IFCONFIG $LAN_IF up

echo "Adding $1 to $BRNAME"
sudo $BRCTL addbr $BRNAME
sudo $BRCTL addif $BRNAME $LAN_IF
sudo $BRCTL addif $BRNAME $1

echo "Bringing up $BRNAME"
sudo $IFCONFIG $BRNAME up

sleep 2

ifdown.sh

#!/bin/sh

BRCTL=/sbin/brctl
IFCONFIG=/sbin/ifconfig
BRNAME=br0

echo "Bringing down $1"
sudo $IFCONFIG $1 down

echo "Removing $1 from $BRNAME"
sudo $BRCTL delif $BRNAME $1

sleep 2

Maybe you see the problem, these scripts introduce. Don't you? We must use "sudo", but "sudo" will prompt for the user's password. To work around this issue, we need to edit /etc/sudoers:

/etc/sudoers

#
# This file MUST be edited with the 'visudo' command as root.
#
# Please consider adding local content in /etc/sudoers.d/ instead of
# directly modifying this file.
#
# See the man page for details on how to write a sudoers file.
#
Defaults        env_reset
Defaults        mail_badpass
Defaults        secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

# Host alias specification

# User alias specification

# Cmnd alias specification
Cmnd_Alias      QEMU=/sbin/ifconfig, \
/sbin/brctl, \
/usr/sbin/tunctl


# User privilege specification
root    ALL=(ALL:ALL) ALL

# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL

# Allow members of group sudo to execute any command
%sudo   ALL=(ALL:ALL) ALL

# See sudoers(5) for more information on "#include" directives:

#includedir /etc/sudoers.d

ruben   ALL=NOPASSWD: QEMU

Important: The last line must be inserted last, otherwise, it may be overridden from other configuration directives!

Of course you need to adapt the username. Now, the specified commands can be executed with sudo without beeing prompted for the password.

Now you can boot the VM, the TAP interface will be brought up and bridged to eth3.

Creating a simple VM startup script

Soon you'll get tired of typing the qemu command over and over, at least I did. This quick and dirty startup script will assist you in starting the VM. Please take care, that you need to adapt it according to your needs.

start.sh

#!/bin/bash
VM=myVM
TAP=tap0
IFUP=$VM"-ifup.sh"
IFDOWN=$VM"-ifdown.sh"
MAC="02:00:00:00:00:01"
IMAGE=$VM".img"
MEMORY=512
MONITOR="unix:"$VM"-monitor,server,nowait"
VNCPORT=1

TUNCTL=/usr/sbin/tunctl

sudo $TUNCTL -t $TAP -u `whoami`

COMMAND="qemu-system-x86_64 \
-enable-kvm -smp 2 -cpu host \
-hda $IMAGE -name $VM -m $MEMORY \
-boot c -net nic,model=virtio,macaddr=$MAC \
-net tap,ifname=$TAP,script=$IFUP,downscript=$IFDOWN \
-k de -monitor $MONITOR -vnc :$VNCPORT" echo $COMMAND $COMMAND

That's it

Yes, that's it.

To come back to the initial motivation, I'd like to add: I created a clean Ubuntu Server install which I can copy every time I need a new machine. No need to install anything. Just copy the image, copy the scripts (start.sh, ifup.sh, ifdown.sh), configure the settings and start the virtual machine. It barely takes two minutes. No more excuses for not trying something ;-)

If you think I'm wrong or something can be done in a better or more sane way, please let me know. I would be glad to add it here.

Go back

My Whishlist

If the information I provided was helpful to you, I would really appreciate if you have a look on my Amazon whishlist.

I'm not begging for anything and I will continue to share my knowledge but of course I would be really happy to see some packages arriving ;-)