In this article by Konstantin Ivanov, author of the book KVM Virtualization Cookbook, we are going to deploy three different network types, explore the network XML format and see examples on how to define and manipulate virtual interfaces for the KVM instances.
(For more resources related to this topic, see here.)
To be able to connect the virtual machines to the host OS, or to each other, we are going to use the Linux bridge and the Open vSwitch (OVS) daemons, userspace tools and kernel modules. Both software bridging technologies are great at creating software defined networks (SDN) of various complexity, in a consistent and easy to manipulate manner. The Linux bridge and OVS both act as a bridge/switch that the virtual interfaces of the KVM guests can connect to.
With all this in mind, lets start by learning more about the software bridges in Linux.
The Linux bridge
The Linux bridge is a software Layer 2 device that provides some of the functionality of a physical bridge device. It can forward frames between KVM guests, the host OS and virtual machines running on other servers, or networks. The Linux bridge consists of two components – a userspace administration tool that we are going to use in this recipe and a kernel module, that performs all the work of connecting multiple Ethernet segments together. Each software bridge we create can have a number of ports attached to it, where network traffic is forwarded to and from. When creating KVM instances we can attach the virtual interfaces that are associated with them to the bridge, which is similar to plugging a network cable from a physical server’s Network Interface Card (NIC) to a bridge/switch device. Being a Layer 2 device, the Linux bridge works with MAC addresses and maintains a kernel structure to keep track of ports and associated MAC addresses in the form of a Content Addressable Memory (CAM) table.
In this recipe we are going to create a new Linux bridge and use the brctl utility to manipulate it.
Getting Ready
For this recipe we are going to need the following:
- Recent Linux kernel with enabled 802.1d Ethernet Bridging options.
To check if your kernel is compiled with those features, or exposed as kernel modules, run:
root@kvm:~# cat /boot/config-`uname -r` | grep -i bridg # PC-card bridges CONFIG_BRIDGE_NETFILTER=y CONFIG_NF_TABLES_BRIDGE=m CONFIG_BRIDGE_EBT_BROUTE=m CONFIG_BRIDGE_EBT_T_FILTER=m CONFIG_BRIDGE_EBT_T_NAT=m CONFIG_BRIDGE_EBT_802_3=m CONFIG_BRIDGE_EBT_AMONG=m CONFIG_BRIDGE_EBT_ARP=m CONFIG_BRIDGE_EBT_IP=m CONFIG_BRIDGE_EBT_IP6=m CONFIG_BRIDGE_EBT_LIMIT=m CONFIG_BRIDGE_EBT_MARK=m CONFIG_BRIDGE_EBT_PKTTYPE=m CONFIG_BRIDGE_EBT_STP=m CONFIG_BRIDGE_EBT_VLAN=m CONFIG_BRIDGE_EBT_ARPREPLY=m CONFIG_BRIDGE_EBT_DNAT=m CONFIG_BRIDGE_EBT_MARK_T=m CONFIG_BRIDGE_EBT_REDIRECT=m CONFIG_BRIDGE_EBT_SNAT=m CONFIG_BRIDGE_EBT_LOG=m # CONFIG_BRIDGE_EBT_ULOG is not set CONFIG_BRIDGE_EBT_NFLOG=m CONFIG_BRIDGE=m CONFIG_BRIDGE_IGMP_SNOOPING=y CONFIG_BRIDGE_VLAN_FILTERING=y CONFIG_SSB_B43_PCI_BRIDGE=y CONFIG_DVB_DDBRIDGE=m CONFIG_EDAC_SBRIDGE=m # VME Bridge Drivers root@kvm:~# - The bridge kernel module.
To verify that the module is loaded and to obtain more information about its version and features, execute:
root@kvm:~# lsmod | grep bridge
bridge 110925 0
stp 12976 2 garp,bridge
llc 14552 3 stp,garp,bridge
root@kvm:~# modinfo bridge
filename: /lib/modules/3.13.0-107-generic/kernel/net/bridge/bridge.ko
alias: rtnl-link-bridge
version: 2.3
license: GPL
srcversion: 49D4B615F0B11CA696D8623
depends: stp,llc
intree: Y
vermagic: 3.13.0-107-generic SMP mod_unload modversions
signer: Magrathea: Glacier signing key
sig_key: E1:07:B2:8D:F0:77:39:2F:D6:2D:FD:D7:92:BF:3B:1D:BD:57:0C:D8
sig_hashalgo: sha512
root@kvm:~#- The bridge-utils package, that provides the tool to create and manipulate the Linux bridge.
- The ability to create new KVM guests using libvirt, or the QEMU utilities
How to do it…
- Install the Linux bridge package, if it is not already present:
root@kvm:~# apt install bridge-utils - Build a new KVM instance using the raw image:
root@kvm:~# virt-install --name kvm1 --ram 1024 --disk path=/tmp/debian.img,format=raw --graphics vnc,listen=146.20.141.158 --noautoconsole --hvm --import Starting install... Creating domain... | 0 B 00:00 Domain creation completed. You can restart your domain by running: virsh --connect qemu:///system start kvm1 root@kvm:~# - List all available bridge devices:
root@kvm:~# brctl show bridge name bridge id STP enabled interfaces virbr0 8000.fe5400559bd6 yes vnet0 root@kvm:~# - Bring the virtual bridge down, delete it and ensure it’s been deleted:
root@kvm:~# ifconfig virbr0 down root@kvm:~# brctl delbr virbr0 root@kvm:~# brctl show bridge name bridge id STP enabled interfaces root@kvm:~# - Create a new bridge and bring it up:
root@kvm:~# brctl addbr virbr0 root@kvm:~# brctl show bridge name bridge id STP enabled interfaces virbr0 8000.000000000000 no root@kvm:~# ifconfig virbr0 up root@kvm:~# - Assign an IP address to the bridge:
root@kvm:~# ip addr add 192.168.122.1 dev virbr0 root@kvm:~# ip addr show virbr0 39: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default link/ether 32:7d:3f:80:d7:c6 brd ff:ff:ff:ff:ff:ff inet 192.168.122.1/32 scope global virbr0 valid_lft forever preferred_lft forever inet6 fe80::307d:3fff:fe80:d7c6/64 scope link valid_lft forever preferred_lft forever root@kvm:~# - List the virtual interfaces on the host OS:
root@kvm:~# ip a s | grep vnet 38: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500 root@kvm:~# - Add the virtual interface vnet0 to the bridge:
root@kvm:~# brctl addif virbr0 vnet0 root@kvm:~# brctl show virbr0 bridge name bridge id STP enabled interfaces virbr0 8000.fe5400559bd6 no vnet0 root@kvm:~# - Enable the Spanning Tree Protocol (STP) on the bridge and obtain more information:
root@kvm:~# brctl stp virbr0 on root@kvm:~# brctl showstp virbr0 virbr0 bridge id 8000.fe5400559bd6 designated root 8000.fe5400559bd6 root port 0 path cost 0 max age 20.00 bridge max age 20.00 hello time 2.00 bridge hello time 2.00 forward delay 15.00 bridge forward delay 15.00 ageing time 300.00 hello timer 0.26 tcn timer 0.00 topology change timer 0.00 gc timer 90.89 flags vnet0 (1) port id 8001 state forwarding designated root 8000.fe5400559bd6 path cost 100 designated bridge 8000.fe5400559bd6 message age timer 0.00 designated port 8001 forward delay timer 0.00 designated cost 0 hold timer 0.00 flags root@kvm:~# - Test connectivity from the KVM instance to the host OS:
root@kvm:~# virsh console kvm1 Connected to domain kvm1 Escape character is ^] root@debian:~# ip a s eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:55:9b:d6 brd ff:ff:ff:ff:ff:ff inet 192.168.122.92/24 brd 192.168.122.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe55:9bd6/64 scope link valid_lft forever preferred_lft forever root@debian:~# root@debian:~# ping 192.168.122.1 -c 3 PING 192.168.122.1 (192.168.122.1) 56(84) bytes of data. 64 bytes from 192.168.122.1: icmp_seq=1 ttl=64 time=0.276 ms 64 bytes from 192.168.122.1: icmp_seq=2 ttl=64 time=0.226 ms 64 bytes from 192.168.122.1: icmp_seq=3 ttl=64 time=0.259 ms --- 192.168.122.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1999ms rtt min/avg/max/mdev = 0.226/0.253/0.276/0.027 ms root@debian:~#
How it works…
When we first installed and started the libvirt daemon, few things happened automatically:
- A new Linux bridge was created with the name and IP address defined in the /etc/libvirt/qemu/networks/default.xml configuration file.
- The dnsmasq service was started with a configuration specified in the /var/lib/libvirt/dnsmasq/default.conf file.
Lets examine the default libvirt bridge configuration:
root@kvm:~# cat /etc/libvirt/qemu/networks/default.xml
<network>
<name>default</name>
<bridge name="virbr0"/>
<forward/>
<ip address="192.168.122.1" netmask="255.255.255.0">
<dhcp>
<range start="192.168.122.2" end="192.168.122.254"/>
</dhcp>
</ip>
</network>
root@kvm:~#This is the default network that libvirt created for us, specifying the bridge name, IP address and the IP range used by the DHCP server that was started. We are going to talk about libvirt networking in much more details later in this article, however we are showing it here to help you understand where all the IP addresses and the bridge name came from.We can see that a DHCP server is running on the host OS and its configuration file, by running:
root@kvm:~# pgrep -lfa dnsmasq
38983 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf
root@kvm:~# cat /var/lib/libvirt/dnsmasq/default.conf
##WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
##OVERWRITTEN AND LOST. Changes to this configuration should be made using:
## virsh net-edit default
## or other application using the libvirt API.
##
## dnsmasq conf file created by libvirt
strict-order
user=libvirt-dnsmasq
pid-file=/var/run/libvirt/network/default.pid
except-interface=lo
bind-dynamic
interface=virbr0
dhcp-range=192.168.122.2,192.168.122.254
dhcp-no-override
dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases
dhcp-lease-max=253
dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile
addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts
root@kvm:~#From the preceding configuration file, notice how the IP address range for the DHCP service and the name of the virtual bridge match what is configured in the default libvirt network file that we just saw.
With all this in mind let’s step through all the actions we performed earlier:
In step 1, we installed the userspace tool brctl that we use to create, configure and inspect the Linux bridge configuration in the Linux kernel.
In step 2, we provisioned a new KVM instance using a custom raw image containing the guest OS.
In step 3, we invoked the bridge utility to list all available bridge devices. From the output we can observe that currently there’s one bridge, named virbr0, which libvirt created automatically. Notice that under the interfaces column we can see the vnet0 interface. This is the virtual NIC that was exposed to the host OS, when we started the KVM instance. This means that the virtual machine is connected to the host bridge.
In step 4, we first bring the bridge down in order to delete it, then we use the brctl command again to remove the bridge and ensure it’s not present on the host OS.
In step 5, we recreated the bridge and brought it back up. We do this to demonstrate the steps required to create a new bridge.
In step 6, we re-assigned the same IP address to the bridge and listed it.
In steps 7 and 8 we list all virtual interfaces on the host OS. Since we only have one KVM guest currently running on the server, we only see one virtual interface – vnet0. We then proceed to add/connect the virtual NIC to the bridge.
In step 9, we enabled the Spanning Tree Protocol (STP) on the bridge. STP is a layer 2 protocol that helps prevent network loops if we have redundant network paths. This is especially useful in larger, more complex network topologies, where multiple bridges are connected together.
Finally, in step 10, we connect to the KVM guest using the console, list its interface configuration and ensure we can ping the bridge on the host OS. Notice that the IP address of the guest OS was automatically assigned by the dnsmasq server running on the host, as it is a part of the IP range defined in the configuration file we saw earlier.
Summary
We have covered the Linux bridge in detail here. We have deployed three different network types, explored the network XML format and have seen examples on how to define and manipulate virtual interfaces for the KVM instances.
Resources for Article:
Further resources on this subject:
- A Virtual Machine for a Virtual World [article]
- Getting Started with Ansible [article]
- Supporting hypervisors by OpenNebula [article]