Some time ago, we had a couple of inquiries about support for running KVM hypervisors with Open vSwitch (OVS). Older versions of OVS included the brcompat module, which made OVS work with regular Linux bridges instead of its own virtual switches. This meant Abiquo would behave as if it were using regular bridges. However, with recent versions of OVS, this brcompat module has been deprecated and it is not working as well as it should. Support for OVS is in our development roadmap, but in the meantime, we will explain how to “hack” an unsupported version of this feature using libvirt “domain events”.

With Abiquo 2.6, many packages have been updated, including libvirt on KVM nodes. The new version of libvirt has native OVS support. We tested it with OVS and some slight changes to the XML definition of the VMs. This means that to use OVS, we just need to change the XML definition for the network part from, say:

[code lang=”xml” light=”true”] <interface type=’bridge’>
<mac address=’52:54:00:a5:db:f9’/>
<source bridge=’abiquo_2’/>
<target dev=’vnet1’/>
<model type=’e1000’/>
<alias name=’net0’/>
<address type=’pci’ domain=’0x0000′ bus=’0x00′ slot=’0x03′ function=’0x0’/>
</interface>
[/code]

To:

[code lang=”xml” light=”true”] <interface type=’bridge’>
<mac address=’52:54:00:a5:db:f9’/>
<source bridge=’abiquo-vSwitch’/>
<vlan>
<tag id=’2’/>
</vlan>
<virtualport type=’openvswitch’>
<parameters interfaceid=’6fb5714c-ca4f-923d-8890-1f0db4a7566a’/>
</virtualport>
<model type=’e1000’/>
<address type=’pci’ domain=’0x0000′ bus=’0x00′ slot=’0x03′ function=’0x0’/>
</interface>
[/code]

Using domain events and the sample code in libvirt’s git for python here, we can create a test implementation just by modifying callback functions.

You can safely comment out the multiple function callback binding declarations and leave only a general callback:

[code lang=”python” light=”true”] #Add 2 callbacks to prove this works with more than just one
vc.domainEventRegister(domainEventCallback,None)
vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE, domainEventCallback, None)
# vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_REBOOT, myDomainEventRebootCallback, None)
# vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_RTC_CHANGE, myDomainEventRTCChangeCallback, None)
# vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR, myDomainEventIOErrorCallback, None)
# vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_WATCHDOG, myDomainEventWatchdogCallback, None)
# vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_GRAPHICS, myDomainEventGraphicsCallback, None)
# vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_DISK_CHANGE, myDomainEventDiskChangeCallback, None)
#vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_TRAY_CHANGE, myDomainEventTrayChangeCallback, None)
#vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_PMWAKEUP, myDomainEventPMWakeupCallback, None)
#vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_PMSUSPEND, myDomainEventPMSuspendCallback, None)
#vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_BALLOON_CHANGE, myDomainEventBalloonChangeCallback, None)
#vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_PMSUSPEND_DISK, myDomainEventPMSuspendDiskCallback, None)
#vc.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED, myDomainEventDeviceRemovedCallback, None)
[/code]

Here we bind any event sent by libvirt to the domainEventCallback function. Then the domainEventCallback function just needs to implement the change described above:

[code lang=”python” light=”true”] print "domainEventCallback EVENT: Domain %s(%s) %s %s" % (dom.name(), dom.ID(),
eventToString(event),
detailToString(event, detail))
print "domainEventCallback REAL event : %s" % event
print "domainEventCallback REAL detail : %s" % detail

if event == 2 and detail == 0: # VM defined
print "Domain : " + dom.name()

domconfig = ET.fromstring(dom.XMLDesc(0))

for interface in domconfig.findall(‘./devices/interface[@type="bridge"]’):
time.sleep(2)
if interface.find("virtualport") == None:
source = interface.find("source").attrib.get("bridge")
mac = interface.find("mac").attrib.get("address")
vlan = string.split(interface.find("source").attrib.get("bridge"), "_")[1] target = interface.find("target").attrib.get("dev")
driver = interface.find("model").attrib.get("type")

print " Interface :"
print " MAC : " + mac
print " Source : " + source
print " VLAN : " + vlan
print " Target DEV : " + target
print " Driver : " + driver

print "Replacing XML : "
print ET.tostring(interface)

#dom.detachDevice(ET.tostring(interface))

interface.find("source").set("bridge", "abiSwitch")
vlannode = ET.SubElement(interface, "vlan")
tagnode = ET.SubElement(vlannode, "tag")
tagnode.set("id", vlan)
alias = interface.find("alias")
interface.remove(alias)

virtualport = ET.SubElement(interface, "virtualport")
virtualport.set("type", "openvswitch")

print "With XML : "
print ET.tostring(interface)

#dom.attachDevice(ET.tostring(interface))
conn.defineXML(ET.tostring(domconfig))

dom.destroy()
dom.create()
else:
print "Already using OVS."
[/code]

This code:

  • Checks the event sent by libvirt
  • When a VM is defined (booted), it looks for the network interface part of the definition and modifies it to match the OVS requirements and sets the VLAN tag (if it isn’t set already)
  • Stops and starts the VM so the changes are applied

We need to stop and start the VM because the ACPI behavior in libvirt may prevent detaching a device and attaching it back right away (see http://patchwork.ozlabs.org/patch/80613/).

With this little hack you can make changes at deploy time (or “power cycle time”) without the need for any modification to the Abiquo code (so you won’t have to wait for the next release).

To test the hack, you’ll need to:

  • Set up a KVM node for 2.6
  • Install OVS
  • Create an OVS switch

Note that the switch name is hard coded in the callback function, so you can change this to get it as a parameter:

[code lang=”shell” light=”true”] # ovs-vsctl add-br abiSwitch
[/code]

The script takes the libvirt URI it will monitor as a parameter, which defaults to localhost if not supplied.

[code lang=”shell” light=”true”] # python events.py qemu+tcp://<libvrt_ip>/system
[/code]

And deploy!