Tag Archives: libvirt

Xen Project now in OpenStack Nova Hypervisor Driver Quality Group B

A few weeks ago, we introduced the Xen Project – OpenStack CI Loop, which is testing Nova commits against the Xen Project Hypervisor and Libvirt. Xen Project community is pleased to announce that we have moved from Quality Group C to B, as we’ve made significant progress in the last few weeks and the Xen Project CI loop is now voting on Nova commits.

This diagram shows the number of OpenStack Nova drivers for Hypervisors, which allow you to choose which Hypervisor(s) to use for your Nova Deployment. Note that these are classified into groups A, B and C. Xen Project is now in Quality Group B.

This diagram shows the number of OpenStack Nova drivers for Hypervisors, which allow you to choose which Hypervisor(s) to use for your Nova Deployment. Note that these are classified into groups A, B and C. Xen Project is now in Quality Group B.

Quality groups are defined as follows:

  • Group C: These drivers have minimal testing and may or may not work at any given time. Test coverage may include unit tests that gate commits. There is no public functional testing.
  • Group B: Test coverage includes unit tests that gate commits. Functional testing is provided by an external system (such as our CI loop) that do not gate commits, but advises patch authors and reviewers of results in OpenStack Gerrit.
  • Group A: Test coverage includes unit tests and functional testing that both gate commits.

What does this mean in practice?

The easiest way to understand what this means in practice, is to look at a real code review as shown in the figure below.

This diagram shows how the functional tests for the Xen Project initially failed, and passed after a new patch was uploaded.

This diagram shows how the functional tests for the Xen Project initially failed, and passed after a new patch was uploaded.

This code review shows that the OpenStack Jenkins instance (running on KVM) and the Xen Project CI Loop failed their respective tests initially, so a new patchset was uploaded and the tests succeeded afterwards.

This diagram shows the status after a new patchset was uploaded. Note that the review of the patchset is still pending manual review.

This diagram shows the status after a new patchset was uploaded. Note that the review of the patchset is still pending manual review.

Also see Merging.Repository Gating in the OpenStack documentation.

Relevant OpenStack Summit Sessions

There are a number of sessions at this week’s OpenStack Summit that are worth attending including:

Hands-on sessions to improve 3rd party CI infrastructure include:

Other third-party CI-related sessions include:

Relevant Regular Meetings

Note that there are also weekly Third Party CI Working Group meetings for all operators of 3rd party CI loops in #openstack-meeting-4 on Wednesdays at 1500/0400 UTC alternating organized by Kurt Taylor (krtaylor). Third party CI operators interested in enhancing documentation, reviewing patches for relevant work, and improving the consumability of infra CI components are encouraged to attend. See here for more information on the working group.

Introducing the Xen Project – OpenStack CI Loop

We recently introduced the new Xen Project Test Lab, a key piece of infrastructure to improve the quality of our code and code coverage. As stated earlier, “we are serious and proactive when it comes to minimising and, whenever possible, eliminating any adverse effects from defects, security vulnerabilities or performance problems”. This also applies to Xen Project integration with OpenStack, which is why the Xen Project Advisory Board made available funds to create the Xen Project – OpenStack CI Loop in January 2015. We started work on setting up our OpenStack CI Loop immediately after the funds were made available, fixed a number of issues in the Xen Project Hypervisor, Libvirt and OpenStack Nova and are excited to announce that our Xen Project – OpenStack CI Loop is now live and in production.

I wanted to thank members of our Test Infrastructure Working Group, comprised of employees from AMD, Amazon Web Services, Citrix, Intel and Oracle, and community members from Rackspace and Suse who have been supporting the creation of the Xen Project – OpenStack CI Loop.

What does an OpenStack CI Loop Do?

An OpenStack external testing platform (or CI Loop) enables third parties to run tests against an OpenStack environment that is configured with that third party’s drivers or hardware and reports the results of those tests on the code review of a proposed OpenStack patch. It is easy to see the benefit of this real-time feedback by taking a look at a code review that shows how these platforms provide feedback.

In this screenshot, you can see a number Verified +1 and one Verified -1 labels added by CI loops to OpenStack Nova.

In this screenshot, you can see a number Verified +1 and one Verified -1 labels added by CI loops to OpenStack Nova.

The figure below, shows the number of OpenStack Nova drivers for Hypervisors, which allow you to choose which Hypervisor(s) to use for your Nova Deployment. Note that these are classified into groups A, B and C.

This diagram shows the different Nova compute drivers and their quality status

This diagram shows the different Nova compute drivers and their quality status

In a nutshell, the groups have the following meaning:

  • Group C: These drivers have minimal testing.
  • Group B: These drivers have unit tests that gate commits and functional testing providing by an external system that does not gate commits, but advises patch authors and reviewers of results in the OpenStack code review system.
  • Group A: These drivers have unit tests that gate commits and functional testing that gate commits.

With the introduction of the Xen Project – OpenStack CI Loop we are close to achieving our first goal of moving the Xen Project Hypervisor from Group C to B. This is a goal we first publicly stated at this year’s FOSDEM’15 where some of you may had had the opportunity to hear Stefano Stabellini talk about using the Xen Project Hypervisor in OpenStack.

What we test against?

Currently, our CI Loop tests all OpenStack Nova commits against Ubuntu Xen 4.4.1 packages with a number of patches applied to them, together Libvirt 1.2.14 with a number of patches applied to them (for more details see this specification). All the patches are already integrated upstream. When off-the shelf packages of these changes are available in Linux distros, we will start testing against them.

What is next?

Monitor the CI loop test results against the official OpenStack CI: we will run the CI loop for a few weeks and monitor, whether there are any discrepancies between the results of the official Nova CI loop and ours. If there are any, we will investigate where the issue is and fix them. This is important, as we had a number of intermittent issues in the Xen Libvirt driver and we want to be sure, we have fixed them all.

Fix two known issues, and enable two disabled tests: We have two test cases related to iSCSI support (test_volume_boot_pattern), which are currently disabled. In one case, we believe the that the Tempest Integration Suite makes some KVM specific assumptions: Xen Project community members are working with the OpenStack community to fix the issue. The second issue is related, but requires further investigation.

Make the Xen Project – OpenStack CI Loop voting: The next logical step, is to work with the OpenStack community to make our CI loop voting. This is the first step to move from group C to B.

Beyond That: Build trust and become an active member of the OpenStack community. Our ultimate goal, is to ensure that the Xen Project Hypervisor moves into group A, alongside KVM. We are also investigating, whether it makes sense to run Xen Project + Libvirt against OpenStack Neutron. We are also looking at integrating OpenStack Tempest into our new Xen Project Test Lab, such that we can ensure that upstream Xen and Libvirt will work with OpenStack Nova.

Can I Help?

If you use Xen Project with OpenStack and Libvirt, feel free to get in touch. Some members of the Xen Project community will be at the Vancouver OpenStack Summit and we are keen to learn, what issues you have and whether we can help fixing them. As community manager, will help arrange meetings and build bridges: feel free to contact me under community dot manager at xenproject dot org. You can also ask questions on xen-devel@ and xen-users@ and connect with Xen Project Developers and Users.

Additional Resources

Using Xen Project on OpenStack “Juno” via Libvirt

By Xing Lin

This document describes steps I took to setup a compute node based on Ubuntu 14.04 for OpenStack “juno”, using the Xen Project via libvirt approach. Openstack does not support this approach well as it is in Group C of the hypervisor support matrix for Openstack. You can hardly find any tutorial online describing this approach and this might be the first. Let’s get started!

Prerequisites

Follow “OpenStack Installation Guide for Ubuntu 14.04″ to setup the control node and network node, following the three-node architecture with OpenStack Networking (neutron). This involves lots of configuration and could take a day or two. Check that the control node and network node is working.

Continue reading

Xen Hypervisor Case Studies

I am starting a new community initiative to collect and write Xen hypervisor case studies to demonstrate the variety of ways that the Xen hypervisor is leveraged in the IT world. The initial case study is from a Swedish company, ATG: ATG Case Study Feb 29, 2008

I have created a new section in the Wiki to store all the case studies that the community or I create. You can get to the Wiki case study section here.  Please feel free to create your own case study and upload into the Wiki site or contact me at stephen.spector@citrix.com if you would like my assistance. Having an updated collection of case studies is a great way for the community to show the power and capabilities of the Xen hypervisor.