Monthly Archives: November 2013

RT-Xen: Real-Time Virtualization in Xen

RT-XenThe researchers at Washington University in St. Louis and University of Pennsylvania are pleased to announce, here on this blog, the release of a new and greatly improved version of the RT-Xen project. Recent years have seen increasing demand for supporting real-time systems in virtualized environments (for example, the Xen-ARM projects and several other real-time enhancements to Xen), as virtualization enables greater flexibility and reduces cost, weight and energy by breaking the correspondence between logical systems and physical systems. As an example of this, check out the video below from the 2013 Xen Project Developer Summit

The video describes how Xen could be used in an in-vehicle infotainement system.

In order to combine real-time and virtualization, a formally defined real-time scheduler at the hypervisor level is needed to provide timing guarantees to the guest virtual machines. RT-Xen bridges the gap between real-time scheduling theory and the virtualization technology by providing a suite of multi-core real-time schedulers to deliver real-time performance to domains running on the Xen hypervisor.

Background: Scheduling in Xen

In Xen, each domain’s core is abstracted as a Virtual CPU (VCPU), and the hypervisor scheduler is responsible for scheduling VCPUs. For example, the default credit scheduler would assign a weight per domain, which decides the proportional share of CPU cycles that a domain would get. The credit scheduler works great for general purpose computing, but is not suitable for real-time applications due to the following reasons:

  1. There is no reservation with credit scheduler. For example, when two VCPUs runs on a 2 GHz physical core, each would get 1 GHz. However, if another VCPU also boots on the same PCPU, the resource share shrinks to 0.66 GHz. The system manager have to carefully configure the number of VMs/VCPUs to ensure that each domain get an appropriate amount of CPU resource;
  2. There is little timing predictability or real-time performance provided to the VM. If a VM is running some real-time workload (video decoding, voice processing, and feedback control loops) which are periodically triggered and have a timing requirement — for example, the VM must be scheduled every 10 ms to process the data — there is no way the VM can express this information to the underlying VMM scheduler. The existing SEDF scheduler can help with this, but it has poor support for multi-core.

RT-Xen: Combining real-time and virtualization

RT-Xen aims to solve this problem by providing a suite of real-time schedulers. The users can specify (budget, period, CPU mask) for each VCPU individually. The budget represents the maximum CPU resource a VCPU will get during a period; the period represents the timing quantum of the CPU resources provided to the VCPU; the CPU mask defines a subset of physical cores a VCPU is allowed to run. For each VCPU, the budget is reset at each starting point of the period (all in milliseconds), consumed when the VCPU is executing, and deferred when the VCPU has budget but no work to do.

Within each scheduler, the users can switch between different priority schemes: earliest deadline first (EDF), where VCPU with earlier deadline has higher priority; or rate monotonic (RM), where VCPU with shorter period has higher priority. As a results, not only the VCPU gets a resource reservation (budget/period), but also an explicit timing information for the CPU resources (period). The real-time schedulers in RT-Xen delivers the desired real-time performance to the VMs based on the resource reservations.

To be more specific, the two multi-core schedulers in RT-Xen are:

  • RT-globalwhich uses a global run queue to hold all VCPUs (in runnable state). It is CPU mask aware, and provides better resource utilization, as VCPU can migrate freely between physical cores (within CPU mask)
  • RT-partition: which uses a run queue per physical CPU. In this way, each physical CPU only looks at its own run queue to make scheduling decisions, which incurs less overhead and potentially better cache performance. However, load-balancing between physical cores is not provided in the current release.

Source Code and References

The developers of RT-Xen are looking closely at how to integrate both schedulers into the Xen mainstream. In the meantime, please check out publications at [EMSOFT’14], [EMSOFT’11], [RTAS’12] and source code.

Xen Project Well Represented at SUSECon and openSUSE Summit

What do a chameleon, a panda, and a mouse have in common?  More than you might imagine, unless you were present at SUSECon and the openSUSE Summit at the Walt Disney Coronado Springs resort last week in Florida.  During the week, it was clear that the SUSE chameleon and Xen panda could happily coexist in Mickey Mouse’s home turf.

Geeko, the openSUSE Chameleon

I had the opportunity to attend and speak at both conferences. The week started with 3 and a half days of SUSECon, an event dedicated to SUSE’s commercial Linux products, and finished with 2 days of openSUSE Summit, which celebrates the Open Source distribution. Key people from both worlds filled the halls, and the schedule boasted an excellent assortment of talks.

At least a half dozen different sessions included some Xen-related content at SUSECon, not to mention additional sessions at openSUSE Summit.  Both the Open Source openSUSE distribution and the commercial SUSE Linux Enterprise Server (SLES) boast support for multiple virtualization engines (Xen, KVM, Linux containers, etc.).  While this might not seem too significant, it is a refreshing departure from many companies in the industry which insist on hawking their preferred virtualization technology. On platforms where hypervisors have equal footing, Xen has the opportunity to shine — and usually does.

Continue reading

Xen Project Developer Summit Slides and Videos are Live

Most of the Xen Project Developer Summit Videos and Presentations are now live on Xenproject.org. I wanted to thank all our speakers and attendees for making this year’s Xen project Developer Summit a success and am looking forward to next year’s event.

A few Highlights


Will Auld from Intel explains how any unmodified OS could be used as HVM Dom0 in Xen


Luwei Cheng talks about his research on TCP performance in VMs

Lovene Bhatia from Samsung shows two Android VMs running on a Nexus 10

Note that we will be running a series of in-depth articles on some of the topics that have been discussed at the Xen Project Developer and User Summits over the coming months. Some presentations, videos and Bof notes are still missing. Please check the Dev Summit page occasionally for updates.

Xen Project will co-host FOSDEM’14 Virtualisation and IaaS DevRoom

fosdem14

We are pleased to announce that the Xen Project will co-host a two-day Virtualisation and IaaS DevRoom at FOSDEM’14, alongside Redhat and the Openstack Foundation.

Call for Papers: Closes December 1st

The Call for Papers for the DevRoom will be open until December 1st. The scope for this devroom is open source, openly-developed projects in the areas of virtualisation and IaaS type clouds, ranging from low level to data center, up to cloud management platforms and cloud resource orchestration.

Sessions proposals should always target a developer audience and proposals for collaborative sessions that would be appealing to developers from multiple projects will be preferred. FOSDEM is about collaboration and unique in that it brings developers from many different projects and technologies together.

We (the DevRoom organizers) are particularly interested in the following themes:

  • Innvovation in low level virtualisation
  • New features in classic and container-based virtualisation technologies
  • New use cases for virtualisation, such as virtualisation in mobile, automotive and embedded in general
  • Other resource virtualisation technologies: networking, storage, … and their integration into other projects
  • Deep technical dives into specific IaaS or virtualisation management project’s features
  • Relationship between IaaS projects and specific dependencies (not just virtualisation)
  • Integration and development leveraging solutions from multiple projects

More information about the CfP and how to submit a proposal here.

FOSDEM

The Xen Project has had a strong presence at FOSDEM in the last two years and FOSDEM is always a lot of fun and very productive. It is entirely volunteer organized and free to attend, which makes the event different from many other open source events. Why not check out our event report from last year and submit a talk proposal for our DevRoom.

My first Xen Developer Summit: expectations vs reality

Last week I was in Edinburgh to attend my first Xen Project Developer Summit. It turned out quite different from my expectations, so I thought I’d share a bit about my experience.

When I first saw the call for participation I wasn’t sure that I had anything sufficiently relevant to contribute that would be of interest to the community. But I decided there wouldn’t be any harm in submitting a proposal for a talk, and it came as a bit of a surprise when I learned that it had been accepted.

My day job is to work on the performance and scalability of XenServer, Citrix’s enterprise-grade virtualisation platform built on Xen. The XenServer engineering team had recently made great strides in improving VM density scalability so it was natural for my talk to be on this topic. The reason for my uncertainty over the relevance of this talk was that it was all about stuff built around the hypervisor and not so much about the hypervisor itself. Evidently I was wrong to assume that this wouldn’t be of interest.

So who’s in the Xen community?

Once my talk had been accepted, I looked forward to attending and seeing what the Xen community looks like. My expectation was that it would be a meeting of the regulars on the xen-devel mailing list verbalising the same kind of discussions as are typically held on there. This wouldn’t have been a bad thing, but I’d have been an outsider on the sidelines, being unable to give or take a lot from this.

But I was wrong. The attendees constituted a surprisingly broad group of people. There were some from academia, others from industry. There were some doing research and others commercialising it. There were some who were hacking on the Xen hypervisor itself and others involved with construction of products built on top of it.

Luwei Cheng talks about his research on TCP performance in VMs

Luwei Cheng talks about his research on TCP performance in VMs

What about the talks?

Compared to similar-sized conferences I’ve attended in other fields of technology in the past, the quality of talks delivered at the Xen Developer Summit was generally very high; I was impressed. And there was an excellent range of presentations, covering a good variety of topics. I was surprised to find that there were many that were relevant to my own work. I was especially interested by talks on:

But it wasn’t all about performance! For those with interest in other topics, there was plenty of content concerning Xen-on-ARM, Android support, GPU integration, security, microkernel VMs, and a lot more.

So I very much enjoyed my first Xen Developer Summit. There was a real buzz around the Xen community. I learned a lot and made some great contacts, and hope that I’ll be able to return to future Summits.