At XenSummit in August, I talked about the new planning process that Xen.org is experimenting with. This is apparently a pretty hot topic, as that presentation on slideshare.net has received over 72,000 views! We’re about half-way through the planned 4.3 development cycle, and while I’ve been regularly tracking the status of various features on the xen-devel mailing list, and also updating them on the 4.3 roadmap wiki page, I thought it might be useful just to give an update on how things are progressing.
After concluding our poll about changes to the security discussion, we determined that “Pre-disclosure to software vendors and a wide set of users” was probably the best fit for the community. A set of concrete changes to the policy have now been discussed on xen-devel (here and here), and we seem to have converged on something everyone finds acceptable.
We are now presenting these changes for public review. The purpose of this review process is to allow feedback on the text which will be voted on, in accordance to the Xen.org governance procedure. Our plan is to leave this up for review until the third week in January. Any substantial updates will be mentioned on the blog and will extend the review time.
All feedback and discussion should happen in public on the xen-devel mailing list. If you have any suggestions for how to improve the proposal, please e-mail the list, and cc George Dunlap (george dot dunlap at citrix.com).
Read on for a summary of the updates, as well as links to the full text of the original and proposed new policies.
In part 1 of this series, I introduced the concepts of full virtualization and paravirtualization (PV), as well as the hardware virtualization (HVM) feature used by Xen (among other things) to implement full virtualization. I also introduced the concept of installing paravirtualized drivers on a fully virtualized system.
This small step, from full virtualization towards paravirtualization, begins to hint at the idea of a spectrum of paravirtualization. In this article, I will cover the historical reasons for the development of PVHVM, and finally of the newest mode, PVH.
Problems with paravirtualization: AMD and x86-64
It comes as a surprise to many people that while 32-bit paravirtualized guests in Xen are faster than 32-bit fully virtualized guests, when running in 64-bit mode, paravirtualized guests can sometimes be slower than fully virtualized guests. This is due to some changes AMD made when designing the architecture which simplified things for them, but made things more difficult for Xen.
Most modern operating systems need just two levels of protection: user mode and kernel mode. Kernel mode memory is protected from user mode memory via the pagetable “supervisor mode” bit.
At XenSummit 2012 in San Diego, Mukesh Rathor from Oracle presented his work on a new virtualization mode, called “PVH”. Adding this mode, there are now a rather dizzying array of different terms thrown about — “HVM”, “PV”, “PVHVM”, “PVH” — what do they all mean? And why do we have so many?
The reason we have all these terms is that virtualization is no longer binary; there is a spectrum of virtualization, and the different terms are different points along the spectrum. Part of the reason the terminology is a little unclear is the history; any language and terminology evolves over time in response to the changing situation. However, changing the terminology after-the-fact once certain usages become common is difficult.
So in this series of articles I will introduce just enough history to understand how the current situation came about, and (hopefully) introduce a consistent set of terminology which may help clear things up, while balancing this against the fact that people will still continue to use existing terminology.
This article will be divided into two parts. The first part will give a general introduction to virtualization, and to paravirtualization, Xen’s unique contribution to the field, as well as the advent of hardware virtualization extensions (HVM). It will also introduce the idea of adding paravirtualized drivers for disk and network. The second mode will cover the motivation and technical descriptions of two more modes which further mix elements of full virtualization and paravirtualization.
In the early days of virtualization (at least in the x86 world), the assumption was that you needed your hypervisor to provide a virtual machine that was functionally nearly identical to a real machine. This included the following aspects:
- Disk and network devices
- Interrupts and timers
- Emulated platform: motherboard, device buses, BIOS
- “Legacy” boot: i.e., starting in 16-bit mode and bootstrapping up to 64-bit mode
- Privileged instructions
- Pagetables (memory access)
In the early days of x86 virtualization, all of this needed to be virtualized: disk and network devices needed to be emulated, as did interrupts and timers, the motherboard and PCI buses, and so on. Guests needed to start in 16-bit mode and run a BIOS which loaded the guest kernel, which (again) ran in 16-bit mode, then bootstrapped its way up to 32-bit mode, and possibly then to 64-bit mode. All privileged instructions executed by the guest kernel needed to be emulated somehow; and the pagetables needed to be emulated in software.
This is mode, where all of the aspects the virtual machine must be functionally identical to real hardware, is what I will call fully virtualized mode.
Xen and paravirtualization
Unfortunately, particularly for x86, virtualizing privileged instructions is very complicated. Many instructions for x86 behave differently in kernel and user mode without generating a trap, meaning that your options for running kernel code were to do full software emulation (incredibly slow) or binary translation (incredibly complicated, and still very slow).
The key question of the original Xen research project at Cambridge University was, “What if instead of trying to fool the guest kernel into thinking it’s running on real hardware, you just let the guest know that it was running in a virtual machine, and changed the interface you provide to make it easier to implement?” To answer that question, they started from the ground up designing a new interface designed for virtualization. Working together with researchers at both the Intel and Microsoft labs, they took both Linux and Windows XP, and ripped out anything that was slow or difficult to virtualize, replacing it with calls into the hypervisor (hypercalls) or other virtualization-friendly techniques. (The Windows XP port to Xen 1.0, as you might imagine, never left Microsoft Research; but it was benchmarked in the original paper.)
One of the issues in any software project can be the disconnect between what users want or would find useful, and the developers’ idea of what users want or would find useful, and Xen is no exception.
To help address this issue, we are starting to experiment with a website called UserVoice. UserVoice allows users to do two things:
- Suggest improvements or features
- “Vote” for which improvements or features they think are most important.
You can find the Xen.org uservoice page here:
You can either create an account, or log in with an existing Google or Facebook account.
Obviously we can’t promise that everything that is suggested or voted highly will be implemented! But hopefully it will give the developers a better idea what our users are thinking.
To make sure that the site is as useful as possible, please try focus on things you want to do, rather than how you want to be able to do them. That will help us choose the best way to help you; or perhaps point you to an existing way of doing something that’s already available.
Monday we closed the poll for the security discussion. Thank you everyone who participated! The process has not turned up a hidden option that everyone agreed on; however, it has helped find what I hope will be a “median” option which best addresses the concerns and desires as the community as a whole. Below I give a description of the results of the poll, and a recommendation as to what I think is the best way forward.