Monthly Archives: May 2011

Xen support upstreamed to QEMU

My name is Anthony Perard and I have been working on up-streaming Xen patches to the QEMU mainline. The good news is that the bulk of the changes for Xen are in the QEMU mainline as of yesterday.

Let me begin a year ago, when I started working on this project. The Xen codeline contained a fork of the QEMU code implementing a “xen” target within QEMU.  The drawback of using a “xen” specific target within the QEMU source tree, was that lots of QEMU code had to be duplicated, in effect increasing the maintenance burden on the QEMU project. Unsurprisingly the code as it was, wasn’t acceptable.

Thus we embarked on a project to rewrite the Xen QEMU code. The goal was to add Xen support to QEMU with the goal to avoid code duplication.  It has taken some time: but after several revisions of the patch series, the code has finally been accepted into the QEMU mainline. A lot of effort has been spent on ensuring the quality of the changes are very high straight from the outset. This is not the end of the road: some additional patches for PCI pass-through and support for VGA dirty bitmaps are still outstanding.

Xen support in QEMU will have advantages for the QEMU as well as the Xen community. First of all, the last year has brought the two communities closer together. Going forward we hope that there will be more collaboration and sharing of ideas. Of course for the Xen community there are also advantages using vanilla QEMU: we will have access to more functionality as it is developed, such as support for the Q35 chipset within QEMU, PCI express support, and others.

I am looking forward to working more on this in the future and also wanted to thank Alexander Graf for feedback and suggestions on QEMU patches.

Xen.org Project Governance Proposal

One of the objectives that I was given, when I started as Xen.org community manager was to learn about the unwritten rules of how Xen.org governance works, write them down and make some improvement proposals. At this point, I wanted to present my findings and also make an improvement proposal. Governance changes can be quite divisive, which is why we are going to consult you and ask for input. The proposal has already been reviewed by key stake-holders in the community and by the Xen Advisory Board. The feedback we have received, has so far been positive.

The detailed proposal can be found on Xen.org and essentially consists of three parts:

  • Basic principles describing how the community works: these are Openness, Transparency, Meritocracy and Consensus Decision Making
  • Key roles in the community: these are Maintainers, Project leads and a lightweight nomination and election process
  • A lightweight project life cycle that shows how a project gets from an idea to maturity and is eventually archived. There is also a section on how this works in practice.

If you are a contributor to Xen.org or an otherwise active community member, please review the proposal. We also created a poll to find out whether the proposal resonates with you. The poll will be open for 4 weeks (until June 7th).

Introducing Jiageng Yu and Linux based stubdom’s

This is a guest blog post by Jiageng Yu, one of our Google Summer of Code students. This will be the first of a series of posts by our students.

I am Jiageng Yu, a PHD candidate of Institute of Software, Chinese Academy of Sciences, Beijing, China. My research areas are operating systems, virtual machines, and cloud computing. Personally, I am fascinated with science-fiction and magic movies, like G-Force, The Lord of the Rings, Transformers and Inception. Besides, I like visiting the interesting places in Beijing at weekend, therefore, I always find something amazing, including delicious food and magic things.

I have been accepted to GSoC 2011 and will work on the project of Linux based stubdom in next three months. I am greatly honored and excited to join the Xen community. Let me talk a bit about the project I will be working on: Linux based stubdom’s.

Stubdoms are very small Xen PV guests used to run some software components that otherwise live in dom0. I plan to implement a linux based stubdom, which provides devices emulation to a particular HVM guest. The major tasks of the this project are described below.

  1. Establish the environment of linux based stubdom with minimal linux kernel and shared libraries, which could contain the upstream qemu distribution exactly as it. Maintain a minimal upstream qemu distribution by removing unnecessary objetss in its configuration phase. The minimal linux kernel, shared libraries and the upstream qemu are packed into the ramdisk, which is the real body of stubdom we need.
  2. Adapt simulation devices of upstream qemu to the relevant frontend devices of stubdom in optimal ways. In this
    scenario, consider devices including network, disk and graphics.
  3. Implement the proper save/restore mechanism for simulation devices. Qemu running in the stubdom writes
    a statefile that has to be returned to the toolstack in dom0. Currently minios based stubdoms use two PV consoles to do that, which have not been supported in Linux. Therefore, instead of using additional consoles, create a dynamic virtual disk to transfer save/restore data between Dom0 and stubdom.
  4. Support pci pass-through devices in the stubdoms, by mapping the memory (including ioport, iomem and irq) of
    pass-through devices into the memory of frontend devices. Note that there could be a few differences to the implementation of MiniOS based stubdoms.
  5. Produce benchmarks to measure the resource usage and performance of linux stubdom in comparison with
    MiniOS stubdoms. Measure the throughput of CPU, disk and network, etc.

You can find my personal web page at cloud-os.org (chinese, translated) and get in touch with me on skype or gmail via yujiageng734.

Event Report: OpenStack Conference and Design Summit

Last week I was at the OpenStack Conference and Design summit: I thought I would share a quick report of what I found cool and interesting. Obviously this won’t be a complete picture and the report is my personal opinion. Before I do this though, I wanted to remind you that Xen and XCP are tightly integrated with OpenStack Compute and OpenStack Object Storage.

Not all presentations and videos have been posted yet: they will be available from OpenStack on their website and vimeo soon. Anyway, here is what I liked:

  • A talk by James Williams (NASA) on “NASA and OpenStack” covering the past, present and future of NASA in OpenStack.
  • A talk by Ryan Lane (Wikimedia) called The site Architecture you can edit” which shows how cloud orchestration tools (based on OpenStack) can lead to a new model of collaborative and community led system administration.
  • Ewan Mellor (Citrix) showed a demo of the OpenStack Virtual Appliance project as part of a talk given by Gordon Mangione
  • Neil Sample (eBay) analyzed the case for the Hybrid Cloud in OpenStack and eBay providing interesting data and figures showing how the Public Cloud reduces internal data center costs.
  • I have always been very interested in governance of open source projects so I was extremely excited to listen to a fairly new community coming to grips with how it is and will be governed in future (see the Governance discussion video)
  • Nati Shalom from Gigaspaces gave a very interesting presentation on PaaS on Openstack discussing different PaaS architectures, their trade-offs, leading into a discussion what architecture OpenStack should follow
  • The developer community led an excellent discussion on whether to use Launchpad or Git in future. Those issues always bring forward lots of passion in any community.
  • A saw a demo and talk of Dell’s Crowbar project given by Rob Hirschfeld. Crowbar is a Cloud Installer come Operations Foundation tool (you will have to watch the video of the demo when it goes on-line).
  • Bernard Golden (HyperStatus) talked aboutCloud Service Providers and OpenStack diving into the choices for service providers when it comes to offering Cloud solutions.
  • Of course there were many more…

I also had the opportunity to meet Xen community members from all across the world for the first time, which was fantastic. Some cool ideas for future events and activities were born. I will let you know more, as the detail develops. Watch this space!