Tag Archives: policy

Updates to Xen Project Security Process

lockBefore Christmas, the Xen Project ran a community consultation to refine its Security Problem Response Process.  We recently approved changes that, in essence, are tweaks to our existing process, which is based on a Responsible Disclosure philosophy.

Responsible Disclosure and our Security Problem Response Process are important components of keeping users of Xen Project-based products and services safe from security exploits. Both ensure that products and services can be patched by members of the pre-disclosure list before details of a vulnerability are published and before said vulnerabilities can be exploited by black hats.

The changes to our response process fall into a number of categories:

  • Clarify whether security updates can be deployed on publicly hosted systems (e.g. cloud or hosting providers) during embargo
  • Sharing of information among pre-disclosure list members
  • Applications procedure for pre-disclosure list membership

The complete discussion leading to the changes, the concrete changes to the process, and the voting records supporting the changes are tracked in Bug #44 -Security policy ambiguities. On February 11, 2015, the proposed changes were approved in accordance with Xen Project governance. Note that some process changes are already implemented, whereas others are waiting for implementation tasks (e.g. new secure mailing lists) before they can fully be put in place. We have however updated our Security Problem Response Process as the most important elements of the process are already in place.

Process Changes Already in Operation

The updated policy makes explicit whether or not patches related to a Xen Security Issue can be deployed by pre-disclosure list members. The concrete policy changes can be found here and here. In practice, every Xen Security Advisory will contain a section such as:

DEPLOYMENT DURING EMBARGO
=========================

Deployment of the patches and/or mitigations described above (or
others which are substantially similar) is permitted during the
embargo, even on public-facing systems with untrusted guest users and
administrators.

Predisclosure list members who wish to deploy significantly different
patches and/or mitigations, please contact the Xen Project Security
Team.

This section will clarify whether deploying fixed versions of Xen during the embargo is allowed. Any restrictions will also be stated in the embargoed advisory. The Security Team will impose deployment restrictions only to prevent the exposure of security vulnerability technicalities, which present a significant risk of vulnerability rediscovery (for example, by visible differences in behaviour). Such situations have been, and are expected, to be rare.

Changes to Application Procedure for Pre-disclosure List Membership

We also made additional changes related to streamlining and simplifying the process of applying for pre-disclosure list membership. Detailed policy changes can be found here and here. Moving forward, future applications to become members of the Xen Project pre-disclosure list have to be made publicly on the predisclosure-applications mailing list. This enables Xen Project community members to provide additional information and also is in line with one of our community’s core principles: transparency. In addition, we’ve clarified our eligibility criteria to make it easier for the Xen Project Security Team, as well as observers of the mailing list, to verify whether applicants are eligible to become members of the list.

Process Changes Not Fully Implemented:  Sharing of Information Among Pre-disclosure List Members

Finally, members of the pre-disclosure list will be explicitly allowed to share fixes to embargoed issues, analysis, and other relevant information with the security teams of other pre-disclosure members. Information sharing will happen on a private and secure mailing list hosted by the Xen Project.  Note the list is not yet in place; more details here.

Will you give Xen a ride … or will Xen give you a ride?

And by “a ride”, we actually mean a ride. Like this:

 

8275748195_4a18513755_z

Like, will Xen run in your car?  Well, it appears it will!

It all started with ARM Support

In fact, Xen Project developers started woking on supporting the ARM architecture (with hardware virtualization capabilities) a couple of years ago. The goal was simple: as soon as ARM server are available, it must be possible to run Xen Project software on them. That goal has been achieved, but that is another story!

It is well known that processors employing the ARM architecture are powering already the vast majority of the so called Embedded Systems, ranging from phones, tablets and smart TVs up to cars or even airplanes. But does that mean that at some point we will start to see virtualization capable chips in cars? And if yes, when? The answers to these questions are “Yes” and “really really really soon”! In fact, the Xen Project Hypervisor is uniquely placed to support this new range of use-cases. Its isolation and security features, flexible virtualization mode and architecture, not to mention driver disaggregation and the fact that it now supports ARM (and does it with only ~90K lines of code), make it a perfect fit for the embedded world.

Some Recent ‘History’

Mobile and embedded virtualization on ARM has a long history within the Xen Project, with research projects such as Samsung’s ARM PV port and the Embedded Xen effort. However these projects were mainly research focused. With ARM support becoming a part of the Xen Project Hypervisor last year and various market factors coming together, Xen Project based products are now on the horizon. Last autumn was pivotal in generating momentum for this concept. A number of companies showed real demos and prototypes at our 2013 Developer Summit, such as

  • The Xen Project Hypervisor running on a Nexus 10 (slides and video)
  • The Xen Project Hypervisor powering an in-vehicle infotainment (IVI) system, and other systems on the TI Jacinto 6 automotive platform designed for cars (slides and video).

Since then, momentum has built within the community – as can be seen on xen-devel mailing list discussions – to port embedded OSes to the Xen Hypervisor  (some examples: FreeRTOS, Erika and QNX). Contributions and patches for making The Xen Project Hypervisor work better in such environments started to arrive too, from individuals, research institutions and small and big companies. Among the companies, GlobalLogic Inc., a full-lifecycle product development services company, has made the largest contribution so far, but we must also mention DornerWorks, GaloisUniversity of Washington and Evidence (in collaboration with the University of Modena).

A summary of the past and ongoing activities of this kind is below:

What about now?

On Monday (we told you: “really really really soon” :-D), The Xen Collaborative Project and The Linux Foundation announced a new Embedded and Automotive initiative. Artem Mygaiev, AVP Development at GlobalLogic, will serve as the Embedded and Automotive Project Lead.

The Embedded and Automotive team within The Xen Project intends to build a platform around the Xen Hypervisor that enables using it for all the non-data center use cases (automotive, internet TV, mobile, etc.) by providing a community focal-point within the Xen Project community as well as within the wider open source community.

The team plans to:

  • develop and upstream necessary changes to The Xen Project Hypervisor and Linux
  • implement new drivers (such as GPU, HID, …), protocols, capabilities and functionality that are needed for a complete automotive/embedded/mobile virtualization stack
  • upstream all necessary changes to support such functionality in operating systems that are needed for these use-cases (e.g. Android, Linux, etc.)

For the occasion, Alex Agizim, CTO of Embedded Systems at GlobalLogic, which also is a member of The Linux Foundation Automotive Grade Linux Steering Committee, said:

With ARM support, Xen Project technology is a perfect fit for embedded systems and automotive use. For example, our Nautilus platform, based on The Xen Project virtualization, enables ourin-vehicle infotainment (IVI) and auto manufacturing partners to quickly and cost-effectively develop hybrid Android/Linux-based systems. Using Nautilus, developers are able to run multiple sandboxed OSes on a single System-on-Chip (SOC). This provides superior functionality and security for both infotainment and operational functions within a car.

The latest demo of GlobalLogic‘s Nautilus Platform has been shown at the latest edition of the Automotive Linux Summit, in Tokyo. Check out the video and slides. We also heard about further use cases for Xen Project Software at this week’s Developer Summit. The rate of innovation in our community in this area is staggering: fasten your seat belts! We will tell you about these more in an upcoming event report. All this activity is also creating many benefits for the cloud and traditional server use use-cases. Certification will lead to quality improvements across shared components. Realtime scheduling can be used for graphics and gaming use-cases in the cloud and for Network Function Virtualization. And so on, and so on, …

Learn More

GlobalLogic, in partnership with The Linux Foundation, will present a free webinar at 9 a.m. PDT, Wednesday, August 27, 2014, titled “Virtualization in the Automotive Industry.” Register today to learn how Xen Project technology adds reliability and security when adopting virtualization for automotive software development.

Vendors and individual developers interested in collaborating on embedded, automotive and mobile use cases are encouraged to join the new Xen Project subproject at http://xenproject.org/developers/teams/embedded-and-automotive.html.

Release management: Risk, intuition, and freeze exceptions

I’ve been release coordinator for Xen’s 4.3 and 4.4 releases. For the 4.5 release, I’ve handed this role off to Konrad Wilk, from Oracle. In this blog, I try to capture some of my thoughts and experience about one aspect of release management: deciding what patches to accept during a freeze.

I have three goals when doing release management:

  1. A bug-free release
  2. An awesome release
  3. An on-time release

One of the most time-consuming seasons of being a release manager is during any kind of freeze. You can read in detail about our release process elsewhere; I’ll just summarize it here. During normal development, any patch which has the approval of the relevant maintainer can be accepted. As the release approaches, however, we want to start being more and more conservative in what patches we accept.

Obviously, no patch would ever be considered for acceptance which didn’t improve Xen in some way, making it more awesome. However, it’s a fact of software that any change, no matter how simple or obvious, may introduce a bug. If this bug is discovered before the release, it may delay the release, making it not on-time; or it may not be found until after the release, making the release not bug-free. The job of helping decide whether to take a patch or not falls to the release coordinator.

But how do you actually make decisions? There are two general things to say about this.

Risk

The only simple rule to follow that will make sure that there are no new bugs introduced is to do no development at all. Since we must do development, we have to learn to deal with risk.

Making decisions about accepting or rejecting patches as release coordinator is about making calculated risks: look at the benefits, look at the potential costs, look at the probabilities, and try to balance them the best you can.

Part of making calculated risks is accepting that sometimes your gamble won’t pay off. You may approve a patch to go in, and it will then turn out to have a bug in it which delays the release. This is not necessarily a failure: if you can look back at your decision and say with honesty, “That was the best decision I could have made given what I knew at the time”, then your choice was the right one.

The extreme example of this kind of thinking that of a poker player: a poker player may make a bet that she knows she only has a 1 in 4 chance of winning, if the pay-off is more than 4 to 1; say, 5 to 1. Even though she loses 75% of the time, the 25% that she does win will pay for the losses. And when she makes the bet and loses (as she will 75% of the time), she knows she didn’t make a mistake; taking risks is just a part of the game.

Obviously as release coordinator, the costs of a bug are generally higher than the benefits of a feature. But the general principle — of taking calculated risks, and accepting occasional failure as the inevitable consequence of doing so — applies.

Intuition

But how do we actually calculate the risks? A poker player frequently deals with known quantities: she can calculate that there is exactly a 25% chance of winning, exactly a 5x monetary pay-off, and do the math; the release coordinator’s decisions are not so quantifiable.

shutterstock_58404295This is where intuition comes in. While there are a handful of metrics that can be applied to patches (e.g., the number of lines in the patch), for the most part the risk and benefit are not very quantifiable at all: expert judgement / intuition is the only thing we have.

Now, research has shown that the intuition of experts can, under the right circumstances, be very good. Intuition can quickly analyze hundreds of independent factors, and compare against thousands of instances, and give you a result which is often very close to the mark.

However, research has also shown that in other circumstances, expert intuition is worse than random guessing, and far worse than a simple algorithm. (For a reference, see the books listed at the bottom.)

One of the biggest ways intuition goes wrong is by only looking at part of the picture. It is very natural for programmers, when looking at a patch, to consider only the benefits. The programmer’s intuition then accurately gives them a good sense of the advantage of taking the patch; but doesn’t warn them about the risk because they haven’t thought about it. Since they have a positive feeling, then they may end up taking a patch even when it’s actually too risky.

The key then is to make sure that your intuition considers the risks properly, as well as the benefits. To help myself with this, during the 4.4 code freeze I developed a sort of checklist of things to think about and consider. They are as follows:

  1. What is the benefit of this patch?
  2. What is the probability this patch has a bug?
  3. If this patch had a bug, what kind of bug might it be?
  4. If this patch had a bug, what is the probability we would find it
    before the release?

When considering the probability of a bug, I look at two things:

  • The complexity of the patch
  • My confidence in my / the other reviewers’ judgement.

Sometimes you’re looking at code you’re very familiar with or is straightforward; sometimes you’re looking at code that is very complicated or you’re not that familiar with. If the patch looks good and it’s code you’re familiar with, it’s probably fine. If the patch looks good but it’s code you’re not familiar with, there’s a risk that your judgement may be off.

When trying to think of what kind of bug it might be, I look at the code that it’s modifying, and consider things on a spectrum:

  1. Error / tool crash on unexpected trusted input; or normal input to library-only commands
  2. Error / tool crash on normal input, secondary commands / new functionality
  3. Error / tool crash on normal input, core commands
  4. Performance
  5. Guest crash / hang
  6. Host crash / hang
  7. Security vulnerability
  8. Data loss

Usually you can tell right away where in the list a bug might be. Modifying xenpm or xenctx? 3 max. Modifying the scheduler? Probably #6. Modifying hypercalls available to guests? #7. And so on.

When asking whether we’d find the bug before the release, consider the kind of testing the codepath is likely to get. Is it tested in osstest? In XenRT? Or is it in a corner case that few people really use?

After thinking through those four questions, and going over the criteria in detail, then my intuition is probably about as well-formed as it’s going to get. Now I ask the fifth question: given the risks, is it worth it to accept this patch?

After giving it it some thought, I went with my best guess. Sometimes I’m just not sure; in which case go away and do something else for a couple of hours, then come back to it (going over again the four questions to make sure they’re fresh in my mind). The first few dozen times this took a very long time; as I gained experience, judgements came faster (although many were still painfully slow).

In some cases, I just didn’t have enough knowledge of the code to make the judgement myself; this happened once or twice with the ARM code in the 4.4 release. In that case, my goal was to try to make sure that those who did have the relevant knowledge were making sound decisions: thinking about both the benefits and the risks and weighing them appropriately.

For those who want to look further into risk and intuition, several books have had a pretty big influence on my thinking in this area. Probably the best one, but also the hardest (most dense) one, is Thinking, Fast and Slow, by Daniel Kahneman. It’s very-well written and accessible, but just contains a huge amount of information that is different to the way you normally think. Not a light read. Another one I would recommend is The Black Swan, by Nassim Nicholas Taleb. And finally, Blink, by Malcolm Gladwell.

How fast is Xen on ARM, really?

With Xen on ARM getting out of the early preview phase and becoming more mature, it is time to run a few benchmarks to check that the design choices paid out, the architecture is sound and the code base is solid. It is time to find out how much is the overhead introduced by Xen on ARM and how it compares with Xen and other hypervisors on x86.
I measured the overhead by running the same benchmark on a virtual machine on Xen on ARM and on native Linux on the same hardware. It takes a bit longer to complete the benchmark inside a VM, but how much longer? The answer to this question is the virtualization overhead.

Setup

I chose AppliedMicro X-Gene as the ARM platform to run the benchmarks on: it is an ARMv8 64-bit SoC with an 8 cores cpu and 16GB of RAM. I had Dom0 running with 8 vcpus and 1GB of RAM, the virtual machine that ran the tests had 2GB of RAM and 8 vcpus. To make sure that the results are comparable I restricted the amount of memory available to the native Linux run, so that Linux had all the 8 physical cores at its disposal but only 2GB of RAM.

For the x86 tests, I used a Dell server with an Intel Xeon x5650, that is a 6 cores HyperThreading cpu. HyperThreading was disabled during the tests for better performances. Similarly to the ARM tests, I had Dom0 running with 6 vcpus and 1GB of RAM and the virtual machine running with 2GB of RAM and 6 vcpus. The native Linux run had 6 physical cores and 2GB of RAM. For the KVM tests I booted the host with 3GB of RAM, then assigned 2GB of RAM to the KVM virtual machine.

In terms of software on both ARMv8 and x86 I used:

  • Linux 3.13 as Dom0, DomU and native kernel
  • Xen 4.4
  • OpenSUSE 13.1
  • QEMU-KVM 1.6.2 (for the KVM tests on x86)

I could not test KVM on ARMv8 because KVM support for X-Gene is not upstream in Linux 3.13.

Benchmarks – lower is better

The y-axis shows the overhead in terms of percentage of native: “0%” means that it is a fast as native. “1%” means that it takes 1% longer than native Linux to complete the benchmark inside a virtual machine. Given that we are dealing with overheads, lower is better.

Kernbench

Kernbench is a popular benchmark that measures the time that it takes to compile the Linux kernel. It is a cpu and memory intensive benchmark.
chart_4 (1)

PBzip2

PBzip2 is a parallel implementation of bzip2. This benchmark measures the time that it takes to compress a 4GB file.
chart_3

SPECjbb2005 (non-compliant)

SPECjbb2005 simulates a Java server workload. It is a cpu and memory bound benchmark.
The runs are non-compliant (therefore cannot be compared with compliant runs) and the overhead is calculated on the peak warehouse alone.
chart_7

Next I ran a couple of disk IO benchmarks, but both X-Gene and the Dell server came with spinning disks for storage: the following tests showed that both disks were too slow to actually measure the virtualization overhead (it is lower than 1%).

FIO

FIO is a popular tool to measure disk performances. This benchmark uses FIO to perform a combination of random reads and writes and measures the overhead on iops.
chart_5

PGBench

PGBench is the PostgresSQL database benchmarking tool. This benchmark is disk IO bound.
chart_6

Conclusions

Developing Xen on ARM we have been focused on correctness and feature completeness rather than performances. Nonetheless it provides a very lower overhead that is already on par or lower than Xen’s on x86, that in turn is lower than KVM’s on x86. Given the benefits that virtualization brings to the table, including ease of deployment and lower downtimes, it really makes sense to deploy Xen on your ARM based cloud.

Summary of the gains of Xen oxenstored over cxenstored

This week, we are reblogging this excellent piece from Luis from SUSE. The article came about because of a discussion Luis had at the Linux Foundation Collaboration Summit in Napa, and he decided to write down some basic generals of the xenstore, a review of its first implementation and a summary of oxenstored. The original post is available here, on Luis’ blog.

OXenstored is the default in Xen, only if the ocaml compiler was installed at build time. This means that in many Linux distributions that ship Xen, cxenstored is most likely your default. You can check whether oxenstored is installed by checking whether the oxenstored process runs in your Dom 0.

It all started with a paper: OXenstored – An Efficient Hierarchical and Transactional Database using Functional Programming with Reference Cell Comparisons

First a general description of the xenstore and its first implementation. The xenstore is where Xen stores the information over its systems. It covers dom0 and guests and it uses a filesystem type of layout kind of how we keep a layout of a system on the Linux kernel in sysfs. The original xenstored, which the paper refers to a Cxenstored was written in C. Since all information needs to be stored in a filesystem layout any library or tool that supports designing a tree to have key value store of information should suffice to upkeep the xenstore. The Xen folks decided to use the Trivial Database, tdb, which as it turns out was designed and implemented by the Samba folks for its own database. Xen then has a daemon sitting in the background which listens to reads / write requests onto this database, that’s what you see running in the background if you ‘ps -ef | grep xen’ on dom0. dom0 is the first host, the rest are guests. dom0 uses Unix domain sockets to talk to the xenstore while guests talk to it using the kernel through the xenbus. The code for opening up a connection onto the c version of the xenstore is in tools/xenstore/xs.c and the the call is xs_open(). The first attempt by code will be to open the Unix domain socket with get_handle(xs_daemon_socket()) and if that fails it will try get_handle(xs_domain_dev()), the later will vary depending on your Operating System and you can override first by setting the environment variable XENSTORED_PATH. On Linux this is at /proc/xen/xenbus. All the xenstore is doing is brokering access to the database. The xenstore represents all data known to Xen, we build it upon bootup and can throw it out the window when shutting down, which is why we should just use a tmpfs for it (Debian does, OpenSUSE should be changed to it). The actual database for the C implementation is by default stored under the directory /var/lib/xenstored, the file that has the database there is called tdb. On OpenSUSE that’s /var/lib/xenstored/tdb, on Debian (as of xen-utils-4.3) that’s /run/xenstored/tdb. The C version of the xenstore therefore puts out a database file that can actually be used with tdb-tools (actual package name for Debian and SUSE). xentored does not use libtdb which is GPLv3+, Xen in-takes the tdb implementation which is licensed under the LGPL and carries a copy under tools/xenstore/tdb.c. Although you shouldn’t be using tdb-tools to poke at the database you can still read from it using these tools, you can read the entire database as follows:

 tdbtool /run/xenstored/tdb dump

The biggest issue with the C version implementation and relying on tdb is that you can live lock it if you have a guest or any entity doing short quick accesses onto the xenstore. We need Xen to scale though and the research and development behind oxenstored was an effort to help with that. What follows next is my brain dump of the paper. I don’t get into the details of the implementation because as can be expected I don’t want to read OCaml code. Keep in mind that if I look for a replacement I’m looking also for something that Samba folks might want to consider.

OXenstored has the following observed gains:

  • 1/5th the size in terms of line of code in comparison to the C xenstored
  • better performance increasing support for the number of guests, it supports 3 times number of guests for an upper limit of 160 guests

The performance gains come from two things:

  • how it deals with transactions through an immutable prefix tree. Each transaction is associated with a triplet (T1, T2, p) where T1 is the root of the database just before a transaction, T2 is the local copy of the database with all updates made by the transaction made up to that point, p is the path to the furthest node from the root T2 whose subtree contains all the updates made by the transaction up that point.
  • how it deals with sharing immutable subtrees and uses ‘reference cell equality’, a limited form of pointer equality, which compares the location of values instead of the values themselves. Two values are shared if they share the same location. Functional programming languages enforce that multiple copies of immutable structures share the same location in memory. oxenstored takes avantage of this functional programming feature to design trie library which enforces sharing of  subtrees as much as possible. This lets them simpilfy how to determine and merge / coalesce concurrent transactions.

The complexity of the algorithms used by oxenstored is confined only to the length of the path, which is rarely over 10. This gives predictable performance regardless of the number of guests present.

XenGT – a Full Graphics Virtualization Solution on Intel ® Processor Graphics

Background

The Graphics Processing Unit (GPU) has become a fundamental building block in today’s computing environment, accelerating tasks from entertainment applications (gaming, video playback, etc.) to general purpose windowing (Windows* Aero*, Compiz Fusion, etc.) and high performance computing (medical image processing, weather forecast, computer aided designs, etc.).

Today, we see a trend toward moving GPU-accelerated tasks to virtual machines (VMs). Desktop virtualization simplifies the IT management infrastructure by moving a worker’s desktop to the VM. In the meantime, there is also demand for buying GPU computing resources from the cloud. Efficient GPU virtualization is required to address the increasing demands.

Enterprise applications (mail, browser, office, etc.) usually demand a moderate level of GPU acceleration capability. When they are moved to a virtual desktop, our integrated GPU can easily accommodate the acceleration requirements of multiple instances.

GPU Background

Let’s first look at the architecture of Intel Processor Graphics:

arch_of_intel_graphics

The render engine represents the GPU acceleration capabilities with fixed pipelines and execution units, which are used through GPU commands queued in command buffers. The display engine routes data from graphics memory to external monitors, and contains states of display attributes (resolution, color depth, etc.). The global state represents all the remaining functionality, including initialization, power control, etc. Graphics memory holds the data, used by the render engine and display engine.

The Intel Processor Graphics uses system memory as the graphics memory, through the graphics translation table (GTT). A single 2GB global virtual memory (GVM) space is available to all GPU components through the global GTT(GGTT). In the meantime, multiple per-process virtual memory (PPVM) spaces are created through the per-process GTTs (PPGTTs), extending the limited GVM resource and enforcing process isolation.

Graphics Virtualization Technologies

Several technologies achieve graphics virtualization, as illustrated in the image below, with more hardware acceleration toward the right.

virtual_gfx_techs

Device emulation is mainly used in server virtualization, with emulation of an old VGA display card. Qemu is the most widely used vehicle. Full emulation of a GPU is almost impossible, because of complexity and extremely poor performance.

API forwarding implements a frontend/backend driver pair. The frontend driver forwards high-level DirectX/OpenGL API calls from the VM to the backend driver in the host through an optimized inter-VM channel. Multiple backend drivers behave like normal 3D applications in the host, so a single GPU can be multiplexed to accelerate multiple VMs. However, the difference between the VM and host graphics stacks easily leads to reduced performance or compatibility issues. Because it is hardware-agnostic, this is the most widely used technology, so far. Actual implementations vary, depending on the level where forwarding happens. For example, VMGL directly forwards GL commands, while VMware vGPU presents itself as a virtual device, with high-level DirectX calls translated to its private SVGA3D protocol. Another recent example is Virgil, with its experimental virtual 3D support for QEMU.

Direct pass-through, based on VT-d, assigns the whole GPU exclusively to a single VM. When achieving the best performance, it sacrifices the sharing capability.

Mediated pass-through extends direct pass-through, using a software approach. Every VM is allowed to access partial device resources without hypervisor intervention, while privileged operations are mediated through a software layer. It sustains the performance of direct pass-through, while still provides the sharing capability. XenGT adopts this technology. 

XenGT

XenGT is a full GPU virtualization solution with mediated pass-through, on Intel Processor Graphics. A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature and sharing capability.

arch_of_xengt

Above figure shows the overall XenGT architecture. Each VM is allowed to access a partial performance critical resource without hypervisor intervention. Privileged operations are trapped by Xen and forwarded to the mediator for emulation. The mediator emulates a virtual GPU instance for each VM. Context switches are conducted by the mediator when switching the GPU between VMs. XenGT implements the mediator in dom0. This avoids adding complex device knowledge to Xen, and also permits a more flexible release model. In the meantime, we want to have a unified architecture to mediate all the VMs, including dom0, itself. So, the mediator is implemented as a separate module from dom0’s graphics driver. It brings a new challenge, that Xen must selectively trap the accesses from dom0’s driver while granting permission to the mediator. We call it a “de-privileged” dom0 mode.

Performance critical resources are passed through to a VM:

  • Part of the global virtual memory space
  • VM’s own per-process virtual memory spaces
  • VM’s own allocated command buffers (actually in graphics memory)

This minimizes hypervisor intervention in the critical rendering path. Even when a VM is not scheduled to use the render engine, that VM can continuously queue commands in parallel.

Other operations are privileged, and must be trapped and emulated by the mediator, including:

  • MMIO/PIO
  • PCI configuration registers
  • GTT tables
  • Submission of queued GPU commands

The mediator maintains the virtual GPU instance based on the traps mentioned above, and schedules use of the render engine among VMs to ensure secure sharing of the single physical GPU.

Current Status

The latest source codes and the setup guide are available at the github repositories:

(The first repository has a XenGT_Setup_Guide.pdf, which supplies step-by-step instructions for getting a system set up.)

Linux: https://github.com/01org/XenGT-Preview-kernel.git

Xen: https://github.com/01org/XenGT-Preview-xen.git

Qemu: https://github.com/01org/XenGT-Preview-qemu.git

Patches are welcomed!

We plan to upstream this work, and are now preparing some cleanup.

 

Changelog

XenGT was first announced in Sep 2013:

http://lists.xen.org/archives/html/xen-devel/2013-09/msg00681.html

It was presented at the 2013 Xen Project Developer Summit, Edinburgh:

http://events.linuxfoundation.org/sites/events/files/slides/XenGT-Xen%20Summit-v7_0.pdf

An update was announced recently in Feb 2014:
http://lists.xen.org/archives/html/xen-devel/2014-02/msg01848.html