Author Archives: Lars Kurth

About Lars Kurth

Lars Kurth is a highly effective, passionate community manager with strong experience of working with open source communities (Symbian, Symbian DevCo, Eclipse, GNU) and currently is community manager for xen.org. Lars has 9 years of experience building and leading engineering teams and a track record of executing several change programs impacting 1000 users. Lars has 16 years of industry experience in the tools and mobile sector working at ARM, Symbian Ltd, Symbian Foundation and Nokia. Lars has strong analytical, communication, influencing and presentation skills, good knowledge of marketing and product management and extensive background in C/C , Java and software development practices which he learned working as community manager, product manager, chief architect, engineering manager and software developer. If you want to know more, check out uk.linkedin.com/in/larskurth. Personally, Lars has a wide range of interests such as literature, theatre, cinema, cooking and gardening. He is particularly fascinated by orchids and carnivorous plants and has built a rather large collection of plants from all over the world. His love for plants extends into a passion for travel, in particular to see plants grow in their native habitats.

Xen Project Spectre/Meltdown FAQ

Updated to v3 on Dec 12th!

Google’s Project Zero announced several information leak vulnerabilities affecting all modern superscalar processors. Details can be found on their blog, and in the Xen Project Advisory 254. To help our users understand the impact and our next steps forward, we put together the following FAQ.

Note that we will update the FAQ as new information surfaces.

Changes since the initial publication:

  • v3: Added information related to Comet mitigation for Variant 3 (Meltdown) – for now Xen 4.10 only
  • v2: Added information related to Vixen mitigation for Variant 3 (Meltdown) – see Are there any patches for the vulnerability?
  • v2: Replaced SPx with Variant x to be in line with the terminology used elsewhere vulnerability?

Is Xen impacted by Meltdown and Spectre?

There are two angles to consider for this question:

  • Can an untrusted guest attack the hypervisor using Meltdown or Spectre?
  • Can a guest user-space program attack a guest kernel using Meltdown or Spectre?

Systems running Xen, like all operating systems and hypervisors, are potentially affected by Spectre (referred to as Variant 1 and 2 in Advisory 254). For Arm Processors information, you can find which processors are impacted here.  In general, both the hypervisor and a guest kernel are vulnerable to attack via Variant 1 and 2.

Only Intel processors are impacted by Meltdown (referred to as Variant 3 in Advisory 254). On Intel processors, only 64-bit PV mode guests can attack Xen using Variant 3. Guests running in 32-bit PV mode, HVM mode, and PVH mode (both v1 and v2) cannot attack the hypervisor using Variant 3. However, in 32-bit PV mode, HVM mode, and PVH mode (both v1 and v2), guest userspaces can attack guest kernels using Variant 3; so updating guest kernels is advisable.

Interestingly, guest kernels running in 64-bit PV mode are not vulnerable to attack using Variant 3, because 64-bit PV guests already run in a KPTI-like mode.

However, keep in mind that a successful attack on the hypervisor can still be used to recover information about the same guest from physical memory.

Is there any risk of privilege escalation?

Meltdown and Spectre are, by themselves, only information leaks. There is no suggestion that speculative execution can be used to modify memory or cause a system to do anything it might not have done already.

Where can I find more information?

We will update this blog post and Advisory 254 as new information becomes available. Updates will also be published on xen-announce@.

We will also maintain a technical FAQ on our wiki for answers to more detailed technical questions that emerge on xen-devel@ (and in particular this e-mail thread) and other communication channels.

Are there any patches for the vulnerability?

We have published a mitigation for Meltdown on Intel CPUs: please refer to Advisory 254. The published solutions are labelled Vixen and Comet (also see README.which-shim). Alternative solutions are being worked on.

A Mitigation for Variant 2/CVE-2017-5715 is available on the xen-devel@ mailing list, but has not yet undergone rigorous enough review to be published as an official patch.

As information related to Meltdown and Spectre is now public, development will continue in public on xen-devel@ and patches will be posted and attached to Advisory 254 as they become available in the next few days.

Can Variant 1/Variant 2 be fixed at all? What plans are there to mitigate them?

Variant 2 can be mitigated in two ways, both of which essentially prevent speculative execution of indirect branches. The first is to flush the branch prediction logic on entry into the hypervisor. This requires microcode updates, which Intel and AMD are in the process of preparing, as well as patches to the hypervisor which are also in process and should be available soon.

The second is to do indirect jumps in a way which is not subject to speculative execution. This requires the hypervisor to be recompiled with a compiler that contains special new features. These new compiler features are also in the process of being prepared for both gcc and clang, and should be available soon.

Variant 1 is much more difficult to mitigate. We have some ideas we’re exploring, but they’re still at the design stage at this point.

Does Xen have any equivalent to Linux’s KPTI series?

Linux’s KPTI series is designed to address Variant 3 only. For Xen guests, only 64-bit PV guests are able to execute a Variant 3 attack against the hypervisor. A KPTI-like approach was explored initially, but required significant ABI changes.  Instead we’ve decided to go with an alternate approach, which is less disruptive and less complex to implement. The chosen approach runs PV guests in a PVH container, which ensures that PV guests continue to behave as before, while providing the isolation that protects the hypervisor from Variant 3. This works well for Xen 4.8 to Xen 4.10, which is currently our priority.

For Xen 4.6 and 4.7, we are evaluating several options, but we have not yet finalized the best solution.

Devicemodel stub domains run in PV mode, so is it still more safe to run device models in a stub domain than in domain 0?

The short answer is, yes, it is still safer to run stub domains than otherwise.

If an attacker can gain control of the device model running in a stub domain, it can indeed attempt to use these processor vulnerabilities to read information from Xen.

However, if an attacker can gain control of a device model running in domain 0 without deprivileging, the attacker can gain control of the entire system.  Even with qemu deprivileging, the qemu process may be able to execute speculative execution attacks against the hypervisor.

So although XSA-254 does affect device model stub domains, they are still safer than not running with a stub domain.

What is the Xen Project’s plan going forward?

The Xen Project is working on finalizing solutions for Variant 3 and Variant 2 and evaluating options for Variant 1. If you would like to stay abreast on our progress, please sign up to xen-announce@. We will update this FAQ as soon as we have more news and updated information. Answers to more detailed technical questions will be maintained in a technical FAQ on our wiki. Thank you for your patience.

How can I ask further questions?

Please respond to this e-mail thread on xen-devel@ or xen-users@.

Announcing the Xen Project 4.10 RC and Test Day Schedules

On Monday, we created Xen 4.10 RC1 and will release a new release candidate every MONDAY, until we declare a release candidate as the final candidate and cut the Xen 4.10 release. We will also hold a Test Day every WEDNESDAY for the release candidate that was released the week prior to the Test Day starting from RC2. Note that RC’s are announced on the following mailing lists: xen-announce, xen-devel and xen-users. This means we will have Test Days on October 25th, Nov 1st, 8th, 15th and 22nd. Your testing is still valuable on other days, so please feel free to send Test Reports as outlined below at any time.

Getting, Building and Installing a Release Candidate

Release candidates are available from our git repository at

git://xenbits.xenproject.org/xen.git (tag 4.10.0-<rc>)

where <rc> is rc1, rc2, rc3, etc. and as tarball from

https://downloads.xenproject.org/release/xen/4.10.0-<rc>/xen-4.10.0-<rc>.tar.gz
https://downloads.xenproject.org/release/xen/4.10.0-<rc>/xen-4.10.0-<rc>.tar.gz.sig

Detailed build and Install instructions can be found on the Test Day Wiki. Make sure you check the known issues section of the instructions before trying to download an RC.

Testing new Features, Test and Bug Reports

You can find Test Instructions for new features on our Test Day Wiki and instructions for general tests on Testing Xen. The following pages provide information on how to report successful tests and how to report bugs and issues.

Happy Testing!

Recap of LinuxCon China and Xen Project’s Growth in the Region

It’s been a very busy month or so for the Xen Project. During mid-June, I was lucky to attend and speak at LinuxCon + ContainerCon China held in Beijing. There I spoke on the topic of securing embedded systems with the hypervisor and live patching, virtual machine introspection and vulnerability management alongside my colleague Cheng Zhang of Citrix.

Open source has grown tremendously in China over the last few years, with Xen Project technology being a key enabler for cloud computing. Most recently, the Xen Project announced Huawei joining the Project’s advisory board. Huawei is one of a growing number of Chinese companies leveraging and contributing to the Xen Project’s software. Other organizations include Alibaba, Fujitsu (China), Intel (China), Tencent, Inspur, and more.

The Xen Project hypervisor currently powers Alibaba Cloud, which is growing at a massive rate with incredible potential.

Many ask why is this growth happening in China and why now? There are many different reasons, but I think the main point is: As key technologies are increasingly built collaboratively, more and more Chinese companies are using open source to leapfrog competitors. By joining Linux Foundation projects, in-country organizations are helping to drive further growth and development.

Collaboration in China and at the Conference

Contributions with the Xen Project have greatly expanded over the last few years, especially in contributions and membership coming from China. In our latest release, Xen Project 4.9, we had 25% more contributors to the core hypervisor, and an increase of 17% of contributions coming from the hypervisor, tests, and other components. We received several contributions from individuals based in China as well as Fujitsu (China), Huawei Technologies, and Intel (China).

We are generally seeing more companies (in China and beyond) participating in the project with an eye toward automotive, embedded, security, and native-cloud computing.

During the conference, I was able to meet up with community members from Alibaba, Huawei, Hyper_, Intel and others.. Key highlights and conversations for me included:

  • In the last year, we have seen very rapid adoption of Xen Project based products in government (e.g. China State Grid), industry (e.g. CTCI), telecoms (e.g. China Mobile), banking/financial (e.g. ICBC, People’s Insurance Company of China) and are starting to see adoption in High Performance Computing. One surprising factor that is leading to rapid adoption of open source in China is that many industries are required to perform code audits on software with the aim of strengthening cybersecurity, which gives open source software a significant edge.
  • I had lots of discussions on the “ins and outs” of Virtual Machine Introspection, after I highlighted that VMI defeated WannaCry/EternalBlue a priori mentioned in my live patching, virtual machine introspection and vulnerability management talk. As I learned most WannaCry victims were based in China including a number of companies such as the China National Petroleum Corporation, which led to 20% of petrol stations across the mainland going offline.
  • Live Patching, and its potential limitations and the complexities of how to build and validate them, were also high on the list of discussions which came up several times.
  • Another highlight was a discussion around the proposed Shared coprocessor framework for Xen, whose design is currently being finalized and will support sharing of GPUs, DSPs, FPGs and security once the prototype has been completed and upstreamed. I had originally assumed that co-processor sharing was mainly of interested either in embedded or for niche cloud use-cases, but was surprised to learn that there may be much more market pull than anticipated.

I’m looking forward for continued collaboration and innovation in this region.

Download server change for Xen releases

The official way to get the Xen hypervisor and other Xen Project downloads is via the the https://www.xenproject.org/ website. If you get Xen via the links on the website, you do not need to read the rest of this message.

We are aware that some users have been visiting the download server directly. That download server is changing.

In the past, the Xen Project has hosted its releases on space kindly provided on bits.xensource.com by Citrix (and, previously, XenSource). For some time now, we have in parallel made available downloads on the Xen Project’s server at https://downloads.xenproject.org/release/xen/.

Starting right away, Xen Project releases will appear only on the Xen Project’s server.

The directory structure remains unchanged. So, you can replace
http://bits.xensource.com/oss-xen/release/
at the start of all urls, with
https://downloads.xenproject.org/release/xen/
in all scripts, bookmarks, etc.

Previously published files will remain on bits.xensource.com, but new releases will not appear there.

Announcing Xen Project 4.9 RC and Test Day Schedule

Today, we created Xen 4.9 RC1 and will release a new release candidate every week, until we declare a release candidate as the final candidate and cut the Xen 4.9 release. We will also hold a Test Day every TUESDAY for the release candidate that was released the week prior to the Test Day starting from RC2. Note that RC’s are announced on the following mailing lists: xen-announce, xen-devel and xen-users. This means we will have Test Days on April 25th, May 2nd, 9th and 16th. Your testing is still valuable on other days, so please feel free to send Test Reports as outlined below at any time.

Getting, Building and Installing a Release Candidate

Release candidates are available from our git repository at

git://xenbits.xenproject.org/xen.git (tag 4.9.0-<rc>)

where <rc> is rc1, rc2, rc3, etc. and as tarball from

https://downloads.xenproject.org/release/xen/4.9.0-<rc>/xen-4.9.0-<rc>.tar.gz
https://downloads.xenproject.org/release/xen/4.9.0-<rc>/xen-4.9.0-<rc>.tar.gz.sig

Detailed build and Install instructions can be found on the Test Day Wiki. Make sure you check the known issues section of the instructions before trying to download an RC.

Testing new Features, Test and Bug Reports

You can find Test Instructions for new features on our Test Day Wiki and instructions for general tests on Testing Xen. The following pages provide information on how to report successful tests and how to report bugs and issues.

Happy Testing!

Now Accepting Submissions for Xen Project Developer and Design Summit 2017

screen-shot-2017-03-14-at-17-02-17 
We’re excited to announce that registration and the call for proposals is open for Xen Project Developer and Design Summit 2017, which will be held in Budapest, Hungary from July 11-13, 2017. The Xen Project Developer and Design Summit combines the formats of Xen Project Developer Summits with Xen Project Hackathons, and brings together the Xen Project’s community of developers and power users.

Submit a Talk

Do you have an interesting use case around Xen Project technology or best practices around the community? There’s a wide variety of topics we are looking for, including security, embedded environments, network function virtualization (NFV), and more. You can find all the suggested topics for presentations and panels here (make sure you select the Topics tab).

Several formats are being accepted for speaking proposals, including:

  • Presentations and Panels
  • Interactive design and problem solving sessions. These sessions can be submitted as part of the CFP, but we will reserve a number of design sessions to be allocated during the event. Proposers of design sessions are expected to host and moderate design sessions following the format we have used at Xen Project Hackathons. If you have not participated in these in the past, check out past event reports from 2016, 2015 and 2013.

Never talked at a conference before? Don’t worry! We encourage new speakers to submit for our events and have plenty of resources to help you prepare for your presentation.

Here are some dates to remember for submissions and in general:

  • CFP Close: April 14, 2017 (correction: was extended to April 21)
  • CFP Notifications: May 5, 2017
  • Schedule Announced: May 16, 2017
  • Event: July 11-13, 2017

Registration

Come join us for this event, and if you register by May 19, you’ll get an early bird discount 🙂 Travel stipends are available for students or individuals that are not associated with a company. If you have any questions, please send a note to community.manager@xenproject.org.