Author Archives: Lars Kurth

About Lars Kurth

Lars Kurth is a highly effective, passionate community manager with strong experience of working with open source communities (Symbian, Symbian DevCo, Eclipse, GNU) and currently is community manager for xen.org. Lars has 9 years of experience building and leading engineering teams and a track record of executing several change programs impacting 1000 users. Lars has 16 years of industry experience in the tools and mobile sector working at ARM, Symbian Ltd, Symbian Foundation and Nokia. Lars has strong analytical, communication, influencing and presentation skills, good knowledge of marketing and product management and extensive background in C/C , Java and software development practices which he learned working as community manager, product manager, chief architect, engineering manager and software developer. If you want to know more, check out uk.linkedin.com/in/larskurth. Personally, Lars has a wide range of interests such as literature, theatre, cinema, cooking and gardening. He is particularly fascinated by orchids and carnivorous plants and has built a rather large collection of plants from all over the world. His love for plants extends into a passion for travel, in particular to see plants grow in their native habitats.

Xen Project 4.10.1 Available

I am pleased to announce the release of 4.10.1. Xen Project Maintenance releases are released in line with our Maintenance Release Policy. We recommend that all users of the 4.10 stable series update to the latest point release.

These releases are available from their git repositories

xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.10 (tag RELEASE-4.10.1)

or from the XenProject download page

www.xenproject.org/downloads/xen-archives/xen-project-410-series/xen-4101.html

These releases contain many bug fixes and improvements. For a complete list of changes, please check the lists of changes on the download pages.

Announcing the Xen Project 4.11 RC and Test Day Schedules

On Tuesday, we created Xen 4.11 RC1 and will release a new release candidate every FRIDAY, until we declare a release candidate as the final candidate and cut the Xen 4.11 release. We will also hold a Test Day every TUESDAY for the release candidate that was released the week prior to the Test Day starting from RC1. Note that RC’s are announced on the following mailing lists: xen-announce, xen-devel and xen-users. This means we will have Test Days on April 24th, May 1st, 8th, 15th and 22nd. Your testing is still valuable on other days, so please feel free to send Test Reports as outlined below at any time.

Getting, Building and Installing a Release Candidate

Release candidates are available from our git repository at

git://xenbits.xenproject.org/xen.git (tag 4.11.0-<rc>)

where <rc> is rc1, rc2, rc3, etc. and as tarball from

https://downloads.xenproject.org/release/xen/4.11.0-<rc>/xen-4.11.0-<rc>.tar.gz
https://downloads.xenproject.org/release/xen/4.11.0-<rc>/xen-4.11.0-<rc>.tar.gz.sig

Detailed build and Install instructions can be found on the Test Day Wiki. Make sure you check the known issues section of the instructions before trying to download an RC.

Testing new Features, Test and Bug Reports

You can find Test Instructions for new features on our Test Day Wiki and instructions for general tests on Testing Xen. The following pages provide information on how to report successful tests and how to report bugs and issues.

Happy Testing!

The Xen Project is participating in 2018 Summer round of Outreachy

This is a quick reminder that the Xen Project is again participating in Outreachy (May 2018 to August 2018 Round). Please check the Outreachy application page for more information.

Outreach Program for Women has been helping women (cis and trans), trans men, and genderqueer people get involved in free and open source software worldwide. Note that the program has been extended and is now also open to people from other groups underrepresented in FOSS: specifically the program is open to residents and nationals of the United States of any gender who are Black/African American, Hispanic/Latin, American Indian, Alaska Native, Native Hawaiian, or Pacific Islander. Information on Eligibility and the application process can be found here.

Meet us at FOSDEM 2018

As in the past, the Xen Project will have a booth at Europe’s biggest open source conference FOSDEM (taking place February 3rd and 4th in Brussels, Belgium).

Where?

During FOSDEM community volunteers will man our booth, which is located in bulding K (level 1, group C).

fosdem-event-2018

Meet the Team!

You will have the opportunity to speak to some of our developers: Anthony Perard (Maintainer of various components and qemu developer, Citrix), Daniel Kiper (Xen and Grub developer, Oracle) – Daniel will be mostly on the Grub booth, Ian Jackson (Committer and maintainer of various components, Citrix), Lars Kurth (Community Manager, Citrix), Roger Pau Monné (Maintainer of various components most recently PVH, Citrix), Simon Kainzer (Project Lead of Unikraft, NEC) and Wim ten Have (Xen and libvirt developer, Oracle). We also have Julien Fontanet (CTO, Vates) and Olivier Lambert (CEO, Vates) from Xen Orchestra at our booth!

We also have a few talks in the Virtualisation Devroom.

Xen Project Spectre / Meltdown FAQ (Jan 22 Update)

This document has been updated since it was published! Updates are marked in blue.

On January 3rd, 2018, Google’s Project Zero announced several information leak vulnerabilities affecting all modern superscalar processors. Details can be found on their blog, and in the Xen Project Advisory 254. To help our users understand the impact and our next steps forward, we put together the following FAQ. We divided the FAQ into several sections to make it easier to consume:

  • General considerations affecting all 3 vulnerabilities
  • “Rogue Data Load” (aka SP3, “Variant 3”, Meltdown, CVE-2017-5754)
  • “Branch Target Injection” (aka SP2, “Variant 2”, Spectre CVE-2017-5715)
  • “Bounds-check bypass” (aka SP1, “Variant 1”, Spectre CVE-2017-5753)

The project has been developing patches in order of exploitability. Our initial focus was on fixes for Meltdown, then on fixes for Spectre Variant 2, and finally on Variant 1.

Generally in the context of Xen based systems, there are many different considerations that have gone into our strategy, such as

  • Can a guest (user or kernel space) attack the hypervisor using Meltdown or Spectre?
  • Can a guest (user or kernel space) attack another guest (user or kernel space) when running in a Xen VM?

Note that impact and mitigations are specific to CPU architectures (and in some cases models) and may also differ depending on virtualization mode. The below FAQ tries to lay this out clearly, but if you have any questions, please reply to this email thread.

Note that we will update or re-issue the FAQ on this blog as new information surfaces.

1) General Questions related to all 3 vulnerabilities

1.1) Is there any risk of privilege escalation?

Meltdown and Spectre are, by themselves, only information leaks. There is no suggestion that speculative execution can be used to modify memory or cause the system to do anything it might not have done already.

1.2) Where can I find more information?

We will update this blog post and Advisory 254 as new information becomes available. Updates will also be published on the xen-announce@ mailing list.

1.3) Where can I ask questions?

This blog post, has been posted in text form on the xen-devel@ mailing list. If you have questions or improvement suggestions, please reply to this email thread.

1.4) Where does development of mitigations happen?

As information related to Meltdown and Spectre is now public, development will continue in public on xen-devel@ and patches will be posted and attached to Advisory 254 as they become available.

2) SP3, “Variant 3”, Meltdown, CVE-2017-5754

2.1) Is Xen impacted by Meltdown (“Variant 3”)?

Only Intel processors are impacted by Meltdown (referred to as SP3 in Advisory 254). On Intel processors, only 64-bit PV mode guests can attack Xen. Guests running in 32-bit PV mode, HVM mode, and PVH mode cannot attack the hypervisor using SP3.

Note that in general, some ARM processors are impacted by Meltdown (see https://developer.arm.com/support/security-update): however these cannot be exploited on Xen.

Guest Type

Is a user space attack from a guest to Xen possible?

Is a kernel space attack from a guest to Xen possible?

Available Mitigations

32 bit PV

No

No

N/A

64 bit PV

Yes

Yes

Several with different trade-offs
See Question 2.2

HVM

No

No

N/A

PVH

No

No

N/A

ARM [1]

No

No

N/A

Notes:
[1] ARM’s security update refers to a subvariant of Meltdown called “Variant 3a”. The impact analysis of this variant is not yet fully complete, but we believe that no sensitive data can be leaked to exploit Xen.

2.2) Are there any patches available for Meltdown (“Variant 3”)?

The project has published five different mitigations with Advisory 254 following different mitigation strategies for Meltdown. Two strategies involve switching from PV guests to PVH or HVM guests. The others require application of patches as outlined in Advisory 254:

  • Vixen: The basic principle is to run PV guests (which can read all of host memory due to Meltdown) as HVM guests (which cannot read memory due to Meltdown) using a hypervisor shim.
  • Comet: The basic principle is to run PV guests (which can read all of host memory due to the hardware bugs) as PVH guests (which cannot read memory due to Meltdown) using a hypervisor shim.
  • PTI or Xen PTI stage-1: This solution implements Page Table Isolation (PTI) for Xen.

Each strategy has different trade-offs and will work well for some use-cases, but not others. A high-level comparison of the different trade-offs for each mitigation, including information about code and documentation can be found in Advisory 254 (under “SP3 MITIGATION OPTIONS SUMMARY TABLE FOR 64-bit X86 PV GUESTS”). Please make sure you carefully read this section and the README files in the advisory.

2.3) How are Xen Guests impacted by Meltdown (“Variant 3”)?

In 32-bit PV mode, HVM mode, and PVH mode, guest user spaces can attack guest kernels using SP3; so updating guest kernels is advisable. Interestingly, guest kernels running in 64-bit PV mode are not vulnerable to attack using SP3, but attacks on user and kernel spaces of other guests are possible.

Guest Type

Is a user space attack on the guest kernel possible (when running in a Xen VM)?

Is a user space attack on other guests possible (when running in a Xen VM)?

Is a kernel space attack on other guests possible (when running in a Xen VM)?

32 bit PV

Yes [1]

No

No

64 bit PV

No [2]

Yes [3]

Yes [3]

HVM

Yes [1]

No

No

PVH

Yes [1]

No

No

ARM

Yes [1]

No

No

Mitigations and notes:
[1] Can be mitigated by the Linux KPTI patch and similar patches for other operating systems
[2] Although, a direct user space attack on the kernel is not possible, user space can indirectly be exploited via [3]. When Vixen and Comet are deployed, all guest memory is mapped by the “shim,” which is itself vulnerable to Meltdown. The Xen PTI patches protect both the hypervisor and the guest kernel from attacks from the guest user (without need for additional guest kernel patches). Note that KPTI is automatically disabled when running in 64 bit PV guests: thus running Xen PTI together with KPTI should not have any adverse effects.
[3] Mitigated by stage-1 Xen PTI

2.4) What is the long-term plan for Meltdown (“Variant 3”)?

Longer term, we will merge Vixen with Comet and release in suitable Xen Point releases with the codename Rudolph. In addition, we will improve PTI. We will likely backport and release PTI in suitable Xen point releases.

Note that Vixen and Comet will not be released in Xen point releases, but only through Advisory 254.

2.5) Does Xen have any equivalent to Linux’s KPTI series?

Linux’s KPTI series is designed to address SP3 only. For Xen guests, only 64-bit PV guests are affected by SP3. We have released a PTI (sometimes called Xen PTI or XPTI) series, which we will continue to improve over the coming weeks.

3) SP2, “Variant 2”, Spectre, CVE-2017-5715

3.1) Is Xen impacted by Spectre (“Variant 2”)?

Both Intel and AMD CPUs are vulnerable to Spectre (both variants). Vulnerability of ARM processors to Spectre (both variants) varies by model and manufacturer.

Guest Type

Is a user space attack from a guest to Xen possible?

Is a kernel space attack from a guest to Xen possible?

Available Mitigations

x86

Yes

Yes

See Question 3.4.1

ARM 32 [1]

Yes

Yes

See Question 3.4.2

ARM 64 [1]

Yes

Yes

Mitigations and notes:
[1] ARM has information on affected models on the following website: https://developer.arm.com/support/security-update. According to Cavium Thunder X1 is not vulnerable to Spectre (both variants).

3.2) How are Xen Guests impacted by Spectre (“Variant 2”)?

Both Intel and AMD CPUs are vulnerable to Spectre (both variants). Vulnerability of ARM processors to Spectre (both variants) varies by model and manufacturer.

Guest Type

Is a user space attack on other user processes or the guest kernel within the same guest possible (when running in a Xen VM)?

Is a user space attack on other guests possible (when running in a Xen VM)?

Is a kernel space attack on other guests possible (when running in a Xen VM)?

x86

Yes [2]

Yes [3]

Yes [3]

ARM 32 [1]

Yes [2]

Yes [4]

Yes [4]

ARM 64 [1]

Yes [2]

Yes [5]

Yes [5]

Mitigations and notes:
[1] ARM has information on affected models on the following website: https://developer.arm.com/support/security-update. According to Cavium Thunder X1 is not vulnerable to Spectre (both variants).
[2] Mitigated by retpoline or firmware based Indirect Branch Control mitigations in guest operating systems (see here for Linux Kernel mitigations)
[3] Mitigated by “Intel and AMD CPUs” approach as outlined in question 3.4.1
[4] Mitigated by “Affected ARM CPUs” (64 bit) approach as outlined in question 3.4.2
[5] Mitigated by “Affected ARM CPUs” (32 bit) approach as outlined in question 3.4.2

3.3) Are mitigations for Spectre possible (“Variant 2”)?

SP2 can be mitigated in two ways, both of which essentially prevent speculative execution of indirect branches. The first is to flush the branch prediction logic on entry into the hypervisor. This requires microcode updates, which Intel and AMD are in the process of preparing, as well as patches to the hypervisor which are also in process and should be available soon. On ARM, firmware updates are required (see here).

The second is to do indirect jumps in a way that is not subject to speculative execution (this approach is called Retpoline). This requires the hypervisor to be recompiled with a compiler that contains special new features. These new compiler features are also in the process of being prepared for both GCC (see here and here) and clang, and should be available soon.

3.4) Are there any patches available for Spectre (“Variant 2”)?

3.4.1) Intel and AMD CPUs:

We have developed prototype patches for x86 CPUs. These patches depend on firmware updates. Our prototype patches were developed against pre-released versions of MSR specifications and were modified to comply with published MSR specifications (see here). This may require changes to our patches. There have also been reports of issues with some published firmware updates (see here and here) leading to frequent reboots of systems where these have been deployed. We are currently evaluating the situation to verify whether Xen based systems with mitigations are affected.

Variant 2 mitigations have been published via Advisory 254. More information on ongoing development can be found on relevant xen-devel@ discussions which are linked to from here.

3.4.2) Affected ARM CPUs:

A framework to mitigate Spectre Variant 2 has been developed (for 64 bit and 32 bit). CPU vendors, will be able to add support for specific CPUs to the framework.

The framework and vendor specific mitigations has been published via Advisory 254 (including support for ARM CPUs). More information on ongoing development can be found on relevant xen-devel@ discussions which are linked to from here.

4) SP1, “Variant 1”, Spectre, CVE-2017-5753

4.1) Is Xen impacted by Spectre (“Variant 1”)?

Both Intel and AMD CPUs are vulnerable to Spectre (both variants). Vulnerability of ARM processors to Spectre (both variants) varies by model and manufacturer.

Guest Type

Is a user space attack from a guest to Xen possible?

Is a kernel space attack from a guest to Xen possible?

Available Mitigations

x86

Yes

Yes

See Question 4.3

ARM [1]

Yes

Yes

Mitigations and notes:
[1] ARM has information on affected models on the following website: https://developer.arm.com/support/security-update. According to Cavium Thunder X1 is not vulnerable to Spectre (both variants).

4.2) How are Xen Guests impacted by Spectre (“Variant 1”)?

Both Intel and AMD CPUs are vulnerable to Spectre (both variants). Vulnerability of ARM processors to Spectre (both variants) varies by model and manufacturer.

Guest Type

Is a user space attack on other user processes or the guest kernel within the same guest possible (when running in a Xen VM)?

Is a user space attack on other guests possible (when running in a Xen VM)?

Is a kernel space attack on other guests possible (when running in a Xen VM)?

x86

Yes [2]

Yes [3]

Yes [3]

ARM [1]

Yes [2]

Yes [3]

Yes [3]

Mitigations and notes:
[1] ARM has information on affected models on the following website: https://developer.arm.com/support/security-update. According to Cavium Thunder X1 is not vulnerable to Spectre (both variants).
[2] Please refer to guest operating specific mitigations (see here for Linux Kernel mitigations)
[3] See question 4.3

4.3) Are mitigations for Spectre possible (“Variant 1”)?

Spectre Variant 1 is much more difficult to mitigate. We have some ideas we’re exploring, but they’re still at the design stage at this point.

Xen Project Spectre/Meltdown FAQ

Updated to v3 on Dec 12th!

Google’s Project Zero announced several information leak vulnerabilities affecting all modern superscalar processors. Details can be found on their blog, and in the Xen Project Advisory 254. To help our users understand the impact and our next steps forward, we put together the following FAQ.

Note that we will update the FAQ as new information surfaces.

Changes since the initial publication:

  • v3: Added information related to Comet mitigation for Variant 3 (Meltdown) – for now Xen 4.10 only
  • v2: Added information related to Vixen mitigation for Variant 3 (Meltdown) – see Are there any patches for the vulnerability?
  • v2: Replaced SPx with Variant x to be in line with the terminology used elsewhere vulnerability?

Is Xen impacted by Meltdown and Spectre?

There are two angles to consider for this question:

  • Can an untrusted guest attack the hypervisor using Meltdown or Spectre?
  • Can a guest user-space program attack a guest kernel using Meltdown or Spectre?

Systems running Xen, like all operating systems and hypervisors, are potentially affected by Spectre (referred to as Variant 1 and 2 in Advisory 254). For Arm Processors information, you can find which processors are impacted here.  In general, both the hypervisor and a guest kernel are vulnerable to attack via Variant 1 and 2.

Only Intel processors are impacted by Meltdown (referred to as Variant 3 in Advisory 254). On Intel processors, only 64-bit PV mode guests can attack Xen using Variant 3. Guests running in 32-bit PV mode, HVM mode, and PVH mode (both v1 and v2) cannot attack the hypervisor using Variant 3. However, in 32-bit PV mode, HVM mode, and PVH mode (both v1 and v2), guest userspaces can attack guest kernels using Variant 3; so updating guest kernels is advisable.

Interestingly, guest kernels running in 64-bit PV mode are not vulnerable to attack using Variant 3, because 64-bit PV guests already run in a KPTI-like mode.

However, keep in mind that a successful attack on the hypervisor can still be used to recover information about the same guest from physical memory.

Is there any risk of privilege escalation?

Meltdown and Spectre are, by themselves, only information leaks. There is no suggestion that speculative execution can be used to modify memory or cause a system to do anything it might not have done already.

Where can I find more information?

We will update this blog post and Advisory 254 as new information becomes available. Updates will also be published on xen-announce@.

We will also maintain a technical FAQ on our wiki for answers to more detailed technical questions that emerge on xen-devel@ (and in particular this e-mail thread) and other communication channels.

Are there any patches for the vulnerability?

We have published a mitigation for Meltdown on Intel CPUs: please refer to Advisory 254. The published solutions are labelled Vixen and Comet (also see README.which-shim). Alternative solutions are being worked on.

A Mitigation for Variant 2/CVE-2017-5715 is available on the xen-devel@ mailing list, but has not yet undergone rigorous enough review to be published as an official patch.

As information related to Meltdown and Spectre is now public, development will continue in public on xen-devel@ and patches will be posted and attached to Advisory 254 as they become available in the next few days.

Can Variant 1/Variant 2 be fixed at all? What plans are there to mitigate them?

Variant 2 can be mitigated in two ways, both of which essentially prevent speculative execution of indirect branches. The first is to flush the branch prediction logic on entry into the hypervisor. This requires microcode updates, which Intel and AMD are in the process of preparing, as well as patches to the hypervisor which are also in process and should be available soon.

The second is to do indirect jumps in a way which is not subject to speculative execution. This requires the hypervisor to be recompiled with a compiler that contains special new features. These new compiler features are also in the process of being prepared for both gcc and clang, and should be available soon.

Variant 1 is much more difficult to mitigate. We have some ideas we’re exploring, but they’re still at the design stage at this point.

Does Xen have any equivalent to Linux’s KPTI series?

Linux’s KPTI series is designed to address Variant 3 only. For Xen guests, only 64-bit PV guests are able to execute a Variant 3 attack against the hypervisor. A KPTI-like approach was explored initially, but required significant ABI changes.  Instead we’ve decided to go with an alternate approach, which is less disruptive and less complex to implement. The chosen approach runs PV guests in a PVH container, which ensures that PV guests continue to behave as before, while providing the isolation that protects the hypervisor from Variant 3. This works well for Xen 4.8 to Xen 4.10, which is currently our priority.

For Xen 4.6 and 4.7, we are evaluating several options, but we have not yet finalized the best solution.

Devicemodel stub domains run in PV mode, so is it still more safe to run device models in a stub domain than in domain 0?

The short answer is, yes, it is still safer to run stub domains than otherwise.

If an attacker can gain control of the device model running in a stub domain, it can indeed attempt to use these processor vulnerabilities to read information from Xen.

However, if an attacker can gain control of a device model running in domain 0 without deprivileging, the attacker can gain control of the entire system.  Even with qemu deprivileging, the qemu process may be able to execute speculative execution attacks against the hypervisor.

So although XSA-254 does affect device model stub domains, they are still safer than not running with a stub domain.

What is the Xen Project’s plan going forward?

The Xen Project is working on finalizing solutions for Variant 3 and Variant 2 and evaluating options for Variant 1. If you would like to stay abreast on our progress, please sign up to xen-announce@. We will update this FAQ as soon as we have more news and updated information. Answers to more detailed technical questions will be maintained in a technical FAQ on our wiki. Thank you for your patience.

How can I ask further questions?

Please respond to this e-mail thread on xen-devel@ or xen-users@.