Category Archives: Announcements

Announcements affecting the Xen Project community

Xen Project Spectre/Meltdown FAQ

Updated to v3 on Dec 12th!

Google’s Project Zero announced several information leak vulnerabilities affecting all modern superscalar processors. Details can be found on their blog, and in the Xen Project Advisory 254. To help our users understand the impact and our next steps forward, we put together the following FAQ.

Note that we will update the FAQ as new information surfaces.

Changes since the initial publication:

  • v3: Added information related to Comet mitigation for Variant 3 (Meltdown) – for now Xen 4.10 only
  • v2: Added information related to Vixen mitigation for Variant 3 (Meltdown) – see Are there any patches for the vulnerability?
  • v2: Replaced SPx with Variant x to be in line with the terminology used elsewhere vulnerability?

Is Xen impacted by Meltdown and Spectre?

There are two angles to consider for this question:

  • Can an untrusted guest attack the hypervisor using Meltdown or Spectre?
  • Can a guest user-space program attack a guest kernel using Meltdown or Spectre?

Systems running Xen, like all operating systems and hypervisors, are potentially affected by Spectre (referred to as Variant 1 and 2 in Advisory 254). For Arm Processors information, you can find which processors are impacted here.  In general, both the hypervisor and a guest kernel are vulnerable to attack via Variant 1 and 2.

Only Intel processors are impacted by Meltdown (referred to as Variant 3 in Advisory 254). On Intel processors, only 64-bit PV mode guests can attack Xen using Variant 3. Guests running in 32-bit PV mode, HVM mode, and PVH mode (both v1 and v2) cannot attack the hypervisor using Variant 3. However, in 32-bit PV mode, HVM mode, and PVH mode (both v1 and v2), guest userspaces can attack guest kernels using Variant 3; so updating guest kernels is advisable.

Interestingly, guest kernels running in 64-bit PV mode are not vulnerable to attack using Variant 3, because 64-bit PV guests already run in a KPTI-like mode.

However, keep in mind that a successful attack on the hypervisor can still be used to recover information about the same guest from physical memory.

Is there any risk of privilege escalation?

Meltdown and Spectre are, by themselves, only information leaks. There is no suggestion that speculative execution can be used to modify memory or cause a system to do anything it might not have done already.

Where can I find more information?

We will update this blog post and Advisory 254 as new information becomes available. Updates will also be published on xen-announce@.

We will also maintain a technical FAQ on our wiki for answers to more detailed technical questions that emerge on xen-devel@ (and in particular this e-mail thread) and other communication channels.

Are there any patches for the vulnerability?

We have published a mitigation for Meltdown on Intel CPUs: please refer to Advisory 254. The published solutions are labelled Vixen and Comet (also see README.which-shim). Alternative solutions are being worked on.

A Mitigation for Variant 2/CVE-2017-5715 is available on the xen-devel@ mailing list, but has not yet undergone rigorous enough review to be published as an official patch.

As information related to Meltdown and Spectre is now public, development will continue in public on xen-devel@ and patches will be posted and attached to Advisory 254 as they become available in the next few days.

Can Variant 1/Variant 2 be fixed at all? What plans are there to mitigate them?

Variant 2 can be mitigated in two ways, both of which essentially prevent speculative execution of indirect branches. The first is to flush the branch prediction logic on entry into the hypervisor. This requires microcode updates, which Intel and AMD are in the process of preparing, as well as patches to the hypervisor which are also in process and should be available soon.

The second is to do indirect jumps in a way which is not subject to speculative execution. This requires the hypervisor to be recompiled with a compiler that contains special new features. These new compiler features are also in the process of being prepared for both gcc and clang, and should be available soon.

Variant 1 is much more difficult to mitigate. We have some ideas we’re exploring, but they’re still at the design stage at this point.

Does Xen have any equivalent to Linux’s KPTI series?

Linux’s KPTI series is designed to address Variant 3 only. For Xen guests, only 64-bit PV guests are able to execute a Variant 3 attack against the hypervisor. A KPTI-like approach was explored initially, but required significant ABI changes.  Instead we’ve decided to go with an alternate approach, which is less disruptive and less complex to implement. The chosen approach runs PV guests in a PVH container, which ensures that PV guests continue to behave as before, while providing the isolation that protects the hypervisor from Variant 3. This works well for Xen 4.8 to Xen 4.10, which is currently our priority.

For Xen 4.6 and 4.7, we are evaluating several options, but we have not yet finalized the best solution.

Devicemodel stub domains run in PV mode, so is it still more safe to run device models in a stub domain than in domain 0?

The short answer is, yes, it is still safer to run stub domains than otherwise.

If an attacker can gain control of the device model running in a stub domain, it can indeed attempt to use these processor vulnerabilities to read information from Xen.

However, if an attacker can gain control of a device model running in domain 0 without deprivileging, the attacker can gain control of the entire system.  Even with qemu deprivileging, the qemu process may be able to execute speculative execution attacks against the hypervisor.

So although XSA-254 does affect device model stub domains, they are still safer than not running with a stub domain.

What is the Xen Project’s plan going forward?

The Xen Project is working on finalizing solutions for Variant 3 and Variant 2 and evaluating options for Variant 1. If you would like to stay abreast on our progress, please sign up to xen-announce@. We will update this FAQ as soon as we have more news and updated information. Answers to more detailed technical questions will be maintained in a technical FAQ on our wiki. Thank you for your patience.

How can I ask further questions?

Please respond to this e-mail thread on xen-devel@ or xen-users@.

Announcing the Windows PV HID Drivers

Some recent patches to the QEMU source fix a long standing problem where the PV vkbd backend was unable to function correctly without the PV fb backend, which effectively made it pointless to implement PV HID (i.e. keyboard and mouse) frontends for HVM guests.
Now that the problem has been fixed, I’m happy to announce that this commit enables enumeration of PV vkbd devices in Windows enabling use of the new XENVKBD and XENHID drivers.

The driver sources are hosted alongside the other PV driver sources on xenbits.xen.org and development builds are available for download here.

Once the new drivers are installed into your Windows guest then you can disable the USB tablet (see xl.cfg manpage) pointing device in your VM configuration and still have the benefit of an absolute pointing device without the overhead of USB emulation.

If you are interested in this then please try the new driver packages and send feedback to the mailing list.

Xen Project Member Spotlight: Bitdefender

The Xen Project is comprised of a diverse set of member companies and contributors that are committed to the growth and success of the Xen Project Hypervisor. The Xen Project Hypervisor is a staple technology for server and cloud vendors, and is gaining traction in the embedded, security and automotive space. This blog series highlights the companies contributing to the changes and growth being made to the Xen Project, and how the Xen Project technology bolsters their business.

contemporary-1850469_1920

Name: Shaun Donaldson
Title: Director of Strategic Alliances
Company: Bitdefender

When did you join the Xen Project and why/how is your organizations involved?
Bitdefender has been collaborating with Linux Foundation for the past three years, and active within the Xen Project community, especially around Virtual Machine Introspection, for about the same time. We officially joined the Xen Project toward the end of 2017. We are focused on security, which is core to the philosophy of the Xen Project.

How does your involvement benefit your company?
A key benefit has been working with the open source community where security is top-of-mind. Rather than developing ideas and approaches in a vacuum, the Bitdefender team of researchers and developers have been able to validate ideas and benefit from the feedback of the talented Xen Project community. There is a deep pool of knowledge around the history of what has worked, along with a wide variety of perspectives. This is why the Xen Project hypervisor is so flexible.

How does the Xen Project’s technology help your business?
Bitdefender is a security company with customers and partners across the globe. The team works on many fronts across the security landscape and our alliance with the Xen Project enables us to be a part of the open source community dedicated to Virtual Machine Introspection (VMI).

VMI is an example of a leap-forward that required creativity, experimentation, and coding by talented people to be realized beyond the academic sphere. Supporting and extending this process, as well as translating the capabilities for commercial offerings benefits everyone, including Bitdefender. To date, Bitdefender is the only commercial security vendor leveraging VMI; and the Xen Project hypervisor VMI capability has been adopted by Citrix within XenServer as Direct Inspect APIs.

The location in the stack where a security solution can gain insight, has shifted from being within the VM to the hypervisor. This fundamentally changes how workloads can be secured, which is a game-changer. VMI provides context, while keeping the security mechanism isolated. It is a best of both worlds scenario: context with isolation.

What are some of the major changes you see with virtualization and the transition to cloud native computing?
The security implications of moving workloads from traditional virtualized environments to cloud native environments, is something organizations challenge us with every day. The concept of oud-native provokes ideas about the myriad of implications of building workloads from the outside-in. This means there is a shift from service-oriented outcomes back to the way a service, whether IaaS, PaaS, or SaaS, is built and, ultimately delivered.

The Bitdefender team works with organizations that are trying to understand how they can avoid grafting traditional security onto new methods of delivering services. A hypervisor which takes full advantage of hardware capabilities, and makes the result of those capabilities consumable, applicable, and actionable to higher-level services, is extremely important.

At the same time, the fundamental elements remain: processes run on supervisors/operating systems, hypervisors run on the underlying hardware, and so-on. In this, there are opportunities to apply security from the root of the stack.

What advice would you give someone considering joining the Xen Project?
If your organization wants to be on the forefront of cloud computing, you must be part of the groups dedicated to advancing the underlying technology, or risk being left behind. Organizations open to participating in groups like the Xen Project, may discover new and unanticipated ways of solving problems.

The Xen Project has over a decade of development and has benefited from thousands of contributions from individuals, users, and organizations. The collective knowledge and passion to move cloud computing forward ensures the best information sharing and access to a community focused on advancing virtualization. This enables organizations to have the advantage of lessons-learned as well as a mature structure to best continue conversations on where the Xen Project hypervisor can continue to evolve.

What excites you most about the future of Xen?
At Bitdefender, we are excited to see the ways and places the Xen Project hypervisor will be used and how it will continue to expand. From the services we consume, to the cars we drive, there is a rich future for Xen, along with the parallel need to secure it, ensuring a long and fruitful alliance with Xen.

You can read more about Bitdefender joining the Xen Project here.

What’s New in the Xen Project Hypervisor 4.10

I am pleased to announce the release of the Xen Project Hypervisor 4.10. As always, we focused on improving code quality, security hardening as well as enabling new features.

The Xen Project Hypervisor 4.10 continues to take a security-first approach with improved architecture and more centralized documentation. The release is equipped with the latest hardware updates from Arm and a more intuitive user interface.

We are also pleased to announce that Juergen Gross will be the next release manager for Xen Project Hypervisor 4.11. Juergen has been an active developer for the past few years, making significant code contributions to advance Xen support in Linux. He is a virtualization kernel developer at Suse and maintainer of Xen subsystem in Linux as well as parvirtualization.

We grouped updates to the Xen Project Hypervisor using the following categories:

  • Hypervisor general
  • Hypervisor Arm
  • Hypervisor x86
  • Toolstack
  • Misc

Hypervisor General

Credit 2 scheduler improvements: Soft-affinity support for the Credit 2 scheduler was added to allow those using the Xen Project in the cloud and server space to specify a preference for running a VM on a specific CPU. This enables NUMA aware scheduling for the Credit 2 scheduler. In addition we added cap support allowing users to set a the maximum amount of CPU a VM will be able to consume, even if the host system has idle CPU cycles.

Null scheduler improvements: The recent updates to the “null” scheduler guarantee near zero scheduling overhead, significantly lower latency, and more predictable performance. Added tracing support enables users to optimise workloads and introduced soft-affinity. Soft affinity adds a flexible way to express placement preference of vcpus on processors, which improves cache and memory performance when configured appropriately.

Virtual Machine Introspection improvements: Performance improvements have been made to VMI. A software page table walker was added to VMI on ARM, which lays the groundwork to alt2pm for ARM CPUs. For more information on alt2pm is available here.

PV Calls Drivers in Linux: In Xen Project 4.9, the Xen Project introduced the PV Calls ABI, which allows forwarding POSIX requests across guests. This enables a new networking model that is a natural fit for cloud-native apps. The PV Calls backend driver was added to Linux 4.14.

Better User Experience through the Xen Project User Interface

The Xen Project community also made significant changes to the hypervisor’s user interface. It is now possible to modify certain boot parameters without the need to reboot Xen. Guest types are now selected using the type option in the configuration file, where users can select a PV, PVH or HVM guest. The builder option is being depreciated in favor of the type option, the PVH option has been removed and a set of PVH specific options have been added.

These changes allow the Xen Project to retain backward compatibility on new hardware without old PV code, providing the same functionality with a much smaller codebase. Additional user interface improvements are detailed in our blog post.

Hypervisor Arm

Support for Latest System-on-chip (SoC) Technology: The Xen Project now supports SoCs based on the 64-bit Armv8-A architecture from Qualcomm Centriq 2400 and Cavium ThunderX.

SBSA UART Emulation for Arm CPUs: Implementation of SBSA UART emulation support in the in the Xen Project Hypervisor makes it accessible through the command line tools. This enables the guest OS to access the console when no PV console driver is present. In addition, the SBSA UART emulation is also required to be compliant with the VM System specification.

ITS support for ARM CPUs: Xen Project 4.10 adds support for ARM’s Interrupt Translation Service (ITS), which accompanies the GICv3 interrupt controller such as the ARM CoreLink GIC-500. ITS support allows the Xen Project Hypervisor to harnesses all of the benefits of the GICv3 architecture, improving interrupt efficiency and allowing for greater virtualization on-chip for both those using the Xen Project for the server and embedded space. ITS support is essential to virtualize systems with large amounts of interrupts. In addition ITS increases isolation of virtual machines by providing interrupt remapping, enabling safe PCI passthrough on ARM..

GRUB2 on 64-bit Armv8-A architecture: The GRUB community merged support to boot Xen on 64-bit Arm-based CPU platforms. GRUB2 support for Armv8-A improves the user experience when installing Xen via distribution package on UEFI platform.

Hypervisor x86

Rearchitecture Creates Smaller Attack Surface and Cleaner Code

Since the introduction of Xen Project Hypervisor 4.8, the project has overhauled the x86 core of its technology. The intention is to create a cleaner architecture, less code and a smaller computing base for security and performance. As part of this re-architecture, Xen Project 4.10 supports PVHv2 DomU. PVHv2 guests have a smaller TCB and attack surface compared to PV and HVM guests.

In Xen Project Hypervisor 4.9, the interface between Xen Project software and QEMU was completely reworked and consolidated via DMOP. For the Xen Project Hypervisor 4.10, the Xen Project community built on DMOP and added a Technology Preview for dm_restrict to constrain what device models, such as QEMU, can do after startup. This feature limits the impact of security vulnerabilities in QEMU. Any previous QEMU vulnerabilities that could normally be used for escalation privileges to the host cannot escape the sandbox.

This work significantly reduces potential security vulnerabilities in the Xen Project software stack.

L2 CAT for Intel CPUs: In Xen 4.10 we added support for Intel’s L2 Cache Allocation Technology(CAT) — available on certain models of (Micro) Server platforms. Xen L2 CAT support provides Xen users a mechanism to partition or share the L2 Cache among virtual machines, if such technology is present on the hardware Xen runs. This allows users to make better use of the shared L2 cache depending on the VM characteristic (e.g. priority).

Local Machine-Check Exception(LMCE) for Intel CPUs: Xen 4.10 provides LMCE support for HVM guests. A LMCE, if the affected vCPU is known, will be injected to related vCPU, otherwise, the LMCE will be broadcasted to all vCPUs running on the host. This allows for more efficient passing of MCE from hypervisor to virtual machines for further handling.

User Mode Instruction Prevention(UMIP) for Intel CPUs: User-Mode Instruction Prevention (UMIP) is a security feature present in new Intel Processors. If enabled, it prevents the execution of certain instructions if the Current Privilege Level (CPL) is greater than 0. Xen 4.10 exposes UMIP to virtual machines to take advantage of this feature.

Misc.

Improved Support Documentation

In Xen Project 4.10, a machine-readable file (support.md) was added to describe support related information in a single document. It defines support status and whether features are security supported and to which degree. For example, a feature may be security supported on x86, but not on Arm.

This file will be back-ported to older Xen releases and will be used to generate support information for Xen Project releases and will be published on xenbits.xen.org/docs/. This effort will both allow users to better understand how they are impacted by security issues, and centralizing security support related information is a pre-condition to become a CVE Numbering authority.

Summary

Despite the shorter release cycle, the community developed several major features, and found and fixed many more bugs. It is also rather impressive to see multiple vendors collaborate on the Xen Project Hypervisor to drive multiple projects forward. Contributions for this release of the Xen Project came from Amazon Web Services, AMD, Aporeto, Arm, BAE Systems, BitDefender, Cavium, Citrix, EPAM, GlobalLogic, Greenhost, Huawei Technologies, Intel, Invisible Things Lab, Linaro, Nokia, Oracle, Red Hat, Suse, US National Security Agency, and a number of universities and individuals.

As in Xen 4.9, we took a security-first approach for Xen 4.10 and spent a lot of energy to improve code quality and harden security. This inevitably slowed down the acceptance of new features somewhat and also delayed the release. However, we believe that we reached a meaningful balance between mature security practices and innovation.

On behalf of the Xen Project Hypervisor team, I would like to thank everyone for their contributions (either in the form of patches, code reviews, bug reports or packaging efforts) to the Xen Project. Please check our acknowledgement page, which recognises all those who helped make this release happen.

The source can be located in the tree (tag RELEASE-4.10.0) or can be downloaded as a tarball from our website. For detailed download and build instructions check out the guide on building Xen 4.10

More information can be found at

Unikraft: Unleashing the Power of Unikernels

This blog post was written by Dr. Felipe Huici, Chief Researcher, Systems and Machine Learning Group, at NEC Laboratories Europe

The team at NEC Laboratories Europe spent quite a bit of time over the last few years developing unikernels, a specialized virtual machine images targeting specific applications. This technology is fascinating to us because of its fantastic performance benefits: tiny memory footprints (hundreds of KBs or a few MBs), boot times compared to those of processes or throughput in the range of 10-40 Gb/s, among many other attributes. Specific metrics can be found in these articles: My VM is Lighter (and Safer) than your ContainerUnikernels Everywhere: The Case for Elastic CDNs, ClickOS and the Art of Network Function Virtualization

The potential of unikernels is great (as you can see from the work above), but there hasn’t been a massive adoption of unikernels. Why? Development time. For example, developing Minipython, a MicroPython unikernel, took the better part of three months to put together and test. ClickOS, a unikernel for NFV, was the result of a couple of years of work.

What’s particularly bad about this development model, besides the considerable time spent, is each unikernel is basically a throwaway. Every time we want to create a new unikernel targeting a different application, developers have to start from scratch. Essentially, there is a lack of shared research and development when it comes to building unikernels.

We (at NEC) wanted to change this, so we started to re-use the work and created a separate repo consisting of a tool stack that would contain functionality useful to multiple unikernels — mostly platform-independent versions of newlib and lwip (a C library and network stack intended for embedded systems).

This got us thinking that we should take our work to a much bigger level. We asked the question: Wouldn’t it be great to be able to very quickly choose, perhaps from a menu, the bits of functionality that we want for an unikernel, and to have a system automatically build all of these pieces together into a working image? It would also be great if we could choose multiple platforms (e.g., Xen, KVM, bare metal) without having to do additional work for each of them.

The result of that thought process is Unikraft. Unikraft decomposes operating systems into elementary pieces called libraries (e.g., schedulers, memory allocators, drivers, filesystems, network stacks, etc.) that users can then pick and choose from, using a menu to quickly build images tailored to the needs of specific applications. In greater detail, Unikraft consists of two basic components (see Figure 1):

  • Library pools contain libraries that the user of Unikraft can select from to create the unikernel. From the bottom up, library pools are organized into (1) the architecture library tool, containing libraries specific to a computer architecture (e.g., x86_64, ARM32 or MIPS); (2) the platform tool, where target platforms can be Xen, KVM, bare metal (i.e. no virtualization), user-space Linux and potentially even containers; and (3) the main library pool, containing a rich set of functionality to build the unikernel. This last library includes drivers (both virtual such as netback/netfront and physical such as ixgbe), filesystems, memory allocators, schedulers, network stacks, standard libs (e.g. libc, openssl, etc.), and runtimes (e.g. a Python interpreter and debugging and profiling tools). These pools of libraries constitute a codebase for creating unikernels. As shown, a library can be relatively large (e.g libc) or quite small (a scheduler), which allows for customization for the unikernel.
  • The Unikraft build tool is in charge of compiling the application and the selected libraries together to create a binary for a specific platform and architecture (e.g., Xen on x86_64). The tool is currently inspired by Linux’s KCONFIG system and consists of a set of Makefiles. It allows users to select libraries, to configure them, and to warn them when library dependencies are not met. In addition, the tool can also simultaneously generate binaries for multiple platforms.

unikraft

Figure 1. Unikraft architecture.

Getting Involved
We are very excited about the recent open source release of Unikraft as a Xen Project Foundation incubator project. The Xen Project is a part of the Linux Foundation umbrella. We welcome developers willing to help improve Unikraft. Whether you’re interested in particular applications, programming languages, platforms, architectures or OS primitive. We are more than happy to build and receive contributions from the community. To get you started, here are a number of available resources:

Please don’t be shy about getting in touch with us, we would be more than happy to answer any questions you may have. You can reach the core Unikraft development team at sysml@listserv.neclab.eu .

Xen Project 4.7.4 and 4.9.1 are available

I am pleased to announce the release of Xen 4.7.4 and 4.9.1. Xen Project Maintenance releases are released in line with our Maintenance Release Policy. We recommend that all users of the 4.7 and 4.9 stable series update to the latest point release.

These releases are available from their git repositories

xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.7 (tag RELEASE-4.7.4)

xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.9 (tag RELEASE-4.9.1)

or from the XenProject download page

www.xenproject.org/downloads/xen-archives/xen-project-47-series/xen-474.html

www.xenproject.org/downloads/xen-archives/xen-project-49-series/xen-491.html

These releases contain many bug fixes and improvements. For a complete list of changes, please check the lists of changes on the download pages.