Monthly Archives: July 2018

A Recap of the Xen Project Developer and Design Summit: Community Health, Development Trends, Coding Changes and More

We were extremely thrilled to host our Xen Project Developer and Design Summit in Nanjing Jiangning, China this June. The event brought together our community and power users under one roof to collaborate and to learn more about the future of our project. It also gave us the opportunity to connect with a large group of our community who is based in China. We’ve seen a steady stream of Xen Project hypervisor adoption in this region.

If you were unable to attend the event, we have recordings of the presentations, and we also have the slideshares from the presentation available. Please check them out!

During our event, we always start with a weather report on the Xen Project. It covers areas that we are improving upon, where we need more support, and also the potential direction of the project. This blog covers information from the weather report as well as next steps and focus areas for the project.

Community Health
Code commits for the hypervisor have on average grown by 11% YoY since 2014. Commits in the first 5 months of 2018 have grown 11% compared to the same period last year. The top 6 contributors to the project since 2011 have been Citrix, Suse, AMD, Arm, Intel, Oracle. This is also true for the last 12 months in which 90% of contributions came from the top 6 players.

However, we have seen a larger than normal volume of contributions from Arm and AMD, which contributed twice as much as in previous years. In addition, EPAM is establishing itself on the top table with first contributions and a significant number of code reviews.  In addition, AWS started to make first significant code contributions in 2017.

Hardware security issues had an impact on the code review process of the project and thus on the project’s capability to take in some code. In other words, x86 related development that was not directly attributed to hardware security issues were slowed down, because developers normally reviewing contributions had less bandwidth to do so.

This has forced the community to make some changes that are starting to have a positive effect: x86 developers across companies are collaborating more and better, meaning that hardware security issues in 2018 had a smaller impact on the community than those in 2017.

Innovation and Development Trends
Unikraft, a Xen Project sub-project, is on a healthy growth projection. Unikraft aims to simplify the process of building unikernels through a unified and customizable code base. It was created after Xen Project Developer and Design Summit 2017.

The project recently upstreamed a significant amount of functionality, including:

  • Scheduling support, better/more complete support for KVM/Xen/Linux. Supporting Xen/KVM allows Unikraft to cater to a larger set of potential users/companies. Linux user-space provides an excellent development environment: Unikraft users can create their Unikraft unikernels as a Linux executable, use Linux’s wide range of debugging and performance optimization tools, and when done simply re-compile as a KVM or Xen unikernel (work on creating x86/Arm bare metal images is ongoing).
  • A release of newlib (a libc-like library) and lwip (a network stack: This support allows Unikraft to compile with most applications. It is a basic requirement to support a potentially wide range of applications.
  • The project is beginning to pick up traction with contributions coming from companies like NEC, Arm, and Oracle.

For more information check out the following two presentations: Unikraft and Unikraft on Arm.

We have been re-writing the x86 core. We are working on adding complex new CPU hardware features such as support for NVDIMMs and SGX. In addition, we are working on making technologies that have been used by security-conscious vendors in non-server environments ready to be used in server virtualization and cloud computing; support for measured boot is an example.

Another key innovation is a project called Panopticon, which aims to re-write some portions of the hypervisor to make Xen resilient to all types of side-channel attacks by removing unnecessary information about guests from the hypervisor.

You can find presentations related to these topics here (x86 evolution) and here (side-channel attacks and mitigations).

Continued Growth in Embedded and Automotive
We are seeing continued contributions within the embedded and automotive space to Xen Project Core with new features and functionality, including:

  • Co-processor (GPU) sharing framework enabling virtualization of co-processors such as FPGAs, DRMs, etc.
  • 2nd generation Power management and HPM on Arm  – this enables a huge reduction in power consumption, which is significant for some embedded market segments.
  • RTOS based Dom0 and code size reduction – this reduces the cost of safety certification significantly and is important for market segments where safety certification is important (such as automotive, avionics, medical, etc). We already managed to get Xen code size on Arm to below 45K SLOC and we expect that Dom0 will also be below 50K SLOC. This makes it possible to safety certify a Xen based stack to DAL C ASIL-B/C standards at a cost equivalent to less than 10 years.
  • Improved startup latency to boot multiple VMs in parallel from the device tree – this opens up the use of Xen to small IoT and embedded devices and allows booting of a complete Xen system in milliseconds compared to seconds. In addition, it halves the cost of safety certifications for systems where a Dom0 is not necessary

You can see the progress of our re-architecture in our latest release, Xen Project hypervisor 4.11. Also, the following summit presentations were relevant: here (Xen and automotive at Samsung) here (CPUFreq) and here (Real-time support).

These are just a few features and updates that make it easier for Xen to be used in embedded environments and market segments where safety certification is relevant. In addition, this will also significantly improve BoM and security in other market segments. On x86 we are also reducing code size, but this is significantly harder because of backward compatibility guarantees for x86 hardware and older operating systems.

Conclusion
The event was a great success with a lot of community and technical topics, like “How to Get Your Code Into Xen” and “The Art of Virtualizing Cache Maintenance.” Find the playlist for the full conference here. Additionally, our design sessions focused on architecture, embedded and safety, security, performance, and working practices and processes. You can find what was discussed, and next steps with these areas on our wiki.

If you want to stay abreast of where and when the Xen Project Developer and Design Summit will be held next year, follow us on Facebook and Twitter.

Xen Project Hypervisor Power Management: Suspend-to-RAM on Arm Architectures

This is the second part of the Xen Project Hypervisor series on power management. The first article focused on how virtualization and power management are coalescing into an energy-aware hypervisor.

In this post, the focus is on a project that was started to lay the foundation for full-scale power management for applications involving the Xen Project Hypervisor on Arm architectures. The group intends to make Xen on Arm’s power management the open source reference design for other Arm hypervisors in need of power management capabilities. Read the full story via Linux.com here.

Xen Project Matrix

Xen Project Hypervisor: Virtualization and Power Management are Coalescing into an Energy-Aware Hypervisor

Power management in the Xen Project Hypervisor historically targets server applications to improve power consumption and heat management in data centers reducing electricity and cooling costs. In the embedded space, the Xen Project Hypervisor faces very different applications, architectures and power-related requirements, which focus on battery life, heat, and size.

Although the same fundamental principles of power management apply, the power management infrastructure in the Xen Project Hypervisor requires new interfaces, methods, and policies tailored to embedded architectures and applications. This post recaps Xen Project power management, how the requirements change in the embedded space, and how this change may unite the hypervisor and power manager functions. Read the full article on Linux.com here.

Xen Project 4.8.4 is available!

I am pleased to announce the release of the Xen 4.8.4. Xen Project maintenance releases are released in line with our Maintenance Release Policy. We recommend that all users of the 4.8 stable series update to the latest point release.

The release is available from its git repositories

xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.8 (tag RELEASE-4.8.4)

or from the Xen Project download page

www.xenproject.org/downloads/xen-archives/xen-project-48-series/xen-484.html

This release contains many bug fixes and improvements. For a complete list of changes, please check the lists of changes on the download page.

What’s New in the Xen Project Hypervisor 4.11

I am pleased to announce the release of the Xen Project Hypervisor 4.11. One of our long-term development goals since the introduction of Xen Project Hypervisor 4.8 has been to create a cleaner architecture for core technology, less code and a smaller computing base for security and performance. The Xen 4.11 release has followed this approach by delivering more PVH related functionality: PVH Dom0 support is now available as experimental feature and support for running unmodified PV guests in a PVH Container has been added. In addition, significant chunks of the ARM port have been rewritten.

Mitigations against Cache Side-channel Attacks

This release contains mitigations for the Meltdown and Spectre vulnerabilities. It is worth noting that we spent a significant amount of time on completing and optimizing fixes for Meltdown and Spectre vulnerabilities. Xen 4.11 contains the following mitigations.

XPTI

We implemented performance optimized XPTI, Xen’s equivalent to KPTI. It is worth noting that only “classic PV” guests need XPTI whereas HVM and PVH can’t attack the hypervisor via Meltdown.

Branch Predictor Hardening

For x86 CPUs, we added a new framework for Intel and AMD microcode related to Spectre mitigations as well as support for Retpoline. By default, Xen will pick the most appropriate mitigations based on compiled in support, loaded microcode, and hardware details, and will virtualise appropriate mitigations for guests to use. Command line controls via the spec-ctrl command line option are available. SP4 (Speculative Store Bypass) mitigations are also available to enable guest software to protect against within-guest information leaks via spec-ctrl=ssbd. In addition, mitigation for Lazy FP state restore (INTEL-SA-00145) are available via spec-ctrl=eager-fpu.

Arm32: Mitigation for Cortex-A15, Cortex-A12, Cortex-A17 are present in Xen 4.7 and later with some caveats (update on the firmware).

Arm64: A PSCI-based mitigation framework for Spectre type vulnerabilities was introduced including concrete mitigations for Cortex-A57, A72, A73 and A75 CPUs for Xen 4.7 to Xen 4.9. An SMCCC 1.1 based mitigation is available for Cortex-A57, Cortex-A72, Cortex-A72, Cortex-A75 for Xen 4.10 and later.

PVH related Features

A key motivation behind PVH was to combine the best of PV and HVM mode, to simplify the interface between operating systems with Xen Support and the Xen Hypervisor and to reduce the attack surface of Xen. This led to the current implementation of PVH. PVH guests are lightweight HVM guests which use Hardware virtualization support for memory and privileged instructions, PV drivers for I/O and native operating system interfaces for everything else. PVH also does not require QEMU.

PVH Dom0

Xen 4.11 adds experimental PVH Dom0 support by calling Xen via dom0=pvh on the command line. Up to now, the only guest type that was capable running as Dom0, were PV guests. HVM guests require QEMU to run in Dom0 to provide some emulated services to the guest, which makes HVM guests unsuitable to run as Dom0 as QEMU is not running when Dom0 boots. PVH guests, in contrast, require no support from anything other than the hypervisor, so it can boot with no other guests running and can take on the responsibilities of Dom0. Running a PVH Dom0 increases security of Xen based systems by removing approximately 1.5 million lines of QEMU code from Xen’s trusted computing base.

Note that enabling a PVH Dom0 requires a PVH Dom0 capable Linux or FreeBSD. Patches for each operating system have been developed and are currently being upstreamed and should be available in the next Linux and FreeBSD versions.

PCI config space emulation in Xen

In Xen 4.11 support for the PCI configuration space has been moved from QEMU to the Hypervisor. Besides enabling PVH Dom0 support, this code will eventually also be available to HVM guests and PVH guests: however, additional security hardening needs to be performed before exposing such functionality to security supported guest types such as PVH or HVM guests.

PV in PVH container (or short: PVH Shim)

Support to run unmodified legacy PV-only guest to be run in PVH mode has been added in Xen 4.11. This allows cloud providers to support old, PV-only distros while only providing support for a single kind of guest (PVH). This simplifies management, reduces the surface of attack significantly, and eventually allows end-users to build a Xen hypervisor configuration with no “classic” PV support at all.

Next Steps

In subsequent releases, you should expect PVH Dom0 to become a supported feature and for PCI passthrough to be enabled in PVH guests. In addition, we will add the capability to compile PV-only and HVM-only versions of Xen.

Other Features

Scheduler Optimizations: Credit1 and Credit2 scheduling decisions when a vCPU is exclusively pinned to a pCPU or when soft-affinity is used have been performance optimised.

Add DMOPs to allow use of VGA with restricted QEMU (x86): Xen 4.9 introduced the Device Model Operation Hypercall (DMOPs) which significantly limits the capability of a compromised QEMU to attack the hypervisor. In Xen 4.11 we added DMOPs that enable the usage of the VGA console, which was previously restricted.

Enable Memory Bandwidth Allocation in Xen (Intel Skylake or newer): Xen 4.11 adds support for Memory Bandwidth Allocation (MBA), that allows Xen to slow misbehaving VMs by using a credit-based throttling mechanism.

Emulator enhancements (x86): support for previously unsupported Intel AVX and AVX2, and for AMD F16C, FMA4, FMA, XOP and 3DNow! instructions have been added to to the x86 emulator.

Guest resource mapping (x86): support for directly mapping Grant tables and IOREQ server pages have been introduced into Xen to improve performance.

Clean-up and future-proofing (Arm): Xen’s VGIC support has been re-implemented. In addition, stage-2 page table handling, memory subsystems and big.LITTLE support have been refactored to make it easier to maintain and update the code in future.

Support for PSCI 1.1 and SMCCC 1.1 compliance (Arm): Xen has been updated to comply with the latest versions of the Arm® Power State Coordination Interface and Secure Monitor Call Calling Conventions that provides an optimised calling convention and optional, discoverable support for mitigating Spectre Variant 2.

Summary

This release contains 1206 commits from 406 patch series. Contributions for this release of the Xen Project came from Citrix, Suse, ARM, AMD, Intel, Amazon, Gentoo Linux, Google, Invisible Things Lab, Oracle, EPAM Systems, Huawei, DornerWorks, Qualcomm, and a number of universities and individuals.

As in Xen 4.10, we took a security-first approach for Xen 4.11 and spent a lot of energy to improve code quality and harden security. Our efforts are not restricted to the current release, but include Xen 4.6 – 4.10: due to mitigations for side-channel attacks an unusually large number of commits – 765 in total – were back-ported to older releases to ensure that users of these releases are not impacted. Despite the disruption caused by Spectre and Meltdown, the community developed several major features and made significant progress towards completing PVH.

On behalf of the Xen Project Hypervisor team, I would like to thank everyone for their contributions (either in the form of patches, code reviews, bug reports or packaging efforts) to the Xen Project.

Please check our acknowledgement page, which recognises all those who helped make this release happen. The source can be located in the tree (tag RELEASE-4.11.0) or can be downloaded as a tarball from our website.

For detailed download and build instructions check out the guide on building Xen 4.11.

More information can be found at

Xen Project 4.7.6 is available!

I am pleased to announce the release of the Xen 4.7.6. Xen Project maintenance releases are released in line with our Maintenance Release Policy. We recommend that all users of the 4.7 stable series update to the latest point release.

The release is available from its git repositories

xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.7 (tag RELEASE-4.7.6)

or from the Xen Project download page

www.xenproject.org/downloads/xen-archives/xen-project-47-series/xen-476.html

This release contains many bug fixes and improvements. For a complete list of changes, please check the lists of changes on the download page.