Xen Project 4.6.6 and 4.7.3 are available

I am pleased to announce the release of 4.6.6 and 4.7.3. Xen Project Maintenance releases are released in line with our Maintenance Release Policy. We recommend that all users of the 4.6 and 4.7 stable series update to the latest point release.

4.6.6
The release is available from its git repository
http://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.6
(tag RELEASE-4.6.6) or from the XenProject download page
http://www.xenproject.org/downloads/xen-archives/xen-46-series/xen-466.html

4.7.3
The release is available from its git repository
http://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.7
(tag RELEASE-4.7.3) or from the XenProject download page
http://www.xenproject.org/downloads/xen-archives/xen-47-series/xen-473.html

These releases contain many bug fixes and improvements. For a complete list of changes, please check the lists of changes on the download pages.

What’s New in the Xen Project Hypervisor 4.9?

I am pleased to announce the release of the Xen Project Hypervisor 4.9. As always, we focused on improving code quality, security hardening as well as enabling new features. Our approach to security is also the reason why we delayed this release by 3 weeks: security issues that were discovered during the hardening phase of this release, were batched and handled using our Security Policy, which requires us to develop fixes for security issues in private and allows organisations on our pre-disclosure list to update their systems and software, before any code is made public. Consequently, we had to wait until June 20, before we could apply security fixes, build the final release candidate and test the final release candidate.

The Xen Project Hypervisor 4.9 release focuses on advanced features for embedded, automotive and native-cloud-computing use cases, enhanced boot configurations for more portability across different hardware platforms, the addition of new x86 instructions to hasten machine learning computing, and improvements to existing functionality related to the ARM® architecture, device model operation hypercall, and more.

We are also pleased to announce that Julien Grall, Senior Software Engineer at ARM, will stay release manager for Xen Project Hypervisor 4.10 release.

We grouped updates to the Xen Project Hypervisor using the following categories

  • New Features
  • Improvements to Existing Functionality
  • Multi-Release Long-Term Development

New Features

Boot Xen on EFI platforms using GRUB2 (x86): From Xen Project 4.9 and GRUB2 2.02 onwards, the Xen Project Hypervisor can be booted using the multiboot2 protocol on legacy BIOS and EFI x86 platforms. Partial support for the multiboot2 protocol was also introduced into network boot firmware (iPXE). This makes the Xen Project boot process much more flexible. Boot configurations can be changed directly from within a bootloader (without having to use text editors) and boot configurations are more portable across different platforms.

Near native latency for embedded and automotive environments: The “null” scheduler enables use-cases where every virtual CPU can be assigned to a physical CPU (commonly needed for embedded and automotive environments) removing almost all of the scheduler overheads in such environments. Usage of the “null” scheduler also guarantees significantly lower latency and more predictable performance. The new vwfi parameter for ARM (virtual Wait For Interrupt) allows fine-grained control of how the Xen Project Hypervisor handles WFI instructions. Setting vwfi to “native” reduces interrupt latency by approximately 60%. Benchmarks on Xilinx Zynq Ultrascale+ MPSoC’s have shown a maximum interrupt latency of less than 2 microseconds, which is extremely close to hardware limits, and should be small enough for the vast majority of embedded use cases.

Xen 4.9 includes new standard ABIs for sharing devices between virtual machines (including reference implementations) for a number of embedded, automotive and cloud native computing use-cases.

For embedded/automotive, a virtual sound ABI was added implementing audio playback and capture as well as volume control and the possibility to mute/unmute audio sources. In addition a new virtual display ABI for complex display devices exposing multiple framebuffers and displays has been added. Multi-touch support has been added to the virtual keyboard/mouse protocol enabling touch screens.

Xen 4.9 also introduces a Xen transport for 9pfs, which is a remote filesystem protocol originally written for Plan 9. During the Xen 4.9 release cycle, a Xen 9pfs frontend was upstreamed in the Linux kernel and a backend in QEMU. It is now possible to share a filesystem (not necessarily a block device) from a virtual machine to another, which is a requirement for adding Xen support to many container engines, such as CoreOS rkt.

The PV Calls ABI has been introduced to allow forwarding POSIX requests across guests: a POSIX function call originating from an app in a DomU can be forwarded and implemented in Dom0. For example, guest networking socket calls can be executed to Dom0, enabling a new networking model which is a natural fit for cloud-native apps.

Improvements to Existing Functionality

Xenstored optimisations: Xenstore daemons allow Dom0 and guests access to system configuration information. C-xenstored scalability limits have been increased to allow large hosts (about >1000 domains) to run efficiently. Transaction handling has been improved for better performance, smaller memory footprint and fewer transaction conflicts. Dynamic debugging capabilities have been added.

DMOP (Device Model Operation Hypercall): In Xen 4.9 the interface between Xen and QEMU was completely re-worked and consolidated. There is now only a single hypercall in Xen (the DMOP hypercall), which is carefully designed to allow the privcmd driver to audit any QEMU memory ranges and parameters that are passed to Xen via DMOP. The Linux privcmd driver enables DMOP auditing, which significantly limits the capability of a compromised QEMU to attack the hypervisor.

Alternative runtime patching and GICv3 support for ARM32: Alternative runtime patching which enables the hypervisor to apply workarounds for erratas affecting the processor and to apply optimizations specific to a CPU and GICv3 support was extended for 32-bit ARM platforms, bringing this functionality to embedded use-cases.

Intel and x86 Feature Support: The latest version of the Xen Project hypervisor adds the support of Neural Network Instructions AVX512_4VNNIW and Multiply Accumulation Single precision AVX512_4FMAPS as subfamilies of AVX512 instruction sets. With these instructions enabled in Xen for both HVM and PV guests, programs in guest OSes can take full advantage of these important instructions to speed up machine learning computing. This Xen release also further enhances VT-d Posted Interrupt (PI) optimization, Machine Check Exception(MCE) handling, and more.

System Error Detection (ARM): Xen on ARM made a step forward in reliability and serviceability with the introduction of System Error detection and reporting, a key feature for customers with highly available systems.

GCOV support: We removed the old GCOV implementation and replaced it with an updated version that supports more formats and exposes a more generic interface.

Re-work and hardening of x86 emulation code for security: Hardware-assisted virtualisation provides hypervisors with the ability to execute most privileged instructions natively and securely. However, for some boundary cases, it is still necessary to emulate x86 instructions in software. In Xen 4.9, the project completely re-worked the x86 emulation code, added support for new instructions, audited the code against security vulnerabilities and created AFL based test fuzzing tests that are regularly run against the emulator.

Updated support for Microsoft’s Hyper-V Hypervisor Top-Level Functional Specification (also known as Viridian Enlightenments): Xen implements a subset of version 5.0 of the Hyper-V Hypervisor TLFS, which enables Xen to run Windows guests at similar performance as it would run on Hyper-V. In addition, this work lays the groundwork to enable us to run Hyper-V within Xen in the future using nested virtualization.

Multi-Release Long-Term Development

This section contains large feature developments that cover several release cycles. It is intended to provide a progress update for larger features.

Transition from PVHv1 to PVHv2: Xen Project 4.8 laid the groundwork for re-architecting and simplifying PVH, focussing on the DomU guest ABI, which enabled Guest operating system developers to start porting their OSes to this mode. Support for FreeBSD is in progress, while support for Linux is committed. Xen 4.9 added Dom0 builder support and support for multiple virtual Intel I/O Advanced Programmable Interrupt Controllers (vIO APIC). PVHv2 for interrupt routing and PCI emulation is currently being peer reviewed and can be expected early in the Xen 4.10 release cycle. This lays the groundwork for a PVHv2 Dom0. For PVHv2 DomU support, PCI Passthrough and a major re-work of the xl/libxl and libvirt user interfaces for PVH have been started. Support for PVHv1 has been removed from the Xen Codebase.

Reworking the Xen-QEMU integration to protect against QEMU security vulnerabilities: In Xen Project 4.8, we embarked on an effort to re-work Xen-QEMU integration which amounts to sandboxing QEMU within Dom0. Significant progress was made in Xen 4.9 towards this goal, with the implementation of DMOP. Other changes such de-privileging QEMU in Dom0 and changes to the Linux privcmd driver have been mostly completed in Xen 4.9. Changes that are currently designed, but net yet implemented, are necessary changes to libxl and QEMU’s usage of XenStore.

Summary

Despite the shorter release cycle, the community developed several major features, and found and fixed many more bugs. Compared to Xen 4.8, which was our first fixed-term release, we have seen increased Development Velocity (in Xen 4.8 developers contributed 1245 changes – in Xen 4.9 developers contributed 1549 changes – a growth of 20%), increased Code Review activity, and more contributors (both individual and organisations contributing). For Xen 4.8 a total of 68 developers from 25 employers contributed, for Xen 4.9 a total of 86 developers from 30 employers contributed.

As in Xen 4.8, we took a security-first approach for Xen 4.9 and spent a lot of energy to improve code quality and harden security. This inevitably slowed down the acceptance of new features somewhat and also delayed the release. However, we believe that we reached a meaningful balance between mature security practices and innovation.

On behalf of the Xen Project Hypervisor team, I would like to thank everyone for their contributions (either in the form of patches, code reviews, bug reports or packaging efforts) to the Xen Project. Please check our acknowledgement page, which recognises all those who helped make this release happen.

The source can be located in the https://xenbits.xenproject.org/gitweb/?p=xen.git;a=shortlog;h=refs/tags/RELEASE-4.9.0 tree (tag RELEASE-4.9.0) or can be downloaded as tarball from our website. For detailed download and build instructions check out the guide on building Xen 4.9

More information can be found at

Automotive, Security and the Future of the Xen Project at The Xen Project Developer and Design Summit

The Xen Developer and Design Summit schedule is now live! This conference combines the formats of the Xen Project Developer Summits with the Xen Project Hackathons. If you are part of the Xen Project’s community of developers and power users, come join us in Budapest, Hungary, July 11 – 13 for this must-attend event!

pandas-656890_1920

The conference will cover many different topic areas including community, embedded/automotive, performance, tooling, hardware, security and more. The format will include traditional panels and presentation, as well as design and problem solving sessions.

Design and problem solving session proposals will be accepted until July 7. This is a great way to meet other developers face-to-face to:

  • Discuss and advance the design and architecture of future functionality
  • Coordinate and plan upcoming features
  • Discuss and share best practices and ideas on how to improve community collaboration
  • Hear interactive sessions covering lessons learned from contributors, users and vendor

Submit your design and problem solving ideas here.

Keynotes this year are coming from Lars Kurth, Xen Project Chairperson and Director of Open Source Solutions at Citrix; Oleksandr Andrushchenko, Lead Software Engineer at EPAM Systems; Stefano Stabellini, Virtualization Architect at Aporeto; and Wei Liu, Senior Software Engineer at Citrix.

Here’s a small sampling of other speaking sessions during the conference:

Automotive

  • Dedicated Secure Domain as an Approach for Certification of Automotive Sector Solutions from Iurii Mykhalskyi of GlobalLogic
  • Harmony of CPU Scheduling Between RT Guest OS and Rich Guest OS in Automotive Virtualization from Sangyun Lee of LG Electronics

Security

  • Hypervisor-Based Security: Bringing Virtualized Exceptions Into the Game from Mihai Dontu of Bitdefender
  • Uniprof: Transparent Unikernel Performance Profiling and Debugging from Florian Schmidt of NEC

Future of Xen

  • Intel GVT-g: From Production to Upstream from Zhi Wang of Intel
  • Recent and Ongoing Xen Related Work in the Linux Kernel from Jürgen Groß of SUSE

General Hypervisor

  • Bring up PCI Passthrough on ARM from Julien Grall of ARM
  • EFI Secure Boot, Shim and Xen: Current Status of Developments from Daniel Kiper of Oracle

You can view the entire schedule here. Early bird specials for tickets (price is $250) are available until May 31st.

A special thank you to our Diamond Sponsor Citrix and Gold sponsors ARM, Intel and Superfluidity. We look forward to seeing you at the event in July, and please stay informed on Xen Project updates by following us on social (Twitter and Facebook) and registering to our xen-announce mailing list.

 

Announcing the Windows PV Console Driver

It has long been the case that all HVM guests under Xen are provided with a PV console. You can attach to this console in the same way that you attach to the console of a PV guest, by typing in the control domain:

xl console name_of_guest

Until recently there has been no Windows PV driver interaction with this console. Starting with this commit support for logging via the PV console was added to the XENBUS driver.

I’m happy to announce that the three commits to XENBUS starting with this one added the necessary infrastructure to support a brand new XENCONS PV driver which exposes the PV console to Windows user-space as a character device.

The XENCONS driver source is hosted alongside the other PV driver sources on xenbits.xen.org and development builds are available for download here.

The XENCONS package also contains a Windows service to monitor the presence of the PV console device and invoke a command shell login process with redirected stdin/stdout. This means that, once the driver package has been installed, if you attach to the PV console and hit ENTER you’ll see a prompt something like this:

DESKTOP-KVEHAKT login:

From this prompt you can log in as any local user and you’ll then be presented with the command shell:

DESKTOP-KVEHAKT login: User
Password:
Microsoft Windows [Version 10.0.15063]
(c) 2017 Microsoft Corporation. All rights reserved.

C:\Users\User>

Be aware that this shell is running in session 0 so does not have access to the interactive session, but you can still use it for many administrative tasks. For instance, you can run netsh to display aspects of your network configuration:


C:\Users\User>netsh
netsh>interface ipv4 show global
Querying active state...

General Global Parameters
---------------------------------------------
Default Hop Limit : 128 hops
Neighbor Cache Limit : 256 entries per interface
Route Cache Limit : 4096 entries per compartment
Reassembly Limit : 33420160 bytes
ICMP Redirects : enabled
Source Routing Behavior : dontforward
Task Offload : enabled
Dhcp Media Sense : enabled
Media Sense Logging : disabled
MLD Level : all
MLD Version : version3
Multicast Forwarding : disabled
Group Forwarded Fragments : disabled
Randomize Identifiers : enabled
Address Mask Reply : disabled
Minimum Mtu : 576
Locality Address Selection : disabled
Flow Label : disabled

Current Global Statistics
---------------------------------------------
Number of Compartments : 1
Number of NL clients : 7
Number of FL providers : 4

Over the coming weeks I intend to add to the functionality that the driver provides. One obvious extension would be some form of hotkey support to link into the XENBUS_DEBUG interface to enable PV drivers to register a callback to be triggered by a particuler key.

If you are interested in this then please try the XENCONS package and send feedback to the mailing list.

Updates on XSA-213, XSA-214 and XSA-215

Today we released three patches for the following vulnerabilities: XSA-213, XSA-214 and XSA-215. Xen Project follows industry-accepted best practices regarding software security. This includes observing an embargo period, during which time the Xen Project security team will assess, respond, and prepare patches fixing the vulnerability, and distribute them privately to software and cloud providers before the public disclosure occurs.

When issuing a Xen Project Security Advisory (XSA), during the embargo this advisory is pre-disclosed to only members on the Xen Project Pre-Disclosure List. Vendors and open source projects who are on the Xen Project pre-disclosure list will not be affected by this security vulnerability and have updated their systems. The Xen Project security team has created fixes for these vulnerabilities, which can be publicly downloaded here: http://xenbits.xen.org/xsa/

Public cloud providers on the Xen Project predisclosure list were notified of these vulnerabilities two weeks ago; if your public cloud provider is on the list it is likely that your VMs are not vulnerable. Distributions and other software providers were also notified; they should have updated packages available soon (if they are not available already).

All three vulnerabilities have the potential to enable a guest virtual machine to break out of the hypervisor isolation. However, in order to exploit this vulnerability, an attacker would need to be running code in kernel mode of one or more VMs on the system. Any system that allows untrusted users to run arbitrary kernels will be particularly vulnerable.

Systems which only allow trusted users (such as IT professionals employed by the company) to run arbitrary kernels are less vulnerable, because an attacker would first need to find one or more exploit in the software running on one of the VMs before being able to then exploit this vulnerability. However, all users are encouraged to update as soon as possible.

Any 64-bit PV guest can exploit the vulnerability with XSA-213. The other two are more constrained. XSA-214 requires an attacker to control two different kinds of guests (either a PV one and an HVM one or a 32-bit PV one and 64-bit PV one). XSA-215 only affects you if your host has a very large amount of memory (either 3.5 TiB or 5 TiB depending on configuration).

Again, even with these constraints, we encourage you to update as soon as possible.

We take security very seriously and have developed security process best practices that are aimed for cloud environments that maximize fairness and transparency. We also have a very strict standard of review when it comes to new code being added to the Xen Project. We run Coverity static analyzer regularly to prevent certain classes of programming errors from being introduced. Additionally, we regularly run a generational fuzzing tool on our instruction emulator.

The Xen Project community developed Live Patching and introduced it into Xen Project 4.7. Now security fixes can be deployed without having to reboot VMs or have significant spare compute capacity to avoid reboots via VM migration.

These vulnerabilities were discovered by Jann Horn, from Google Project Zero.

Download server change for Xen releases

The official way to get the Xen hypervisor and other Xen Project downloads is via the the https://www.xenproject.org/ website. If you get Xen via the links on the website, you do not need to read the rest of this message.

We are aware that some users have been visiting the download server directly. That download server is changing.

In the past, the Xen Project has hosted its releases on space kindly provided on bits.xensource.com by Citrix (and, previously, XenSource). For some time now, we have in parallel made available downloads on the Xen Project’s server at https://downloads.xenproject.org/release/xen/.

Starting right away, Xen Project releases will appear only on the Xen Project’s server.

The directory structure remains unchanged. So, you can replace
http://bits.xensource.com/oss-xen/release/
at the start of all urls, with
https://downloads.xenproject.org/release/xen/
in all scripts, bookmarks, etc.

Previously published files will remain on bits.xensource.com, but new releases will not appear there.