Category Archives: Releases

Information about a new release of software

Xen Project 4.7 and 4.6.3 Release

I’m pleased to announce the release of Xen Project Hypervisor 4.7 and Xen Project Hypervisor 4.6.3.

Xen Project Hypervisor 4.7

This new release focuses on improving code quality, security hardening, security features, live migration support, usability improvements and support for new hardware features — this is also the first release of our fixed term June – December release cycle.

We continue to strive to make Xen Project Hypervisor the most secure open source hypervisor to match the security challenges in cloud computing, and for embedded and IoT use-cases. We are continuing to improve upon the performance and scalability for our users, and aim to continuously bring many new features to our users in a timely manner.

To make it easier to understand the major changes during this release cycle, I’ve grouped them below into several categories:

  • Security Features
  • Migration Support
  • Performance and Workloads
  • Support for new Hardware Features
  • Drivers and Devices (Linux, FreeBSD and other)

Security Features

Reboot-free Live Patching: Xen Project Hypervisor 4.7 comes equipped with Live Patching, a technology that enables re-boot free deployment of security patches to minimize disruption and downtime during security upgrades for system administrators and DevOps practitioners. Xen Project 4.7 implements version 1 of the Xen Project’s Live Patching specification, which is designed to encode the vast majority of security patches (approximately 90%) as Live Patching payloads. This version ships with a Live Patching enabled hypervisor and payload deployment tools and is available as a technology preview.

KCONFIG support: For security, embedded automotive and IoT use cases, Xen Project introduced the ability to remove core Xen Hypervisor features at compile time via KCONFIG. This ability creates a more lightweight hypervisor and eliminates extra attack surfaces that are beneficial in security-first environments, microservice architectures and environments that have heavy compliance and certification needs, like automotive.

Improvements to the Virtual Machine Introspection (VMI) subsystem: A number of performance, scalability, robustness and interface improvements have been added to the Virtual Machine Introspection subsystem, that was introduced in Xen 4.5. In addition, Bitdefender Hypervisor Introspection leveraging Xen Project Virtual Machine Introspection, has recently been released as a new enterprise security solution to discover and remedy deep threats that remain hidden via traditional endpoint security tools.

Foundation work to tolerate a restartable Dom0: Several key components in a Xen Project system run in Dom0, which make Dom0 the single point of failure. Xen Project has been able to run xenstored, the daemon for managing the hypervisor’s central settings repository on a Xen Project host, in a sandboxed Virtual Machine called xenstored stub domain since Xen Project version 4.2. In Xen 4.7, we have made it easier to build xenstored stub domains and for them to tolerate a Dom0 restart. This will make Dom0 less critical to a Xen Project system and help us move towards a more robust and secure architecture in the future. More work in this area is expected in subsequent releases.

Migration Support

Improved Migration support: CPU ID Levelling enables migration of VM’s between a larger range of non-identical hosts than previously supported.

Fault Tolerance / Coarse-grained Lock-stepping (COLO): Xen 4.5 laid the foundation for COLO while improving the Xen Project’s Hypervisors Live Migration and Remus High Availability support. The COLO Manager, which introduces a relaxed approach to checkpointing that avoids unnecessary checkpoints enabling near native performance for many workloads, has been fully integrated as an experimental feature into Xen 4.7. Note that the COLO Block Replication and COLO Proxy components, both of which are QEMU components, are currently still reviewed by the QEMU community. Both components are available as out-of-tree add-ons to the Xen Project Hypervisor, until fully integrated into QEMU.

Performance and Workloads

Support for a wider range of workloads and applications: The PV guest limit restriction of 512GB has been removed to allow the creation of huge PV domains in the TB range. TB sized VMs, coupled with Xen Project’s existing support for 512 vCPUs per VM, enable execution of memory and compute intensive workloads such as big data analytics workloads and in-memory databases.

Improved Credit 2 scheduler: The Credit2 scheduler is one (big) step closer to being ready for production use. It is now possible to instruct the scheduler to organize its runqueues and perform load balancing at core, socket or NUMA node granularity. More fine grained (core) configurations, deliver more aggressive load balancing, and are best suited for medium size systems. This feature has been proven to enable very good performance, especially if Hyper Threading is present.

Less fine grained configurations entail less overhead, and is suitable for larger servers or when no Hyper Threading is available. In addition, Credit2 has been extended to allow pinning of vCPUs to pCPUs (also known as “hard affinity”), allowing system administrators to configure the system in the exact way they want, and achieve the best setup for a given workload (for instance, a guarantee that a certain subset of vCPUs are always able to run when they need to run).

Improved RTDS scheduler: The RTDS scheduler is a real-time CPU scheduler built to provide guaranteed CPU capacity to guest VMs on SMP hosts, which primarily targets embedded, real-time and low-latency workloads. In Xen Project 4.7, the scheduling model has been changed from a quantum-driven to an event-driven model, which reduces scheduling overhead and thus scalability and performance for embedded and realtime workloads. In addition, per-VCPU parameter configuration has been added to allow better scheduler control for specialised workloads.

Per-cpu reader-writer lock: This new infrastructure allows for the fast path read case to have low overhead by only setting/clearing a per-cpu variable for using the read lock. After transforming various hypervisor locks to this infrastructure, VM-VM network transfer with 16 queues jumped from 15 gbit/s to 48 gbit/s on a 2 socket Haswell-EP host.

Usability Improvements

PVUSB Support: In Xen Project 4.7, a new XL command line interface to manage PVUSB devices has been introduced to manage PVUSB devices for PV guests. Both in kernel PVUSB backend and QEMU backend are supported.

Hot plugging of QEMU disk backends: Xen Project now enables hot-plugging of USB devices as well as QEMU disk backends, such as drbd, iscsi, and more in HVM guests. This new feature allows users to add and remove disk backends to virtual machines without the need to reboot the guest.

Soft-reset: The soft reset feature for HVM guests allows for a more graceful shutdown and restart of the HVM guest.

New Hardware Support

Features specific to the ARM Architecture

SBBR Compliance: Xen Project now supports booting on hosts that expose ACPI 6.0 (and later) information. The ARM Server Base Boot Requirements (SBBR) stipulate that compliant systems need to express hardware resources with ACPI; thus this support will come in useful for ARM Servers. This effort was carried out by Shannon Zhao of Linaro with minor patches from Julien Grall of ARM.

PCSI 1.0 Compatibility: PSCI 1.0 compatibility allows Xen Project software to operate on systems that expose PSCI 1.0 methods. Now, all 1.x versions of PSCI will be compatible with Xen Project software. More information on Power State Coordination Interface can be found here. This effort was also carried out by Julien Grall with a patch from Dirk Behme of Bosch.

vGIC-v3: Virtual Generic Interrupt Controller version 3. Reworked to be spec-compliant and optimised in some code paths.

Wallclock support: ARM guest can now get wallclock time directly from Xen Project via shared info page.

Features specific to Intel® Xeon® processor product family

Improved Interrupt Efficiency: Xen Project 4.7 supports VT-d Posted Interrupts, which provides hardware-level acceleration to increase interrupt virtualization efficiency. It reduces latency and improves user experience through performance improvements, especially for interrupt-intensive front-end workloads such as web servers. Note that Posted Interrupts in Xen Project 4.7 are still experimental and disabled by default.

Code and Data Prioritization: Xen Project 4.7 is the first to include Code and Data Prioritization (CDP), part of the Intel® Resource Director Technology (RDT) Framework and an extension of Cache Allocation Technology (CAT), first introduced in Xen Project 4.6. The introduction of CDP allows isolation of code/data within the shared L3 cache of multi-tenant environments, reducing contention and improving performance.

Other Intel Features: Additional features specific to the Intel Xeon processor family in Xen Project 4.7 include: VMX TSC Scaling, which allows for easier migration between machines with different CPU frequencies and support for Memory Protection Keys, a new security feature for hardening the software stack.

Drivers and Devices (Linux, FreeBSD and other)

During the Xen Project 4.7 release cycle, we made significant improvements to major operating systems and components we rely on to improve interoperability. During this development cycle 1494 Xen Project only related changesets – mostly bug fixes and small improvements – were applied to Linux, FreeBSD, NetBSD, QEMU and the Windows PV drivers: more than twice as many as in the 4.6 release cycle.

Summary

With dozens of major improvements, many more bug fixes and small improvements, and significant improvements to Drivers and Devices, Xen Project 4.7 reflects a thriving community around the Xen Project Hypervisor.

We are extremely proud of achieving the highest quality of the release while increasing development velocity across the hypervisor and its upstream dependencies by about 16%. In particular, our latest security related features enable Xen Project software to compete in the security appliance market and help answer some of the difficult questions regarding security in the cloud era.

We set out at the beginning of this release cycle to foster greater collaboration among vendors, individual developers, upstream maintainers, other projects and distributions. During this release cycle we continued to see an increasing influx of patches and newcomers such as Star Lab, Bosch and Netflix. We had a significant amount of contributions from cloud providers, software vendors, hardware vendors, academic researchers and individuals to help with this release. Major contributors for this particular release come from Citrix, SUSE, Intel, Star Lab, Oracle, Linaro, Fujitsu, Bitdefender, Red Hat, Huawei, ARM, Novetta, Broadcom, Xilinx, Bosch, AMD, GlobalLogic, NSA, Netflix and a number of universities and individuals. Thank you to all who participated.

As the release manager, I would like to thank everyone for their contributions (either in the form of patches, bug reports or packaging efforts) to the Xen Project. This release wouldn’t have happened without contributions from so many people around the world. Please check out our 4.7 contributor acknowledgement page.

The source can be located in the http://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.7 tree (tag RELEASE-4.7.0) or can be downloaded as tarball from our website. More information can be found at

Xen Project Hypervisor 4.6.3

The Xen Project 4.6.3 release is a maintenance release which comprises bug fixes and security updates. This is release is available immediately from its git repository
http://xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.6
(tag RELEASE-4.6.3) or from the Xen Project download page
http://www.xenproject.org/downloads/xen-archives/xen-46-series/xen-463.html
(where a list of changes can also be found).

We recommend all users of the 4.6 stable series which do not wish to upgrade to Xen 4.7, to update to this latest point release.

Note regarding version numbering: an issue was found late in the release process,
after one of the affected qemu trees was already tagged with a signed release git tag. Signed git tags provide a secure way of accounting for the source code, but once created they cannot be removed. Thus, the project could have released this maintenance release with a known issue, or fix the issue and skip a version number. We opted for the latter and decided to skip version 4.6.2.

Xen Project 4.7 Planning Opens

With Xen 4.6 released in October, we are already one month into the new cycle. Which means it is time to start planning for the next release. You may remember that one of the goals of the 4.6 release planning was to create smoother developer experience and to release Xen 4.6 on time. Both goals were achieved, so it was time to think where to go from here. Thus, the Xen community underwent a thorough discussion on how to manage future releases from xen-unstable and its impact on stable releases. The takeaway message of those lengthy threads is that we should continue to work on making the release cycle shorter and more predictable.

As such, the timeline for 4.7 is:

  • Development starts: October 13, 2015
  • Last posting date: March 18, 2016
  • Hard code freeze: April 1, 2016
  • Release date: June 3, 2016

After the 4.7 release, we will start to release Xen every 6 months: at the beginning of June and December. A regular 6 monthly release schedule has worked well for Ubuntu, OpenStack and many other projects. The idea behind it is a simple one: set a hard date and modify your goals to match that timeline. Which is also, why we dropped feature freeze exceptions, which create overheads and introduce unnecessary risk and debate. In addition, the new fixed release schedule will help open source projects and commercial vendors who consume Xen to plan their own releases better. And it allows us to set a schedule that ensures that every single release cycle is only affected by a single holiday period and that we have a Xen Project developer event (be it a Hackathon or Xen Project Developer Summit) during each release cycle. The stable release scheme is unchanged: 18 months full support, plus 18 months security fixes afterwards.

For more information, check out the slides that explain our release process and how it is changing for Xen 4.7 and beyond. To follow the roadmap in the coming months, be sure to check the Xen 4.7 Roadmap page on our wiki. Get involved on xen-devel@ and happy hacking!

For more updates, follow @XenProject.org on Twitter.

Xen Project 4.5.2 Maintenance Release Available

I am pleased to announce the release of Xen 4.5.2. Xen Project Maintenance releases are released roughly every 4 months, in line with our Maintenance Release Policy. We recommend that all users of the 4.5 stable series update to this point release.

Xen 4.5.2 is available immediately from its git repository:

    xenbits.xenproject.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.5
    (tag RELEASE-4.5.2)

or from the Xen Project download page at www.xenproject.org/downloads/xen-archives/xen-45-series/xen-452.html.

This release contains many bug fixes and improvements. For a complete list of changes in this release, please check lists of changes on the download page.

Best Quality and Quantity of Contributions in the New Xen Project 4.6 Release

I’m pleased to announce the release of Xen Project Hypervisor 4.6. This release focused on improving code quality, security hardening, enablement of security appliances, and release cycle predictability — this is the most punctual release we have ever had. We had a significant amount of contributions from cloud providers, software vendors, hardware vendors, academic researchers and individuals to help with this release. We continue to strive to make Xen Project Hypervisor the most secure open source hypervisor to match the security challenges in cloud computing, and for embedded and IoT use-cases. We are also continuing to improve upon the performance and scalability for our users, and aim to continuously bring many new features to our users in a timely manor.

Despite an increase of new features compared to previous releases, the Xen Project Hypervisor codebase has only increased by 6KLOC compared to Xen 4.5. In addition, we were able to increase the number of changesets that we integrated into Xen from 178/month (1812 in total) for Xen 4.5 to 259/month (2247 in total). In addition, the quality of Xen 4.6 was higher than in the past, enabling the CentOS 7 Virtualization SIG and XenServer to include Xen into their upcoming releases.

To make it easier to understand the major changes during this release cycle I have grouped the major updates into several categories:

  • Hypervisor
  • Toolstack
  • Xen Project Test Lab
  • Linux, FreeBSD and other OSes that utilise the new features
  • Greater Ecosystem

General Hypervisor Updates

  • The memory event subsystem has been reworked and extended to a new VM event subsystem. The new VM event subsystems supports both the ARM and x86 architectures. It can be used to intercept all sorts of VM events, such as memory access, register access and more. This enables security applications such as zero-footprint guest introspection, host-wide monitoring and many others. Have a look at Tamas’s presentations and Steve’s presentations on this topic to get more insights.
  • The Xen Security Modules (XSM) now have a default policy that is regularly tested in the Xen Project Test Lab to make sure it is not broken by mistake. This will enable us to switch on XSM by default in the future.
  • vTPM 2.0 support has been contributed by Intel and BitDefender [ 1 ]. To learn more about how to use vTPM and how it can make your host more secure, go to our wiki.
  • Grant table scalability has been improvement significantly by using finer-grained locks in grant tables. In some scenarios aggregate intrahost network throughput has been shown to improve by 100%. Other I/O drivers in Xen should potentially show significant performance improvements as well.
  • We introduced ticket lock to improve fairness, which provides better support of massive workloads from up to hundreds or thousands of VMs on a single host.
  • The unused SEDF scheduler has been removed from the hypervisor and toolstack. The Xen Project is committed to actively remove unused code to keep the code base small and to minimize security risks.
  • We removed Mini-OS from the Xen code base into its own tree. Mini-OS started as a demonstration OS, but received significant contributions in recent years (e.g. it is used by many Unikernels). We decide to treat it as a separately maintained independent project with it’s own mailing list and code tree to make it easier to consume. We hope this will help unikernel communities to more easily consume and contribute to Mini-OS, while reducing the Xen Project Hypervisor footprint.

x86-specific Hypervisor Updates

  • The Intel alternate P2M framework is a new capability for VM Introspection, Security and Privacy in Xen that gives Xen the ability to host up to 10 alternate guest to physical memory domains mappings for a specific guest-domain. It is one of the key technologies to enable zero-footprint VM introspection. It can also help Xen to implement faster NFV applications.
  • Intel Page Modification Logging Technology offloads the page dirty logging duty to hardware. Microbenchmark shows about 7% improvement in SPECJbb and should be particularly beneficial for Live Migration.
  • Intel Cache Allocation Technology allows system administrators to assign more L3 cache capacity to individual VMs, resulting in lower latency and higher performance for high-priority workloads such as NFV, real-time and video-on-demand applications.
  • Intel Memory Bandwidth Monitoring allows system administrators to identify memory bandwidth saturation on a Xen host that may be caused by several memory-intensive VMs running on the same host. Taking corrective actions, such as migrating VMs to a different Xen host, increases scalability and performance in the data center.
  • Intel Reserve Memory Region reporting provides a mechanism to report and reserve memory regions for legacy devices to allow for safe device passthrough.
  • Virtual Performance Monitoring Unit support makes it possible to profile the Xen Project Hypervisor with the Linux perf tool. Note that some work still needs to be completed within Linux to make perf fully functional.
  • Virtual NUMA for HVM guest is a continuation of the NUMA work performed in Xen 4.5 and previous releases. In this release, we exposed the functionality through the XL toolstack and added firmware changes to make the feature fully functional.

ARM-specific Hypervisor Updates

  • The supported number of VCPUs has been increased from 8 to 128 VCPUs on ARM64 platforms.
  • Passthrough for non-PCI devices allows users to passthrough devices via partial device trees. Full support for PCI device passthrough is currently being worked on.
  • ARM GICv2 on GICv3 support.
  • 32 bit userspace in 64 bit guest support.
  • OVMF for ARM contributed by Linaro.
  • 64K page ARM guest support.
  • Support for the following new Hardware Platforms has been added: Renesas R-Car Gen2, Thunder X, Huawei hip04-d04 and Xilinx ZynqMP SoC.

Toolstack Updates

  • Live Migration support in libxc / libxl and has been replaced with a completely new implementation (Migration v2). The new version respects different layers in the Xen Software stack and has been designed to be more robust and extensible to better support next-generation infrastructures and work planned in subsequent hypervisor releases.
  • Remus – our High Availability solution – has been reworked and is now based on Migration v2.
  • Libxl asynchronous operations can now be cancelled. This allows libxl users to cancel long-running asynchronous operations and benefits tool stacks such as libvirt and is beneficial for integration with cloud orchestration stacks.
  • Improved SPICE/QXL support.
  • AHCI disk controller support.
  • A new host I/O topology query interface gives upper layer in the software stack the ability to identify the I/O topology of underlying hardware platform.
  • Addition of Xenalyze, which is a tool for analyzing Hypervisor trace buffers and can be used for debugging and optimization, has been added to the Xen Project codebase as a maintained feature.

Xen Project Test Lab Updates

During the Xen 4.6 release cycle, the Xen Project created an Advisory Board funded Continuous Integration Test Lab. It currently has 24 hosts and is going to expanded in the future. This has led to significant improvements in Xen code quality and has allowed the project to expand automated test coverage. The number of test cases doubled during the 4.6 cycle. Some interesting new test cases that have been added are:

  • XSM
  • Stub Domain
  • VM migration using libvirt between two hosts is now tested.
  • Live Migration between hosts of different Xen versions is now tested and will help identify any breakage in our migration code or specification.
  • Test with different disk formats such as QCOW2, VHD and raw format.

More test cases are in the pipeline, including test case for OpenStack’s devstack, performance and scalability tests, FreeBSD Dom0 etc.

Linux, FreeBSD and other OSes

During the Xen 4.6 release cycle, we made significant improvements to major operating systems we rely on to improve interoperability. Some highlights on Linux kernel development spanning from Linux 3.18 to 4.3 were:

  • Xen blkfront multiqueue and multipage ring support.
  • Xen SCSI frontend and backend support.
  • VPMU kernel support.
  • Performance improvement in mmap call.
  • P2M in PV guest can address 512GB or more.

For FreeBSD there were these improvements:

  • Experimental PVH Dom0/DomU support.
  • Removal of classic i386 PV port by FreeBSD developer John Baldwin.
  • Blkfront indirect descriptor support by FreeBSD developer Colin Percival.
  • Removal of broken FreeBSD specific blkfront/back extensions.
  • ARM32 and ARM64 guest support are underway.

Greater Ecosystem

Summary

With dozens of major improvements, many more bug fixes and small improvements, efforts in other projects as well as a greater ecosystem, Xen 4.6 reflects a thriving community around the Xen Project Hypervisor. We are extremely proud of achieving the highest quality of the release while increasing development velocity. In particular, our latest security related features enable Xen to compete in the security appliance market and help answer some of the difficult questions regarding security in the cloud era.

We set out at the beginning of this release cycle to foster greater collaboration among vendors, individual developers, upstream maintainers, other projects and distributions. During this release cycle we continued to see an increasing influx of patches and newcomers. As the release manager, I would like to thank everyone for their contributions (either in the form of patches, bug reports or packaging efforts) to Xen. This release wouldn’t have happened without contributions from so many people around the world. Please check out our 4.6 contributor acknowledgement page.

The source can be located in the xen.git tree (tag RELEASE-4.6.0) or can be downloaded tarball from our website. More information can be found at


[ 1 ] Note that when this article was published, the contribution was mistakenly attributed to the US National Security Agency, instead of BitDefender.

Xen Project Test Day for 4.6 RC4 Scheduled for October 1

Our Fourth (and Possibly Final) 4.6 Release Candidate to be Tested This Thursday

TestDayOur Xen Project Test Days help insure that upcoming releases are ready for production, beyond what our automated testing through our Test Lab can accomplish. It is particularly important that our users test out the upcoming release in their own environment. We rely on your functional testing of features, stress-testing, edge case testing, and performance testing to prove that the code is ready for consumption. And this is your opportunity to verify that the new code will continue to work well in your particular situation.

Xen Project 4.6 Release Candidate 4 Testing

Continuing our current release cycle, the Test Day for Xen Project 4.6 RC4 has been set for Thursday, October 1, 2015.

This may be the final RC before release, so the time to test the software is now!

Test Day Information

Additrional information about Test Days can be found here:

Join us on Tuesday in #xentest on Freenode IRC!
Test a Release Candidate! Help others, get help! And have fun!

Our Next Test Day is September 15: Xen Project 4.6 RC3

The Third 4.6 Release Candidate to be Tested on Tuesday

TestDayOur Xen Project Test Days help insure that upcoming releases are ready for production, beyond what our automated testing through our Test Lab can accomplish. It is particularly important that our users test out the upcoming release in their own environment. We rely on your functional testing of features, stress-testing, edge case testing, and performance testing to prove that the code is ready for consumption. And this is your opportunity to verify that the new code will continue to work well in your particular situation.

Xen Project 4.6 Release Candidate 3 Testing

Continuing our current release cycle, the Test Day for Xen Project 4.6 RC3 has been set for Tuesday, September 15, 2015.

Additional Test Days are expected to be scheduled roughly ever other week until Xen Project 4.6 is ready for release.

Test Day Information

Additrional information about Test Days can be found here:

Join us on Tuesday in #xentest on Freenode IRC!
Test a Release Candidate! Help others, get help! And have fun!
If you can’t make Tuesday, remember that Test and Issue Reports are welcome any time.