Category Archives: Announcements

Announcements affecting the Xen Project community

Unikraft: Unleashing the Power of Unikernels

This blog post was written by Dr. Felipe Huici, Chief Researcher, Systems and Machine Learning Group, at NEC Laboratories Europe

The team at NEC Laboratories Europe spent quite a bit of time over the last few years developing unikernels, a specialized virtual machine images targeting specific applications. This technology is fascinating to us because of its fantastic performance benefits: tiny memory footprints (hundreds of KBs or a few MBs), boot times compared to those of processes or throughput in the range of 10-40 Gb/s, among many other attributes. Specific metrics can be found in these articles: My VM is Lighter (and Safer) than your ContainerUnikernels Everywhere: The Case for Elastic CDNs, ClickOS and the Art of Network Function Virtualization

The potential of unikernels is great (as you can see from the work above), but there hasn’t been a massive adoption of unikernels. Why? Development time. For example, developing Minipython, a MicroPython unikernel, took the better part of three months to put together and test. ClickOS, a unikernel for NFV, was the result of a couple of years of work.

What’s particularly bad about this development model, besides the considerable time spent, is each unikernel is basically a throwaway. Every time we want to create a new unikernel targeting a different application, developers have to start from scratch. Essentially, there is a lack of shared research and development when it comes to building unikernels.

We (at NEC) wanted to change this, so we started to re-use the work and created a separate repo consisting of a tool stack that would contain functionality useful to multiple unikernels — mostly platform-independent versions of newlib and lwip (a C library and network stack intended for embedded systems).

This got us thinking that we should take our work to a much bigger level. We asked the question: Wouldn’t it be great to be able to very quickly choose, perhaps from a menu, the bits of functionality that we want for an unikernel, and to have a system automatically build all of these pieces together into a working image? It would also be great if we could choose multiple platforms (e.g., Xen, KVM, bare metal) without having to do additional work for each of them.

The result of that thought process is Unikraft. Unikraft decomposes operating systems into elementary pieces called libraries (e.g., schedulers, memory allocators, drivers, filesystems, network stacks, etc.) that users can then pick and choose from, using a menu to quickly build images tailored to the needs of specific applications. In greater detail, Unikraft consists of two basic components (see Figure 1):

  • Library pools contain libraries that the user of Unikraft can select from to create the unikernel. From the bottom up, library pools are organized into (1) the architecture library tool, containing libraries specific to a computer architecture (e.g., x86_64, ARM32 or MIPS); (2) the platform tool, where target platforms can be Xen, KVM, bare metal (i.e. no virtualization), user-space Linux and potentially even containers; and (3) the main library pool, containing a rich set of functionality to build the unikernel. This last library includes drivers (both virtual such as netback/netfront and physical such as ixgbe), filesystems, memory allocators, schedulers, network stacks, standard libs (e.g. libc, openssl, etc.), and runtimes (e.g. a Python interpreter and debugging and profiling tools). These pools of libraries constitute a codebase for creating unikernels. As shown, a library can be relatively large (e.g libc) or quite small (a scheduler), which allows for customization for the unikernel.
  • The Unikraft build tool is in charge of compiling the application and the selected libraries together to create a binary for a specific platform and architecture (e.g., Xen on x86_64). The tool is currently inspired by Linux’s KCONFIG system and consists of a set of Makefiles. It allows users to select libraries, to configure them, and to warn them when library dependencies are not met. In addition, the tool can also simultaneously generate binaries for multiple platforms.

unikraft

Figure 1. Unikraft architecture.

Getting Involved
We are very excited about the recent open source release of Unikraft as a Xen Project Foundation incubator project. The Xen Project is a part of the Linux Foundation umbrella. We welcome developers willing to help improve Unikraft. Whether you’re interested in particular applications, programming languages, platforms, architectures or OS primitive. We are more than happy to build and receive contributions from the community. To get you started, here are a number of available resources:

Please don’t be shy about getting in touch with us, we would be more than happy to answer any questions you may have. You can reach the core Unikraft development team at sysml@listserv.neclab.eu .

Xen Project 4.7.4 and 4.9.1 are available

I am pleased to announce the release of Xen 4.7.4 and 4.9.1. Xen Project Maintenance releases are released in line with our Maintenance Release Policy. We recommend that all users of the 4.7 and 4.9 stable series update to the latest point release.

These releases are available from their git repositories

xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.7 (tag RELEASE-4.7.4)

xenbits.xen.org/gitweb/?p=xen.git;a=shortlog;h=refs/heads/stable-4.9 (tag RELEASE-4.9.1)

or from the XenProject download page

www.xenproject.org/downloads/xen-archives/xen-project-47-series/xen-474.html

www.xenproject.org/downloads/xen-archives/xen-project-49-series/xen-491.html

These releases contain many bug fixes and improvements. For a complete list of changes, please check the lists of changes on the download pages.

Announcing the Xen Project 4.10 RC and Test Day Schedules

On Monday, we created Xen 4.10 RC1 and will release a new release candidate every MONDAY, until we declare a release candidate as the final candidate and cut the Xen 4.10 release. We will also hold a Test Day every WEDNESDAY for the release candidate that was released the week prior to the Test Day starting from RC2. Note that RC’s are announced on the following mailing lists: xen-announce, xen-devel and xen-users. This means we will have Test Days on October 25th, Nov 1st, 8th, 15th and 22nd. Your testing is still valuable on other days, so please feel free to send Test Reports as outlined below at any time.

Getting, Building and Installing a Release Candidate

Release candidates are available from our git repository at

git://xenbits.xenproject.org/xen.git (tag 4.10.0-<rc>)

where <rc> is rc1, rc2, rc3, etc. and as tarball from

https://downloads.xenproject.org/release/xen/4.10.0-<rc>/xen-4.10.0-<rc>.tar.gz
https://downloads.xenproject.org/release/xen/4.10.0-<rc>/xen-4.10.0-<rc>.tar.gz.sig

Detailed build and Install instructions can be found on the Test Day Wiki. Make sure you check the known issues section of the instructions before trying to download an RC.

Testing new Features, Test and Bug Reports

You can find Test Instructions for new features on our Test Day Wiki and instructions for general tests on Testing Xen. The following pages provide information on how to report successful tests and how to report bugs and issues.

Happy Testing!

A Brief Introduction to the Xen Project and Virtualization from Mohsen Mostafa Jokar

Mohsen Mostafa Jokar is a Linux administrator who works at the newspaper Hamshahri as a network and virtualization administrator. His interest in virtualization goes back to when he was at school and saw a Microsoft Virtual PC for the first time. He installed it on a PC with 256 MB of RAM and used it to virtualize Windows 98 and DOS.

Beyond Linux and virtualization, Mohsen is also familiar with Fedora Core, Knoppix, RedHat, bochs, Qemu, Xen, Citrix XenServer and VMWare ESXi.

He recently wrote an introductory book on Xen called “Hello Xen Project,” which is now available via this wiki. It provides a brief history of virtualization and the Xen Project, dom(s) and grub, using the Xen Project, and having fun with Xen. He hopes that more Xen users and experts will share their expertise and knowledge about this strong, stable and reliable virtualization platform through this wikipedia page.

A celebration of Xen Project 4.9 and the wiki book.

A celebration of Xen Project 4.9 and the wiki book.

In addition to his fascination with virtualization, Mohsen is a translator and an author. He has translated and written books for beginners and professional users that focus on virtualization, security and Linux. A few books that he has worked on as a technical reviewer include “Elixir in Action,” “Learn Git in a Month of Lunches,” “Mesos in Action” and “Reverse Engineering for Beginners.”

Announcing the Windows PV Console Driver

It has long been the case that all HVM guests under Xen are provided with a PV console. You can attach to this console in the same way that you attach to the console of a PV guest, by typing in the control domain:

xl console name_of_guest

Until recently there has been no Windows PV driver interaction with this console. Starting with this commit support for logging via the PV console was added to the XENBUS driver.

I’m happy to announce that the three commits to XENBUS starting with this one added the necessary infrastructure to support a brand new XENCONS PV driver which exposes the PV console to Windows user-space as a character device.

The XENCONS driver source is hosted alongside the other PV driver sources on xenbits.xen.org and development builds are available for download here.

The XENCONS package also contains a Windows service to monitor the presence of the PV console device and invoke a command shell login process with redirected stdin/stdout. This means that, once the driver package has been installed, if you attach to the PV console and hit ENTER you’ll see a prompt something like this:

DESKTOP-KVEHAKT login:

From this prompt you can log in as any local user and you’ll then be presented with the command shell:

DESKTOP-KVEHAKT login: User
Password:
Microsoft Windows [Version 10.0.15063]
(c) 2017 Microsoft Corporation. All rights reserved.

C:\Users\User>

Be aware that this shell is running in session 0 so does not have access to the interactive session, but you can still use it for many administrative tasks. For instance, you can run netsh to display aspects of your network configuration:


C:\Users\User>netsh
netsh>interface ipv4 show global
Querying active state...

General Global Parameters
---------------------------------------------
Default Hop Limit : 128 hops
Neighbor Cache Limit : 256 entries per interface
Route Cache Limit : 4096 entries per compartment
Reassembly Limit : 33420160 bytes
ICMP Redirects : enabled
Source Routing Behavior : dontforward
Task Offload : enabled
Dhcp Media Sense : enabled
Media Sense Logging : disabled
MLD Level : all
MLD Version : version3
Multicast Forwarding : disabled
Group Forwarded Fragments : disabled
Randomize Identifiers : enabled
Address Mask Reply : disabled
Minimum Mtu : 576
Locality Address Selection : disabled
Flow Label : disabled

Current Global Statistics
---------------------------------------------
Number of Compartments : 1
Number of NL clients : 7
Number of FL providers : 4

Over the coming weeks I intend to add to the functionality that the driver provides. One obvious extension would be some form of hotkey support to link into the XENBUS_DEBUG interface to enable PV drivers to register a callback to be triggered by a particuler key.

If you are interested in this then please try the XENCONS package and send feedback to the mailing list.

Updates on XSA-213, XSA-214 and XSA-215

Today we released three patches for the following vulnerabilities: XSA-213, XSA-214 and XSA-215. Xen Project follows industry-accepted best practices regarding software security. This includes observing an embargo period, during which time the Xen Project security team will assess, respond, and prepare patches fixing the vulnerability, and distribute them privately to software and cloud providers before the public disclosure occurs.

When issuing a Xen Project Security Advisory (XSA), during the embargo this advisory is pre-disclosed to only members on the Xen Project Pre-Disclosure List. Vendors and open source projects who are on the Xen Project pre-disclosure list will not be affected by this security vulnerability and have updated their systems. The Xen Project security team has created fixes for these vulnerabilities, which can be publicly downloaded here: http://xenbits.xen.org/xsa/

Public cloud providers on the Xen Project predisclosure list were notified of these vulnerabilities two weeks ago; if your public cloud provider is on the list it is likely that your VMs are not vulnerable. Distributions and other software providers were also notified; they should have updated packages available soon (if they are not available already).

All three vulnerabilities have the potential to enable a guest virtual machine to break out of the hypervisor isolation. However, in order to exploit this vulnerability, an attacker would need to be running code in kernel mode of one or more VMs on the system. Any system that allows untrusted users to run arbitrary kernels will be particularly vulnerable.

Systems which only allow trusted users (such as IT professionals employed by the company) to run arbitrary kernels are less vulnerable, because an attacker would first need to find one or more exploit in the software running on one of the VMs before being able to then exploit this vulnerability. However, all users are encouraged to update as soon as possible.

Any 64-bit PV guest can exploit the vulnerability with XSA-213. The other two are more constrained. XSA-214 requires an attacker to control two different kinds of guests (either a PV one and an HVM one or a 32-bit PV one and 64-bit PV one). XSA-215 only affects you if your host has a very large amount of memory (either 3.5 TiB or 5 TiB depending on configuration).

Again, even with these constraints, we encourage you to update as soon as possible.

We take security very seriously and have developed security process best practices that are aimed for cloud environments that maximize fairness and transparency. We also have a very strict standard of review when it comes to new code being added to the Xen Project. We run Coverity static analyzer regularly to prevent certain classes of programming errors from being introduced. Additionally, we regularly run a generational fuzzing tool on our instruction emulator.

The Xen Project community developed Live Patching and introduced it into Xen Project 4.7. Now security fixes can be deployed without having to reboot VMs or have significant spare compute capacity to avoid reboots via VM migration.

These vulnerabilities were discovered by Jann Horn, from Google Project Zero.