Tag Archives: XSA-197

What You Need to Know about Recent Xen Project Security Advisories

Today the Xen Project announced eight security advisories: XSA-191 to XSA-198. The bulk of these security advisories were discovered and fixed during the hardening phase of the Xen Project Hypervisor 4.8 release (expected to come out in early December). The Xen Project has implemented a security-first approach when publishing new releases.

In order to increase the security of future releases, members of the Xen Project Security Team and key contributors to the Xen Project, actively search and fix security bugs in code areas where vulnerability were found in past releases. The contributors use techniques such as code inspections, static code analysis, and additional testing using fuzzers such as American Fuzzy Lop. These fixes are then backported to older Xen Project releases with security support and published in bulk to make it easier for downstreams consumers to apply security fixes.

Before we declare a new Xen Project feature as supported, we perform a security assessment (see declare the Credit2 scheduler as supported). In addition, the contributors focused on security have started crafting tests for each vulnerability and integrated them into our automated regression testing system run regularly on all maintained versions of Xen. This ensures that the patch will be applied to every version which is vulnerable, and also ensures that no bug is accidentally reintroduced as development continues to go forward.

The Xen Project’s mature and robust security response process is optimized for cloud environments and downstream Xen Project consumers to maximize fairness, effectiveness and transparency. This includes not publicly discussing any details with security implications during our embargo period. This encourages anyone to report bugs they find to the Xen Project Security team, and allows the Xen Project Security team to assess, respond, and prepare patches, before before public disclosure and broad compromise occurs.

During the embargo period, the Xen Project does not publicly discuss any details with security implications except:

  • when co-opting technical assistance from other parties;
  • when issuing a Xen Project Security Advisory (XSA). This is pre-disclosed to only members on the Xen Project Pre-Disclosure List (see www.xenproject.org/security-policy.html); and
  • when necessary to coordinate with other projects affected

The Xen Project security team will assign and publicly release numbers for vulnerabilities. This is the only information that is shared publicly during the embargo period. See this url for “XSA Advisories, Publicly Released or Pre-Released”: xenbits.xen.org/xsa.

Xen’s latest XSA-191, XSA-192, XSA-193, XSA-194, XSA-195, XSA-196, XSA-197 and XSA-198 Advisory can all be found here:
xenbits.xen.org/xsa

Any Xen-based public cloud is eligible to be on our “pre-disclosure” list. Cloud providers on the list were notified of the vulnerability and provided a patch two weeks before the public announcement in order to make sure they all had time to apply the patch to their servers.

Distributions and other major software vendors of Xen Project software were also given the patch in advance to make sure they had updated packages ready to download as soon as the vulnerability was announced. Private clouds and individuals are urged to apply the patch or update their packages as soon as possible.

All of the above XSAs that affect the hypervisor can be deployed using the Xen Project LivePatching functionality, which enables re-boot free deployment of security patches to minimize disruption and downtime during security upgrades for system administrators and DevOps practitioners. The Xen Project encourages its users to download these patches.

More information about the Xen Project’s Security Vulnerability Process, including the embargo and disclosure schedule, policies around embargoed information, information sharing among pre-disclosed list members, a list of pre-disclosure list members, and the application process to join the list, can be found at: www.xenproject.org/security-policy.html

Mirage OS v2.0: The new features

The first release of Mirage OS back in December 2013 introduced the prototype of the unikernel concept, which realised the promise of a safe, flexible mechanism to build highly optimized software stacks purpose-built for deployment in the public cloud (see the overview of Mirage OS for some background). Since then, we’ve been hard at work using and extending Mirage for real projects and the community has been steadily growing.

Today, we’re thrilled to announce the release of Mirage OS v2.0! Over the past few weeks the team has been hard at work writing about all the new features in this latest release, which I’ve been busy co-ordinating. Below are summaries of those features and links to in-depth blog posts where you can learn more:

Thomas Leonard's Cubieboard2

Thomas Leonard’s Cubieboard2

ARM device support: While the first version of Mirage was specialised towards conventional x86 clouds, the code generation and boot libraries have now been made portable enough to operate on low-power embedded ARM devices such as the Cubieboard 2. This is a key part of our efforts to build a safe, unified multiscale programming model for both cloud and mobile workloads as part of the Nymote project. We also upstreamed the changes required to the Xen Project so that other unikernel efforts like HalVM or ClickOS can benefit.

Irmin – distributed, branchable storage: Unikernels usually execute in a distributed, disconnection-prone environment (particularly with the new mobile ARM support). We therefore built the Irmin library to explicitly make synchronization easier via a Git-like persistence model that can be used to build and easily trace the operation of distributed applications across all of these diverse environments.

OCaml TLS: The philosophy of Mirage is to construct the entire operating system in a safe programming style, from the device drivers up. This continues in this release with a comprehensive OCaml implementation of Transport Layer Security, the most widely deployed end-to-end encryption protocol on the Internet (and one that is very prone to bad security holes). The series of posts is written by Hannes Mehnert and David Kaloper.

Modularity and communication: Mirage is built on the concept of a library operating system, and this release provides many new libraries to flexibly extend applications with new functionality.

  • Fitting the modular Mirage TCP/IP stack together” by Mindy Preston explains the rather unique modular architecture of our TCP/IP stack that lets you swap between the conventional Unix sockets API, or a complete implementation of TCP/IP in pure OCaml.
  • Vchan: low-latency inter-VM communication channels” by Jon Ludlam shows how unikernels can communicate efficiently with each other to form distributed clusters on a multicore Xen host, by establishing shared memory rings with each other.
  • Modular foreign function bindings” by Jeremy Yallop continues the march towards abstraction by expaining how to interface safely with code written in C, without having to write any unsafe C bindings! This forms the basis for allowing Xen unikernels to communicate with existing libraries that they may want to keep at arm’s length for security reasons.

All the libraries required for these new features are regularly released into the OPAM package manager, so just follow the installation instructions to give them a spin. A release this size probably introduces minor hiccups that may cause build failures, so we very much encourage bug reports on our issue tracker or questions to our mailing lists. Don’t be shy: no question is too basic, and we’d love to hear of any weird and wacky uses you put this new release to! And finally, the lifeblood of Mirage is about sharing and publishing libraries that add new functionality to the framework, so do get involved and open-source your own efforts.

Xen Project @ FOSDEM’14: an Event Report

As usual, the first weekend of February (1st & 2nd Feb this year) is FOSDEM weekend. Taking place at “ULB Solbosch Campus, Brussels, Belgium, Europe, Earth”, FOSDEM is the Open Source event of the year. At least for Europe: the website claims that FOSDEM hosts 5,000+ geeks and hackers and 512 lectures!

But it doesn’t stop here: the main wireless network provided (essid FOSDEM) was IPv6 only… as announced in the opening keynote(video at online already!). And in fact it took a while for me to figure out why my Android phone could connect but not get any IP ?!?!

Like 20140202_100927_good last year, I went to FOSDEM to see all the cool stuff and to help man the The Xen Project booth. Although this time I also had a my own talk to deliver. This was only my second FOSDEM, so this may still be my “nubiennes” talking, but it is really hard to describe how big and amazing the event is. Even more so, if you consider it is entirely run and organized by volunteers. And members of the Xen Project helped organize parts of FOSDEM beyond our booth : Ian Jackson and Tim Mackey were Video Volunteers and Lars Kurth helped organize a DevRoom.

It is amazing to have the chance to choose from such a huge amount of super high quality talks and presentations. It is great to see what the most thriving Open Source projects have to show off at their booths, in the exposition area, and to collect some gadgets. Being one of those, The Xen Project was giving away T-Shirts and some other gadgets, for free (some project use FOSDEM to raise funds). And that, like last year, has been quite a success!

T-Shirts 20140201_114724_goodapart, this year we decided we really wanted to do our best to show everyone what Xen is really capable of. That is why we decided to invite community members to host demos. It was an experiment: and it has been a great success! Basically, we invited Xen related projects and or companies that use Xen for their solutions and products, to submit a request for an exclusive slot at the booth. They would then show what they do to FOSDEM attendees. I’m calling it a success because we were fully booked and able to show demos of the following projects: Xen on an Android tablet (a SAMSUNG NEXUS 10), Xen Orchestra, Mirage OS, OSv, Qubes OS and ClickOS. I think we really should do this again next year. I ran the QubesOS demo myself, and it was very pleasant to notice people appreciating what high level of isolation and security this very special Linux distribution provides. It leverages some of the most advanced Xen features, such as stub and driver domains (a couple of people were amazed when I showed them how untrusted PDF conversion works in Qubes! :-D). The presence of two Cloud Operating Systems, MirageOS and OSv (both running on top of Xen, of course) also raised quite some interest. Many attendees were impressed about the responsiveness of SAMSUNG’s solution for virtualizing the GPU in their dual Android tablet demo and the performances and the super quick boot time of ClickOS. Others liked the clean and effective interface of Xen Orchestra. One person even commented that Xen Orchestra looks more impressive than what VMWare provides.

It was great to have the chance to talk with many people that know and use Xen happily and fruitfully, and that were willing to acknowledge all our efforts: technical and not. And of course, to grab a free T-Shirt! Having done pretty much the same last year allows me to run a comparison. There is little doubt that there were more people interested in Xen, and much more awareness about the project doing well and actually expanding (e.g. into embedded and automotive). It was also good, as a developer, to get the chance to talk to users (of any kind) and other members of the Xen community. Like the volunteers showing the demos. For example. I had a great discussion with developers from Samsung about helping them upstream some (at least the kernel and Xen parts) of the incredible work they have done with GPU virtualization on the NEXUS 10.

20140202_101132_good

Even when I wasn’t at the booth, there have been  many chances to hear about Xen, in the various talks given in the Virtualization, BSD and Automotive devrooms, as was announced in this post from some weeks ago. Oh, speaking of the talks, another pretty awesome fact: this year, everything –I mean, every single talk– has been recorded, and videos will be available online soon.

Most of the action, as far as the Xen Project was concerned, happened in the Virtualization & IaaS devroom. Let’s not forget that members of the Xen Project helped record these. A big thank you to them and also a big thank you to the whole FOSDEM video team. Check out out the streams on the FOSDEM website (some are available already). Of course, videos and slides for the Xen talks will be available on the usual aggregators (vimeo and slideshare).

marks-pic

All in all, I personally very much enjoyed having been at FOSDEM. For The Xen  Project, I think we really did good in teaming together with all the members of the community and presenting ourselves to current and prospective users in very good shape. Can’t wait for next year!

Xen on ARM and the Device Tree vs. ACPI debate

ACPI vs. Device Tree on ARM

Some of you may have seen the recent discussions on the linux-arm-kernel mailing list (and others) about the use of ACPI vs DT on the ARM platform. As always LWN have a pretty good summary (currently subscribers only, becomes freely available on 5 December) of the situation with ACPI on ARM.

Device Tree (or DT) and Advanced Configuration & Power Interface (or ACPI) are both standards which are used for describing a hardware platform e.g. to an operating system kernel. At their core both technologies provide a tree like data structure containing a hierarchy of devices and specifying what type they are and a set of “bindings” for that device. A binding is essentially a schema for specifying I/O regions, interrupt mappings, GPIOs and clocks etc.

For the last few years Linux on ARM has been moving away from hardcoded “board files” (a bunch of C code for each platform) towards using Device Tree instead. In the ARM space ACPI is the new kid on the block and has many unknowns. Given this the approach to ACPI which appears to have been reached by the Linux kernel maintainers, which is essentially to wait and see how the market pans out, seems sensible.

On the Xen side we started the port to ARM around the time that Linux’s transition from board files to Device Tree was starting and made the decision early on to go directly to device tree (ACPI wasn’t even on the table at this point, at least not publicly). Xen DT to discover all of the hardware on the system, both that which it intends to use itself and that which it intends to pass to domain 0. As well as consuming DT itself Xen also creates a filleted version of the host DT which it passes to the domain 0 kernel. DT is simple and yet powerful enough to allow us to do this relatively easily.

DT is also used by some of the BSD variants in their ARM ports as well.

My Position as Xen on ARM Maintainer

The platform configuration mechanism supported by Xen on ARM today is Device Tree. Device Tree is a good fit for our requirements and we will continue to support it as our primary hardware description mechanism.

Given that a number of operating system vendors and hardware vendors care about ACPI on ARM and are pushing hard for it, especially in the ARM server space, it is possible, perhaps even likely, that we will eventually find ourselves needing support ACPI as well. On systems which support both ACPI and DT we will continue to prefer Device Tree. Once ARM hardware platforms that only support ACPI are available, we will obviously need to support ACPI.

The Xen Project works closely with the Linux kernel and other open source upstreams as well as organisations such as Linaro. Before Xen on ARM can support ACPI I would like see it gaining some actual traction on ARM. In particular I would like to see it get to the point where it has been accepted by the Linux kernel maintainers. It is clearly not wise for Xen to be pioneering the use of ACPI before to it becoming clear whether or not it is going to gain any traction in the wider ecosystem.

So if you are an ARM silicon or platform vendor and you care about virtualization and Xen in particular, I encourage you to provide a complete device tree for your platform.

Note that this only applies to Xen on ARM. I cannot speak for Xen on x86 but I think it is pretty clear that it will continue to support ACPI so long as it remains the dominant hardware description on that platform.

It should also be noted that ACPI on ARM is primarily a server space thing at this stage. Of course Xen and Linux are not just about servers: both communities have sizable communities of embedded vendors (on the Xen side we had several interesting presentations at the recent Xen Developer Summit on embedded uses of Xen on ARM). Essentially no one is suggesting that the embedded use cases should move from DT to ACPI and so, irrespective of what happens with ACPI, DT has a strong future on ARM.

ACPI and Type I Hypervisors

Our experience on x86 has shown that the ACPI model is not a good fit for Type I hypervisors such as Xen, and the same is true on ARM. ACPI essentially enforces a model where the hypervisor, the kernel, the OSPM (the ACPI term for the bit of an OS which speaks ACPI) and the device drivers all must reside in the same privileged entity. In other words it effectively mandates a single monolithic entity which controls everything about the system. This obviously precludes such things as dividing hardware into that which is owned and controlled by the hypervisor and that which is owned and controlled by a virtual machine such as dom0. This impedance mismatch is probably not insurmountable but experience with ACPI on x86 Xen suggests that the resulting architecture is not going to be very agreeable.

UEFI

Due to their history on x86 ACPI and UEFI are often lumped together as a single thing when in reality they are mostly independent. There is no reason why UEFI cannot also be used with Device Tree. We would expect Xen to support UEFI sooner rather than later.

Xen @ Linaro Connect Europe 2013

My name is Julien Grall.  I joined the Citrix Open Source team few months ago to work on Xen on ARM with Ian Campbell and Stefano Stabellini. Since Citrix has joined the Linaro Enterprise Group (LEG), I’m also part of the virtualization team which takes care of Xen, KVM and QEMU within Linaro.

A couple of weeks ago, I have attended my first Linaro Connect Europe, held in Dublin from 8th to 12th of July.  All the major players in the ARM world came together to discuss the future of the industry and build an healthy Open Source ecosystem for ARM.

Continue reading

Bringing Xen on the Chromebook and the Arndale Board: the journey so far

I’ve started to work on the Xen on ARM project by trying to get Xen running on the nice little Samsung Chromebook which run an ARM processor with the virtualization extensions. The Chromebook uses Exynos 5250 dual core platform with 2GB of RAM, which could be perfect for a good demo of Xen on ARM.

But porting Xen to new ARM platform without any useful debug output of some kind was really hard at the beginning. There is no accessible serial port available on the Chromebook and Xen cannot display anything on the screen. But there is a trick, write your log to specific area in RAM, and read it back after a reboot. It is really annoying, does take more time to have any feedback from Xen and you don’t know when you can reboot the machine to get those logs. This is how we did it, until we received an Arndale Development board.

So, we started the development on Chromebook by simply installing a Linux on it (on an SD card). There were few tutorial on internet, so it was not too hard. Then we recompiled a kernel, as it will be needed to integrate dom0 support. And last, we tried to understand how to use the bootloader on the machine to load Xen where we need it to be and execute it, it took some time and many tries. When we finally get something from Xen, we could continue make it work on the Chromebook. We had bugs like loading Xen at the wrong place in RAM or missing information in the device tree. Then it took sometime to load Linux correctly and we had to use some hack to actually find the Linux blob in memory as the default bootloader was not very helpful for that.

We found several problems were a page table was written in the middle of the binary in memory because of too many hack on how to load Linux for dom0. Before starting to work with the Arndale, we’ve been able to run some of Linux code, but fail early in Linux when it tried to load the device tree.

The Arndale board finally arrived, we had to start over, different bootloader and this time there is a serial port!! The Arndale use the same hardware platform, namely, the Exynos 5.

So first thing, how to start Xen on it? It appears to be relatively easy to load your own kernel, Xen here. So let’s hack a serial driver for Xen, in assembly first :-). We found out that the bootloader delivered with the board does not let Xen switch to hypervisor mode, so we had to recompile a u-boot for the board. Not a big deal, Linaro has done some work on the board as well, so we could get u-boot from their source tree and finally have Xen get its hypervisor mode.

After the first bits resolved, we could get along and figured out all the hack/fix needed in Xen in order to run it on this second platform, the first one was a Versatile Express. There were a lot of assumption about the hardware. So to get things running, we just replace those by others because having a proper way to handle different platform take a lot of time.

We had an issue with the time on this platform. The ARM Generic Timer was not working, it was always giving the same time. After trying to use the Multi-Core Timer of the Exynos platform, we realise there was the same, but we had to set a bit in the MCT register in order for the timer to start counting.

After resolving few other issues, we could finally get a prompt! Next, we worked on getting more, like starting few guest.

There is yet another hack for both the Arndale and the Chromebook to get a working network interface or having something on the screen of the Chromebook, it’s a 1:1 mapping of the memory of dom0. This is when the virtual address (seen by dom0) match the physical address (seen by the hardware). Normaly, those two addresses are different for any guest.

To conclude, it has been a lot of fun working on those two. We could get a working demo for both the Arndale and the Chromebook of Xen capabilities. There is still a lot of work ahead of us, but we are getting there.

Much more happened in the latest weeks, during which I concentrated on something different than Xen on ARM or the Chromebook. For instance, there is now a better multi-platform support in Xen and SMP support is getting there as well.

Here is some wiki page on how to get started: