Tag Archives: hypervisor

Xen Project Matrix

Xen Project Hypervisor: Virtualization and Power Management are Coalescing into an Energy-Aware Hypervisor

Power management in the Xen Project Hypervisor historically targets server applications to improve power consumption and heat management in data centers reducing electricity and cooling costs. In the embedded space, the Xen Project Hypervisor faces very different applications, architectures and power-related requirements, which focus on battery life, heat, and size.

Although the same fundamental principles of power management apply, the power management infrastructure in the Xen Project Hypervisor requires new interfaces, methods, and policies tailored to embedded architectures and applications. This post recaps Xen Project power management, how the requirements change in the embedded space, and how this change may unite the hypervisor and power manager functions. Read the full article on Linux.com here.

Xen Project Announces Schedule for its Annual Developer and Design Summit

Today, we are excited to announce the program and speakers for the Xen Project Developer and Design Summit. The summit brings together developers, engineers, and Xen Project power users for in-person collaboration and educational presentations. The event will take place in Nanjing Jiangning, China from June 20 -22, 2018.

This is the fifth annual Xen Project Summit with presentations and panels focusing on hypervisor performance and development, security, automotive and much more. The conference will kick-off with a weather report from Lars Kurth, chairperson of the Xen Project and director of open source at Citrix.

At last year’s Xen Project Developer Summit in Budapest, Hungary.

A sample of presentations include:

  • Sung-Min Lee, principal engineer at Samsung Electronics, will present a production-ready automotive virtualization solution with Xen.
  • Marek Marczykowski-Górecki, senior systems developer, Invisible Things Lab, will present on linux-based device model stubdomains in Qubes OS.
  • Julien Grall, senior software virtualization engineer at Arm, will share capabilities that were added to the latest revision of the ARmv7-A architectures and how Arm has been improving virtualization support with incremental versions of the Armv8 architecture.
  • Felipe Huici, chief researcher at NEC, and Florian Schmidt, research scientist at NEC, will co-present on Unikraft, a sub-project of the Xen Project aimed at automativing the process of building customized unikernels tailored to a specific applications.
  • Bo Zhang, business analyst at Huawei, will introduce Huawei Cloud’s optimization on the Xen platform to solve regular problems that occur in customer scenarios

You can view the full schedule here.

Beyond panels and presentations, the Xen Project will be running design sessions that share a similar format to Xen Project hackathons. Attendees of the conference have the opportunity to propose design sessions during the conference. Current design topics already include Making Safety Certifications for Xen Easier, From Hobbyist to Maintainer: Why and How and Reworking x86 in Xen (Current and Future Plans).

If you’ve never attended a Xen Project Developer and Design Summit, check out last year’s presentations to get a better feel for the event.

A special thank you Citrix for being a diamond sponsor of the summit.

 

Summer = Xen Project Internships!

We received a lot of amazing submissions for our summer Outreachy internship program and have accepted Dafna Hirschfeld to join us in creating new execution targets for Unikraft. Unikraft is a Xen Project incubation project that aims to simplify the process of building unikernels through a unified and customizable code base.

Currently, Unikraft supports building images that can be executed as a virtual machine on Xen and KVM, and as an ELF binary within the Linux user space environment. Support for more execution targets is done by providing more platform libraries that can be chosen during build. Dafna will be fairly free to choose a platform, perhaps based on familiarity or curiosity. Examples for platform choice include bare-metal, ARM, X86_64, VMware, Microsoft Hyper-V, and bhyve.

If you are unfamiliar with Outreachy, it provides three-month internships for people from groups traditionally underrepresented in tech. Interns work remotely with mentors from Free and Open Source Software (FOSS) communities. Xen Project interns have later gone on to work at companies like Oracle, Google and Citrix.

In addition to Outreachy, we are excited to announce a few new interns that will be working on the Xen Project hypervisor through Google Summer of Code. Although the Xen Project was not a mentoring organisation for Google Summer of Code this year, FreeBSD and The Honeypot Project were and had a number of Xen Project related projects. Google Summer of Code is a global program focused on bringing more student developers into open source software development.

Interns that are a part of the Google Summer of Code and working on pushing Xen Project technologies forward include:

  • Kristaps Civkulis who will be working on enabling the EFI loader to load FreeBSD Xen Dom0. There are two parts to the project – you can learn more about it here. The organization supporting this is the FreeBSD Project.
  • Pratyush Yadav who will import the Xen grant-table bus_dma(9) handlers from OpenBSD. FreeBSD Project is supporting Pratyush.
  • Lele Ma who will Port LibVMI to Xen MiniOS. In this project, the core functionalities of the LibVMI will be ported to Xen MiniOS. After ported, Xen MiniOS will have the basic capabilities of introspecting the memory of other guest virtual machines. Honeynet Project is supporting Lele.
  • Honeynet Project is also supporting Stewart Sentanoe who is working on stealth monitoring with Xen altp2m based on previous work that has been done – see here. And Ulrich Fourier who is working on adding support for ARM introspection, which is a follow-up to a 2016 GSoC project that developed altp2m support to Xen on ARM.

Working in open source is a great way to start your career in technology. In a recent survey from HackerRank 84% of respondents (including CEOs, CTOs and company founders) said they look to an applicant’s GitHub project work as an indicator of a prospective employee’s on-the-job skills.

We want to thank everyone who applied to our Outreachy scholarship, and look forward to sharing the accomplishments of our interns. Welcome to open source!

 

Call for Proposals Open for the Xen Project Developer and Design Summit Happening in June!

Registration and the call for proposals are open for the Xen Project Developer and Design Summit 2018, which will be held in Nanjing Jiangning, China from June 20 – 22, 2018. The Xen Project Developer and Design Summit combines the formats of Xen Project Developer Summits with Xen Project Hackathons, and brings together the Xen Project’s community of developers and power users.

Submit a Talk

Do you have an interesting use case around Xen Project technology or best practices around the community? There’s a wide variety of topics we are looking for, including cloud, server virtualization, unikernels, automotive, security, embedded environments, network function virtualization (NFV), and more. You can find all the suggested topics for presentations and panels here (make sure you select the Topics tab).

Several formats are being accepted for speaking proposals, including:

  • Presentations and panels
  • Interactive design and problem solving sessions. These sessions can be submitted as part of the CFP, but we will reserve a number of design sessions to be allocated during the event.
    • Proposers of design sessions are expected to host and moderate design sessions following the format we have used at Xen Project Hackathons. If you have not participated in these in the past, check out past event reports from 2016, 2015 and 2013.

Never talked at a conference before? Don’t worry! We encourage new speakers to submit for our events!

Here are some dates to remember for submissions and in general:

  • CFP Close: April 13, 2018
  • CFP Notifications: April 30 – May 2, 2018
  • Schedule Announced: May 3, 2018
  • Event: June 20 – 22, 2018

Registration

Come join us for this event, and if you register by May 2, you’ll get an early bird discount of $125/ 800 Yuan Travel stipends are available for students or individuals that are not associated with a company. If you have any questions, please send a note to community.manager@xenproject.org.

Curious about last year’s event? Check out a few of our presentations last year here!

PV Calls: a new paravirtualized protocol for POSIX syscalls

Let’s take a step back and look at the current state of virtualization in the software industry. X86 hypervisors were built to run a few different operating systems on the same machine. Nowadays they are mostly used to execute several instances of the same OS (Linux), each running a single server application in isolation. Containers are a better fit for this use case, but they expose a very large attack surface. It is possible to reduce the attack surface, however it is a very difficult task, one that requires minute knowledge of the app running inside. At any scale it becomes a formidable challenge. The 15-year-old hypervisor technologies, principally designed for RHEL 5 and Windows XP, are more a workaround than a solution for this use case. We need to bring them to the present and take them into the future by modernizing their design.

The typical workload we need to support is a Linux server application which is packaged to be self contained, complying to the OCI Image Format or Docker Image Specification. The app comes with all required userspace dependencies, including its own libc. It makes syscalls to the Linux kernel to access resources and functionalities. This is the only interface we must support.

Many of these syscalls closely correspond to function calls which are part of the POSIX family of standards. They have well known parameters and return values. POSIX stands for “Portable Operating System Interface”: it defines an API available on all major Unixes today, including Linux. POSIX is large to begin with and Linux adds its own set of non-standard calls on top of it. As a result a Linux system has a very high number of exposed calls and, inescapably, also a high number of vulnerabilities. It is wise to restrict syscalls by default. Linux containers struggle with it, but hypervisors are very accomplished in this respect. After all hypervisors don’t need to have full POSIX compatibility. By paravirtualizing hardware interfaces, Xen provides powerful functionalities with a small attack surface. But PV devices are the wrong abstraction layer for Docker apps. They cause duplication of functionalities between the guest and the host. For example, the network stack is traversed twice, first in DomU then in Dom0. This is unnecessary. It is better to raise hypervisor abstractions by paravirtualizing a small set of syscalls directly.

PV Calls

It is far easier and more efficient to write paravirtualized drivers for syscalls than to emulate hardware because syscalls are at a higher level and made for software. I wrote a protocol specification called PV Calls to forward POSIX calls from DomU to Dom0. I also wrote a couple of prototype Linux drivers for it that work at the syscall level. The initial set of calls covers socket, connect, accept, listen, recvmsg, sendmsg and poll. The frontend driver forwards syscalls requests over a ring. The backend implements the syscalls, then returns success or failure to the caller. The protocol creates a new ring for each active socket. The ring size is configurable on a per socket basis. Receiving data is copied to the ring by the backend, while sending data is copied to the ring by the frontend. An event channel per ring is used to notify the other end of any activity. This tiny set of PV Calls is enough to provide networking capabilities to guests.

We are still running virtual machines, but mainly to restrict the vast majority of applications syscalls to a safe and isolated environment. The guest operating system kernel, which is provided by the infrastructure (it doesn’t come with the app), implements syscalls for the benefit of the server application. Xen gives us the means to exploit hardware virtualization extensions to create strong security boundaries around the application. Xen PV VMs enable this approach to work even when virtualization extensions are not available, such as on top of Amazon EC2 or Google Compute Engine instances.

This solution is as secure as Xen VMs but efficiently tailored for containers workloads. Early measurements show excellent performance. It also provides a couple of less obvious advantages. In Docker’s default networking model, containers’ communications appear to be made from the host IP address and containers’ listening ports are explicitly bound to the host. PV Calls are a perfect match for it: outgoing communications are made from the host IP address directly and listening ports are automatically bound to it. No additional configurations are required.

Another benefit is ease of monitoring. One of the key aspects of hardening Linux containers is keeping applications under constant observation with logging and monitoring. We should not ignore it even though Xen provides a safer environment by default. PV Calls forward networking calls made by the application to Dom0. In Dom0 we can trivially log them and detect misbehavior. More powerful (and expensive) monitoring techniques like memory introspection offer further opportunities for malware detection.

PV Calls are unobtrusive. No changes to Xen are required as the existing interfaces are enough. Changes to Linux are very limited as the drivers are self-contained. Moreover, PV Calls perform extremely well! Let’s take a look at a couple of iperf graphs (higher is better):

iperf client

iperf server

The first graph shows network bandwidth measured by running an iperf server in Dom0 and an iperf client inside the VM (or container in the case of Docker). PV Calls reach 75 gbit/sec with 4 threads, far better than netfront/netback.

The second graph shows network bandwidth measured by running an iperf server in the guest (or container in the case of Docker) and an iperf client in Dom0. In this scenario PV Calls reach 55 gbit/sec and outperform not just netfront/netback but even Docker.

The benchmarks have been run on an Intel Xeon D-1540 machine, with 8 cores (16 threads) and 32 GB of ram. Xen is 4.7.0-rc3 and Linux is 4.6-rc2. Dom0 and DomU have 4 vcpus each, pinned. DomU has 4 GB of ram.

For more information on PV Calls, read the full protocol specification on xen-devel. You are welcome to join us and participate in the review discussions. Contributions to the project are very appreciated!

Virtualization Mini Summit, July 22, 2008 at linuxsymposium 2008

 

linuxsymposium 2008, Ottawa, Canada, July 23 – July 26, 2008 http://www.linuxsymposium.org

The intent of the Virtualization MiniSummit is to provide a forum for attendees to explore all aspects of Linux virtualization. Whether that be the underlying technology, application of the technology in their environment or new tools for managing and doing interesting and new things with virtualized servers.

Attendees can range from those developing virtualization technologies, using virtualization, managing virtualized environments to wanting to learn more about virtualization.

A potential list of topics:

* Review and/or provide deep insight into the fundamentals of specific virtualization technologies
* Exploration of project development opportunities
* Discussion of ideas to improve virtualization technologies
* How virtualized environments can be made manageable
* What’s worked and what has not worked
* New and emerging ideas for virtualizing Linux systems

Presentations

Presentation time slots will be on the order of 50 minutes and should include time for questions (10-15 minutes).

The proposal submission process requires that you submit a proposal and a personal biography that will be displayed on the Virtualization Mini Summit web site at: http://virtminisummit.linux.hp.com. Proposal submissions will be accepted until June 20, 2008. Early submitters will be given preferential consideration.

The proposal is your opportunity to show that your topic has merit and that you have the background to provide an excellent presentation at the virtualization Mini Summit.

Specifications:

Proposal:
# Maximum of 200 words
# Two paragraphs: The first should describe the topic you will be presenting in concise detail; The second should explain why your topic will be of interest to the attendees of the Virtualization Mini Summit.

Biography:
# Maximum of 100 words
# Written in 3rd person
# One paragraph describing your professional work experience, and related projects you are currently or have been involved in.

Papers:
Submission of a paper is not required, but would be greatly appreciated be the Virtualization Mini Summit attendees. Papers and/or Presentations must be submitted by July 16, 2008 so that they can be posted to the Virtualization Mini Summit in time for the event.

Accepted presenters will be provided instructions on submission of presentation, and paper upon notification.

Presentation Submissions

Please login to http://virtminiconf.linux.hp.com and submit to the Virtualization Mini Summit. Please create a login account via the “new user?” link located in the login box on the main page. Login and navigate to the “Call for Presentations” -> “Submit Here” folder. In the upper right drop-down menu “add to folder” your “file” or new “page” with your submittal (see the example template). In the same upper right menu bar, ensure that the content is the state of “private”, only the admin will have access to review the proposal. Please include the following information in your proposal:

Name:
Email Address (will be obscured in posting):
Company/Affiliation:
Title of Proposal:
Short Presentation Abstract: (200 words)
Short Biography: (100 words max.)

Submission Timeline:

  • Abstract submission deadline: Friday, June 20, 2008
  • Presentation acceptance notification before: Wednesday, June 27, 2008
  • Program schedule and abstracts posted: Wednesday, July 2, 2008
  • Paper/Presentation submission: Wednesday, July 16, 2008

Virtualization mini summit: Tuesday, July 22, 2008