Monthly Archives: August 2014

OPW: The Xen Project Developer Summit

This is a repost from Arianna’s blog, which contains a lot of in-depth technical articles related to the Xen Project.

So, about halfway through my OPW internship, I was informed that my wonderful mentor, Konrad Wilk, and Xen Project Community Manager Lars Kurth thought to allow me to attend the Xen Project Developer Summit that was to be held in Chicago on the 18th and 19th of August. I actually went there, had fun and learned a lot: now it’s time to write a blog post about it!

The Xen Project Developer Summit

The keynote session featured talks from Xen community and development coordinators, which aimed to provide an insight of the main improvements and of the new features introduced in the Xen Project. Lars Kurth described the challenges of coordinating such a large and growing project and the efforts made towards better cooperation. George Dunlap, release coordinator for Xen 4.3 and 4.4, and Konrad Rzeszutek Wilk, that took over the task for the upcoming 4.5, detailed the main improvements made to the Xen hypervisor during the last development cycles. They also explained and commented on the rationale behind the feature selection work performed by release coordinators and some of the toughest decisions that are required by their duty.Linux x86 co-maintainer David Vrabel provided a status update on the newest features added to Linux domain support and set up a wishlist for future features to be developed. During the keynote session I had the chance to meet a former Xen Project intern, Elena Ufimtseva, who worked on the implementation of vNUMA in Xen during the 2013 OPW round, mentored by Citrix Senior Engineer Dario Faggioli. She also presented her work in 2013 at the Xen Developer Summit. I could gather details and her opinion about her experience with OPW and in finding a job after the end of the internship, and I could discuss with her some of the challenges experienced by an intern and the best ways to exploit those difficult aspects by turning them into stronger development skills.

The following talks provided in-depth details on the main research aspects of Xen development. Some of them covered the performance of Xen with respect to networking, such as Jun Nakajima‘s talk on the main bottlenecks found while experimenting with Xen as a Network Virtualization Functions platform component and the solutions that were implemented by Intel. Other talks focused on storage, as Felipe Franciosi‘s insight on memory grant technologies available in Xen that can contribute to optimising aggregate workloads of several GB/s per guest (he actually allowed me to take part in the BoF session that followed his talk, therefore giving me the chance to hear further opinions and learn even more on the storage performance achieved by Xen guests). Still on the same trend, Filipe Manco presented NEC Europe’s work towards tracking down performance limitations and bottlenecks that increase startup latencies of Xen guests, when they are run in bursts of thousands; he also proposed a prototype reimplementation of some Xen components to prove his points. Anil Madhavapeddy showed the benefits of the new Irminsule distributed transactional filesystem, that allows to handle storage accesses in a version control system fashion, letting unikernels running in isolated stubdomains, such as MirageOS, use a common and consistent API. More talks covered security aspects of virtualization, as Mihai Dontu‘s presentation, that proposed a zero-footprint implementation of memory introspection for Xen domUs that can allow a supervisor domain to perform run-time detection of malware on Xen-based guests; James Bielman described Galois’ implementation of Mandatory Access Control for the Xenstore, showing how it can be managed by a centralized security server as it does not benefit from the XSM security policy. James Fehlig‘s talk, instead, covered the important topic of virtualization management tools, providing an overview of libvirt, a status update on the libxenlight driver and a roadmap proposal. Moving on to the topic of architecture and hardware support, Daniel Kiper approached the subject of EFI, outlining how Xen efficiently uses its infrastructure and what can be improved in the support provided by the hypervisor. Wei Liu instead described the status of vNUMA support in Xen, giving an in-depth report of its implementation and of its importance with relevant statistics.

The main session opened with a detailed overview of the Verizon Cloud architecture provided by Don Slutz, which described what features are used and the optimization it provides to both Xen and QEMU. It also featured a report on the Linux kernel delta that SUSE supports for Xen and a proposal on how to address it, delivered by Luis Rodriguez. Following another trend were some Xen-on-ARM-related talks, as the presentation by Stefano Stabellini, that provided an insight on the current state of the project and how it performs on the newest ARMv8 64-bit platforms, and the one by Julien Grall, which detailed the process of porting an OS as a Xen-on-ARM guest. Jonathan Daugherty also described, in his talk, his experience in porting FreeRTOS to Xen on a Cortex A15-based platform. More talks were performance-related, as Zoltan Kiss‘ presentation on network improvements made in XenServer and Feng Wu‘s on Intel’s work on introducing interrupt posting with its virtualization technology. John Else explained his work about efficient inter-domain communication of performance data, his findings about the XenStore being the bottleneck in the current technique and proposed a lock-free, efficient solution to the issue. Talks also included the relevant topic of testing for a software ecosystem as complex as the Xen one: Ian Jackson presented Xen’s automatic testing facility, osstest, outlining its last development steps and the wider set of configurations it now supports. Some of the talks were related, instead, to safety aspects of using Xen in an environment with real-time constraints. Nathan Studer and Robert VanVossen presented DornerWorks’ efforts on certifying Xen for automotive, medical and avionics, the challenges behind the task, a proposed roadmap to overcome the most tricky aspects and the current state of the project. Sisu Xi described the Washington University’s work on RT-Xen with the aim of combining real-time and virtualization. Willing to give, instead, a more detailed insight on unikernels, Adam Wick outlined their features and described the general rules that establish whether a unikernel is the right choice for a software infrastructure component. Glauber Costa introduced the topic of LibraryOSs, highlighting their benefits in terms of performance, lightness and scalability, describing which applications they support and how can prove to be useful to the Xen community. Philip Tricca explained the drawbacks of the static configuration used to isolate system components in OpenXT, a collection of hardened Linux-on-Xen virtual machines providing a user platform for client devices, and a new toolkit to enhance the platform’s flexibility.

During the main session I met some of my fellow OPW interns. I had the chance to talk to the brilliant Mindy Preston, who worked on MirageOS’s network stack fixing bugs and implementing missing RFCs, about her experience and exchange opinions about ARM-based boards. I had the chance to take part in the final OPW/GSoC-related panel with her; it also featured the very professional GSoC intern Jyotsna Prakash, who worked on cloud API support for MirageOS by implementing cloud API bindings for OCaml, along with some of our mentors and Lars Kurth as a host. The panel gave us interns the chance to provide feedback to our mentors and to the program’s organization and to express our opinion about what we learned from it. It also covered very important aspects of participating in a large open-source project within a heterogeneous and just as big community: George Dunlap thoroughly explained the lights and shadows of Linus Torvalds’ approach to commenting bad code, while Konrad Wilk delivered a thoughtful insight about how cultural differences can influence the interaction between developers during software review.

What Did I Learn

Being able to attend the conference was a highly educational experience. It allowed me to get a better idea of how the community is organized, to get involved even more and hear about the experience of other attendees. I also could benefit from my mentor’s advice on how to interact with other developers. Having to speak in front of an audience also has always been one of the aspects of working on a project that I feared the most; the chance to take part in the panel and my mentor’s very useful advice make a huge addition to my experience and will allow me to fully exploit the opportunity to share my findings and my enthusiasm with others on future occasions.

As a final note, I’d like to thank my very patient mentor, Konrad Wilk, for allowing me to take part in OPW (even if I applied to him as a candidate on the very last day before the deadline) and for his invaluable guidance during the program; I’d like to thank also the GNOME Foundation and Xen Community Manager Lars Kurth for granting me the opportunity to attend the conference, and Elena Ufimtseva for giving me the benefit of her own experience. Last, but not least, I’d like to thank my always so helpful advisor, Paolo Valente, and Citrix Senior Engineer Dario Faggioli for introducing me to the internship program.

Links

Xen Developer Summit – Chicago, 2014 – Schedule
Slides used for many of the talks (Xen Project’s official SlideShare)

Will you give Xen a ride … or will Xen give you a ride?

And by “a ride”, we actually mean a ride. Like this:

 

8275748195_4a18513755_z

Like, will Xen run in your car?  Well, it appears it will!

It all started with ARM Support

In fact, Xen Project developers started woking on supporting the ARM architecture (with hardware virtualization capabilities) a couple of years ago. The goal was simple: as soon as ARM server are available, it must be possible to run Xen Project software on them. That goal has been achieved, but that is another story!

It is well known that processors employing the ARM architecture are powering already the vast majority of the so called Embedded Systems, ranging from phones, tablets and smart TVs up to cars or even airplanes. But does that mean that at some point we will start to see virtualization capable chips in cars? And if yes, when? The answers to these questions are “Yes” and “really really really soon”! In fact, the Xen Project Hypervisor is uniquely placed to support this new range of use-cases. Its isolation and security features, flexible virtualization mode and architecture, not to mention driver disaggregation and the fact that it now supports ARM (and does it with only ~90K lines of code), make it a perfect fit for the embedded world.

Some Recent ‘History’

Mobile and embedded virtualization on ARM has a long history within the Xen Project, with research projects such as Samsung’s ARM PV port and the Embedded Xen effort. However these projects were mainly research focused. With ARM support becoming a part of the Xen Project Hypervisor last year and various market factors coming together, Xen Project based products are now on the horizon. Last autumn was pivotal in generating momentum for this concept. A number of companies showed real demos and prototypes at our 2013 Developer Summit, such as

  • The Xen Project Hypervisor running on a Nexus 10 (slides and video)
  • The Xen Project Hypervisor powering an in-vehicle infotainment (IVI) system, and other systems on the TI Jacinto 6 automotive platform designed for cars (slides and video).

Since then, momentum has built within the community – as can be seen on xen-devel mailing list discussions – to port embedded OSes to the Xen Hypervisor  (some examples: FreeRTOS, Erika and QNX). Contributions and patches for making The Xen Project Hypervisor work better in such environments started to arrive too, from individuals, research institutions and small and big companies. Among the companies, GlobalLogic Inc., a full-lifecycle product development services company, has made the largest contribution so far, but we must also mention DornerWorks, GaloisUniversity of Washington and Evidence (in collaboration with the University of Modena).

A summary of the past and ongoing activities of this kind is below:

What about now?

On Monday (we told you: “really really really soon” :-D), The Xen Collaborative Project and The Linux Foundation announced a new Embedded and Automotive initiative. Artem Mygaiev, AVP Development at GlobalLogic, will serve as the Embedded and Automotive Project Lead.

The Embedded and Automotive team within The Xen Project intends to build a platform around the Xen Hypervisor that enables using it for all the non-data center use cases (automotive, internet TV, mobile, etc.) by providing a community focal-point within the Xen Project community as well as within the wider open source community.

The team plans to:

  • develop and upstream necessary changes to The Xen Project Hypervisor and Linux
  • implement new drivers (such as GPU, HID, …), protocols, capabilities and functionality that are needed for a complete automotive/embedded/mobile virtualization stack
  • upstream all necessary changes to support such functionality in operating systems that are needed for these use-cases (e.g. Android, Linux, etc.)

For the occasion, Alex Agizim, CTO of Embedded Systems at GlobalLogic, which also is a member of The Linux Foundation Automotive Grade Linux Steering Committee, said:

With ARM support, Xen Project technology is a perfect fit for embedded systems and automotive use. For example, our Nautilus platform, based on The Xen Project virtualization, enables ourin-vehicle infotainment (IVI) and auto manufacturing partners to quickly and cost-effectively develop hybrid Android/Linux-based systems. Using Nautilus, developers are able to run multiple sandboxed OSes on a single System-on-Chip (SOC). This provides superior functionality and security for both infotainment and operational functions within a car.

The latest demo of GlobalLogic‘s Nautilus Platform has been shown at the latest edition of the Automotive Linux Summit, in Tokyo. Check out the video and slides. We also heard about further use cases for Xen Project Software at this week’s Developer Summit. The rate of innovation in our community in this area is staggering: fasten your seat belts! We will tell you about these more in an upcoming event report. All this activity is also creating many benefits for the cloud and traditional server use use-cases. Certification will lead to quality improvements across shared components. Realtime scheduling can be used for graphics and gaming use-cases in the cloud and for Network Function Virtualization. And so on, and so on, …

Learn More

GlobalLogic, in partnership with The Linux Foundation, will present a free webinar at 9 a.m. PDT, Wednesday, August 27, 2014, titled “Virtualization in the Automotive Industry.” Register today to learn how Xen Project technology adds reliability and security when adopting virtualization for automotive software development.

Vendors and individual developers interested in collaborating on embedded, automotive and mobile use cases are encouraged to join the new Xen Project subproject at http://xenproject.org/developers/teams/embedded-and-automotive.html.

Time to Register for Xen Project User Summit on Sept 15, 2014

Great Session Lineup Awaits Attendees in New York City!

xpus-join_us-2014

It’s time to make your travel plans to New York City for the September 15 arrival of the Xen Project User Summit!

The Lighthouse Executive Conference Center will play host to the only full day user-centric Xen Project event on this year’s calendar.  If you are a power user, a new user, or you just want to see what Xen Project can possibly do for you, you’ll want to be there.

Attendees will find an excellent selection of talks waiting for them at this year’s event.

Is Your Head in the Clouds?

We have a number of terrific cloud-related talks on the schedule!

Listen to SUSE‘s Peter Linnell talk about Xen, OpenStack, and the SUSE Cloud.  And are you thinking about unikernels in the cloud?  Adam Wick of Galois, provider of HaLVM, will expound on the virtues of tiny VMs providing a single-purpose operating environment (imagine many of the advantages of Linux Containers but with the security of a real hypervisor).  And Don Marti from Cloudius Systems will talk about using the OSv unikernel to create a C and Java environment directly on the hypervisor.

Or is Security Your Focus?

Then you’ll want to hear about the Zazen security framework as described by Steven Maresca of Zentific.  Plus, there’s my talk discussing the Advanced Security Features of Xen Project, most of which are easy to use, but most users don’t even seem aware of them.

Is Your Mind on the Future?

That’s good, because Oracle‘s Konrad Wilk, the Release Manager for the upcoming 4.5 Xen Project Release, will give us the run down of the features to expect.  There’s an impressive list of improvements in the works, so Konrad will help us to understand what is likely to make the cut.  And our own Community Manager Lars Kurth from Citrix will give us a healthcheck on the project itself, including fascinating information about how diverse our development ecosystem has become in the past few years.

Or Maybe Distribution Support is Your Thrust?

You’ll want to hear Johnny Hughes of the CentOS Project discuss the Xen4CentOS effort with a quick and easy cookbook for restoring our favorite hypervisor to the CentOS world.  He will also address how the effort has given birth to the new CentOS Virtualization SIG with an eye on making CentOS a key platform for all types of virtualization technologies.

Could XenServer be Your Concern?

If so, then you will want to hear XenServer Community Manager Tim Mackey discuss where XenServer has gone in the year since it became Open Source — and, more importantly, where it is expected to go from here.  And you will definitely want to hear from Olivier Lambert of Vates as he discusses the Xen Orchestra project, an interface for controlling XenServer and XAPI installations from a web-based interface.

Or High Availability?

Then you’ll definitely want to hear Will Auld of Intel discuss the COLO project.  Still in development, COLO aims to create lock-step VMs to deliver non-stop availability during a VM failure.  This is critical capability when you have a VM which needs to keep running no matter what.

Or Deployment?

Don’t miss Grant McWilliams‘ talk about deploying XAPI-based security devices.  It is actually an amazing tale describing the design and deployment of some amazing devices using the Xen Project Hypervisor to make magic happen.

Then You Need to Register Today!

For just $79 — less than the price of a good dinner in Manhattan — you can get all this, plus the User Jam Session.  User Jam is your time to give feedback about the project and the presentations given.

And tell a coworker to sign up, too!  We have two tracks most of the day, so bring a friend who can go to the sessions you can’t.

See the full schedule and register on the Linux Foundation website:

And we hope to see you in New York City on September 15!

Customers Call the Shots — Verizon Cloud Adds Business Value with Quality of Service

dslutz

Both businesses and consumers rely on public clouds for a range of tasks and activities from collaboration and video streaming to gmail and Netflix. New companies are born with just a dozen employees, a laptop and an Internet connection practically overnight. This is all thanks to cloud computing.

It’s no surprise that in the next six years, almost 90 percent of new spending on Internet and communications technologies, a $5 trillion global business, will be on cloud-based technology, according to industry analyst firm IDC. Cloud applications will also account for 90 percent of total mobile data traffic by 2018, according to the Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2013–2018.

The benefits for users are almost too numerous to count, but most IT professionals agree that cloud computing epitomizes constant change. Its ability to provide ubiquitous, on-demand access to a shared pool of networks, servers, storage, and services whenever and wherever they are needed is creating both market opportunity and market upheaval.

To temper the turbulence, capitalize on the opportunities and best prepare for any number of cloud unknowns, several of the world’s largest public providers including Amazon Web Services, Rackspace, IBM/SoftLayer and Verizon Terremark rely on Xen Project virtualization. Open source Xen Project software offers superior IT efficiencies, workload balancing, hyperscalability and tight security by running VMs on a cloud service.

While today the media is focusing on price wars and the possible commoditization of infrastructure as a service (IaaS), cloud providers like Verizon Terremark are innovating with novel Quality of Service agreements and new levels of automation. In his talk in Chicago at our Xen Project Developer Summit, Verizon Terremark’s Don Slutz will present an overview of the Verizon Cloud architecture based on Xen.

“It’s the core foundation of the Verizon Cloud, allowing our users to run any type or size workload they’d like to. Xen is critical to Verizon. Competing solutions were either too cost prohibitive or lacked the security controls that Xen has,” Don said.

Verizon Terremark is a long-time advocate of open standards and is more actively involved than ever before in the open source ecosystem. Verizon sponsors and participates in Xen Project software, invests in CloudStack and most recently joined the Cloud Foundry Foundation, hoping to see the cloud market mature quickly and provide businesses with cloud-based offerings that address specific needs like performance, choice, cost and flexibility.

For the past three years, Don has worked on integrating and designing Xen for the Verizon Cloud architecture along with seven full-time engineers. Today, clients are fully deployed on Verizon’s IaaS based on Xen. A focal point of his talk will be Verizon’s Quality of Service (QoS) goals with CPU, memory, network and disk performance.

“Often clouds end up requiring far too much support personnel, which we are trying to rectify. With our QoS agreement, we allow users to set the performance parameters their business requires and guarantee that Verizon will back these up at all times. Instead of focusing on speed or load size, we’ll guarantee certain CPU, memory, network or disk performance. This is really unique in the industry,” he added.

In addition to delivering workload efficiency, security and cost savings to its cloud customers, Verizon is also giving back to the Xen Project community.

“We’re working to make Verizon Cloud a high capacity service that allows people to move existing VMs easily onto it it,” Don said. “Our goal is to add enough VMWare support so that a guest can be exported from VMWare and automatically run without any changes on Xen.”

Verizon’s VMWare code is currently in review and in the past year has contributed 40 change sets that totals 4,300 lines of code.

Proof that demand for cloud services is growing and spurring more change, Don will also address Verizon’s design goals to move from three to seven data centers in the near future. If you’re interested in learning more, be sure to register today for the Xen Project Developer Summit to hear Don present on Tuesday, August 19 from 9 to 9:45 a.m.

About Don Slutz
Currently, Don works for Verizon Terremark enhancing Xen, which is the basis for Verizon Cloud. He got started early (1970) in computers because of his father Dr. Ralph J. Slutz and spent 16 years at Prime Computer in operating systems. He has extensive networking, performance, and testing experience.

2 weeks to Xen Project Developer Summit – Chicago 18-19, IL, USA

Only last week, the Xen Project team was at OSCON where we launched Mirage OS 2.0 (event report to follow soon, but in the meantime check out the following sessions Nymote and Mirage, Floss Weekly on Mirage OS and Community War Stories) and now our Developer Summit is just round the corner. As we have seen tremendous community growth in the last 12 months (>30%) and the most feature reach Xen Project Hypervisor release coming up soon, I thought I’d share what you can expect.

xpds14
(click image to go to event website)

What to expect?

Xen Project Developer Summits are packed with highly technical content where the core developers of the Xen Project community come together to discuss the evolution of the Xen Project. The conference is a mixture of talks and interactive sessions in un-conference format (which we call BoFs). Newcomers and those who are interested in the progress and future of the Xen Project, it’s sub projects (Hypervisor on ARM and x86, Upstreams and Downstreams, Embedded and Automotive variants, Cloud Operating Systems such as Mirage OS) usually will get tremendous value from attending the event.

Besides roadmap, feature updates and developer topics, this year features a few themes:

  • Network Function Virtualization
  • Security
  • Performance and Scalability
  • Cloud Operating Systems
  • Topics that are important for automotive/embedded/mobile use-cases, such as Real-time virtualization, certification and ARM support

Why not check out the agenda or watch last year’s sessions to get a sense of what is coming. Note that BoF’s and discussion groups will be published next week.

How to get the most out of the Summit?

Our developer events are designed to help you make connections and to participate. A good way to network are our evening social event and to network during the breaks. Another great way to get the most out of the summit is to submit a BoF/discussion groups about a topic you care about or to participate in a BoF/discussion group. BoF submissions are open until August 11 and the BoF schedule will be published the week before the event. Most of our talks will have an extensive and interactive Q&A portion, which is another way to engage.