Monthly Archives: May 2011

Daniel Kiper: Recovery of crashed Linux

This is a guest blog post by Daniel Kiper, one of our Google Summer of Code students. Daniel’s GSoC project is called Recovery of crashed Linux. Please welcome Daniel into the community.

My name is Daniel Kiper. I was born and live in Poland. I am a PhD student at Warsaw University of Technology, Faculty of Electronics and Information Technology. My research and thesis are focused on Air Navigation Services (ANS) delivery with the emphasis on Air Traffic Management (ATM) during contingencies. However, my core education and interests have been connected with IT and electronics since I was at high engineering school. What most interests me are operating systems (especially *NIX of any kind; over my professional career I have used Linux, AIX, Solaris and SCO Unix) and all types of virtualization. The best solution for me is to work on issues with a thin line between hardware and software.

Sometimes people are surprised that my PhD thesis is focused on Air Traffic Management. They think this topic is far away from computer science. But this is not true. Today, ATM and ANS, in a wider sense, strongly depend on a bleeding edge technology, especially on electronics and IT.

So far I have seen a few operational ATM centres and I have always been amazed how the technology can be used up to its limits to provide critical services. And there is no place for any error. There is no excuse !!! Everything must be perfect.

Additionally, many ideas, methods and algorithms used in IT are very similar to the ones used in the ATM, for example, a traffic simulation base on network flows and shortest path problems which are very often used also in computer

What about my hobbies? I am passionate about my studies. Besides my studies, I am interested in science, technology and engineering issues of any kind (as I mentioned earlier especially IT and electronics). Additionally, I am a railway fan.
I am member of the Polish Association of Railway Enthusiasts. Last but not least, I am interested in Poland and the world’s history of the 20th century.

My GSoC 2011 Project

It is the second opportunity for me to take part in the Google Summer of Code. In 2010 my proposal, Migration from memory ballooning to memory hotplug in Xen, was chosen as the GSoC 2010 project. Jeremy Fitzhardinge
from Citrix was the mentor in this project. Additionally, I was strongly supported by Konrad Rzeszutek Wilk. Currently, the project is very mature and it will be included in the mainline Linux Kernel shortly. The participation in GSoC 2010 allowed me to broaden my knowledge in Xen hypervisor and Linux Kernel. Moreover, I was able to learn something about open source project maintenance and management. I think this knowledge can enable me to participate in this years GSoC 2011 more fruitfully and it can bring in better results for the Xen and Linux Kernel community.

My GSoC 2011 project is Recovery of crashed Linux and is mentored by Konrad Rzeszutek Wilk. The work plan is outlined, however, it can change a bit (maybe more than a bit) while carrying out the project, since some unexpected things may come out. It looks as follows:

  1. see how the HYPERVISOR_kexec_op hypercall works with PV kernel,
    baremetal kernel and Xen hypervisor
  2. integrate HYPERVISOR_kexec_op hypercall into mainline Linux Kernel; probably it is possible to use some code from Xen Linux Kernel Ver. 2.6.18
  3. create proper “transition” page table initialization
  4. ACPI/APIC/DMA/PCI/… initialization during reload of PV Linux Kernel as dom0; Xen hypervisor is not reloaded
  5. ACPI/APIC/DMA/PCI/… initialization during reload of PV Linux Kernel as dom0 and Xen hypervisor
  6. block device (PV) initialization (optional)
  7. network device (PV) initialization (optional)
  8. working kexec/kdump implementation on HVM domain with PV drivers (optional).

I am going to maintain kexec/kdump support for Xen after finishing the GSoC 2011. Also, I am looking forward to any comments and questions.

Picture Credits: ipa, PANSA, Marcin Wilkowski

Introducing the VirtIO on Xen project by Wei Liu

This is a guest blog post by Wei Liu, one of our Google Summer of Code students. Please welcome Wei into the community.

Hi, all. I’m Wei Liu, a graduate student from Wuhan University, Wuhan, Hubei, China. Our university is said to be one of the most beautiful universities in China. I have been doing Xen development for the last two years. My research interests include virtual machine, operating system and their security. When I’m not doing my job, I read science fiction and see movies. My favorite science fiction is “Three bodies”. I’m also a football player and play the harmonica.

It’s my honor to be accepted to GSoC 2011 and work with Xen community. My project is VirtIO on Xen. Let me talk a little bit about my project.

As you all know, VirtIO is a generic library for paravirtualization mainly used in KVM. But it should not be too hard to port VirtIO to Xen. When done, Xen will have access to the Linux kernel’s VirtIO interfaces and developers will have an alternative way to deliver PV drivers besides from the original ring buffer flavor.

This project requires to:

  • Modification of upstream QEMU
  • Replacing the KVM-specific interface with a generic QEMU interface
  • Modification Xen / Xentools to support VirtIO
  • Modifications to the Linux kernel VirtIO interfaces.

The project will take two usage scenarios into consideration: PV-on-HVM and Normal PV. These two scenarios require working on different set of functions:

  • XenBus vs VirtualPCI, it’s about how to create a channel;
  • PV vs HVM, it’s about how events are handled.

In the PV on HVM case, the Virtual PCI bus will be used to establish a channel between Dom0 and DomU. In some sense, it makes no differences on the Linux kernel side.

In the normal PV case, QEMU needs to use event channel to get / send notifications, and foreign mapping functions in libxc / libxl to map memory pages. XenBus / Xenstore will be used to establish a channel between Dom0 and DomU. The Linux VirtIO driver should use Xen’s event channel as kick / notify function.

When the porting is finished, I will carry on some performance tests with standardized tools such as ioperf, netperf and kernbench. A short report will be written based on the results.

This is a brief introduction to the project. Any comments are welcomed.

You can always reach me via “liuw AT liuw DOT name”. Also check out my blog (Chinesetranslated).

XenSummit Program Management Committee

XenSummit is approaching: the Call For Participation will be open until May 31, 2011 20:00 UTC+1. More information about XenSummit can be found here.

Program Management Committee:

I wanted to also take the opportunity to introduce this year’s Program Management Committee.

  • Boris Quiroz, Senior SysAdmin, Xen fan and evangelist since 2006. Currently trying to put Xen and Cloud Computing to the serve of online banking in Chile. His interests includes: Linux and Xen sysadmin, cloud computing and playing Xbox360.
  • Dan Magenheimer has been hacking on Xen since 2004 and completed the first port of Xen to another architecture (Itanium).  His current interest is physical memory utilization, both in Xen and in Linux, e.g. Transcendent Memory (aka “tmem”).
  • Eddie Dong has been working on Xen Hardware Virtual Machine and performance tuning since 2004. His latest focus is Xen SR-IOV and nested virtualization support.
  • Jose Renato Santos is a principal research scientist at HP Labs, where he is currently a member of the Intelligent Infrastructure Lab. He was a key contributor to the development of XenOprofile, which enables system wide performance profiling in Xen using hardware performance counters and contributed to performance analysis and improvements of Xen network para-virtualized drivers.
  • Pasi Kärkkäinen has been using Linux since 1994 and Xen since 2003. Pasi has been contributing documentation (wiki additions/fixes) and tutorials for, and also testing efforts of new hypervisor releases and pvops Linux kernels. You can find Pasi active on various Xen related mailinglists.
  • Stephen Spector is currently the OpenStack community manager and previously held the community manager position.

There will also be a representative from each sponsor: Amazon Web Services and Citrix.

Xen Hackathon, Munich 13-15 September

Since the first Xen Hackathon in Cambridge this year, I have been asked a few times whether we will be organizing another Hackathon this year. The short answer is yes!

Fujitsu will host the next Xen Hackathon in Munich. The Hackathon will take place September 13-15, 2011 (the week before the Oktoberfest). If you want to attend, save the date. I wanted to thank Fujitsu and in particular Jürgen Groß for making the Munich Hackathon happen.

The aim of the Hackathon is to give developers the opportunity to meet face to face to discuss development, coordinate, write code and collaborate with other developers as well as allowing everyone to put names with faces. There is no registration fee. However as an attendee you will need to cover your own travel, accommodation and other costs such as evening meals etc. More details will follow and will be communicated in due course on the blog, mailing lists and via the wiki page.  Some more information is already available on the xen-devel mailing list.

Fujitsu is playing an active role in development of Xen by maintaining the IA64 architecture, CPU pools and pv USB drivers. Fujitsu is using Xen as the base for the SQ mainframes supporting Windows and Linux guests together with Fujitsu’s mainframe OS BS2000, which was ported to run as a pvHVM guest on x86 hardware with Xen.

If you do want to attend, drop Jürgen or me a line.

Budapest and the Ubuntu Developer Summit

Last week I was at the Ubuntu Developer Summit in Budapest.

The best news is: the next release of Ubuntu, oneiric ocelot, will have dom0-capable kernels in main and Xen (4.1, very likely) hypervisor/tools in universe. The plan is to have the hypervisor and tools in main in the next LTS. This is a great step forward.

I’d just like to say a big thank you to all those of you who’ve worked so hard (and had your patience so sorely tested!) getting Xen dom0 support into Linux upstream. Now we are really starting to reap the rewards.

“Cloud” topics were quite prominent in the agenda generally. Both
OpenStack and Eucalyptus were well represented, and there’s exciting new stuff in the field of “cloud orchestration” (eg Ensemble). We need to do some background work to make a readily buildable set of Xapi packages for OpenStack at least, as OpenStack’s Xen integration is via Xapi.

Of course as a longstanding Ubuntu and Debian developer, I attended various other sessions where I was able to help out, and I met a lot of old and new friends. Also outside of the formal programme, there was a lot of interest in Xen.

Perhaps the highlight of the conference’s social programme for me was the Invisible Exhibition fringe event: at this exhibition, which is entirely in the dark, we wandered through a series of scenes which we explored by touch. One aim of the exhibition is to help understand the experience of blind and visually impaired people.

On the personal side: I had nearly two weekends in Budapest, which is an excellent and exciting city; I recommend it. I was particularly impressed by the Children’s Railway and Memento Park.

Daniel Castro’s GSoC project: Add Xen PV block device support to SeaBIOS

This is a guest blog post by Daniel Castro, one of our Google Summer of Code students. We will aim to publish a GSoC student introduction every Monday. Later in the year, we will publish project updates.

My Name is Daniel Castro, from Bogota Colombia. I have been accepted into Pohang University of Science and Technology for a PhD in Computer Science, as part of the Korean Government Scholarship Program for International Students. My research interests are virtualization and networks. I love cats and motorbikes, and also enjoy reading CS textbooks and Sci-fi.

This is my first time in Google Summer of Code and my first time contributing to an open source project. I hope I will keep contributing to Xen Hypervisor for many years to come. I worked with Citrix XenServer for several years, but never worked with the open source version, and I have big expectations for Xen.

The GSoC Project

Currently Xen is using a BIOS called ROMBIOS for HVM Virtual machines. Xen is going to introduce SeaBIOS and work on upstreaming necessary changes to Qemu. SeaBIOS runs natively in Qemu, has Qemu upstream support and is also the BIOS of choice for  KVM.

The project requires modifications to the SeaBIOS hard drive abstraction layer to allow usage of  the Xen PV Drivers instead of the emulated device used in qemu-dm. Once SeaBIOS abstraction layer is in place I will work on the following tasks:

  • Provide PV frontend drivers for SeaBIOS in order to discourage the use of the emulated device.
  • Remove the emulated device to remove the subtle confusion that can arise from booting from one device (emulated) but (apparently) running from another (PV frontend).
  • Around July present the community with patches that enable to test the new functionality.
  • Test for each kind of VM (a compatibility matrix will be presented as part of the documentation).
  • All activities and reports will be published on the SeaBIOS wiki

I am also very interested in HVM Guests of Xen and all the knowledge I gather will be posted on the hvmloader wiki page.