Tag Archives: testing

Xen Project Test Day for 4.6 RC4 Scheduled for October 1

Our Fourth (and Possibly Final) 4.6 Release Candidate to be Tested This Thursday

TestDayOur Xen Project Test Days help insure that upcoming releases are ready for production, beyond what our automated testing through our Test Lab can accomplish. It is particularly important that our users test out the upcoming release in their own environment. We rely on your functional testing of features, stress-testing, edge case testing, and performance testing to prove that the code is ready for consumption. And this is your opportunity to verify that the new code will continue to work well in your particular situation.

Xen Project 4.6 Release Candidate 4 Testing

Continuing our current release cycle, the Test Day for Xen Project 4.6 RC4 has been set for Thursday, October 1, 2015.

This may be the final RC before release, so the time to test the software is now!

Test Day Information

Additrional information about Test Days can be found here:

Join us on Tuesday in #xentest on Freenode IRC!
Test a Release Candidate! Help others, get help! And have fun!

Our Next Test Day is September 15: Xen Project 4.6 RC3

The Third 4.6 Release Candidate to be Tested on Tuesday

TestDayOur Xen Project Test Days help insure that upcoming releases are ready for production, beyond what our automated testing through our Test Lab can accomplish. It is particularly important that our users test out the upcoming release in their own environment. We rely on your functional testing of features, stress-testing, edge case testing, and performance testing to prove that the code is ready for consumption. And this is your opportunity to verify that the new code will continue to work well in your particular situation.

Xen Project 4.6 Release Candidate 3 Testing

Continuing our current release cycle, the Test Day for Xen Project 4.6 RC3 has been set for Tuesday, September 15, 2015.

Additional Test Days are expected to be scheduled roughly ever other week until Xen Project 4.6 is ready for release.

Test Day Information

Additrional information about Test Days can be found here:

Join us on Tuesday in #xentest on Freenode IRC!
Test a Release Candidate! Help others, get help! And have fun!
If you can’t make Tuesday, remember that Test and Issue Reports are welcome any time.

Introducing Xen Project’s New Test Lab

One of Xen Project’s highest priorities is to continually work to improve the quality of our code and code coverage. We’re serious and proactive when it comes to minimising and, whenever possible, eliminating any adverse effects from defects, security vulnerabilities or performance problems. Our users run some of the largest cloud and datacenter operations in the world, so we know that reboots and service interruptions are more than mere glitches in operations.

That’s why the Xen Project Advisory Board has made a significant investment to set up a Xen Project-owned Test Lab based on OSSTEST. We’re excited to announce that the lab is now live and in production.

Our new test lab will allows us to improve upstream quality of the Xen Project Hypervisor, while developers from different organizations now have access to a system that can be used to investigate test failures, schedule test jobs, etc. The test lab is also being used for regression testing of the Xen Project Hypervisor against different upstream and downstream environments. The Xen Project also tests against upstream and downstream open source projects such as the Linux Kernel and OpenStack to ensure that the Xen Project Hypervisor works seamlessly with important key technologies.

To help kick-start the lab, we created the Test Infrastructure Working Group, which helped source a COLO, spec out the hardware requirements for the test system and helped procure the machines needed to run OSSTEST. A special thank you goes to Ian Jackson, who has been instrumental in getting this effort off the ground over the past six months. We also appreciate contributions from our committee comprised of employees from AMD, Amazon Web Services, Citrix, Intel and Oracle, who have worked closely with our maintainers, project leads and wider community to create the new testing facility. As a result, we’ve decommissioned the old system run by Citrix as a service to the Xen Project community.

In this blog, we’ll describe our platform in more detail, identify what’ we’re doing today with OSSTEST and new goals from our Test Infrastructure Working Group.

Xen Project’s New Test Lab

This picture shows a section of our test lab, which is co-located at Earthlink

This picture shows a section of our test lab, which is co-located at the Earthlink Boston Data Centre

Currently the Xen Project owns a rack with a mixture of 24 x86 Intel and AMD hosts, and 8 ARM hosts (a custom-made set-up that includes 4 Cubietruck and 4 Arndale boards). Although we are currently only at 50 percent of planned capacity, the new test system has a much more diverse set of test machines and already offers more capacity than the old one did. It is also significantly more reliable than the old system.

Setting up a test lab with several different architectures and suppliers as well as a globally dispersed community is simply a challenge. For example, we discovered a number of bugs in the Linux kernel, FreeBSD and Hypervisor, which are currently being addressed. We also tripped over a number of unexpected issues, such as hardware issues and BIOS bugs in some of the machines in the COLO.

How Do We Use OSSTEST Today?

As mentioned earlier, the test lab is being used for regression testing. We’re testing different versions of Xen Project Hypervisor against different versions of the Linux Kernel, Linux distros, FreeBSD, Windows, QEMU, Rump Kernels and other open source projects we interface with. We also do test specific features such as Xen Security Modules, Credit 2 scheduler and other Xen Project features. And more recently we added functionality for performance testing of the Xen Project Hypervisor.

The set-up is essentially following a continuous integration paradigm; changes to Xen Project trees are pushed onto a staging branch. When tests succeed, changes are automatically pushed to the master branch. If there are failures, a bisection algorithm is run, which in many cases will identify the changeset that is causing the issue and developers are then notified that a specific changeset has created a problem.

This figure shows how the interaction between the test lab, the staging branch and the master branch.

This figure shows how the contribution workflow and its interaction between the test lab, the staging branch and the master branch.

What’s Next for Xen Project’s Test Lab

Bring the test lab to 100% capacity. As we work through the remaining issues, new machines will be taken into production as issues are resolved. The increased capacity will enable us to add on-demand testing capability, which will enable community members to test on hardware that they may not have themselves.

Broaden Access: The new system is in a community-operated COLO, rather than one hosted inside the Citrix private network. This, and the increased capacity, will mean that we can to grant access to the facility to Xen Project community members regardless of their affiliation, and provide an on-demand testing capability. To do this, we will need to agree upon processes to enable community members to access the test lab, in a similar way as we allow access to Coverity Scan data.

More Test Coverage: This increased capacity will allow us to further increase the breadth and depth of our testing. We have a number of enhancements in the pipeline, including for example testing of FreeBSD hosts, nested virtualisation, and compatibility testing with a variety of OS distributions.

Expand the infrastructure and add another rack. The Xen Project Advisory Board agreed to fund another rack and to procure further test machines in 2015. We expect that some 64-bit ARM machines will be part of the next tranche of machines that the Xen Project will buy.

Xen Project OpenStack CI loop: In addition to the Xen Project Test Lab, the Advisory Board is also funding a 3rd party continuous integration loop to improve Xen Project virtualization and OpenStack integration. A prototype of the CI loop is already running at jenkins.openstack.xenproject.org. Xen Project community members are currently fixing a number of issues in OpenStack Nova, Libvirt and Xen Project, such that the CI loop can become voting on OpenStack Nova commits.

We also continue to run the Coverity Scan static analysis service, which helps us quickly find, review and resolve Xen Project Hypervisor code defects and vulnerabilities. We’ve done this since 2013 and continue to regularly review and fix reported bugs.

If you’re interested in testing Xen, we encourage developers to join the xen-devel mailing list and ask how you can help. We’re always looking for help extending the set of test cases to tune performance for as wide a set of configurations and features as possible.

October 29: Xen Project 4.5 RC1 Test Day

TestDay

The Xen Project team is pleased to announce the first Test Day for 4.5 Release Candidate 1 will be held on October 29, 2014.  The 4.5 release is just a few weeks away, so this is an important event in our development calendar.

Test Days insure that the upcoming release is ready for production.  It also allows power users to test out the upcoming release in their own environment.

Subsequent Test Days are expected to be scheduled roughly ever other week until it is determined that the software is ready for release.

General Information about Test Days can be found here:
http://wiki.xenproject.org/wiki/Xen_Project_Test_Days

and specific instructions for this Test Day are located here:
http://wiki.xenproject.org/wiki/Xen_4.5_RC1_test_instructions

XEN 4.5 FEATURE DEVELOPERS:

If you have a new feature which is cooked and ready for testing in RC1, we need to know about it and how to test it. Either edit the instructions page or send me a few lines describing the feature and how it should be tested.

Currently, this Test Day is focused on general tests (e.g., “Does the software compile, install, and do the things it normally does, regardless of hardware platform?”).  If you have specific new functionality which needs testing in RC1, we need to know about it and how to test it.

EVERYONE:

Please join us on Wednesday, October 29, and help make sure the next release of Xen Project software is the best one yet!  To make room for the Test Day, we’ve moved Document Day back to November 5, so join us then as we improve documentation with a special theme of integration this month.

Mark Your Calendars! Great Xen Project Events Coming in August and September 2014

If you use — or are just interested in learning about — the Xen Project Hypervisor, you will want to mark your calendar now for two great events coming later this summer.

September 15: Xen Project User Summit, New York City, NY

The Xen Project User Summit is great for:

  • Users who employ Xen Project software in their datacenter or cloud
  • Integrators who leverage the Xen Project Hypervisor in their solution or service
  • Architects who need to know how to best utilize the software in their solution architecture
  • Managers who want to understand what current and future features will enable their goals

At just US$79 for one day of sessions, the User Summit is an unmatched training opportunity for users of Xen Project software. Located at the Lighthouse Executive Conference Center in the heart of New York City, this event is a fantastic way to increase your knowledge of all things relating to Xen Project.  The lineup of talks will be unlike any other event this year, so it makes sense to take a long weekend in Manhattan in September.

Plus, this will be the best event this year for Xen Project users to meet and greet one another.  Excellent things happen when the talents of the community are allowed to cross-pollinate at User Summit!

Expected topics will include:

  • What’s Coming in Xen Project 4.5
  • Understanding and Using Xen4CentOS
  • Xen Project on OpenStack in the SUSE Cloud
  • High Availability with Xen Project
  • Using Cloud Operating Systems
  • plus many more!

The full schedule will be announced shortly.

To register for User Summit, click here.

August 18-19: Xen Project Developer Summit, Chicago, IL

For those who delve into the innermost parts of Xen Project, there is the Xen Project Developer Summit in Chicago, IL USA in August.  Co-located with LinuxCon North America and CloudOpen North America, this two-day event is the place to be for those who write code for or around Xen Project.

Some of the topics at this year’s Developer Summit include:

  • Xen Project 4.5 Roadmap
  • libvirt support for libxenlight
  • Scaling Xen’s Aggregate Storage Performance
  • Xen on ARM: Status and Performance
  • An Overview of the Verizon Cloud Architecture
  • osstest, Xen’s Automatic Testing Facility
  • and much, much more!

Click here for the full schedule.

At US$99 for two days of highly technical talks, this is an extradinary value for Xen Project software engineers.

Click here to register for Developer Summit.

And Sponsorship Opportunities Are Still Available!

If your organization has a project, product, or service which would be of interest to either a group of Xen Project users or developers, there are still sponsorship opportunies available.  For as little as US$1000, you can get your name in front of a group of Xen Project users, integrators, service providers, and developers.

Send email to russell(dot)pavlicek(at)XenProject.org for more information.

And we’ll be looking for you at one of the Xen Project Summits this year!

Where Would You Like to See the Next Xen Project User Summit Held?

In 2013, we held the first major Xen event aimed specifically at users: the Xen Project User Summit. In 2014, we want to do it again — but where and when?

The Xen Project wants to hold its second Xen Project User Summit.  We’d like to hold it somewhere which is accessible by a large percentage of our user community.  And we’d like to schedule it at a time which makes sense, possibly in coordination with some existing conference.

We need your help to pick the time and place.  Give us your preferences in a very quick 2 minute survey found here:

https://www.surveymonkey.com/s/YJQCHJ6

It’s very quick and easy to do.  And you may just find that the next User Summit is too convenient for you to pass up.