Virtualization and Grid Computing

If you’re feeling adventurous, there’s a workspace service pre-release out (click on the pic):

From http://blog.xen.org/index.php/2008/04/28/xen-api-community-project/:

Several community members have contacted me recently about the Xen-API utilities. I looked into this and discovered a great opportunity for community members looking for a project to contribute to. So, I am announcing a new community effort to complete the development of the Xen-API utilities. If you are interested in working on the Xen-API project please email me at stephen.spector@xen.org and I will call a meeting in mid-May with all people interested to get the project underway.

I’m pleased to announce the availability of Xen versions 3.1.4 and 3.2.1 — bugfix releases in the Xen 3.1 and 3.2 series.

The source repositories are available using mercurial from:

• http://xenbits.xensource.com/xen-3.1-testing.hg (tag ‘RELEASE-3.1.4′)
• http://xenbits.xensource.com/xen-3.2-testing.hg (tag ‘RELEASE-3.2.1′)

Tarballs are available for download (and will soon be linked from the xen.org web pages):

• http://bits.xensource.com/oss-xen/release/3.1.4/xen-3.1.4.tar.gz
• http://bits.xensource.com/oss-xen/release/3.2.1/xen-3.2.1.tar.gz

Thanks to everyone who has contributed!

Regards,
Keir

Virtual Workspaces Tutorial at Open Source Grid Cluster (May 12-16, 2008)

There will be a Virtual Workspaces tutorial at the Open Source Grid Cluster conference in Oakland, CA. The conference is May 12-16, 2008. The Virtualization and Cloud Computing with Globus session is on Wednesday, May 14th, from 4:30-6:00 pm. We hope to see you there!

Quoting from the summary:

One of the primary obstacles users face in grid computing is that Grids provide access to many diverse resources, their applications often require a very specific, customized environment. This disconnect can lead to resource underutilization, user frustration, and much wasted effort spent on bridging the gap between applications and resources. Virtual Workspaces describe the environment required for the execution of an application that can be dynamically deployed across a variety of resources creating a working and consistent platform for grid applications.

This tutorial will introduce the Globus Toolkit workspace service that implements workspaces as Xen virtual machines and enables authorized grid clients to dynamically deploy them and manage their resources. Further, we will describe and demonstrate the workspace “cloudkit” that provides a user-friendly interface on top of the workspace service allowing authorized users to easily provision and run VMs on the available community clouds. Finally, we will describe how the process of contextualization can be used to provide on-demand functioning clusters and give examples of its use by applications.

There’s been a lot of talk about the dangers of getting locked in to cloud platforms, developing an application that is only suited to one platform.

Here’s a, let’s say… “embellished” example: Gangsta cloud wars could pivot on the traffic-driving power of Google and Microsoft/Yahoo.

When you’re using VMs like Xen (e.g. on EC2), if you design things for it you “should be able to” move without a ton of hassle (research. plan.). The workspace project has been working on portability and usability (see The first one-click STAR production cluster) and one of the things we can do now is use the same VM image on a regular cluster (such as on the Teraport cloud) and EC2. The contextualization software can be configured to sense if it is on EC2 or not (and will bootstrap accordingly). It “would be nice” if such things were standardized but this is not a real problem right now (IMHO).

About something more “strongly typed” like Google’s AppEngine. Application migration might be a bit harder, but not if the APIs are well known and repeatable. Google’s SDK is even Apache 2 licensed.

To that point, have a look at Announcing AppDrop.com (host Google App Engine projects on EC2). It’s not there yet (database is a flat file) but, hey, it was developed in a few days. Cool. Read more at http://appdrop.com.

The long term idea is not that this would solve all your problems magically but that such things are possible, and if there’s a real market for choices, it seems like more work on things of this nature are also inevitable.

I’m no datacenter business expert, but the biggest problem right now seems to be that few people will be able to compete on costs/efficiencies of scale with Google/Amazon/Microsoft/eBay. (<predictions…>) It feels like it would naturally approach the straight web hosting business, though. Let’s say a standard, open source cloud computing infrastructure emerges (such as Apache httpd in the analogy). There will be various levels of players as far as the capital they have and certainly better and worse companies to choose from (including those that differentiate on service etc). But if you’re really sweating the savings an enormous company could provide with such efficiencies vs. a normal size company/datacenter, you’re probably at the point where you could save a whole lot more by buying your own computers.(</predictions…>)

Miscellaneous point about lock-in: something user-facing that ties you to a provider does not seem like a wise idea (e.g. Google’s Users API).

Fifth (and hopefully final!) release candidates are available from the public xen-3.1-testing.hg and xen-3.2-testing.hg trees.

Please give these ones a spin. I hope to roll the releases early next week.

— Keir

[update: http://www.gridvm.org/xen-314-and-321.html]

This month’s OGF newsletter has an article about the Cloud Systems BoF.

If you scroll down to the bottom of the latter link, there are slides and PDFs to view.

The mailing list URL in the newsletter is currently broken, this is the right one: http://www.ogf.org/mailman/listinfo/clouds-bof

New KVM release today: kvm-65.

Anthony Ligouri notes some interesting things @ KVM for the Mainframe.