Virtualization and Grid Computing

There’s been a lot of talk about the dangers of getting locked in to cloud platforms, developing an application that is only suited to one platform.

Here’s a, let’s say… “embellished” example: Gangsta cloud wars could pivot on the traffic-driving power of Google and Microsoft/Yahoo.

When you’re using VMs like Xen (e.g. on EC2), if you design things for it you “should be able to” move without a ton of hassle (research. plan.). The workspace project has been working on portability and usability (see The first one-click STAR production cluster) and one of the things we can do now is use the same VM image on a regular cluster (such as on the Teraport cloud) and EC2. The contextualization software can be configured to sense if it is on EC2 or not (and will bootstrap accordingly). It “would be nice” if such things were standardized but this is not a real problem right now (IMHO).

About something more “strongly typed” like Google’s AppEngine. Application migration might be a bit harder, but not if the APIs are well known and repeatable. Google’s SDK is even Apache 2 licensed.

To that point, have a look at Announcing AppDrop.com (host Google App Engine projects on EC2). It’s not there yet (database is a flat file) but, hey, it was developed in a few days. Cool. Read more at http://appdrop.com.

The long term idea is not that this would solve all your problems magically but that such things are possible, and if there’s a real market for choices, it seems like more work on things of this nature are also inevitable.

I’m no datacenter business expert, but the biggest problem right now seems to be that few people will be able to compete on costs/efficiencies of scale with Google/Amazon/Microsoft/eBay. (<predictions…>) It feels like it would naturally approach the straight web hosting business, though. Let’s say a standard, open source cloud computing infrastructure emerges (such as Apache httpd in the analogy). There will be various levels of players as far as the capital they have and certainly better and worse companies to choose from (including those that differentiate on service etc). But if you’re really sweating the savings an enormous company could provide with such efficiencies vs. a normal size company/datacenter, you’re probably at the point where you could save a whole lot more by buying your own computers.(</predictions…>)

Miscellaneous point about lock-in: something user-facing that ties you to a provider does not seem like a wise idea (e.g. Google’s Users API).

They’ve taken the application level approach (Python currently).

And unlike Sun’s attempt (which also needed porting of app to a platform instead of the looser requirements of EC2 style), there is an interesting entry incentive:

“It’s free to get started. Every Google App Engine application can use up to 500MB of persistent storage and enough bandwidth and CPU for 5 million monthly page views.”

http://code.google.com/appengine/

http://code.google.com/appengine/docs/whatisgoogleappengine.html

http://googleappengine.blogspot.com/2008/04/introducing-google-app-engine-our-new.html

http://appgallery.appspot.com/

http://groups.google.com/group/google-appengine

Here is a good set of pointers about Google’s recent Seattle Conference on Scalability:

http://glinden.blogspot.com/2007/06/more-on-google-scalability-conference.html

The Google patent search interface was launched recently. I would like to find time to search through more comprehensively, it has a nice, helpful interface. Searching for grid computing yields some interesting results.