It would be interesting to have a lot of input on this survey if you have some time, it’s only a few questions.
From: Stephen Childs
Subject: Survey on use of virtualization in production grids
Hi all,
Apologies for the slightly off-topic post. I am giving presentations soon on the use of virtualization in production grids and was thinking it might be nice to have some data to present!
I have set up a survey on the usage of virtualization in production grids at http://www.surveymonkey.com/s.aspx?sm=Sukm0sCdun6EAtpi7IXtwA_3d_3d
The survey will remain open until Friday August 29th. It shouldn’t take more than a few minutes to complete, and I think the results should be quite interesting for our community. I get the impression virtualization usage has rocketed in the last year, but haven’t seen any figures on uptake to date.
Please feel free to forward this to anyone else who may be interested in responding.
If you have any comments, please feel free to email me.
Stephen
–
Dr. Stephen Childs,
Research Fellow, EGEE Project, phone: +353-1-8961797
Computer Architecture Group, email: Stephen.Childs @ cs.tcd.ie
Trinity College Dublin, Ireland web: http://www.cs.tcd.ie/Stephen.Childs
More about cloud computing…
Here’s a short quote from Billy Marshall (rPath): A Big Switch or a Gradual Shift?
The historical metaphor that Carr effectively uses to demonstrate the likelihood of this pending change is the switch from locally produced electrical power to regionally produced electrical power delivered via a high performing electrical grid infrastructure. In Carr’s metaphor electricity is analogous to applications and the electrical grid is analogous to the Internet. There are clearly some parallels, but I believe the metaphor is flawed because information applications are more analogous to hair dryers, drill presses, and die stamping machines (i.e. applications that consume electricity) as opposed to the electricity itself.
Billy goes on to point out how companies are always going to need specific things from these electricity-consuming objects, hypervisors are more like the power transformers that convert and reliably step down electricity (into standardized, repeatable delivery units), and virtual appliances are more like the hair dryers, drill presses, and die stamping machines:
When applications can reliably plug into a grid to receive “power” in a standardized and repeatable manner, it will be increasingly popular to let someone else deliver the power of the grid while the individual companies focus on the “design of the application” (i.e. the drill press, the chip digester, the ore smelter).
I think it’s a good way to frame things, an expansion I’d offer is that it is not just hypervisors that are this transformation/delivery mechanism, but also all of the other cluster infrastructure needed to make a leasable datacenter. The security, scheduling, efficiency, and enforcement mechanisms/policies that must be in effect. The hypervisor is in all likelihood going to be the most popular core technology, but there’s a lot more to making a safe, solvent, and usefully leasable cluster.
At the edge of the cluster and beyond, there’s also all the technology and lessons of grid computing to draw from. A field where virtualization is a mechanism being incorporated in a larger pre-established context (cf. papers from our group and many others). In the analogy, facets of grid computing perhaps get us into “buying clubs”, “electricity markets”, “consumer protection”, etc. (and how about rolling blackouts).
At home, I wanted a reliable disk solution for backups and also wanted a big, blank and resizable storage system for virtual machines. I knew I wanted to be able to get at the shared disk remotely from other nodes and wanted to be able to replace broken hardware quickly if something failed. I also didn’t want to spend a lot of time reconfiguring OSs and software in the case of a total system failure.
I have two cheap computers and so I put some big disks in them and mirrored the disks over the network. Instead of using one file server node and RAID1, this is something like a “whole system RAID”. If anything at all breaks in either computer, hosted services can keep running and data is unharmed except for whatever was unsynced in RAM.
To accomplish the disk mirror I used DRBD. DRBD is a special block device that is designed for highly available clusters, it mirrors activity directly at the block device level across the network to another disk. So like a RAID1 configuration over the network. It lets you build something like the shared storage devices on a SAN, but without any special hardware. This provides the basic reliability layer.
Linux Logical Volume Management (LVM) is a popular tool that lets you flexibly manage disk space. Instead of just partitioning the disk, using LVM lets us do on-the-fly logical partition resizing, snapshots (including hosting snapshot+diffs), and adding more physical disks into the volume group as needs grow (you can even resize a logical partition across multiple underlying disks). Each logical partition is formatted with a filesystem of its own. Using LVM avoided some future headaches I think.
That is how the disk is setup, now how to access it remotely? You could run a shared filesystem of course, exporting via an NFS server on host A (or B). Instead, having heard good things about Global Network Block Device (GNBD) on the Xen mailing lists, I chose to export the logical block devices (from LVM) directly over the network with GNBD. Another node makes a GNBD import and the block device appears to be a local block device there, ready to mount into the file hierarchy. This is like iSCSI but it is a snap to set up and use.
And if that other node is a Xen domain 0, that block device is very handily ready to be used as a VM image, just as if it was a raw partition on that node.
Here’s an example Xen configuration using the imported block device:
disk = [ ‘phy:/dev/gnbd/vmimage001,sda1,w’]
The guest VM needs no awareness of all these tools, it just sees its sda1 and mounts it like anything else Xen presents to it as one of its “hardware” partitions.
Instead of just using the file store for backups and VMs that are used intermittently, I’m also running persistent services like websites, the incremental-backup server and a media server in VMs stored there.
First, this allows for basic backups of the LAN services without any backup software, that’s nice to have, although I really prefer a combination of incremental backups and RAID1. (Here we also avoid a Russell’s paradox situation with the backup server).
Second, keeping time-consuming-to-configure services in a VM allows me to replace hardware quickly, including whole computers in the event of a total failure: the only software I’ll ever need to reinstall is {Linux, DRBD, LVM, GNBD} for a file server node and {Linux, Xen} for a VMM node.
As long as net latency is really low (here it is sub-ms) it doesn’t really matter that the disk is remote for any of my uses. The VMs always respond very well.
(I should mention: you could of course take GNBD out of the picture and run the VMs on host A and B if Xen were installed there)
Another bonus: using GNBD, you can live-migrate the VM to any node that can do a GNBD import. This is nice to have. I only live migrate manually, though. Both DRBD and GNBD have some features that allow for seamless failover but I don’t really need any of this at home.
To learn more about that, check out this paper on the new DRBD (it is interesting): http://www.drbd.org/fileadmin/drbd/publications/drbd8_wpnr.pdf
Thinking about high availability in this kind of setup for a minute, a possible and simple to execute arrangement for services that need to be up at all times would be to take two DRBD mirrored nodes, run VMs on one or both of them, and have the physical nodes heartbeat each other. This is a simpler approach than a centralized file server with block device export, here we just have two peer VMMs that are “watching out” for each other.
You’d have two master/slave arrangements, so in the normal operating case: one VMM with partition A as DRBD master and partition B as secondary, then on the other VMM you have partition A as secondary and partition B as master. VMs run from a partition that is the DRBD master.
Let’s say you split four services into four VMs and put two VMs on each physical node. One of the physical node’s disks fail entirely and a monitor process notices. The heartbeat script makes sure the OK node is now the DRBD master for both partition A and B. Then it boots the two VMs previously hosted on the failed node on the OK node, re-allocating RAM for the time being to accomodate all four VMs.
The applications in those VMs recover just as if they went through a normal hard system reset (their network addresses can stay the same since both physical nodes are on the same LAN). Once the administrator gets the alert email and puts a new disk in, another script is ready to resync DRBD and then migrate two of the VMs back to their normal place.
This seems like something to consider for a highly hammered and important head node (like a Globus GRAM node for example). All it takes is another node, commodity hardware and open source software!
I noticed this blog entry on ZDNet, Virtualization begins to haunt hardware makers, that guesses what you might expect: the current wave of companies consolidating via virtualization is what’s been eating into hardware sales.
Yet Jonathan Schwartz (CEO of Sun) recently took the position that:
if you can double server utilization via Solaris Containers or VMWare [sic], people don’t buy fewer computers - they buy more. The value of innovation, at least to our core customers, is growing so fast that if the price declines, the overall return (value/price) goes through the roof - encouraging a feedback loop.
Could it be that Schwartz is still right but we’ll just first see a noticeable dip (”correction”) this year as most people merely consolidate instead of buying more hardware?
Open source VMs (and their accompanying management infrastructure) are eventually going to be just another standard part of the datacenter toolbox. It seems that value/price would stabilize and people would get used to it; this “feedback loop” can’t feed off itself forever.
Another major shift looks possible: the datacenter could see big power savings in the next decade. The this 80 core processor that Ian Foster pointed out “uses less energy than a quad-core processor and has teraflop performance capabilities.” The ability to sleep and awake cores as demand changes (and the ability to migrate computations away from cores that are getting too hot) results in huge power savings. Sounds like a mini virtual cluster on a chip.
IBM’s developerWorks has a new article that covers the basics of the many virtualization options for Linux:
Virtual Linux: An overview of virtualization methods, architectures, and implementations
“This article explores the ideas behind virtualization and then discusses some of the many ways to implement virtualization. We also look at some of the other virtualization technologies out there, such as operating system virtualization on Linux.”
http://www-128.ibm.com/developerworks/library/l-linuxvirt/index.html
This is old news, but I wanted to remind you that there is a counter argument to the blue pill “100% undetectable malware” prototype that generated a lot of press this year:
http://www.virtualization.info/2006/08/debunking-blue-pill-myth.html
http://x86vmm.blogspot.com/2006/08/blue-pill-is-quasi-illiterate.html
Also, since malware requires an attack vector in the first place: if you don’t have extreme performance requirements, consider putting all network facing services in VMs (my websites are, save one SSH port on a dedicated IP). This should eliminate the ability for a blue pill/subvirt style attack to take hold in the first place (unless there’s an egregious networking stack issue in the VMM (if the VMM is even involved in networking which is not always the case)).
I hope that network facing VMs for the desktop become commonplace which will happen en masse when Microsoft likes the idea I guess (and makes it transparent to the user). Boot from a saved, clean slate every session; perhaps with versioned, non-executable storage shared between host and guest VM for user data updates.
Here is a ready to go web browsing virtual appliance from VMware and another one from rPath.
