Entries in 'appliance'
— Next Page »
A lot of developments with the workspace service and science clouds recently!
The cluster technology lets you bootstrap generic images into new network and security contexts on the fly. We built a sample cluster on top of the technology that lets you create the cluster and be immediately ready to submit jobs to a Torque cluster fronted by GRAM and GridFTP that use a newly created self-signed certificate:
-
cloud-client.sh –run –hours 12 –cluster base-cluster.xml
- Wait a few minutes, once launched note the head-node hostname
-
scp -r root@HOSTNAME:certs/* lib/certs/
(SSH was bootstrapped end to end already)
- Make sure your grid tools trust this certificate and then submit work
This can be done with nearly anything that can run on a non-virtual cluster. Check out these links for more information:
From http://www.gridtoday.com/grid/2370981.html
rPath Enables Cloud Computing for DoE, CERN
RALEIGH, N.C., June 4 — rPath, whose unique technology simplifies application distribution and management through virtual appliances, today announced that the U.S. Department of Energy (DOE) and the European Organization for Nuclear Research (CERN) have been using rBuilder to deliver virtual appliances to both scientists’ desktops and computational clouds. The use of rBuilder in these environments reduces the effort required to support users and allows researchers to take advantage of underutilized computational resources.
rBuilder is the first and only product that simplifies and automates the creation of virtual appliances. A virtual appliance is an application with a streamlined operating system, offered in a format that runs in virtualized environments.
CERN turned to virtual appliances to facilitate the analysis of data created by the Large Hadron Collider (LHC) experiments. The complete software environment needed by the LHC applications is assembled by rBuilder and distributed to run as a virtual machine on physicists’ desktops. Virtual appliances provide a consistent application environment for the LHC applications while, at the same time, allowing scientists to use their desktops for analysis, regardless of operating system.
“The coupling between the LHC applications and the operating system is very strong,” stated Predrag Buncic, virtualization R&D project leader. “By distributing these applications as virtual appliances, we are able to isolate the application from the underlying desktop or laptop operating system, allowing the researchers to run the applications on systems that normally would not be supported.”
The DOE is exploring the concept of using virtual appliances to provide customized environments for scientific applications. Scientific applications are turned into virtual appliances using rPath’s rBuilder. The “Science Clouds” project (http://workspace.globus.org/clouds) provides resources capable of hosting multiple scientific appliances using the Globus Virtual Workspaces software. Scientists submit their virtual appliances to any available resource, knowing that the application environment is controlled and isolated from the underlying system. By relying on portable appliances, the scientists can leverage the resources of science clouds, and seamlessly move to commercial providers, such as Amazon’s EC2, when additional resources are needed.
“For a proof-of-concept, anybody can just configure a virtual machine image by hand,” said Kate Keahey, a scientist at Argonne National Laboratory. “But providing appliance management and maintenance that will scale to many thousands of appliances and that will be truly interoperable between different resource providers requires a new approach.”
About rPath
For application providers that want to accelerate license growth, expand into new markets, and reduce support and development costs, rPath’s platform transforms applications into virtual appliances. A virtual appliance is an application combined with just enough operating system (JeOS) for it to run optimally in any virtualized environment. Virtual appliances eliminate the hassles of installing, configuring and maintaining complex application environments. Only rPath’s technology simplifies application distribution, lowers the customer service costs of maintenance and management, and produces multiple virtual machine formats. The company is headquartered in Raleigh, N.C. For more information, visit www.rpath.com.
—–
Source: rPath
More about cloud computing…
Here’s a short quote from Billy Marshall (rPath): A Big Switch or a Gradual Shift?
The historical metaphor that Carr effectively uses to demonstrate the likelihood of this pending change is the switch from locally produced electrical power to regionally produced electrical power delivered via a high performing electrical grid infrastructure. In Carr’s metaphor electricity is analogous to applications and the electrical grid is analogous to the Internet. There are clearly some parallels, but I believe the metaphor is flawed because information applications are more analogous to hair dryers, drill presses, and die stamping machines (i.e. applications that consume electricity) as opposed to the electricity itself.
Billy goes on to point out how companies are always going to need specific things from these electricity-consuming objects, hypervisors are more like the power transformers that convert and reliably step down electricity (into standardized, repeatable delivery units), and virtual appliances are more like the hair dryers, drill presses, and die stamping machines:
When applications can reliably plug into a grid to receive “power” in a standardized and repeatable manner, it will be increasingly popular to let someone else deliver the power of the grid while the individual companies focus on the “design of the application” (i.e. the drill press, the chip digester, the ore smelter).
I think it’s a good way to frame things, an expansion I’d offer is that it is not just hypervisors that are this transformation/delivery mechanism, but also all of the other cluster infrastructure needed to make a leasable datacenter. The security, scheduling, efficiency, and enforcement mechanisms/policies that must be in effect. The hypervisor is in all likelihood going to be the most popular core technology, but there’s a lot more to making a safe, solvent, and usefully leasable cluster.
At the edge of the cluster and beyond, there’s also all the technology and lessons of grid computing to draw from. A field where virtualization is a mechanism being incorporated in a larger pre-established context (cf. papers from our group and many others). In the analogy, facets of grid computing perhaps get us into “buying clubs”, “electricity markets”, “consumer protection”, etc. (and how about rolling blackouts).
If you’re on the workspace-announce list, you will have already seen the “Science Cloud Available at the University of Chicago” email.
Built with the workspace service, we’ve made some nice client enhancements to get to “cloud simplicity” and it’s up and running on 16 nodes and already serving guests. See the the documentation for command samples, the idea is to make it as simple as possible. On the service side, Nimbus uses TP1.3.1 with some very small additions (mostly this differs because of a new authorization plugin). Building cloud computing solutions is the main business of the workspace service.
Have a look!
[UPDATE: using TP1.3.3.1 now which enables one-click clusters]
This Better Know a VM entry, Virtual Cluster Appliances, gives an overview of VM contextualization technology which is scheduled to be part of the next workspace service release. This is not just relevant to classic grid computing, but any situation where you’d like to automatically launch many virtual machines that work together and want them to securely organize themselves and adapt to the deployment environment. It can even be used for one VM, we’ll look at such cases later.
Borja Sotomayor, my esteemed colleague, has an interesting article No CPU Left Behind about VMs and education. Check it out.
It’s been busy lately, attended the first dev.Globus All Hands Meeting and TeraGrid ‘07 right here in Madison.
At TG07, Kate gave a talk which is online. The paper she presented discusses among other things contextualization, the structure and mechanisms by which an appliance/workspace is “told” what it needs in order to adapt to its deployed environment. This is not just adaptation to site specific services but also to other appliances that may be deployed with it such as in a virtual cluster deployment.
Amidst the bustle we implemented a new backend to the Workspace Service, to Amazon’s Elastic Compute Cloud (EC2). We’ve deployed it to the University of Chicago’s Teraport cluster and will currently pay for usage by selected collaborators.
Besides being somewhat fun to implement (including getting the Globus and Amazon Secure Message stacks on the same wavelength), I think it’s going to be interesting.
Because grid resources are cautiously approaching the pioneering switch to virtualizing resources [1], even in part, it is going to be interesting and educational to see what people will be able to accomplish with workspaces when a large pool of resources is actually available on tap — today.
Because the same deployment protocols can be used for both native and EC2 resources, there are of course capacity overflow use cases. In the right situations, VMs are a good mechanism for providers to dynamically reach more consumers as the need arises.
For a feature list and description, see What is the EC2 backend?
——-
[1] and some would say inevitable switch, even with the performance costs. Consider also that ‘virtualizing resources’ may mean physical node re-imaging, cf. Virtual Workspaces: Achieving Quality of Service and Quality of Life in the Grid.
Our paper about virtual appliance configuration and management was accepted to the TeraGrid 2007 conference and is now online: A Scalable Approach To Deploying And Managing Appliances.
This paper examines configuration and security issues that large and heterogeneous deployments of virtual appliances/workspaces will face.
From the introduction:
The goal of this paper is to develop a holistic approach that would provide scalable and sustainable ways of managing and deploying virtual workspaces implemented as VM images. We will discuss ways of leveraging existing configuration management tools, exemplified by the Bcfg2 system, for VM image lifecycle management that will allow systems staff to deploy robust virtualized resources for their users. We will also describe the process of contextualization — integration of an appliance in its deployment context — and discuss its reference implementation using Bcfg2 and the Workspace Service.
— Next Page »
