Virtualization and Grid Computing

This Better Know a VM entry, Virtual Cluster Appliances, gives an overview of VM contextualization technology which is scheduled to be part of the next workspace service release. This is not just relevant to classic grid computing, but any situation where you’d like to automatically launch many virtual machines that work together and want them to securely organize themselves and adapt to the deployment environment. It can even be used for one VM, we’ll look at such cases later.

The GT development team is pleased to make its newest development release available for download. The 4.1.3 release contains early looks at new features destined for GT 4.2. However, this release is not covered by the same level of support and documentation as stable GT releases. and as such is not recommended for production environments. 4.1.3 features the latest versions of the WS-Addressing and WSRF specs.

Note also that the features and public interfaces in this release are not frozen and may change in future 4.1.x releases.

Relevant links:

To download this release, go to the development Download page at
http://www.globus.org/toolkit/downloads/development/

For information on installing this release, go to the 4.1.3 Installation Guide at http://www.globus.org/toolkit/docs/development/4.1.3/admin/docbook/

Full documentation for this release starts at http://www.globus.org/toolkit/docs/development/4.1.3/

See http://gridshib.globus.org/ for news updates, several new items were just added.

Announce [2007-11-11]
Tom Scavo gives a presentation of a paper entitled A Grid Authorization Model for Science Gateways at the GCE07 Workshop at SC07.

Announce [2007-11-01]
The OASIS membership approves the Metadata Profile for the OASIS Security Assertion Markup Language (SAML) V1.x and the Metadata Extension for SAML V2.0 and V1.x Query Requesters as OASIS Standards. (GridShib implements both of these Standards.)

Announce [2007-10-31]
The OGSA Attribute Exchange Profile Version 1.0 is submitted to the OGF Authz-WG.

Announce [2007-10-12]
A paper by Tom Scavo and Von Welch entitled A Grid Authorization Model for Science Gateways has been accepted by the Grid Computing Environments (GCE) Workshop at SC07.

Announce [2007-10-01]
The GridShib SAML Tools have been integrated into the MAEViz portal.

Announce [2007-09-14]
The GridShib SAML Tools have been integrated into SimpleCred, a component of the SimpleGrid portal framework.

Kate Keahey writes:

On behalf of the workspace team, I am happy to announce the TP 1.3 release of the Workspace Service. You can download the new release from:http://workspace.globus.org/downloads/index.html

This release adds significant new features. First, the workspace service can now start multiple VM instances grafted off of the same VM image in one request; the VM instances can be managed as a group or individually. Second, we added accounting functionality and services allowing users to query accounting information. Last but not least, we added configuration enhancements to make service administration easier, as well as numerous functionality and usability enhancements for the client.

These new features and enhancements necessitated some WSDL changes as well as an addition of a new namespace. The current release has a technology preview status: both interfaces and implementation are likely to change to some extent.

This new 1.3 release provides a baseline for a cycle of releases that aim to break up the service into several replacable components that could be easily and/or independently used. In particular, the next releases will include the following functionality:

• the contextualization service allowing VMs to be adapted at deployment time to the context of a site, an organization, or other VMs
• the workspace pilot which can be used in conjunction with existing batch schedulers (such as e.g. PBS) to run VMs as well as normal jobs on the same cluster

These technologies are currently undergoing alpha testing by selected friendly communities. If you are interested in testing, and don’t mind a little bit of adventure, give us a call.

We welcome comments, feedback, and bug reports. Information about the project, software downloads, documentation and instructions on how to join the workspace-user mailing list for support questions can be found
at: http://workspace.globus.org/

Groups:

(click on the picture to read more)

Regarding the last entry, Workspace Service TP1.3, release candidate 1, the user and administrator documentation is now ready to go.

Mail sent to workspace-dev earlier:

We are busy preparing TP1.3 documentation and have put release candidate #1 online in the meantime. You can download RC1 here:

http://workspace.globus.org/vm/TP1.3/index.html

Currently the changelog, new interface description pages, and plugin description page are prepared (still considered drafts though). However, the new user and administrator guides are not online yet, see the RC1 notes at the link above.

We are only recommending past workspace service installers try out the release candidate.

The changelog contains a summary section at its beginnng, see:

http://workspace.globus.org/vm/TP1.3/index.html#changelog

Some of the additions: group deployments, usage tracking modules, configuration changes to make administration easier, client enhancements, and documentation enhancements.

Instead of a single allocation, EC2 announced you can run several different kinds of instances.

See the EC2 home page for details:

$0.10 - Small Instance (Default)

1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform

$0.40 - Large Instance

7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of instance storage, 64-bit platform

$0.80 - Extra Large Instance

15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform

In many cases it may be more cost effective to still get the small instance but just get a lot of them, this will be interesting for our workspace EC2 adapter and contextualization users (and us!). Once we make the small alterations to accomodate requesting these types, it will be just as easy to get 100 x small instance as 25 x large instance, or whatever combination, because deployment configurations can be coordinated on the fly. What would be best for what situation would have to be examined closely. An extra large instance for the virtual cluster head node(s) or storage/transfer node(s) could be extremely useful for the typical grid-cluster bottlenecks.

Quoting from workspace news:

The STAR community successfully completed its first production-size deployment of a VM-based virtual cluster managed by the workspace service and backed by EC2 resources.

The 100 node cluster was composed of a headnode and workernodes based on the OSG 0.6.0 grid middleware stack and Torque. Its deployment-time configuration was securely coordinated by the new workspace contextualization technology.

[UPDATE, related: http://www.gridvm.org/virtual-cluster-appliances.html]

[UPDATE, see: One-click clusters, VWS TP1.3.3]

Tom Scavo writes:

The GridShib Project is pleased to announce the immediate release of GridShib SAML Tools v0.2.0 Technology Preview 2, the first release of the GridShib SAML Tools specifically for developers.

http://gridshib.globus.org/downloads/gridshib-saml-tools-0_2_0-tp2-src.tar.gz
http://gridshib.globus.org/downloads/gridshib-saml-tools-0_2_0-tp2-src.zip

http://gridshib.globus.org/docs/gridshib-saml-tools-0.2.0-tp2/readme.html
http://gridshib.globus.org/docs/gridshib-saml-tools-0.2.0-tp2/install.html

http://viewcvs.globus.org/viewcvs.cgi/gridshib/saml/tool/java/doc/CHANGES.txt?revision=1.4&view=markup

Technology Preview 2 includes the following new features:

• Support for multi-valued attributes
• Complete source code distribution, including the Globus SAML Library
• New GridShib Common Java API (gridshib-common-0_2_0.jar) includes:
  • the GridShib Security Framework, a standalone implementation of the X.509 Binding for SAML Assertions
  • the Loadable interface and its implementations
  • the EntityMap interface and its implementations
  • the GridShib Entity Mapper, a container for EntityMap implementations
• Extensive javadoc documentation
• New top-level build file for developers (available from CVS only)

This is the last TP-level release of v0.2.0. We anticipate the final version of GridShib SAML Tools v0.2.0 will be released Aug 24, 2007.

VDT 1.8.0 has been released.

http://vdt.cs.wisc.edu/releases/1.8.0/release.html

Highlights of the new features since 1.6.1:

* Red Hat Enterprise Linux 5 is now supported.

* We are now using Globus 4.0.5. This release incorporates many (but not all) of the patches that we used to apply to Globus. In particular, it incorporates nearly all of the “community branch” (for TeraGrid) changes that we used apply as a set of patches. This change will make it easier to maintain our Globus build. Also, WS-GRAM received a number of fixes to improve reliability and scalability. You can see our patches to Globus. (VDT Ticket 2721)

* Java 5 is now the default instead of Java 4, except for web applications running in Tomcat 5.0. (VDT Ticket 2718)

* VOMS nows runs on 64-bit computers.

* For OSG installations, batch job managers (Condor, PBS, SGE, and LSF) will reset GLOBUS_LOCATION for jobs based on OSG_GRID. This allows GLOBUS_LOCATION to be different for the gatekeeper and the worker nodes. This has been tested on all platforms, but it might not be working on LSF — the current behavior on LSF seems to simply not change GLOBUS_LOCATION, so it is no worse than it used to be.

* There is now an RPM for installing the CA certificates. It has been lightly tested. If you want to use it, it’s best to tell the VDT not to install the CA certificates at all, and then just use the RPM.

* There is now a program, vdt-update-certs, for automatically updating the CA Certificates. It is a cron job that is optionally enabled.

* The way in which web applications are installed into Tomcat 5.0 and 5.5 is now more robust and more consistent with Tomcat best practices.

* Pacman 3.20 is now the minimum supported version of Pacman. It is substantially faster than previous versions of pacman, which makes a VDT install faster.

VDT Team

Tom Scavo writes of two new GridShib technology preview releases:

The GridShib Project is pleased to announce the simultaneous release of GridShib SAML Tools v0.2.0 Technology Preview 1 and GridShib for GT v0.6.0 Technology Preview 4. These two components work together to seamlessly integrate SAML into a Globus grid environment.

To evaluate this new software technology, please refer to this Quick Start:

http://gridshib.globus.org/docs/gridshib-gt-0.6.0-tp4/quick-start.html

The Quick Start guide gives detailed instructions for installing, configuring, and using Globus Java WS Core 4.0.5, GridShib for GT v0.6.0, and GridShib SAML Tools v0.2.0.

For more information and links, see http://www.globus.org/mail_archive/gridshib-dev/2007/07/msg00047.html