Interesting #13 here (well, they’re all interesting): 25 radical network research projects you should know about.
This points us to Cloud Control with Distributed Rate Limiting which is a paper about distributed bandwidth management.
From the conclusion:
As cloud-based services transition from marketing vaporware to real, deployed systems, the demands on traditional Web-hosting and Internet service providers are likely to shift dramatically. In particular, current models of resource provisioning and accounting lack the flexibility to effectively support the dynamic composition and rapidly shifting load enabled by the software as a service paradigm. We have identified one key aspect of this problem, namely the need to rate limit network traffc in a distributed fashion, and provided two novel algorithms to address this pressing need.
Check out the summary at networkworld but also here is an excerpt from a UCSD post about it:
If half your company’s bandwidth is allocated to your mirror in New York, and it’s the middle of the night there, and your sites in London and Tokyo are slammed, that New York bandwidth is going to waste. UC San Diego computer scientists have designed, implemented, and evaluated a new bandwidth management system for cloud-based applications capable of solving this problem.
The UCSD algorithm enables distributed rate limiters to work together to enforce global bandwidth rate limits, and dynamically shift bandwidth allocations across multiple sites or networks, according to current network demand.”
Instead of a single allocation, EC2 announced you can run several different kinds of instances.
See the EC2 home page for details:
$0.10 - Small Instance (Default)
1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform
$0.40 - Large Instance
7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of instance storage, 64-bit platform
$0.80 - Extra Large Instance
15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform
In many cases it may be more cost effective to still get the small instance but just get a lot of them, this will be interesting for our workspace EC2 adapter and contextualization users (and us!). Once we make the small alterations to accomodate requesting these types, it will be just as easy to get 100 x small instance as 25 x large instance, or whatever combination, because deployment configurations can be coordinated on the fly. What would be best for what situation would have to be examined closely. An extra large instance for the virtual cluster head node(s) or storage/transfer node(s) could be extremely useful for the typical grid-cluster bottlenecks.
I’m looking forward to reading more about Blue Gene/P (coming to Argonne!). Excerpt from the press release:
ARMONK, NY - 26 Jun 2007: IBM (NYSE: IBM) today announced Blue Gene/P, the second generation of the world’s most powerful supercomputer. Blue Gene/P nearly triples the performance of its predecessor, Blue Gene/L — currently the world’s fastest computer — while remaining the most energy-efficient and space-saving computing package ever built.
The IBM® System Blue Gene®/P Solution scales to operate continuously at speeds exceeding one “petaflop” — or one-quadrillion operations per second. The system is 100,000 times more powerful than a home PC and can process more operations in one second than the combined power of a stack of laptop computers nearly 1.5 miles high. The Blue Gene/P supercomputer can be configured to reach speeds in excess of three petaflops, a performance level that many thought unattainable only a few short years ago.
In Torvalds releases 2.6.21 kernel, Steven J. Vaughan-Nichols reports at length on the 2.6.21 kernel’s clockevents and dyntick (dynamic ticks) patches.
The result of these merges is a uniform timekeeping/scheduling interface (clockevents) and the ability to take the CPU into a true idle state if nothing is going on, shutting down the timer event and instead waiting for a regular interrupt or a scheduled, future event (this is dynticks).
This could apparently help with virtualization scheduling:
In the future, both these features will be used to improve virtualization. The virtualization manager — rather than scheduling by HZ — will determine which program or virtual operating system should have the lion’s share of the processor’s time. This is not a pie in the sky idea. The technique was already being used with Linux on IBM mainframes years ago, when trying to deal with a thousand virtual Linux servers at once using HZ scheduling. It lead to situations where the timer interrupt overhead alone was using up almost all of the processors’ time.
I’d imagine a thousand virtual servers splitting the system at once could do that :-)
He points to this informative LWN article from February. That further explains clockevents and dynticks — but also suggests that going tickless during idle times is just the beginning for dynticks:
What’s in 2.6.21 is, thus, not a full dynamic tick implementation. Eliminating the tick during idle times is a good step forward, but there is value in getting rid of the tick while the system is running as well - especially on virtualized systems which may be sharing a host with quite a few other clients. The dynamic tick documentation file suggests that the developers have this goal in mind
Well. On to more important stuff this may promise: battery life! I am joining this LWN commenter, crossing my fingers:
I haven’t seen benchmarks on this in particular, but I remember reading a piece a while ago where a guy claimed that his laptop battery lasts TWICE as long with a 100Hz tick than with a 1000Hz tick… So, I’m expecting great things…
Quoting from this press release:
AgileDelta, Inc., the leading provider of software for lightning-fast delivery of more content to more locations, today announced the release of its Axis 1.x and Axis2 Web Services Integration Kits for Efficient XML, the basis for the emerging web standard for binary XML. These simple plug-in integration kits allow the quick and easy addition of Efficient XML into any web service running on Apache’s Axis 1.x or Axis2 engine. The integration kits automatically detect and use Efficient XML for clients that support it and fall back to text XML for clients that do not.
Efficient XML enables applications and web services to encode and decode XML using a cross-platform binary XML format that makes XML data much more compact and transmits that data faster, using less bandwidth and battery life than standard XML. Efficient XML documents can be over 100 times smaller and faster than XML.
Note their performance and features page.
I am still warmed by the approach taken by VTD-XML or TDX (and many others I’m sure): improving XML parsers themselves. Plaintext XML is not going anywhere and these efforts are making progress.
But Efficient XML looks poised to make a big impact, especially with smooth fallback mechanisms for supporting traditional clients.
