MorphLabs, the Meta-Cloud Vendor?

Fascinating!  I just got an email from G2iX, the parent company of MorphLabs, inviting me to a webinar (Wed, Dec. 9, 2009 from 11AM - 12PM PST) in which they plan to demonstrate how to "Morph your Data Center into a Cloud Vendor":

Traditional data center and co-location vendors are under tremendous threat in the face of Amazon EC2 and related Web Services, and those providers who’re unable to adapt to some sort of cloud computing model face extinction as competing Infrastructure as a Service products become even more prevalent.

This webinar will demonstrate how the Morph Cloud Computing Platform can help regional data centers transform their business using their existing assets with minimal cost and hardware investments so that they can provide a cutting edge cloud solution to a broader customer base with rates that are more than competitive while still maintaining great profit margin.

For those who are unfamiliar with the company, MorphLabs provides a hosting platform that runs on top of Amazon's EC2 web service.  So, assuming I'm not mistaken, G2iX plans to help data centers other than Amazon's set up the Morph platform to run on their own servers.  My first thought was, "Why would they do that?  Won't this just create competition for MorphLabs?!"  However, once I mulled it over, I realized that if they structure things right, it could be quite beneficial to MorphLabs.  What if these Morph platform deployments simply became new pools of servers to which Morph apps could be deployed?  Viewed that way, it actually sounds quite clever.  I'm almost tempted to attend the webinar, even though it has no direct relevance to me. :)

AWS Elastic MapReduce Webinar @Noon ET on May 28

Amazon's new Elastic MapReduce service makes it easier to run Map/Reduce jobs within EC2. If that sounds interesting, you can find out more about it by registering and "attending" the webinar they're holding on Thursday, May 28 at Noon ET. I'll be there, in a virtual sense. :)

High Performance, Parallel, & Grid Computing

On Wednesday night (May 6th), I attended an alumni event held by the UMass Boston CS Department in the beautiful five year old Campus Center.  It was great to see my professors and former classmates, along with talking with current students about their software engineering projects.  Toward the end of the event, there was a talk given by an alumnus named Richard Anderson, the CTO of Symmetric Computing, a company that makes hardware and OS-level software for High Performance Computing (HPC).  Their slogan is something along the lines of “Supercomputing for the Masses”.  The speaker gave a fairly typical overview of the history of CPU performance, including references to Moore’s Law, hitting the gigahertz wall, and the emergence of multi-core chips, followed by a discussion of parallel programming.  He mentioned several times that the company would be making one of their 48 core/350GB systems available to the university for academic use.  It was a decent talk, if not extremely exciting.  It made me think of concurrent programming, grid/cloud computing, and the cool compute appliances made by Azul Systems.  Based on questions asked by myself and others, I was disappointed to discover that although the technical gap between HPC and the rest of the computing world is narrowing, most of the HPC world seems unaware of this.  A lot of work has gone into making clusters of commodity hardware work together and coding for these architectures easier (think Google, Map/Reduce, etc.).  It would be nice if people in HPC and distributed computing could do a better job of exchanging ideas.  Even if typical clusters have to deal with latency issues and the inability to directly access large amounts of memory, that doesn’t explain why someone should code an HPC app in Fortran or C on a Symmetric box instead of in Java on a 432 core/384GB Azul box.
I realize that this is a bit of a rant.  If any readers can provide counter-arguments or counter-examples, I’d be happy (and reassured) to read and know about them.
Update: I forgot to mention that if you’re doing academic research or teaching a class in clustering/distributed computing, you should check out the fairly new Amazon Web Services in Education program.  They provide free credits to educators, researchers, and students.  Pretty cool!

I'm Quoted in PCWorld!

Juan Carlos Perez from IDG News Service (the parent company of PCWorld) contacted me today to ask if I'd be interested in speaking with him about the Amazon Web Services offerings. He explained that he was writing an article on the subject and wanted to hear how customers felt about it. We had a nice chat late this morning. I let him know that I've experimented with EC2 and S3, but haven't done any production work with either. I made sure to plug the Boston Scalability User Group (of which I've been a member since the first meeting) and mentioned that we'd already had Mike Culver of Amazon as a speaker.

The result was an article which appeared in PCWorld: "Amazon.com Eyes CIOs With Its AWS Cloud IT Services". My quotes were pretty clunky (apparently me thinks and me writes more gooder than me talks :) ), but I think readers will at least be able to figure out what I meant.

Update: the article also appears in InfoWorld and ComputerWorld Norway

Another Update: I contacted the reporter about the clunkiness of my quotes.  He said that since the article had already been published, he would have to run any changes by his editors and it would effectively be treated as a retraction/correction.  That's obviously overkill, so I guess I'll just have to live it and follow his suggestion for next time: ask the author to run quotes by me before the article is filed.

Amazon EC2 Reserved Instances

There have been some exciting developments recently in the world of Amazon EC2. For a long while, they've had compelling offerings for batch-oriented services and ones requiring dynamic scaling, but weren't as strong from a pricing perspective for interactive services with fairly steady demands. Well, that's no longer the case. Amazon just announced the introduction of "reserved instances" on EC2, which are different from regular ones in two primary ways. Reserved instances are guaranteed to be available when you ask to activate them and their pricing structure involves an up-front fee in return for a lower hourly operating cost. The availability guarantees are certainly an interesting development, but I'm not going to discuss those here. What currently excites me are the pricing options and what they mean for some architectures and demand patterns.

Others have already done the math comparing Amazon's reserved instance pricing with other hosting options and with standard EC2 on-demand pricing, but I've haven't seen anyone pointing out where the break-even point lies for a reserved instance versus an on-demand one. For example, a small instance costs 10 cents/hour on-demand and $325 (1 year plan) or $500 (3 year plan) and 3 cents/hour reserved. You'll only need to use a reserved instance for 4643 hours on a 1 year plan or 7143 hours on a 3 year plan before the reserved instance starts being cheaper per hour on average than an on-demand one. I've based this on the following equation: .1 * hours = upFrontCost + .03 * hours. For a 1 year plan, that works out to be 193.5 days/year, 16.1 days/month, or 53% of the time. For a 3 year plan, it's 2381 hours/year, 99.2 days/year, 8.3 days/month, or 27.2% of the time. (I'm not factoring in the time value of money here [paying $325 or $500 up-front is not the same as being able to spread it out over 1 to 3 years]. I'll leave it to the reader to do that.)

What does this all mean? If your architecture or typical demand requires 3 instances be running all the time, but you want to be able to dynamically scale up your instance usage, you can reserve the 3 consistently required instances and use on-demand ones to handle spikes in demand. In fact, it might even be worth reserving more instances even if you expect (with higher than 53% confidence), but aren't sure, you'll need them.

Update: On August 20, Amazon announced lower pricing for reserved instances.  I'd redo the math above, but Thomas Brox Røst did a particularly nice job in a post on his blog, so I'll let that do the talking.

RightScale - managing Amazon EC2 instances

I was recently reading some postings on High Scalability and came upon one discussing some cool new features in EC2. It also mentioned a company, RightScale, which has a product for automatically managing your EC2 instances, including automatically launching and killing them based on spikes in demand. The company also has a good EC2-related blog, which unfortunately seems not to have an available RSS feed (odd!).